Commit Graph

531 Commits

Author SHA1 Message Date
sam boyer
e5a6547a94
coremodels: Update to latest Thema with generics (#56602)
* Update thema to latest

* Deal with s/Library/*Runtime/

* Commit new, working results of codegen
2022-10-11 09:45:07 +01:00
Artur Wierzbicki
bf264d2f76
Query library: requiresDevMode dummy backend (#56466)
* query library - dummy backend

* fix tests

* dont explicitly marshall backend dataresponse

* skip integration tests

* null check for tests

* added query library to codeowners

* null check for tests

* lint
2022-10-07 11:31:45 -07:00
Joey Tawadrous
74c809f544
Plugins: Introduce new Flame graph panel (#56376)
* Flamegraph

* Updated flame graph width/height values

* Fix top table rendering issue

* Add feature toggle for flamegraph in explore

* Update tests

* Hide flamegraph from dash panel viz list if feature toggle not enabled

* Show table if no flameGraphFrames

* Add flame graph to testdata ds

* Minor improvement
2022-10-07 11:39:14 +01:00
Will Browne
1038ac7308
DataSources: Remove unused cfg from core Grafana data source (#56188)
* remove unused cfg from grafana ds

* fix test
2022-10-04 06:32:25 -04:00
Will Browne
4087ad413f
re-use fake trace + feature toggles (#56186) 2022-10-04 10:15:10 +02:00
Torkel Ödegaard
e31cb93ec0
NavTree: Make it possible to configure where in nav tree plugins live (#55484)
* NewIA: Plugin nav config

* progress

* Progress

* Things are working

* Add monitoring node

* Add alerts and incidents

* added experiment with standalone page

* Refactoring by adding a type for navtree root

* First test working

* More tests

* more tests

* Progress on richer config and sorting

* Sort weight working

* Path config

* Improving logic for not including admin or cfg nodes, making it the last step so that enterprise can add admin nodes without having to worry about the section not existing

* fixed index routes

* removed file

* Fixes

* Fixing tests

* Fixing more tests and adding support for weight config

* Updates

* Remove unused fake

* More fixes

* Minor tweak

* Minor fix

* Can now control position using sortweight even when existing items have no sortweight

* Added tests for frontend standalone page logic

* more tests

* Remove unused fake and fixed lint issue

* Moving reading settings to navtree impl package

* remove nav_id setting prefix

* Remove old test file

* Fix trailing newline

* Fixed bug with adding nil node

* fixing lint issue

* remove some code we have to rethink

* move read settings to PrivideService and switch to util.SplitString
2022-09-28 08:29:35 +02:00
Kristin Laemmert
701f6d5436
UserService: use the UserService instead of calling sqlstore directly (#55745)
* UserService: update callers to use the UserService instead of calling sqlstore directly

There is one major change hiding in this PR. UserService.Delete originally called a number of services to delete user-related records. I moved everything except the actual call to the user table, and moved those into the API. This was done to avoid dependencies cycles; many of our services depend on the user service, so the user service itself should have as few dependencies as possible.
2022-09-27 07:58:49 -04:00
Will Browne
d0d8544ded
Plugins: Create single point of entry for adding / removing plugins (#55463)
* split out plugin manager

* remove whitespace

* fix tests

* split up tests

* updating naming conventions

* simplify manager

* tidy

* explorations

* fix build

* tidy

* fix tests

* add logger helper

* pass the tests

* tidying

* fix tests

* tidy and re-add test

* store depends on loader

* enrich tests

* fix test

* undo gomod changes
2022-09-23 14:27:01 +02:00
Artur Wierzbicki
03af63d52e
SearchV2: add tracing to background jobs (#55250)
* searchv2: add tracing to background jobs

* searchv2: lint

* searchv2: lint

* searchv2: fix context passing

* searchv2: add init org index span

* searchv2: add traceid to logs

* searchv2: add db count to logs
2022-09-20 16:49:44 +02:00
idafurjes
7ce7c9b64c
Chore: Move SearchOrgs to org service (#55416)
* Chore: Move SearchOrgs to org service

* Fix lint

* Fix lint 2
2022-09-20 09:55:40 +02:00
Will Browne
cd7a464f7e
Plugins: Ensure plugin uninstall success for "pre-installed" community plugins (#55322)
* track pre-installed external plugins in store

* fix tests
2022-09-19 12:14:30 +02:00
Will Browne
29327cbba2
Plugins: Use error plane for api/ds/query (#54750)
* plugin client returns error base

* fix api test

* add plugin client test

* add fallback err

* fix linting

* wip

* replace bad query

* template is an error

* failing test of templated error

* add one test passing

* fix failing test

* move test

* rename ErrBadQuery to ErrQueryValidationFailure

* tidy diff

* Change to one error per specific error kind

* last err + fix test

* fix imports

* more tests

* keep req vars together

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-09-14 18:19:57 +02:00
sam boyer
ced53a8dc2
plugins: Introduce generated, static core plugin registry (#54118)
* Refactor towards template/codegen framework

* Add templates for plugin gen

* Add Go codegen for plugins; overhaul framework, too

* Add new codegen output; assorted framework fixes

* Regenerate after merge

* Remove accidental commit file, update templates

* Export the pfs.Tree loader from plugin types

* Print details from cuetsy errors

* Generate loaders for all plugins and list in registry

* Use pfs_gen.go over lineage_gen.go

* Un-un-ignore main file

* Introduce simple List static registry for plugins

* Last tweaks to codegen

* remove unused tvars

* Ensure loop-local instances for both vars

* Generate pfs parsing in-place in registry

* Stop generating pfs_gen.go

* Move Tree into pfs, rename subdir

* Change package name to match dir

* Ignore gocyclo on HTTPServer.getNavTree
2022-09-14 10:15:09 -04:00
Will Browne
0571d98bba
[WIP] Plugins: Introduce Plugins specific config (#54854) 2022-09-14 14:35:35 +01:00
Emil Tullstedt
b287047052
Chore: Upgrade Go to 1.19.1 (#54902)
* WIP

* Set public_suffix to a pre Ruby 2.6 version

* we don't need to install python

* Stretch->Buster

* Bump versions in lib.star

* Manually update linter

Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.

Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.

* undo reformatting

* Various lint improvements

* More from the linter

* goimports -w ./pkg/

* Disable gocritic

* Add/modify linter exceptions

* lint + flatten nested list

Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
2022-09-12 12:03:49 +02:00
Gabriel MABILLE
101349fe49
RBAC: Add permissions to install and configure plugins (#51829)
* RBAC: Allow app plugins restriction

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* Moving declaration to HttpServer

Co-Authored-By: marefr <marcus.efraimsson@gmail.com>

* Picking changes from the other branch

Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>

* Rename plugins.settings to plugins

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* Account for PluginAdminExternalManageEnabled

Co-authored-by: Will Browne <will.browne@grafana.com>

* Set metadata on instantiation

Co-authored-by: Jguer <joao.guerreiro@grafana.com>

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: marefr <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Jguer <joao.guerreiro@grafana.com>
2022-09-09 09:44:50 +02:00
Will Browne
15756ecf86
remove unused cfg (#54770) 2022-09-06 08:59:28 -04:00
Will Browne
ecdcafb258
manager exposes renderer + secrets manager (#54629) 2022-09-02 14:20:10 +02:00
Will Browne
43987e7f8c
Plugins: Fix plugin registered startup log line (#54634)
* fix plugin registered log

* remove stale test
2022-09-02 14:01:52 +02:00
Ryan McKinley
55fe1506ba
Plugins: unlock on plugin startup failure (#54555) 2022-08-31 16:02:07 -04:00
Will Browne
4a707e2a88
Plugins: Split plugin manager into smaller components (#54384)
* split out plugin manager

* remove whitespace

* fix tests

* split up tests

* updating naming conventions

* simplify manager

* tidy

* add more fakes

* testing time

* add query verif to int test

* renaming

* add process tests

* tidy up manager tests

* add extra case to int test

* add more coverage to store and process tests

* remove comment

* fix capatilization

* init on provide

* remove addfromsource from API
2022-08-30 17:30:43 +02:00
Joan López de la Franca Beltran
6ec06f66b9
Rendering: Add support for renderer token (#54425)
(cherry picked from commit a4f75cc0438712c90b02d24740416f8615e3a0cb)
2022-08-30 12:09:38 +02:00
Giordano Ricci
c68d7f1e35
Correlations: Add CorrelationSettings Page (#53821)
* GrafanaUI: add option to close DeleteButton on confirm click

* add datasource readOnly info to frontend settings

* move isTruthy utility type guard

* add generic non-visualization table component

* Add correlations settings page

* add missing readOnly in mock

* Fix typo

* avoid reloading correlations after add/remove

* use DeepPartial from rhf

* validate source data source

* fix validation logic

* fix navmodel test

* add missing readonly property

* remove unused styles

* handle multiple clicks on elements

* better UX for loading states

* fix remove handler

* add glue icon
2022-08-26 11:27:28 +01:00
Karl Persson
5a1b9d2283
RBAC: Remove DeclareFixedRoles wrapper on Access control and inject service (#54153)
* RBAC: Remove DeclareFixedRoles wrapper on Access control and inject service when needed
2022-08-26 09:59:34 +02:00
Marcus Efraimsson
cd5f27a25d
Plugins: Propagate distributed tracing headers to backend plugins (#54107) 2022-08-24 11:55:49 +02:00
Guilherme Caulada
d90600c454
Secrets: Add fallback to secrets kvstore plugin (#54056)
* Add fallback to secrets kvstore plugin

* Fix linter issues

* Fix linter issues

* Add deletion error to bool to fake secrets kvstore

* Add fallback to fake secrets kvstore

* Fix fake secrets kvstore fallback setter

* Use Key on Item message for secrets manager protobuf

* Add clarifying comment about fallback
2022-08-23 12:21:54 -03:00
Will Browne
26dfdd5af3
Plugins: Refactor plugin download/installation (#43046)
* installer -> repo

* add semver format checking

* add plugin callbacks in test

* remove newline

* post install only scans new directories

* remove unused stuff

* everything in own package

* add missing cli params

* make grafana version part of the API

* resolve conflicts

* tidy up logger

* fix cli and tidy log statements

* rename log package

* update struct name

* fix linter issue

* fs -> filestore

* reorder imports

* alias import

* fix test

* fix test

* inline var

* revert jsonc file

* make repo dep of manager

* actually inject the thing

* accept all args for compatability checks

* accept compat from store

* pass os + arch vals

* don't inject fs

* tidy up

* tidy up

* merge with main and tidy fs storage

* fix test

* fix packages

* fix comment + field name

* update fs naming

* fixed wire

* remove unused func

* fix mocks

* fix storage test

* renaming

* fix log line

* fix test

* re-order field

* tidying

* add test for update with same version

* fix wire for CLI

* remove use of ioutil

* don't pass field

* small tidy

* ignore code scanning warn

* fix testdata link

* update lgtm code
2022-08-23 11:50:50 +02:00
sam boyer
4d433084a5
plugins: New static scanner and validator, with Thema slot support (#53754)
* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add slot definitions

* Start sketching out pfs package

* Rerun codegen with fixes, new cuetsy

* Catch up dashboard with new cuetsy

* Update to go1.18

* Use new vmuxers in thema

* Add slot system in Go

* Draft finished implementation of pfs

* Collapse slot pkg into coremodel dir; add PluginInfo

* Add the mux type on top of kernel

* Refactor plugin generator for extensibility

* Change models.cue package, numerous debugs

* Bring new output to parity with old

* Remove old plugin generation logic

* Misc tweaking

* Reintroduce generation of shared schemas

* Drop back to go1.17

* Add globbing to tsconfig exclude

* Introduce pfs test on existing testdata

* Make most existing testdata tests pass with pfs

* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add APIType control concept, regen pluginmeta

* Use proper numeric types for schema fields

* Make pluginmeta schema follow Go type breakdown

* More decomposition into distinct types

* Add test case for no plugin.json file

* Fix missing ref to #Dependencies

* Remove generated TS for pluginmeta

* Update dependencies, rearrange go.mod

* Regenerate without Model prefix

* Use updated thema loader; this is now runnable

* Skip app plugin with weird include

* Make plugin tree extractor reusable

* Split out slot lineage load/validate logic

* Add myriad tests for new plugin validation failures

* Add test for zip fixtures

* One last run of codegen

* Proper delinting

* Ensure validation order is deterministic

* Let there actually be sorting

* Undo reliance on builtIn field (#54009)

* undo builtIn reliance

* fix tests

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 12:11:45 -04:00
idafurjes
a14621fff6
Chore: Add user service method SetUsingOrg and GetSignedInUserWithCacheCtx (#53343)
* Chore: Add user service method SetUsingOrg

* Chore: Add user service method GetSignedInUserWithCacheCtx

* Use method GetSignedInUserWithCacheCtx from user service

* Fix lint after rebase

* Fix lint

* Fix lint error

* roll back some changes

* Roll back changes in api and middleware

* Add xorm tags to SignedInUser ID fields
2022-08-11 13:28:55 +02:00
Jo
ca72cd570e
Remove ioutil.ReadDir from usage (#53550)
* add depguard rule for ioutil

* replace ioutil.ReadDir with os.ReadDir

* use legacy option in depguard supported in golangci-lint v1.40

* replace ioutil.ReadDir with os.ReadDir

* return error for file info
2022-08-11 07:21:12 -04:00
Jo
062d255124
Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
idafurjes
6afad51761
Move SignedInUser to user service and RoleType and Roles to org (#53445)
* Move SignedInUser to user service and RoleType and Roles to org

* Use go naming convention for roles

* Fix some imports and leftovers

* Fix ldap debug test

* Fix lint

* Fix lint 2

* Fix lint 3

* Fix type and not needed conversion

* Clean up messages in api tests

* Clean up api tests 2
2022-08-10 11:56:48 +02:00
Will Browne
6d066a7aa8
Plugins: Add validation for plugin manifest (#52787)
* add validation for plugin manifest

* no more semver checking

* undo go.mod changes

* undo go.mod changes

* only validate v2 fields where necessary

* remove manifest version field check
2022-07-27 12:56:52 +02:00
Will Browne
843ff932b7
Plugins: Add signature wildcard globbing for dedicated private plugin type (#52163)
* it's glob time

* add more tests

* fix warn log

* support new sig type
2022-07-22 10:48:16 +02:00
Todd Treece
b5d57c45e3
Plugins: Add support for HTTP logger (#46578) 2022-07-21 09:46:47 -04:00
Will Browne
5d052be6ff
Plugins: Always validate root URL if specified in signature manfiest (#52332)
* always validate root URL if specified in signature

* tidy imports
2022-07-15 17:42:20 +02:00
Michael Mandrus
9aa6ce2a50
Datasource: Propagate datasource secret decryption errors to the frontend (#52068)
* update decrypt secrets function signature and add secrets error handling

* remove a couple instances of unnecessary logging since errors are properly handled now

* add unit test

* fix linting issues
2022-07-13 09:27:03 -04:00
Gabriel MABILLE
5975c4bc6d
RBAC: Allow app plugins access restriction (#51524)
* RBAC: Allow app plugins restriction

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* Fix tests

* Imports

* WIP

* Adding RBAC to AppPluginsRoutes

* Switching middleware order

* Restrict access to resources

* Nit

* Cosmetic changes

* Fix fallback

* Moving declaration to HttpServer

Co-Authored-By: marefr <marcus.efraimsson@gmail.com>

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: marefr <marcus.efraimsson@gmail.com>
2022-07-08 13:24:09 +02:00
Artur Wierzbicki
e96f67ae2e
Renderer: Add sanitize API (#50936)
* svg fun

* #50597: add proto

* #50597: add sanitizer methods

* #50597: add provider

* #50597: use sanitizer

* #50597: use sanitizer

* update grafana to match new api

* add comments

* add capability check

* add timing

* update sanitize path

* improve log message

* strings.HasPrefix rather than filepath.IsAbs

* filepath.Clean + filepath.ToSlash for windows

* read 404

* remove `path.clean` from `getPathAndScope`

* add resp body close

* remove unneeded prop

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* remove test files

* filepath.ToSlash correct wrapping

* filepath.ToSlash correct wrapping

* filepath.ToSlash comment

* compilation error

* lint fix

* fix error message

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* add `image/svg+xml` mime type

* refactored log

* refactored log

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>
2022-07-07 15:32:18 +04:00
Will Browne
0b40c8e51e
Plugins: Add missing asserts for Plugin Errors field (#51702)
* make sure to check plugin errors

* remove unused test
2022-07-04 11:03:50 +00:00
Michael Mandrus
7ef21662f9
Plugins: Show that Secrets Manager Plugin is active in the UI (#50953)
* add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not

* show disabled badge in front end if the plugin is not enabled

* Only show error in disabled badge hover if one is present (otherwise it shows "undefined")

* refactor to make use of fields already available in the DTO

* fix typo

* if there is no error returned for the plugin, just show 'disabled'

* fix typo

* Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update frontendsettings.go

add clarifying comment

* fix unit test

* rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled

* Update helpers.test.ts

revert test change

* fix unit test

* bogus commit to trigger precommit

* undo commit

* run precommit manually

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-06-29 08:48:23 -04:00
Kristin Laemmert
945f015770
backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
Marcus Efraimsson
04df634ef5
Plugins: Improved handling of symlinks (#51324)
Improves how we handle symlinks in plugin management.
2022-06-27 12:24:02 +02:00
Ryan McKinley
dd5a3b7747
Heatmap: Replace the heatmap panel with new implementation (#50229) 2022-06-23 11:48:28 -07:00
Marcus Efraimsson
a8eb29f1d7
Plugins: Use a Grafana specific SDK logger implementation for core plugins (#51229)
Upgrade grafana-aws-sdk to v0.10.6
2022-06-23 14:34:25 +02:00
Marcus Efraimsson
b47ec36d0d
CLI: Allow relative symlinks in zip archives when installing plugins (#50537)
Earlier we only allowed symlinks in plugins starting with grafana- in zip archives when 
installing plugins using the grafana-cli. This changes so that symlinks in zip archives 
containing relative links to files in the zip archive are always allowed when installing 
plugins. The reasoning behind this is that Grafana per default doesn't load a plugin 
that has an invalid plugin signature meaning that any symlink must be included in 
the plugin signature manifest.

Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-06-15 14:38:59 +02:00
Ben Kochie
68691d7775
Convert some metrics to Histograms (#50420)
Because Summary metrics can not be aggreated, convert them to histograms
so that users with HA deployments can use these metrics.
* Convert metrics registration to promauto.
* Improve help text style.

Signed-off-by: SuperQ <superq@gmail.com>
2022-06-15 13:19:43 +02:00
Michael Mandrus
d886141d69
Plugins: Refactor secrets plugin .proto file (#50722)
* refactor .proto file according to prior PR suggestions

* re-generate protobuf files and fix compilation errors

* rename proto functions to be more descriptive
2022-06-14 11:53:04 -04:00
Gábor Farkas
8fd9cb4854
loki: add feature-toggle check to live-mode (#49012) 2022-06-13 08:33:46 +02:00
Michael Mandrus
f376c33903
WIP: Add private Secret Manager Plugins support to plugin platform (#49544)
* Add protobuf config and generated code, and client wrapper

* wire up loading of secretsmanager plugin, using renderer plugin as a model

* update kvstore provider to check if we should use the grpc plugin. return false always in OSS

* add OSS remote plugin check

* refactor wire gen file

* log which secrets manager is being used

* Fix argument types for remote checker

* Turns out if err != nil, then the result is always nil. Return empty values if there is an error.

* remove duplicate import

* Update pkg/services/secrets/kvstore/kvstore.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update pkg/services/secrets/kvstore/kvstore.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* refactor RemotePluginCheck interface to just return the Plugin client directly

* rename struct to something less silly

* Update pkg/plugins/backendplugin/secretsmanagerplugin/secretsmanager.proto

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-06-09 13:19:27 -04:00