Plugins: Angular detector: Remote patterns fetching (#69843)

* Plugins: Angular detector: Remote patterns fetching

* Renamed PatternType to GCOMPatternType

* Renamed files

* Renamed more files

* Moved files again

* Add type checks, unexport GCOM structs

* Cache failures, update log messages, fix GCOM URL

* Fail silently for unknown pattern types, update docstrings

* Fix tests

* Rename gcomPattern.Value to gcomPattern.Pattern

* Refactoring

* Add FlagPluginsRemoteAngularDetectionPatterns feature flag

* Fix tests

* Re-generate feature flags

* Add TestProvideInspector, renamed TestDefaultStaticDetectorsInspector

* Add TestProvideInspector

* Add TestContainsBytesDetector and TestRegexDetector

* Renamed getter to provider

* More tests

* TestStaticDetectorsProvider, TestSequenceDetectorsProvider

* GCOM tests

* Lint

* Made detector.detect unexported, updated docstrings

* Allow changing grafana.com URL

* Fix API path, add more logs

* Update tryUpdateRemoteDetectors docstring

* Use angulardetector http client

* Return false, nil if module.js does not exist

* Chore: Split angualrdetector into angularinspector and angulardetector packages

Moved files around, changed references and fixed tests:
- Split the old angulardetector package into angular/angulardetector and angular/angularinspector
- angulardetector provides the detection structs/interfaces (Detector, DetectorsProvider...)
- angularinspector provides the actual angular detection service used directly in pluginsintegration
- Exported most of the stuff that was private and now put into angulardetector, as it is not required by angularinspector

* Renamed detector.go -> angulardetector.go and inspector.go -> angularinspector.go

Forgot to rename those two files to match the package's names

* Renamed angularinspector.ProvideInspector to angularinspector.ProvideService

* Renamed "harcoded" to "static" and "remote" to "dynamic"

from PR review, matches the same naming schema used for signing keys fetching

* Fix merge conflict on updated angular patterns

* Removed GCOM cache

* Renamed Detect to DetectAngular and Detector to AngularDetector

* Fix call to NewGCOMDetectorsProvider in newDynamicInspector

* Removed unused test function newError500GCOMScenario

* Added angularinspector service definition in pluginsintegration

* Moved dynamic inspector into pluginsintegration

* Move gcom angulardetectorsprovider into pluginsintegration

* Log errUnknownPatternType at debug level

* re-generate feature flags

* fix error log
This commit is contained in:
Giuseppe Guerra 2023-06-26 15:33:21 +02:00 committed by GitHub
parent 903af7e29c
commit cca9d89733
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
25 changed files with 949 additions and 282 deletions

View File

@ -72,50 +72,51 @@ Some features are enabled by default. You can disable these feature by setting t
These features are early in their development lifecycle and so are not yet supported in Grafana Cloud.
Experimental features might be changed or removed without prior notice.
| Feature toggle name | Description |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| `live-service-web-worker` | This will use a webworker thread to processes events rather than the main thread |
| `queryOverLive` | Use Grafana Live WebSocket to execute backend queries |
| `lokiExperimentalStreaming` | Support new streaming approach for loki (prototype, needs special loki build) |
| `storage` | Configurable storage for dashboards, datasources, and resources |
| `newTraceViewHeader` | Shows the new trace view header |
| `datasourceQueryMultiStatus` | Introduce HTTP 207 Multi Status for api/ds/query |
| `traceToMetrics` | Enable trace to metrics links |
| `prometheusWideSeries` | Enable wide series responses in the Prometheus datasource |
| `canvasPanelNesting` | Allow elements nesting |
| `scenes` | Experimental framework to build interactive dashboards |
| `disableSecretsCompatibility` | Disable duplicated secret storage in legacy tables |
| `logRequestsInstrumentedAsUnknown` | Logs the path for requests that are instrumented as unknown |
| `redshiftAsyncQueryDataSupport` | Enable async query data support for Redshift |
| `athenaAsyncQueryDataSupport` | Enable async query data support for Athena |
| `showDashboardValidationWarnings` | Show warnings when dashboards do not validate against the schema |
| `mysqlAnsiQuotes` | Use double quotes to escape keyword in a MySQL query |
| `showTraceId` | Show trace ids for requests |
| `alertingBacktesting` | Rule backtesting API for alerting |
| `editPanelCSVDragAndDrop` | Enables drag and drop for CSV and Excel files |
| `lokiQuerySplitting` | Split large interval queries into subqueries with smaller time intervals |
| `lokiQuerySplittingConfig` | Give users the option to configure split durations for Loki queries |
| `individualCookiePreferences` | Support overriding cookie preferences per user |
| `onlyExternalOrgRoleSync` | Prohibits a user from changing organization roles synced with external auth providers |
| `traceqlSearch` | Enables the 'TraceQL Search' tab for the Tempo datasource which provides a UI to generate TraceQL queries |
| `timeSeriesTable` | Enable time series table transformer & sparkline cell type |
| `prometheusResourceBrowserCache` | Displays browser caching options in Prometheus data source configuration |
| `influxdbBackendMigration` | Query InfluxDB InfluxQL without the proxy |
| `clientTokenRotation` | Replaces the current in-request token rotation so that the client initiates the rotation |
| `disableSSEDataplane` | Disables dataplane specific processing in server side expressions. |
| `alertStateHistoryLokiSecondary` | Enable Grafana to write alert state history to an external Loki instance in addition to Grafana annotations. |
| `alertStateHistoryLokiPrimary` | Enable a remote Loki instance as the primary source for state history reads. |
| `alertStateHistoryLokiOnly` | Disable Grafana alerts from emitting annotations when a remote Loki instance is available. |
| `unifiedRequestLog` | Writes error logs to the request logger |
| `pyroscopeFlameGraph` | Changes flame graph to pyroscope one |
| `extraThemes` | Enables extra themes |
| `lokiPredefinedOperations` | Adds predefined query operations to Loki query editor |
| `pluginsFrontendSandbox` | Enables the plugins frontend sandbox |
| `cloudWatchLogsMonacoEditor` | Enables the Monaco editor for CloudWatch Logs queries |
| `exploreScrollableLogsContainer` | Improves the scrolling behavior of logs in Explore |
| `recordedQueriesMulti` | Enables writing multiple items from a single query within Recorded Queries |
| `alertingLokiRangeToInstant` | Rewrites eligible loki range queries to instant queries |
| `flameGraphV2` | New version of flame graph with new features |
| Feature toggle name | Description |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| `live-service-web-worker` | This will use a webworker thread to processes events rather than the main thread |
| `queryOverLive` | Use Grafana Live WebSocket to execute backend queries |
| `lokiExperimentalStreaming` | Support new streaming approach for loki (prototype, needs special loki build) |
| `storage` | Configurable storage for dashboards, datasources, and resources |
| `newTraceViewHeader` | Shows the new trace view header |
| `datasourceQueryMultiStatus` | Introduce HTTP 207 Multi Status for api/ds/query |
| `traceToMetrics` | Enable trace to metrics links |
| `prometheusWideSeries` | Enable wide series responses in the Prometheus datasource |
| `canvasPanelNesting` | Allow elements nesting |
| `scenes` | Experimental framework to build interactive dashboards |
| `disableSecretsCompatibility` | Disable duplicated secret storage in legacy tables |
| `logRequestsInstrumentedAsUnknown` | Logs the path for requests that are instrumented as unknown |
| `redshiftAsyncQueryDataSupport` | Enable async query data support for Redshift |
| `athenaAsyncQueryDataSupport` | Enable async query data support for Athena |
| `showDashboardValidationWarnings` | Show warnings when dashboards do not validate against the schema |
| `mysqlAnsiQuotes` | Use double quotes to escape keyword in a MySQL query |
| `showTraceId` | Show trace ids for requests |
| `alertingBacktesting` | Rule backtesting API for alerting |
| `editPanelCSVDragAndDrop` | Enables drag and drop for CSV and Excel files |
| `lokiQuerySplitting` | Split large interval queries into subqueries with smaller time intervals |
| `lokiQuerySplittingConfig` | Give users the option to configure split durations for Loki queries |
| `individualCookiePreferences` | Support overriding cookie preferences per user |
| `onlyExternalOrgRoleSync` | Prohibits a user from changing organization roles synced with external auth providers |
| `traceqlSearch` | Enables the 'TraceQL Search' tab for the Tempo datasource which provides a UI to generate TraceQL queries |
| `timeSeriesTable` | Enable time series table transformer & sparkline cell type |
| `prometheusResourceBrowserCache` | Displays browser caching options in Prometheus data source configuration |
| `influxdbBackendMigration` | Query InfluxDB InfluxQL without the proxy |
| `clientTokenRotation` | Replaces the current in-request token rotation so that the client initiates the rotation |
| `disableSSEDataplane` | Disables dataplane specific processing in server side expressions. |
| `alertStateHistoryLokiSecondary` | Enable Grafana to write alert state history to an external Loki instance in addition to Grafana annotations. |
| `alertStateHistoryLokiPrimary` | Enable a remote Loki instance as the primary source for state history reads. |
| `alertStateHistoryLokiOnly` | Disable Grafana alerts from emitting annotations when a remote Loki instance is available. |
| `unifiedRequestLog` | Writes error logs to the request logger |
| `pyroscopeFlameGraph` | Changes flame graph to pyroscope one |
| `extraThemes` | Enables extra themes |
| `lokiPredefinedOperations` | Adds predefined query operations to Loki query editor |
| `pluginsFrontendSandbox` | Enables the plugins frontend sandbox |
| `cloudWatchLogsMonacoEditor` | Enables the Monaco editor for CloudWatch Logs queries |
| `exploreScrollableLogsContainer` | Improves the scrolling behavior of logs in Explore |
| `recordedQueriesMulti` | Enables writing multiple items from a single query within Recorded Queries |
| `pluginsDynamicAngularDetectionPatterns` | Enables fetching Angular detection patterns for plugins from GCOM and fallback to hardcoded ones |
| `alertingLokiRangeToInstant` | Rewrites eligible loki range queries to instant queries |
| `flameGraphV2` | New version of flame graph with new features |
## Development feature toggles

View File

@ -101,6 +101,7 @@ export interface FeatureToggles {
cloudWatchLogsMonacoEditor?: boolean;
exploreScrollableLogsContainer?: boolean;
recordedQueriesMulti?: boolean;
pluginsDynamicAngularDetectionPatterns?: boolean;
alertingLokiRangeToInstant?: boolean;
flameGraphV2?: boolean;
}

View File

@ -11,6 +11,7 @@ import (
"github.com/grafana/grafana-azure-sdk-go/azsettings"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angularinspector"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/infra/db"
@ -22,7 +23,6 @@ import (
pluginClient "github.com/grafana/grafana/pkg/plugins/manager/client"
"github.com/grafana/grafana/pkg/plugins/manager/fakes"
"github.com/grafana/grafana/pkg/plugins/manager/loader"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angulardetector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/assetpath"
"github.com/grafana/grafana/pkg/plugins/manager/loader/finder"
"github.com/grafana/grafana/pkg/plugins/manager/registry"
@ -67,10 +67,12 @@ func TestCallResource(t *testing.T) {
pCfg, err := config.ProvideConfig(setting.ProvideProvider(cfg), cfg, featuremgmt.WithFeatures())
require.NoError(t, err)
reg := registry.ProvideService()
angularInspector, err := angularinspector.NewStaticInspector()
require.NoError(t, err)
l := loader.ProvideService(pCfg, fakes.NewFakeLicensingService(), signature.NewUnsignedAuthorizer(pCfg),
reg, provider.ProvideService(coreRegistry), finder.NewLocalFinder(pCfg), fakes.NewFakeRoleRegistry(),
assetpath.ProvideService(pluginscdn.ProvideService(pCfg)), signature.ProvideService(pCfg, statickey.New()),
angulardetector.NewDefaultPatternsListInspector())
angularInspector)
srcs := sources.ProvideService(cfg, pCfg)
ps, err := store.ProvideService(reg, srcs, l)
require.NoError(t, err)

View File

@ -45,7 +45,8 @@ type Cfg struct {
func NewCfg(devMode bool, pluginsPath string, pluginSettings setting.PluginSettings, pluginsAllowUnsigned []string,
awsAllowedAuthProviders []string, awsAssumeRoleEnabled bool, azure *azsettings.AzureSettings, secureSocksDSProxy setting.SecureSocksDSProxySettings,
grafanaVersion string, logDatasourceRequests bool, pluginsCDNURLTemplate string, tracing Tracing, features plugins.FeatureToggles, angularSupportEnabled bool) *Cfg {
grafanaVersion string, logDatasourceRequests bool, pluginsCDNURLTemplate string, tracing Tracing, features plugins.FeatureToggles, angularSupportEnabled bool,
grafanaComURL string) *Cfg {
return &Cfg{
log: log.New("plugin.cfg"),
PluginsPath: pluginsPath,
@ -60,7 +61,7 @@ func NewCfg(devMode bool, pluginsPath string, pluginSettings setting.PluginSetti
LogDatasourceRequests: logDatasourceRequests,
PluginsCDNURLTemplate: pluginsCDNURLTemplate,
Tracing: tracing,
GrafanaComURL: "https://grafana.com",
GrafanaComURL: grafanaComURL,
Features: features,
AngularSupportEnabled: angularSupportEnabled,
}

View File

@ -0,0 +1,69 @@
package angulardetector
import (
"bytes"
"context"
"regexp"
)
var (
_ AngularDetector = &ContainsBytesDetector{}
_ AngularDetector = &RegexDetector{}
_ DetectorsProvider = &StaticDetectorsProvider{}
_ DetectorsProvider = SequenceDetectorsProvider{}
)
// AngularDetector implements a check to see if a js file is using angular APIs.
type AngularDetector interface {
// DetectAngular takes the content of a js file and returns true if the plugin is using Angular.
DetectAngular(js []byte) bool
}
// ContainsBytesDetector is an AngularDetector that returns true if module.js contains the "pattern" string.
type ContainsBytesDetector struct {
Pattern []byte
}
// DetectAngular returns true if moduleJs contains the byte slice d.pattern.
func (d *ContainsBytesDetector) DetectAngular(moduleJs []byte) bool {
return bytes.Contains(moduleJs, d.Pattern)
}
// RegexDetector is an AngularDetector that returns true if the module.js content matches a regular expression.
type RegexDetector struct {
Regex *regexp.Regexp
}
// DetectAngular returns true if moduleJs matches the regular expression d.regex.
func (d *RegexDetector) DetectAngular(moduleJs []byte) bool {
return d.Regex.Match(moduleJs)
}
// DetectorsProvider can provide multiple AngularDetectors used for Angular detection.
type DetectorsProvider interface {
// ProvideDetectors returns a slice of AngularDetector.
ProvideDetectors(ctx context.Context) []AngularDetector
}
// StaticDetectorsProvider is a DetectorsProvider that always returns a pre-defined slice of AngularDetector.
type StaticDetectorsProvider struct {
Detectors []AngularDetector
}
func (p *StaticDetectorsProvider) ProvideDetectors(_ context.Context) []AngularDetector {
return p.Detectors
}
// SequenceDetectorsProvider is a DetectorsProvider that wraps a slice of other DetectorsProvider, and returns the first
// provided result that isn't empty.
type SequenceDetectorsProvider []DetectorsProvider
func (p SequenceDetectorsProvider) ProvideDetectors(ctx context.Context) []AngularDetector {
for _, provider := range p {
if detectors := provider.ProvideDetectors(ctx); len(detectors) > 0 {
return detectors
}
}
return nil
}

View File

@ -0,0 +1,120 @@
package angulardetector
import (
"context"
"regexp"
"testing"
"github.com/stretchr/testify/require"
)
var testDetectors = []AngularDetector{
&ContainsBytesDetector{Pattern: []byte("PanelCtrl")},
&ContainsBytesDetector{Pattern: []byte("QueryCtrl")},
}
func TestContainsBytesDetector(t *testing.T) {
detector := &ContainsBytesDetector{Pattern: []byte("needle")}
t.Run("contains", func(t *testing.T) {
require.True(t, detector.DetectAngular([]byte("lorem needle ipsum haystack")))
})
t.Run("not contains", func(t *testing.T) {
require.False(t, detector.DetectAngular([]byte("ippif")))
})
}
func TestRegexDetector(t *testing.T) {
detector := &RegexDetector{Regex: regexp.MustCompile("hello world(?s)")}
for _, tc := range []struct {
name string
s string
exp bool
}{
{name: "match 1", s: "hello world", exp: true},
{name: "match 2", s: "bla bla hello world bla bla", exp: true},
{name: "match 3", s: "bla bla hello worlds bla bla", exp: true},
{name: "no match", s: "bla bla hello you reading this test code", exp: false},
} {
t.Run(tc.s, func(t *testing.T) {
r := detector.DetectAngular([]byte(tc.s))
require.Equal(t, tc.exp, r, "DetectAngular result should be correct")
})
}
}
func TestStaticDetectorsProvider(t *testing.T) {
p := StaticDetectorsProvider{Detectors: testDetectors}
detectors := p.ProvideDetectors(context.Background())
require.NotEmpty(t, detectors)
require.Equal(t, testDetectors, detectors)
}
type fakeDetectorsProvider struct {
calls int
returns []AngularDetector
}
func (p *fakeDetectorsProvider) ProvideDetectors(_ context.Context) []AngularDetector {
p.calls += 1
return p.returns
}
func TestSequenceDetectorsProvider(t *testing.T) {
for _, tc := range []struct {
name string
fakeProviders []*fakeDetectorsProvider
exp func(t *testing.T, fakeProviders []*fakeDetectorsProvider, detectors []AngularDetector)
}{
{
name: "returns first non-empty provided angularDetectors (first)",
fakeProviders: []*fakeDetectorsProvider{
{returns: testDetectors},
{returns: nil},
},
exp: func(t *testing.T, fakeProviders []*fakeDetectorsProvider, detectors []AngularDetector) {
require.NotEmpty(t, detectors)
require.Len(t, detectors, len(fakeProviders[0].returns))
require.Equal(t, fakeProviders[0].returns, detectors)
require.Equal(t, 1, fakeProviders[0].calls, "fake provider 0 should be called")
require.Zero(t, fakeProviders[1].calls, "fake provider 1 should not be called")
},
},
{
name: "returns first non-empty provided angularDetectors (second)",
fakeProviders: []*fakeDetectorsProvider{
{returns: nil},
{returns: testDetectors},
},
exp: func(t *testing.T, fakeProviders []*fakeDetectorsProvider, detectors []AngularDetector) {
require.NotEmpty(t, detectors)
require.Len(t, detectors, len(fakeProviders[1].returns))
require.Equal(t, fakeProviders[1].returns, detectors)
for i, p := range fakeProviders {
require.Equalf(t, 1, p.calls, "fake provider %d should be called", i)
}
},
},
{
name: "returns nil if all providers return empty",
fakeProviders: []*fakeDetectorsProvider{
{returns: nil},
{returns: []AngularDetector{}},
},
exp: func(t *testing.T, fakeProviders []*fakeDetectorsProvider, detectors []AngularDetector) {
require.Empty(t, detectors, "should not return any angularDetectors")
for i, p := range fakeProviders {
require.Equalf(t, 1, p.calls, "fake provider %d should be called", i)
}
},
},
} {
t.Run(tc.name, func(t *testing.T) {
seq := make(SequenceDetectorsProvider, 0, len(tc.fakeProviders))
for _, p := range tc.fakeProviders {
seq = append(seq, DetectorsProvider(p))
}
detectors := seq.ProvideDetectors(context.Background())
tc.exp(t, tc.fakeProviders, detectors)
})
}
}

View File

@ -0,0 +1,78 @@
package angularinspector
import (
"context"
"errors"
"fmt"
"io"
"regexp"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
)
// Inspector can inspect a plugin and determine if it's an Angular plugin or not.
type Inspector interface {
// Inspect takes a plugin and checks if the plugin is using Angular.
Inspect(ctx context.Context, p *plugins.Plugin) (bool, error)
}
// PatternsListInspector is an Inspector that matches a plugin's module.js against all the patterns returned by
// the detectorsProvider, in sequence.
type PatternsListInspector struct {
// DetectorsProvider returns the detectors that will be used by Inspect.
DetectorsProvider angulardetector.DetectorsProvider
}
func (i *PatternsListInspector) Inspect(ctx context.Context, p *plugins.Plugin) (isAngular bool, err error) {
f, err := p.FS.Open("module.js")
if err != nil {
if errors.Is(err, plugins.ErrFileNotExist) {
// We may not have a module.js for some backend plugins, so ignore the error if module.js does not exist
return false, nil
}
return false, err
}
defer func() {
if closeErr := f.Close(); closeErr != nil && err == nil {
err = fmt.Errorf("close module.js: %w", closeErr)
}
}()
b, err := io.ReadAll(f)
if err != nil {
return false, fmt.Errorf("module.js readall: %w", err)
}
for _, d := range i.DetectorsProvider.ProvideDetectors(ctx) {
if d.DetectAngular(b) {
isAngular = true
break
}
}
return
}
// defaultDetectors contains all the detectors to DetectAngular Angular plugins.
// They are executed in the specified order.
var defaultDetectors = []angulardetector.AngularDetector{
&angulardetector.ContainsBytesDetector{Pattern: []byte("PanelCtrl")},
&angulardetector.ContainsBytesDetector{Pattern: []byte("ConfigCtrl")},
&angulardetector.ContainsBytesDetector{Pattern: []byte("app/plugins/sdk")},
&angulardetector.ContainsBytesDetector{Pattern: []byte("angular.isNumber(")},
&angulardetector.ContainsBytesDetector{Pattern: []byte("editor.html")},
&angulardetector.ContainsBytesDetector{Pattern: []byte("ctrl.annotation")},
&angulardetector.ContainsBytesDetector{Pattern: []byte("getLegacyAngularInjector")},
&angulardetector.RegexDetector{Regex: regexp.MustCompile(`["']QueryCtrl["']`)},
}
// NewDefaultStaticDetectorsProvider returns a new StaticDetectorsProvider with the default (static, hardcoded) angular
// detection patterns (defaultDetectors)
func NewDefaultStaticDetectorsProvider() angulardetector.DetectorsProvider {
return &angulardetector.StaticDetectorsProvider{Detectors: defaultDetectors}
}
// NewStaticInspector returns the default Inspector, which is a PatternsListInspector that only uses the
// static (hardcoded) angular detection patterns.
func NewStaticInspector() (Inspector, error) {
return &PatternsListInspector{DetectorsProvider: NewDefaultStaticDetectorsProvider()}, nil
}

View File

@ -0,0 +1,145 @@
package angularinspector
import (
"context"
"strconv"
"testing"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
)
type fakeDetector struct {
calls int
returns bool
}
func (d *fakeDetector) DetectAngular(_ []byte) bool {
d.calls += 1
return d.returns
}
func TestPatternsListInspector(t *testing.T) {
plugin := &plugins.Plugin{
FS: plugins.NewInMemoryFS(map[string][]byte{"module.js": nil}),
}
for _, tc := range []struct {
name string
fakeDetectors []*fakeDetector
exp func(t *testing.T, r bool, err error, fakeDetectors []*fakeDetector)
}{
{
name: "calls the detectors in sequence until true is returned",
fakeDetectors: []*fakeDetector{
{returns: false},
{returns: true},
{returns: false},
},
exp: func(t *testing.T, r bool, err error, fakeDetectors []*fakeDetector) {
require.NoError(t, err)
require.True(t, r, "inspector should return true")
require.Equal(t, 1, fakeDetectors[0].calls, "fake 0 should be called")
require.Equal(t, 1, fakeDetectors[1].calls, "fake 1 should be called")
require.Equal(t, 0, fakeDetectors[2].calls, "fake 2 should not be called")
},
},
{
name: "calls the detectors in sequence and returns false as default",
fakeDetectors: []*fakeDetector{
{returns: false},
{returns: false},
},
exp: func(t *testing.T, r bool, err error, fakeDetectors []*fakeDetector) {
require.NoError(t, err)
require.False(t, r, "inspector should return false")
require.Equal(t, 1, fakeDetectors[0].calls, "fake 0 should not be called")
require.Equal(t, 1, fakeDetectors[1].calls, "fake 1 should not be called")
},
},
{
name: "empty detectors should return false",
fakeDetectors: nil,
exp: func(t *testing.T, r bool, err error, fakeDetectors []*fakeDetector) {
require.NoError(t, err)
require.False(t, r, "inspector should return false")
},
},
} {
t.Run(tc.name, func(t *testing.T) {
detectors := make([]angulardetector.AngularDetector, 0, len(tc.fakeDetectors))
for _, d := range tc.fakeDetectors {
detectors = append(detectors, angulardetector.AngularDetector(d))
}
inspector := &PatternsListInspector{
DetectorsProvider: &angulardetector.StaticDetectorsProvider{Detectors: detectors},
}
r, err := inspector.Inspect(context.Background(), plugin)
tc.exp(t, r, err, tc.fakeDetectors)
})
}
}
func TestDefaultStaticDetectorsInspector(t *testing.T) {
// Tests the default hardcoded angular patterns
type tc struct {
name string
plugin *plugins.Plugin
exp bool
}
var tcs []tc
// Angular imports
for i, content := range [][]byte{
[]byte(`import { MetricsPanelCtrl } from 'grafana/app/plugins/sdk';`),
[]byte(`define(["app/plugins/sdk"],(function(n){return function(n){var t={};function e(r){if(t[r])return t[r].exports;var o=t[r]={i:r,l:!1,exports:{}};return n[r].call(o.exports,o,o.exports,e),o.l=!0,o.exports}return e.m=n,e.c=t,e.d=function(n,t,r){e.o(n,t)||Object.defineProperty(n,t,{enumerable:!0,get:r})},e.r=function(n){"undefined"!=typeof`),
[]byte(`define(["app/plugins/sdk"],(function(n){return function(n){var t={};function e(r){if(t[r])return t[r].exports;var o=t[r]={i:r,l:!1,exports:{}};return n[r].call(o.exports,o,o.exports,e),o.l=!0,o.exports}return e.m=n,e.c=t,e.d=function(n,t,r){e.o(n,t)||Object.defineProperty(n,t,{enumerable:!0,get:r})},e.r=function(n){"undefined"!=typeof Symbol&&Symbol.toSt`),
[]byte(`define(["react","lodash","@grafana/data","@grafana/ui","@emotion/css","@grafana/runtime","moment","app/core/utils/datemath","jquery","app/plugins/sdk","app/core/core_module","app/core/core","app/core/table_model","app/core/utils/kbn","app/core/config","angular"],(function(e,t,r,n,i,a,o,s,u,l,c,p,f,h,d,m){return function(e){var t={};function r(n){if(t[n])return t[n].exports;var i=t[n]={i:n,l:!1,exports:{}};retur`),
[]byte(`exports_1("QueryCtrl", query_ctrl_1.PluginQueryCtrl);`),
[]byte(`exports_1('QueryCtrl', query_ctrl_1.PluginQueryCtrl);`),
} {
tcs = append(tcs, tc{
name: "angular " + strconv.Itoa(i),
plugin: &plugins.Plugin{
FS: plugins.NewInMemoryFS(map[string][]byte{
"module.js": content,
}),
},
exp: true,
})
}
// Not angular (test against possible false detections)
for i, content := range [][]byte{
[]byte(`import { PanelPlugin } from '@grafana/data'`),
// React ML app
[]byte(`==(null===(t=e.components)||void 0===t?void 0:t.QueryCtrl)};function`),
} {
tcs = append(tcs, tc{
name: "not angular " + strconv.Itoa(i),
plugin: &plugins.Plugin{
FS: plugins.NewInMemoryFS(map[string][]byte{
"module.js": content,
}),
},
exp: false,
})
}
inspector := PatternsListInspector{DetectorsProvider: NewDefaultStaticDetectorsProvider()}
for _, tc := range tcs {
t.Run(tc.name, func(t *testing.T) {
isAngular, err := inspector.Inspect(context.Background(), tc.plugin)
require.NoError(t, err)
require.Equal(t, tc.exp, isAngular)
})
}
t.Run("no module.js", func(t *testing.T) {
p := &plugins.Plugin{FS: plugins.NewInMemoryFS(map[string][]byte{})}
_, err := inspector.Inspect(context.Background(), p)
require.NoError(t, err)
})
}

View File

@ -1,28 +1,32 @@
package angulardetector
package angularinspector
import "github.com/grafana/grafana/pkg/plugins"
import (
"context"
"github.com/grafana/grafana/pkg/plugins"
)
// FakeInspector is an inspector whose Inspect function can be set to any function.
type FakeInspector struct {
// InspectFunc is the function called when calling Inspect()
InspectFunc func(p *plugins.Plugin) (bool, error)
InspectFunc func(ctx context.Context, p *plugins.Plugin) (bool, error)
}
func (i *FakeInspector) Inspect(p *plugins.Plugin) (bool, error) {
return i.InspectFunc(p)
func (i *FakeInspector) Inspect(ctx context.Context, p *plugins.Plugin) (bool, error) {
return i.InspectFunc(ctx, p)
}
var (
// AlwaysAngularFakeInspector is an inspector that always returns `true, nil`
AlwaysAngularFakeInspector = &FakeInspector{
InspectFunc: func(p *plugins.Plugin) (bool, error) {
InspectFunc: func(_ context.Context, _ *plugins.Plugin) (bool, error) {
return true, nil
},
}
// NeverAngularFakeInspector is an inspector that always returns `false, nil`
NeverAngularFakeInspector = &FakeInspector{
InspectFunc: func(p *plugins.Plugin) (bool, error) {
InspectFunc: func(_ context.Context, _ *plugins.Plugin) (bool, error) {
return false, nil
},
}

View File

@ -0,0 +1,35 @@
package angularinspector
import (
"context"
"testing"
"github.com/grafana/grafana/pkg/plugins"
"github.com/stretchr/testify/require"
)
func TestFakeInspector(t *testing.T) {
t.Run("FakeInspector", func(t *testing.T) {
var called bool
inspector := FakeInspector{InspectFunc: func(_ context.Context, _ *plugins.Plugin) (bool, error) {
called = true
return false, nil
}}
r, err := inspector.Inspect(context.Background(), &plugins.Plugin{})
require.True(t, called)
require.NoError(t, err)
require.False(t, r)
})
t.Run("AlwaysAngularFakeInspector", func(t *testing.T) {
r, err := AlwaysAngularFakeInspector.Inspect(context.Background(), &plugins.Plugin{})
require.NoError(t, err)
require.True(t, r)
})
t.Run("NeverAngularFakeInspector", func(t *testing.T) {
r, err := NeverAngularFakeInspector.Inspect(context.Background(), &plugins.Plugin{})
require.NoError(t, err)
require.False(t, r)
})
}

View File

@ -1,46 +0,0 @@
package angulardetector
import (
"bytes"
"regexp"
"github.com/grafana/grafana/pkg/plugins"
)
var (
_ detector = &containsBytesDetector{}
_ detector = &regexDetector{}
)
// detector implements a check to see if a plugin uses Angular.
type detector interface {
// Detect takes the content of a moduleJs file and returns true if the plugin is using Angular.
Detect(moduleJs []byte) bool
}
// containsBytesDetector is a detector that returns true if module.js contains the "pattern" string.
type containsBytesDetector struct {
pattern []byte
}
// Detect returns true if moduleJs contains the byte slice d.pattern.
func (d *containsBytesDetector) Detect(moduleJs []byte) bool {
return bytes.Contains(moduleJs, d.pattern)
}
// regexDetector is a detector that returns true if the module.js content matches a regular expression.
type regexDetector struct {
regex *regexp.Regexp
}
// Detect returns true if moduleJs matches the regular expression d.regex.
func (d *regexDetector) Detect(moduleJs []byte) bool {
return d.regex.Match(moduleJs)
}
// Inspector can inspect a module.js and determine if it's an Angular plugin or not.
type Inspector interface {
// Inspect open module.js and checks if the plugin is using Angular by matching against its source code.
// It returns true if module.js matches against any of the detectors in angularDetectors.
Inspect(p *plugins.Plugin) (bool, error)
}

View File

@ -1,95 +0,0 @@
package angulardetector
import (
"strconv"
"testing"
"github.com/grafana/grafana/pkg/plugins"
"github.com/stretchr/testify/require"
)
func TestAngularDetector_Inspect(t *testing.T) {
type tc struct {
name string
plugin *plugins.Plugin
exp bool
}
var tcs []tc
// Angular imports
for i, content := range [][]byte{
[]byte(`import { MetricsPanelCtrl } from 'grafana/app/plugins/sdk';`),
[]byte(`define(["app/plugins/sdk"],(function(n){return function(n){var t={};function e(r){if(t[r])return t[r].exports;var o=t[r]={i:r,l:!1,exports:{}};return n[r].call(o.exports,o,o.exports,e),o.l=!0,o.exports}return e.m=n,e.c=t,e.d=function(n,t,r){e.o(n,t)||Object.defineProperty(n,t,{enumerable:!0,get:r})},e.r=function(n){"undefined"!=typeof`),
[]byte(`define(["app/plugins/sdk"],(function(n){return function(n){var t={};function e(r){if(t[r])return t[r].exports;var o=t[r]={i:r,l:!1,exports:{}};return n[r].call(o.exports,o,o.exports,e),o.l=!0,o.exports}return e.m=n,e.c=t,e.d=function(n,t,r){e.o(n,t)||Object.defineProperty(n,t,{enumerable:!0,get:r})},e.r=function(n){"undefined"!=typeof Symbol&&Symbol.toSt`),
[]byte(`define(["react","lodash","@grafana/data","@grafana/ui","@emotion/css","@grafana/runtime","moment","app/core/utils/datemath","jquery","app/plugins/sdk","app/core/core_module","app/core/core","app/core/table_model","app/core/utils/kbn","app/core/config","angular"],(function(e,t,r,n,i,a,o,s,u,l,c,p,f,h,d,m){return function(e){var t={};function r(n){if(t[n])return t[n].exports;var i=t[n]={i:n,l:!1,exports:{}};retur`),
[]byte(`exports_1("QueryCtrl", query_ctrl_1.PluginQueryCtrl);`),
[]byte(`exports_1('QueryCtrl', query_ctrl_1.PluginQueryCtrl);`),
} {
tcs = append(tcs, tc{
name: "angular " + strconv.Itoa(i),
plugin: &plugins.Plugin{
FS: plugins.NewInMemoryFS(map[string][]byte{
"module.js": content,
}),
},
exp: true,
})
}
// Not angular (test against possible false detections)
for i, content := range [][]byte{
[]byte(`import { PanelPlugin } from '@grafana/data'`),
// React ML app
[]byte(`==(null===(t=e.components)||void 0===t?void 0:t.QueryCtrl)};function`),
} {
tcs = append(tcs, tc{
name: "not angular " + strconv.Itoa(i),
plugin: &plugins.Plugin{
FS: plugins.NewInMemoryFS(map[string][]byte{
"module.js": content,
}),
},
exp: false,
})
}
inspector := NewDefaultPatternsListInspector()
for _, tc := range tcs {
t.Run(tc.name, func(t *testing.T) {
isAngular, err := inspector.Inspect(tc.plugin)
require.NoError(t, err)
require.Equal(t, tc.exp, isAngular)
})
}
t.Run("no module.js", func(t *testing.T) {
p := &plugins.Plugin{FS: plugins.NewInMemoryFS(map[string][]byte{})}
_, err := inspector.Inspect(p)
require.ErrorIs(t, err, plugins.ErrFileNotExist)
})
}
func TestFakeInspector(t *testing.T) {
t.Run("FakeInspector", func(t *testing.T) {
var called bool
inspector := FakeInspector{InspectFunc: func(p *plugins.Plugin) (bool, error) {
called = true
return false, nil
}}
r, err := inspector.Inspect(&plugins.Plugin{})
require.True(t, called)
require.NoError(t, err)
require.False(t, r)
})
t.Run("AlwaysAngularFakeInspector", func(t *testing.T) {
r, err := AlwaysAngularFakeInspector.Inspect(&plugins.Plugin{})
require.NoError(t, err)
require.True(t, r)
})
t.Run("NeverAngularFakeInspector", func(t *testing.T) {
r, err := NeverAngularFakeInspector.Inspect(&plugins.Plugin{})
require.NoError(t, err)
require.False(t, r)
})
}

View File

@ -1,60 +0,0 @@
package angulardetector
import (
"fmt"
"io"
"regexp"
"github.com/grafana/grafana/pkg/plugins"
)
// defaultDetectors contains all the detectors to detect Angular plugins.
// They are executed in the specified order.
var defaultDetectors = []detector{
&containsBytesDetector{pattern: []byte("PanelCtrl")},
&containsBytesDetector{pattern: []byte("ConfigCtrl")},
&containsBytesDetector{pattern: []byte("app/plugins/sdk")},
&containsBytesDetector{pattern: []byte("angular.isNumber(")},
&containsBytesDetector{pattern: []byte("editor.html")},
&containsBytesDetector{pattern: []byte("ctrl.annotation")},
&containsBytesDetector{pattern: []byte("getLegacyAngularInjector")},
&regexDetector{regex: regexp.MustCompile(`["']QueryCtrl["']`)},
}
// PatternsListInspector matches module.js against all the specified patterns, in sequence.
type PatternsListInspector struct {
detectors []detector
}
// NewDefaultPatternsListInspector returns a new *PatternsListInspector using defaultDetectors as detectors.
func NewDefaultPatternsListInspector() *PatternsListInspector {
return &PatternsListInspector{detectors: defaultDetectors}
}
func ProvideService() Inspector {
return NewDefaultPatternsListInspector()
}
func (i *PatternsListInspector) Inspect(p *plugins.Plugin) (isAngular bool, err error) {
f, err := p.FS.Open("module.js")
if err != nil {
return false, fmt.Errorf("open module.js: %w", err)
}
defer func() {
if closeErr := f.Close(); closeErr != nil && err == nil {
err = fmt.Errorf("close module.js: %w", closeErr)
}
}()
b, err := io.ReadAll(f)
if err != nil {
return false, fmt.Errorf("module.js readall: %w", err)
}
for _, d := range i.detectors {
if d.Detect(b) {
isAngular = true
break
}
}
return
}

View File

@ -6,13 +6,14 @@ import (
"fmt"
"path"
"strings"
"time"
"github.com/grafana/grafana/pkg/infra/metrics"
"github.com/grafana/grafana/pkg/infra/slugify"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/plugins/config"
"github.com/grafana/grafana/pkg/plugins/log"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angulardetector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angularinspector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/assetpath"
"github.com/grafana/grafana/pkg/plugins/manager/loader/finder"
"github.com/grafana/grafana/pkg/plugins/manager/loader/initializer"
@ -36,7 +37,7 @@ type Loader struct {
log log.Logger
cfg *config.Cfg
angularInspector angulardetector.Inspector
angularInspector angularinspector.Inspector
errs map[string]*plugins.SignatureError
}
@ -44,7 +45,7 @@ type Loader struct {
func ProvideService(cfg *config.Cfg, license plugins.Licensing, authorizer plugins.PluginLoaderAuthorizer,
pluginRegistry registry.Service, backendProvider plugins.BackendFactoryProvider, pluginFinder finder.Finder,
roleRegistry plugins.RoleRegistry, assetPath *assetpath.Service, signatureCalculator plugins.SignatureCalculator,
angularInspector angulardetector.Inspector) *Loader {
angularInspector angularinspector.Inspector) *Loader {
return New(cfg, license, authorizer, pluginRegistry, backendProvider, process.NewManager(pluginRegistry),
roleRegistry, assetPath, pluginFinder, signatureCalculator, angularInspector)
}
@ -53,7 +54,7 @@ func New(cfg *config.Cfg, license plugins.Licensing, authorizer plugins.PluginLo
pluginRegistry registry.Service, backendProvider plugins.BackendFactoryProvider,
processManager process.Service, roleRegistry plugins.RoleRegistry,
assetPath *assetpath.Service, pluginFinder finder.Finder, signatureCalculator plugins.SignatureCalculator,
angularInspector angulardetector.Inspector) *Loader {
angularInspector angularinspector.Inspector) *Loader {
return &Loader{
pluginFinder: pluginFinder,
pluginRegistry: pluginRegistry,
@ -182,10 +183,14 @@ func (l *Loader) loadPlugins(ctx context.Context, src plugins.PluginSource, foun
// initialize plugins
initializedPlugins := make([]*plugins.Plugin, 0, len(verifiedPlugins))
for _, p := range verifiedPlugins {
// Detect angular for external plugins
// detect angular for external plugins
if p.IsExternalPlugin() {
var err error
p.AngularDetected, err = l.angularInspector.Inspect(p)
cctx, canc := context.WithTimeout(ctx, time.Minute*1)
p.AngularDetected, err = l.angularInspector.Inspect(cctx, p)
canc()
if err != nil {
l.log.Warn("could not inspect plugin for angular", "pluginID", p.ID, "err", err)
}

View File

@ -9,6 +9,7 @@ import (
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angularinspector"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/plugins"
@ -16,7 +17,6 @@ import (
"github.com/grafana/grafana/pkg/plugins/config"
"github.com/grafana/grafana/pkg/plugins/log"
"github.com/grafana/grafana/pkg/plugins/manager/fakes"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angulardetector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/assetpath"
"github.com/grafana/grafana/pkg/plugins/manager/loader/finder"
"github.com/grafana/grafana/pkg/plugins/manager/loader/initializer"
@ -438,7 +438,7 @@ func TestLoader_Load(t *testing.T) {
reg := fakes.NewFakePluginRegistry()
procPrvdr := fakes.NewFakeBackendProcessProvider()
procMgr := fakes.NewFakeProcessManager()
l := newLoader(tt.cfg, func(l *Loader) {
l := newLoader(t, tt.cfg, func(l *Loader) {
l.pluginRegistry = reg
l.processManager = procMgr
l.pluginInitializer = initializer.New(tt.cfg, procPrvdr, &fakes.FakeLicensingService{})
@ -521,7 +521,7 @@ func TestLoader_Load_CustomSource(t *testing.T) {
Module: "plugin-cdn/grafana-worldmap-panel/0.3.3/public/plugins/grafana-worldmap-panel/module",
}}
l := newLoader(cfg)
l := newLoader(t, cfg)
got, err := l.Load(context.Background(), &fakes.FakePluginSource{
PluginClassFunc: func(ctx context.Context) plugins.Class {
return plugins.ClassBundled
@ -672,7 +672,7 @@ func TestLoader_Load_MultiplePlugins(t *testing.T) {
reg := fakes.NewFakePluginRegistry()
procPrvdr := fakes.NewFakeBackendProcessProvider()
procMgr := fakes.NewFakeProcessManager()
l := newLoader(tt.cfg, func(l *Loader) {
l := newLoader(t, tt.cfg, func(l *Loader) {
l.pluginRegistry = reg
l.processManager = procMgr
l.pluginInitializer = initializer.New(tt.cfg, procPrvdr, fakes.NewFakeLicensingService())
@ -793,7 +793,7 @@ func TestLoader_Load_RBACReady(t *testing.T) {
reg := fakes.NewFakePluginRegistry()
procPrvdr := fakes.NewFakeBackendProcessProvider()
procMgr := fakes.NewFakeProcessManager()
l := newLoader(tt.cfg, func(l *Loader) {
l := newLoader(t, tt.cfg, func(l *Loader) {
l.pluginRegistry = reg
l.processManager = procMgr
l.pluginInitializer = initializer.New(tt.cfg, procPrvdr, fakes.NewFakeLicensingService())
@ -872,7 +872,7 @@ func TestLoader_Load_Signature_RootURL(t *testing.T) {
reg := fakes.NewFakePluginRegistry()
procPrvdr := fakes.NewFakeBackendProcessProvider()
procMgr := fakes.NewFakeProcessManager()
l := newLoader(&config.Cfg{}, func(l *Loader) {
l := newLoader(t, &config.Cfg{}, func(l *Loader) {
l.pluginRegistry = reg
l.processManager = procMgr
l.pluginInitializer = initializer.New(&config.Cfg{}, procPrvdr, fakes.NewFakeLicensingService())
@ -956,7 +956,7 @@ func TestLoader_Load_DuplicatePlugins(t *testing.T) {
reg := fakes.NewFakePluginRegistry()
procPrvdr := fakes.NewFakeBackendProcessProvider()
procMgr := fakes.NewFakeProcessManager()
l := newLoader(&config.Cfg{}, func(l *Loader) {
l := newLoader(t, &config.Cfg{}, func(l *Loader) {
l.pluginRegistry = reg
l.processManager = procMgr
l.pluginInitializer = initializer.New(&config.Cfg{}, procPrvdr, fakes.NewFakeLicensingService())
@ -1055,7 +1055,7 @@ func TestLoader_Load_SkipUninitializedPlugins(t *testing.T) {
}
}
procMgr := fakes.NewFakeProcessManager()
l := newLoader(&config.Cfg{}, func(l *Loader) {
l := newLoader(t, &config.Cfg{}, func(l *Loader) {
l.pluginRegistry = reg
l.processManager = procMgr
l.pluginInitializer = initializer.New(&config.Cfg{}, procPrvdr, fakes.NewFakeLicensingService())
@ -1097,24 +1097,24 @@ func TestLoader_Load_Angular(t *testing.T) {
t.Run(cfgTc.name, func(t *testing.T) {
for _, tc := range []struct {
name string
angularInspector angulardetector.Inspector
angularInspector angularinspector.Inspector
shouldLoad bool
}{
{
name: "angular plugin",
angularInspector: angulardetector.AlwaysAngularFakeInspector,
angularInspector: angularinspector.AlwaysAngularFakeInspector,
// angular plugins should load only if allowed by the cfg
shouldLoad: cfgTc.cfg.AngularSupportEnabled,
},
{
name: "non angular plugin",
angularInspector: angulardetector.NeverAngularFakeInspector,
angularInspector: angularinspector.NeverAngularFakeInspector,
// non-angular plugins should always load
shouldLoad: true,
},
} {
t.Run(tc.name, func(t *testing.T) {
l := newLoader(cfgTc.cfg, func(l *Loader) {
l := newLoader(t, cfgTc.cfg, func(l *Loader) {
l.angularInspector = tc.angularInspector
})
p, err := l.Load(context.Background(), fakePluginSource)
@ -1208,7 +1208,7 @@ func TestLoader_Load_NestedPlugins(t *testing.T) {
reg := fakes.NewFakePluginRegistry()
procPrvdr := fakes.NewFakeBackendProcessProvider()
procMgr := fakes.NewFakeProcessManager()
l := newLoader(&config.Cfg{}, func(l *Loader) {
l := newLoader(t, &config.Cfg{}, func(l *Loader) {
l.pluginRegistry = reg
l.processManager = procMgr
l.pluginInitializer = initializer.New(&config.Cfg{}, procPrvdr, fakes.NewFakeLicensingService())
@ -1386,7 +1386,7 @@ func TestLoader_Load_NestedPlugins(t *testing.T) {
reg := fakes.NewFakePluginRegistry()
procPrvdr := fakes.NewFakeBackendProcessProvider()
procMgr := fakes.NewFakeProcessManager()
l := newLoader(&config.Cfg{}, func(l *Loader) {
l := newLoader(t, &config.Cfg{}, func(l *Loader) {
l.pluginRegistry = reg
l.processManager = procMgr
l.pluginInitializer = initializer.New(&config.Cfg{}, procPrvdr, fakes.NewFakeLicensingService())
@ -1437,11 +1437,13 @@ func Test_setPathsBasedOnApp(t *testing.T) {
})
}
func newLoader(cfg *config.Cfg, cbs ...func(loader *Loader)) *Loader {
func newLoader(t *testing.T, cfg *config.Cfg, cbs ...func(loader *Loader)) *Loader {
angularInspector, err := angularinspector.NewStaticInspector()
require.NoError(t, err)
l := New(cfg, &fakes.FakeLicensingService{}, signature.NewUnsignedAuthorizer(cfg), fakes.NewFakePluginRegistry(),
fakes.NewFakeBackendProcessProvider(), fakes.NewFakeProcessManager(), fakes.NewFakeRoleRegistry(),
assetpath.ProvideService(pluginscdn.ProvideService(cfg)), finder.NewLocalFinder(cfg),
signature.ProvideService(cfg, statickey.New()), angulardetector.NewDefaultPatternsListInspector())
signature.ProvideService(cfg, statickey.New()), angularInspector)
for _, cb := range cbs {
cb(l)

View File

@ -10,6 +10,7 @@ import (
"github.com/grafana/grafana-azure-sdk-go/azsettings"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana-plugin-sdk-go/backend/httpclient"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angularinspector"
"github.com/stretchr/testify/require"
"gopkg.in/ini.v1"
@ -21,7 +22,6 @@ import (
"github.com/grafana/grafana/pkg/plugins/manager/client"
"github.com/grafana/grafana/pkg/plugins/manager/fakes"
"github.com/grafana/grafana/pkg/plugins/manager/loader"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angulardetector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/assetpath"
"github.com/grafana/grafana/pkg/plugins/manager/loader/finder"
"github.com/grafana/grafana/pkg/plugins/manager/registry"
@ -118,10 +118,12 @@ func TestIntegrationPluginManager(t *testing.T) {
require.NoError(t, err)
reg := registry.ProvideService()
lic := plicensing.ProvideLicensing(cfg, &licensing.OSSLicensingService{Cfg: cfg})
angularInspector, err := angularinspector.NewStaticInspector()
require.NoError(t, err)
l := loader.ProvideService(pCfg, lic, signature.NewUnsignedAuthorizer(pCfg),
reg, provider.ProvideService(coreRegistry), finder.NewLocalFinder(pCfg), fakes.NewFakeRoleRegistry(),
assetpath.ProvideService(pluginscdn.ProvideService(pCfg)), signature.ProvideService(pCfg, statickey.New()),
angulardetector.NewDefaultPatternsListInspector())
angularInspector)
srcs := sources.ProvideService(cfg, pCfg)
ps, err := store.ProvideService(reg, srcs, l)
require.NoError(t, err)

View File

@ -565,6 +565,13 @@ var (
Stage: FeatureStageExperimental,
Owner: grafanaObservabilityMetricsSquad,
},
{
Name: "pluginsDynamicAngularDetectionPatterns",
Description: "Enables fetching Angular detection patterns for plugins from GCOM and fallback to hardcoded ones",
Stage: FeatureStageExperimental,
FrontendOnly: false,
Owner: grafanaPluginsPlatformSquad,
},
{
Name: "alertingLokiRangeToInstant",
Description: "Rewrites eligible loki range queries to instant queries",

View File

@ -82,5 +82,6 @@ sqlDatasourceDatabaseSelection,preview,@grafana/grafana-bi-squad,false,false,fal
cloudWatchLogsMonacoEditor,experimental,@grafana/aws-plugins,false,false,false,true
exploreScrollableLogsContainer,experimental,@grafana/observability-logs,false,false,false,true
recordedQueriesMulti,experimental,@grafana/observability-metrics,false,false,false,false
pluginsDynamicAngularDetectionPatterns,experimental,@grafana/plugins-platform-backend,false,false,false,false
alertingLokiRangeToInstant,experimental,@grafana/alerting-squad,false,false,false,false
flameGraphV2,experimental,@grafana/observability-traces-and-profiling,false,false,false,true

1 Name Stage Owner requiresDevMode RequiresLicense RequiresRestart FrontendOnly
82 cloudWatchLogsMonacoEditor experimental @grafana/aws-plugins false false false true
83 exploreScrollableLogsContainer experimental @grafana/observability-logs false false false true
84 recordedQueriesMulti experimental @grafana/observability-metrics false false false false
85 pluginsDynamicAngularDetectionPatterns experimental @grafana/plugins-platform-backend false false false false
86 alertingLokiRangeToInstant experimental @grafana/alerting-squad false false false false
87 flameGraphV2 experimental @grafana/observability-traces-and-profiling false false false true

View File

@ -339,6 +339,10 @@ const (
// Enables writing multiple items from a single query within Recorded Queries
FlagRecordedQueriesMulti = "recordedQueriesMulti"
// FlagPluginsDynamicAngularDetectionPatterns
// Enables fetching Angular detection patterns for plugins from GCOM and fallback to hardcoded ones
FlagPluginsDynamicAngularDetectionPatterns = "pluginsDynamicAngularDetectionPatterns"
// FlagAlertingLokiRangeToInstant
// Rewrites eligible loki range queries to instant queries
FlagAlertingLokiRangeToInstant = "alertingLokiRangeToInstant"

View File

@ -0,0 +1,157 @@
package angulardetector
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"regexp"
"time"
"github.com/grafana/grafana-plugin-sdk-go/backend/httpclient"
"github.com/grafana/grafana/pkg/plugins/log"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
)
const (
// gcomAngularPatternsPath is the relative path to the GCOM API handler that returns angular detection patterns.
gcomAngularPatternsPath = "/api/plugins/angular_patterns"
)
var _ angulardetector.DetectorsProvider = &GCOMDetectorsProvider{}
// GCOMDetectorsProvider is a DetectorsProvider which fetches patterns from GCOM.
type GCOMDetectorsProvider struct {
log log.Logger
httpClient *http.Client
baseURL string
}
// NewGCOMDetectorsProvider returns a new GCOMDetectorsProvider.
// baseURL is the GCOM base url, without /api and without a trailing slash (e.g.: https://grafana.com)
func NewGCOMDetectorsProvider(baseURL string) (angulardetector.DetectorsProvider, error) {
cl, err := httpclient.New()
if err != nil {
return nil, fmt.Errorf("httpclient new: %w", err)
}
return &GCOMDetectorsProvider{
log: log.New("plugins.angulardetector.gcom"),
baseURL: baseURL,
httpClient: cl,
}, nil
}
// ProvideDetectors gets the dynamic angular detectors from the remote source.
// If an error occurs, the function fails silently by logging an error, and it returns nil.
func (p *GCOMDetectorsProvider) ProvideDetectors(ctx context.Context) []angulardetector.AngularDetector {
patterns, err := p.fetch(ctx)
if err != nil {
p.log.Warn("Could not fetch remote angular patterns", "error", err)
return nil
}
detectors, err := p.patternsToDetectors(patterns)
if err != nil {
p.log.Warn("Could not convert angular patterns to angularDetectors", "error", err)
return nil
}
return detectors
}
// fetch fetches the angular patterns from GCOM and returns them as gcomPatterns.
// Call angularDetectors() on the returned value to get the corresponding angular detectors.
func (p *GCOMDetectorsProvider) fetch(ctx context.Context) (gcomPatterns, error) {
st := time.Now()
reqURL, err := url.JoinPath(p.baseURL, gcomAngularPatternsPath)
if err != nil {
return nil, fmt.Errorf("url joinpath: %w", err)
}
p.log.Debug("Fetching dynamic angular detection patterns", "url", reqURL)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, nil)
if err != nil {
return nil, fmt.Errorf("new request with context: %w", err)
}
resp, err := p.httpClient.Do(req)
if err != nil {
return nil, fmt.Errorf("http do: %w", err)
}
defer func() {
if closeErr := resp.Body.Close(); closeErr != nil {
p.log.Error("response body close error", "error", err)
}
}()
var out gcomPatterns
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
return nil, fmt.Errorf("json decode: %w", err)
}
p.log.Debug("Fetched dynamic angular detection patterns", "patterns", len(out), "duration", time.Since(st))
return out, nil
}
// patternsToDetectors converts a slice of gcomPattern into a slice of angulardetector.AngularDetector, by calling
// angularDetector() on each gcomPattern.
func (p *GCOMDetectorsProvider) patternsToDetectors(patterns gcomPatterns) ([]angulardetector.AngularDetector, error) {
var finalErr error
detectors := make([]angulardetector.AngularDetector, 0, len(patterns))
for _, pattern := range patterns {
d, err := pattern.angularDetector()
if err != nil {
// Fail silently in case of an errUnknownPatternType.
// This allows us to introduce new pattern types without breaking old Grafana versions
if errors.Is(err, errUnknownPatternType) {
p.log.Debug("Unknown angular pattern", "name", pattern.Name, "type", pattern.Type, "error", err)
continue
}
// Other error, do not ignore it
finalErr = errors.Join(finalErr, err)
}
detectors = append(detectors, d)
}
if finalErr != nil {
return nil, finalErr
}
return detectors, nil
}
// gcomPatternType is a pattern type returned by the GCOM API.
type gcomPatternType string
const (
gcomPatternTypeContains gcomPatternType = "contains"
gcomPatternTypeRegex gcomPatternType = "regex"
)
// errUnknownPatternType is returned when a pattern type is not known.
var errUnknownPatternType = errors.New("unknown pattern type")
// gcomPattern is an Angular detection pattern returned by the GCOM API.
type gcomPattern struct {
Name string
Pattern string
Type gcomPatternType
}
// angularDetector converts a gcomPattern into an AngularDetector, based on its Type.
// If a pattern type is unknown, it returns an error wrapping errUnknownPatternType.
func (p *gcomPattern) angularDetector() (angulardetector.AngularDetector, error) {
switch p.Type {
case gcomPatternTypeContains:
return &angulardetector.ContainsBytesDetector{Pattern: []byte(p.Pattern)}, nil
case gcomPatternTypeRegex:
re, err := regexp.Compile(p.Pattern)
if err != nil {
return nil, fmt.Errorf("%q regexp compile: %w", p.Pattern, err)
}
return &angulardetector.RegexDetector{Regex: re}, nil
}
return nil, fmt.Errorf("%q: %w", p.Type, errUnknownPatternType)
}
// gcomPatterns is a slice of gcomPattern s.
type gcomPatterns []gcomPattern

View File

@ -0,0 +1,144 @@
package angulardetector
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
)
var mockGCOMResponse = []byte(`[{
"name": "PanelCtrl",
"type": "contains",
"pattern": "PanelCtrl"
},
{
"name": "QueryCtrl",
"type": "regex",
"pattern": "[\"']QueryCtrl[\"']"
}]`)
func mockGCOMHTTPHandlerFunc(writer http.ResponseWriter, request *http.Request) {
if request.URL.Path != "/api/plugins/angular_patterns" {
writer.WriteHeader(http.StatusNotFound)
return
}
_, _ = writer.Write(mockGCOMResponse)
}
func checkMockGCOMResponse(t *testing.T, detectors []angulardetector.AngularDetector) {
require.Len(t, detectors, 2)
d, ok := detectors[0].(*angulardetector.ContainsBytesDetector)
require.True(t, ok)
require.Equal(t, []byte(`PanelCtrl`), d.Pattern)
rd, ok := detectors[1].(*angulardetector.RegexDetector)
require.True(t, ok)
require.Equal(t, `["']QueryCtrl["']`, rd.Regex.String())
}
type gcomScenario struct {
gcomHTTPHandlerFunc http.HandlerFunc
gcomHTTPCalls int
}
func (s *gcomScenario) newHTTPTestServer() *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
s.gcomHTTPCalls++
s.gcomHTTPHandlerFunc(w, r)
}))
}
func newDefaultGCOMScenario() *gcomScenario {
return &gcomScenario{gcomHTTPHandlerFunc: mockGCOMHTTPHandlerFunc}
}
func TestGCOMDetectorsProvider(t *testing.T) {
t.Run("returns value returned from gcom api", func(t *testing.T) {
scenario := newDefaultGCOMScenario()
srv := scenario.newHTTPTestServer()
t.Cleanup(srv.Close)
gcomProvider, err := NewGCOMDetectorsProvider(srv.URL)
require.NoError(t, err)
detectors := gcomProvider.ProvideDetectors(context.Background())
require.Equal(t, 1, scenario.gcomHTTPCalls, "gcom api should be called")
checkMockGCOMResponse(t, detectors)
})
t.Run("error handling", func(t *testing.T) {
for _, tc := range []struct {
*gcomScenario
name string
}{
{name: "http error 500", gcomScenario: &gcomScenario{
gcomHTTPHandlerFunc: func(writer http.ResponseWriter, request *http.Request) {
writer.WriteHeader(http.StatusInternalServerError)
},
}},
{name: "invalid json", gcomScenario: &gcomScenario{
gcomHTTPHandlerFunc: func(writer http.ResponseWriter, request *http.Request) {
_, _ = writer.Write([]byte(`not json`))
},
}},
{name: "invalid regex", gcomScenario: &gcomScenario{
gcomHTTPHandlerFunc: func(writer http.ResponseWriter, request *http.Request) {
_, _ = writer.Write([]byte(`[{"name": "test", "type": "regex", "pattern": "((("}]`))
},
}},
} {
t.Run(tc.name, func(t *testing.T) {
srv := tc.newHTTPTestServer()
t.Cleanup(srv.Close)
gcomProvider, err := NewGCOMDetectorsProvider(srv.URL)
require.NoError(t, err)
detectors := gcomProvider.ProvideDetectors(context.Background())
require.Equal(t, 1, tc.gcomHTTPCalls, "gcom should be called")
require.Empty(t, detectors, "returned AngularDetectors should be empty")
})
}
})
t.Run("handles gcom timeout", func(t *testing.T) {
gcomScenario := &gcomScenario{
gcomHTTPHandlerFunc: func(writer http.ResponseWriter, request *http.Request) {
time.Sleep(time.Second * 1)
_, _ = writer.Write([]byte(`[{"name": "test", "type": "regex", "pattern": "((("}]`))
},
}
srv := gcomScenario.newHTTPTestServer()
t.Cleanup(srv.Close)
gcomProvider, err := NewGCOMDetectorsProvider(srv.URL)
require.NoError(t, err)
// Expired context
ctx, canc := context.WithTimeout(context.Background(), time.Second*-1)
defer canc()
detectors := gcomProvider.ProvideDetectors(ctx)
require.Zero(t, gcomScenario.gcomHTTPCalls, "gcom should be not called due to request timing out")
require.Empty(t, detectors, "returned AngularDetectors should be empty")
})
t.Run("unknown pattern types do not break decoding", func(t *testing.T) {
// Tests that we can introduce new pattern types in the future without breaking old Grafana versions.
scenario := gcomScenario{gcomHTTPHandlerFunc: func(writer http.ResponseWriter, request *http.Request) {
_, _ = writer.Write([]byte(`[
{"name": "PanelCtrl", "type": "contains", "pattern": "PanelCtrl"},
{"name": "Another", "type": "unknown", "pattern": "PanelCtrl"}
]`))
}}
srv := scenario.newHTTPTestServer()
t.Cleanup(srv.Close)
gcomProvider, err := NewGCOMDetectorsProvider(srv.URL)
require.NoError(t, err)
detectors := gcomProvider.ProvideDetectors(context.Background())
require.Equal(t, 1, scenario.gcomHTTPCalls, "gcom should be called")
require.Len(t, detectors, 1, "should have decoded only 1 AngularDetector")
d, ok := detectors[0].(*angulardetector.ContainsBytesDetector)
require.True(t, ok, "decoded pattern should be of the correct type")
require.Equal(t, []byte("PanelCtrl"), d.Pattern, "decoded value for known pattern should be correct")
})
}

View File

@ -0,0 +1,45 @@
package angularinspector
import (
"fmt"
"github.com/grafana/grafana/pkg/plugins/config"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angularinspector"
"github.com/grafana/grafana/pkg/services/featuremgmt"
pAngularDetector "github.com/grafana/grafana/pkg/services/pluginsintegration/angulardetector"
)
type Service struct {
angularinspector.Inspector
}
// newDynamicInspector returns the default dynamic Inspector, which is a PatternsListInspector that will:
// 1. Try to get the Angular detectors from GCOM
// 2. If it fails, it will use the static (hardcoded) detections provided by defaultDetectors.
func newDynamicInspector(cfg *config.Cfg) (angularinspector.Inspector, error) {
dynamicProvider, err := pAngularDetector.NewGCOMDetectorsProvider(cfg.GrafanaComURL)
if err != nil {
return nil, fmt.Errorf("NewGCOMDetectorsProvider: %w", err)
}
return &angularinspector.PatternsListInspector{
DetectorsProvider: angulardetector.SequenceDetectorsProvider{
dynamicProvider,
angularinspector.NewDefaultStaticDetectorsProvider(),
},
}, nil
}
func ProvideService(cfg *config.Cfg) (*Service, error) {
var underlying angularinspector.Inspector
var err error
if cfg.Features != nil && cfg.Features.IsEnabled(featuremgmt.FlagPluginsDynamicAngularDetectionPatterns) {
underlying, err = newDynamicInspector(cfg)
} else {
underlying, err = angularinspector.NewStaticInspector()
}
if err != nil {
return nil, err
}
return &Service{underlying}, nil
}

View File

@ -0,0 +1,42 @@
package angularinspector
import (
"context"
"testing"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/plugins/config"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angulardetector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angularinspector"
"github.com/grafana/grafana/pkg/services/featuremgmt"
pAngularDetector "github.com/grafana/grafana/pkg/services/pluginsintegration/angulardetector"
)
func TestProvideService(t *testing.T) {
t.Run("uses hardcoded inspector if feature flag is not present", func(t *testing.T) {
inspector, err := ProvideService(&config.Cfg{
Features: featuremgmt.WithFeatures(),
})
require.NoError(t, err)
require.IsType(t, inspector.Inspector, &angularinspector.PatternsListInspector{})
patternsListInspector := inspector.Inspector.(*angularinspector.PatternsListInspector)
detectors := patternsListInspector.DetectorsProvider.ProvideDetectors(context.Background())
require.NotEmpty(t, detectors, "provided detectors should not be empty")
})
t.Run("uses dynamic inspector with hardcoded fallback if feature flag is present", func(t *testing.T) {
inspector, err := ProvideService(&config.Cfg{
Features: featuremgmt.WithFeatures(featuremgmt.FlagPluginsDynamicAngularDetectionPatterns),
})
require.NoError(t, err)
require.IsType(t, inspector.Inspector, &angularinspector.PatternsListInspector{})
require.IsType(t, inspector.Inspector.(*angularinspector.PatternsListInspector).DetectorsProvider, angulardetector.SequenceDetectorsProvider{})
seq := inspector.Inspector.(*angularinspector.PatternsListInspector).DetectorsProvider.(angulardetector.SequenceDetectorsProvider)
require.Len(t, seq, 2, "should return the correct number of providers")
require.IsType(t, seq[0], &pAngularDetector.GCOMDetectorsProvider{}, "first AngularDetector provided should be gcom")
require.IsType(t, seq[1], &angulardetector.StaticDetectorsProvider{}, "second AngularDetector provided should be static")
staticDetectors := seq[1].ProvideDetectors(context.Background())
require.NotEmpty(t, staticDetectors, "provided static detectors should not be empty")
})
}

View File

@ -42,6 +42,7 @@ func ProvideConfig(settingProvider setting.Provider, grafanaCfg *setting.Cfg, fe
tracingCfg,
featuremgmt.ProvideToggles(features),
grafanaCfg.AngularSupportEnabled,
grafanaCfg.GrafanaComURL,
), nil
}

View File

@ -12,7 +12,7 @@ import (
"github.com/grafana/grafana/pkg/plugins/manager/client"
"github.com/grafana/grafana/pkg/plugins/manager/filestore"
"github.com/grafana/grafana/pkg/plugins/manager/loader"
"github.com/grafana/grafana/pkg/plugins/manager/loader/angulardetector"
pAngularInspector "github.com/grafana/grafana/pkg/plugins/manager/loader/angular/angularinspector"
"github.com/grafana/grafana/pkg/plugins/manager/loader/assetpath"
"github.com/grafana/grafana/pkg/plugins/manager/loader/finder"
"github.com/grafana/grafana/pkg/plugins/manager/process"
@ -25,6 +25,7 @@ import (
"github.com/grafana/grafana/pkg/services/caching"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/services/oauthtoken"
"github.com/grafana/grafana/pkg/services/pluginsintegration/angularinspector"
"github.com/grafana/grafana/pkg/services/pluginsintegration/clientmiddleware"
"github.com/grafana/grafana/pkg/services/pluginsintegration/config"
"github.com/grafana/grafana/pkg/services/pluginsintegration/keyretriever"
@ -52,7 +53,8 @@ var WireSet = wire.NewSet(
coreplugin.ProvideCoreRegistry,
pluginscdn.ProvideService,
assetpath.ProvideService,
angulardetector.ProvideService,
angularinspector.ProvideService,
wire.Bind(new(pAngularInspector.Inspector), new(*angularinspector.Service)),
loader.ProvideService,
wire.Bind(new(loader.Service), new(*loader.Loader)),
wire.Bind(new(plugins.ErrorResolver), new(*loader.Loader)),