IntenseWebs/grafana
3
0
Fork 0
mirror of https://github.com/grafana/grafana.git synced 2025-02-25 18:55:37 -06:00
Code Issues Packages Projects Releases Wiki Activity

Caching: Add feature toggle for memory efficient cache payload serialization (#76145)

Browse Source 
* add feature toggle for smart cache serialization

* re-add toggle after merge conflict

* switch feature toggle stage to experimental

* incorporate PR feedback
This commit is contained in:
Michael Mandrus 2023-10-12 12:56:49 -04:00 committed by GitHub
parent 5a79e70d20
commit 94ce87571d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 117 additions and 103 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
docs/sources/setup-grafana/configure-grafana/feature-toggles/index.md
View File

@ -20,7 +20,7 @@ This page contains a list of available feature toggles. To learn how to turn on
Some features are enabled by default. You can disable these feature by setting the feature flag to "false" in the configuration. Some features are enabled by default. You can disable these feature by setting the feature flag to "false" in the configuration.
| Feature toggle name | Description | Enabled by default | | Feature toggle name | Description | Enabled by default |
| ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ |
| `disableEnvelopeEncryption` | Disable envelope encryption (emergency only) | | | `disableEnvelopeEncryption` | Disable envelope encryption (emergency only) | |
| `featureHighlights` | Highlight Grafana Enterprise features | | | `featureHighlights` | Highlight Grafana Enterprise features | |
| `dataConnectionsConsole` | Enables a new top-level page called Connections. This page is an experiment that provides a better experience when you install and configure data sources and other plugins. | Yes | | `dataConnectionsConsole` | Enables a new top-level page called Connections. This page is an experiment that provides a better experience when you install and configure data sources and other plugins. | Yes |
@ -28,18 +28,18 @@ Some features are enabled by default. You can disable these feature by setting t
| `redshiftAsyncQueryDataSupport` | Enable async query data support for Redshift | Yes | | `redshiftAsyncQueryDataSupport` | Enable async query data support for Redshift | Yes |
| `athenaAsyncQueryDataSupport` | Enable async query data support for Athena | Yes | | `athenaAsyncQueryDataSupport` | Enable async query data support for Athena | Yes |
| `cloudwatchNewRegionsHandler` | Refactor of /regions endpoint, no user-facing changes | Yes | | `cloudwatchNewRegionsHandler` | Refactor of /regions endpoint, no user-facing changes | Yes |
| `nestedFolderPicker` | Enables the new folder picker to work with nested folders. Requires the nestedFolders feature flag | Yes | | `nestedFolderPicker` | Enables the new folder picker to work with nested folders. Requires the nestedFolders feature toggle | Yes |
| `accessTokenExpirationCheck` | Enable OAuth access_token expiration check and token refresh using the refresh_token | | | `accessTokenExpirationCheck` | Enable OAuth access_token expiration check and token refresh using the refresh_token | |
| `emptyDashboardPage` | Enable the redesigned user interface of a dashboard page that includes no panels | Yes | | `emptyDashboardPage` | Enable the redesigned user interface of a dashboard page that includes no panels | Yes |
| `disablePrometheusExemplarSampling` | Disable Prometheus exemplar sampling | | | `disablePrometheusExemplarSampling` | Disable Prometheus exemplar sampling | |
| `logsContextDatasourceUi` | Allow datasource to provide custom UI for context view | Yes | | `logsContextDatasourceUi` | Allow datasource to provide custom UI for context view | Yes |
| `gcomOnlyExternalOrgRoleSync` | Prohibits a user from changing organization roles synced with Grafana Cloud auth provider | | | `gcomOnlyExternalOrgRoleSync` | Prohibits a user from changing organization roles synced with Grafana Cloud auth provider | |
| `prometheusMetricEncyclopedia` | Adds the metrics explorer component to the Prometheus query builder as an option in metric select | Yes | | `prometheusMetricEncyclopedia` | Adds the metrics explorer component to the Prometheus query builder as an option in metric select | Yes |
| `prometheusDataplane` | Changes responses to from Prometheus to be compliant with the dataplane specification. In particular it sets the numeric Field.Name from 'Value' to the value of the `__name__` label when present. | Yes | | `prometheusDataplane` | Changes responses to from Prometheus to be compliant with the dataplane specification. In particular, when this feature toggle is active, the numeric `Field.Name` is set from 'Value' to the value of the `__name__` label. | Yes |
| `lokiMetricDataplane` | Changes metric responses from Loki to be compliant with the dataplane specification. | Yes | | `lokiMetricDataplane` | Changes metric responses from Loki to be compliant with the dataplane specification. | Yes |
| `dataplaneFrontendFallback` | Support dataplane contract field name change for transformations and field name matchers where the name is different | Yes | | `dataplaneFrontendFallback` | Support dataplane contract field name change for transformations and field name matchers where the name is different | Yes |
| `alertingNotificationsPoliciesMatchingInstances` | Enables the preview of matching instances for notification policies | Yes | | `alertingNotificationsPoliciesMatchingInstances` | Enables the preview of matching instances for notification policies | Yes |
| `useCachingService` | When turned on, the new query and resource caching implementation using a wire service inject will be used in place of the previous middleware implementation | | | `useCachingService` | When active, the new query and resource caching implementation using a wire service inject replaces the previous middleware implementation. | |
| `enableElasticsearchBackendQuerying` | Enable the processing of queries and responses in the Elasticsearch data source through backend | Yes | | `enableElasticsearchBackendQuerying` | Enable the processing of queries and responses in the Elasticsearch data source through backend | Yes |
| `advancedDataSourcePicker` | Enable a new data source picker with contextual information, recently used order and advanced mode | Yes | | `advancedDataSourcePicker` | Enable a new data source picker with contextual information, recently used order and advanced mode | Yes |
| `cloudWatchLogsMonacoEditor` | Enables the Monaco editor for CloudWatch Logs queries | Yes | | `cloudWatchLogsMonacoEditor` | Enables the Monaco editor for CloudWatch Logs queries | Yes |
@ -85,7 +85,7 @@ These features are early in their development lifecycle and so are not yet suppo
Experimental features might be changed or removed without prior notice. Experimental features might be changed or removed without prior notice.
| Feature toggle name | Description | | Feature toggle name | Description |
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `live-service-web-worker` | This will use a webworker thread to processes events rather than the main thread | | `live-service-web-worker` | This will use a webworker thread to processes events rather than the main thread |
| `queryOverLive` | Use Grafana Live WebSocket to execute backend queries | | `queryOverLive` | Use Grafana Live WebSocket to execute backend queries |
| `lokiExperimentalStreaming` | Support new streaming approach for loki (prototype, needs special loki build) | | `lokiExperimentalStreaming` | Support new streaming approach for loki (prototype, needs special loki build) |
@ -149,6 +149,7 @@ Experimental features might be changed or removed without prior notice.
| `navAdminSubsections` | Splits the administration section of the nav tree into subsections | | `navAdminSubsections` | Splits the administration section of the nav tree into subsections |
| `recoveryThreshold` | Enables feature recovery threshold (aka hysteresis) for threshold server-side expression | | `recoveryThreshold` | Enables feature recovery threshold (aka hysteresis) for threshold server-side expression |
| `awsDatasourcesNewFormStyling` | Applies new form styling for configuration and query editors in AWS plugins | | `awsDatasourcesNewFormStyling` | Applies new form styling for configuration and query editors in AWS plugins |
| `cachingOptimizeSerializationMemoryUsage` | If enabled, the caching backend gradually serializes query responses for the cache, comparing against the configured `[caching]max_value_mb` value as it goes. This can can help prevent Grafana from running out of memory while attempting to cache very large query responses. |
## Development feature toggles ## Development feature toggles

1
packages/grafana-data/src/types/featureToggles.gen.ts
View File

@ -142,4 +142,5 @@ export interface FeatureToggles {
navAdminSubsections?: boolean; navAdminSubsections?: boolean;
recoveryThreshold?: boolean; recoveryThreshold?: boolean;
awsDatasourcesNewFormStyling?: boolean; awsDatasourcesNewFormStyling?: boolean;
cachingOptimizeSerializationMemoryUsage?: boolean;
} }

13
pkg/services/featuremgmt/registry.go
View File

@ -236,7 +236,7 @@ var (
}, },
{ {
Name: "nestedFolderPicker", Name: "nestedFolderPicker",
Description: "Enables the new folder picker to work with nested folders. Requires the nestedFolders feature flag", Description: "Enables the new folder picker to work with nested folders. Requires the nestedFolders feature toggle",
Stage: FeatureStageGeneralAvailability, Stage: FeatureStageGeneralAvailability,
Owner: grafanaFrontendPlatformSquad, Owner: grafanaFrontendPlatformSquad,
FrontendOnly: true, FrontendOnly: true,
@ -346,7 +346,7 @@ var (
}, },
{ {
Name: "prometheusDataplane", Name: "prometheusDataplane",
Description: "Changes responses to from Prometheus to be compliant with the dataplane specification. In particular it sets the numeric Field.Name from 'Value' to the value of the `__name__` label when present.", Description: "Changes responses to from Prometheus to be compliant with the dataplane specification. In particular, when this feature toggle is active, the numeric `Field.Name` is set from 'Value' to the value of the `__name__` label.",
Expression: "true", Expression: "true",
Stage: FeatureStageGeneralAvailability, Stage: FeatureStageGeneralAvailability,
Owner: grafanaObservabilityMetricsSquad, Owner: grafanaObservabilityMetricsSquad,
@ -431,7 +431,7 @@ var (
}, },
{ {
Name: "useCachingService", Name: "useCachingService",
Description: "When turned on, the new query and resource caching implementation using a wire service inject will be used in place of the previous middleware implementation", Description: "When active, the new query and resource caching implementation using a wire service inject replaces the previous middleware implementation.",
Stage: FeatureStageGeneralAvailability, Stage: FeatureStageGeneralAvailability,
Owner: grafanaOperatorExperienceSquad, Owner: grafanaOperatorExperienceSquad,
RequiresRestart: true, RequiresRestart: true,
@ -867,5 +867,12 @@ var (
FrontendOnly: true, FrontendOnly: true,
Owner: awsDatasourcesSquad, Owner: awsDatasourcesSquad,
}, },
{
Name: "cachingOptimizeSerializationMemoryUsage",
Description: "If enabled, the caching backend gradually serializes query responses for the cache, comparing against the configured `[caching]max_value_mb` value as it goes. This can can help prevent Grafana from running out of memory while attempting to cache very large query responses.",
Stage: FeatureStageExperimental,
Owner: grafanaOperatorExperienceSquad,
FrontendOnly: false,
},
} }
) )

1
pkg/services/featuremgmt/toggles_gen.csv
View File

@ -123,3 +123,4 @@ kubernetesPlaylists,experimental,@grafana/grafana-app-platform-squad,false,false
navAdminSubsections,experimental,@grafana/grafana-frontend-platform,false,false,false,false navAdminSubsections,experimental,@grafana/grafana-frontend-platform,false,false,false,false
recoveryThreshold,experimental,@grafana/alerting-squad,false,false,true,false recoveryThreshold,experimental,@grafana/alerting-squad,false,false,true,false
awsDatasourcesNewFormStyling,experimental,@grafana/aws-datasources,false,false,false,true awsDatasourcesNewFormStyling,experimental,@grafana/aws-datasources,false,false,false,true
cachingOptimizeSerializationMemoryUsage,experimental,@grafana/grafana-operator-experience-squad,false,false,false,false

1 Name Stage Owner requiresDevMode RequiresLicense RequiresRestart FrontendOnly
123 navAdminSubsections experimental @grafana/grafana-frontend-platform false false false false
124 recoveryThreshold experimental @grafana/alerting-squad false false true false
125 awsDatasourcesNewFormStyling experimental @grafana/aws-datasources false false false true
126 cachingOptimizeSerializationMemoryUsage experimental @grafana/grafana-operator-experience-squad false false false false

10
pkg/services/featuremgmt/toggles_gen.go
View File

@ -144,7 +144,7 @@ const (
FlagNestedFolders = "nestedFolders" FlagNestedFolders = "nestedFolders"
// FlagNestedFolderPicker // FlagNestedFolderPicker
// Enables the new folder picker to work with nested folders. Requires the nestedFolders feature flag // Enables the new folder picker to work with nested folders. Requires the nestedFolders feature toggle
FlagNestedFolderPicker = "nestedFolderPicker" FlagNestedFolderPicker = "nestedFolderPicker"
// FlagAccessTokenExpirationCheck // FlagAccessTokenExpirationCheck
@ -208,7 +208,7 @@ const (
FlagClientTokenRotation = "clientTokenRotation" FlagClientTokenRotation = "clientTokenRotation"
// FlagPrometheusDataplane // FlagPrometheusDataplane
// Changes responses to from Prometheus to be compliant with the dataplane specification. In particular it sets the numeric Field.Name from 'Value' to the value of the `__name__` label when present. // Changes responses to from Prometheus to be compliant with the dataplane specification. In particular, when this feature toggle is active, the numeric `Field.Name` is set from 'Value' to the value of the `__name__` label.
FlagPrometheusDataplane = "prometheusDataplane" FlagPrometheusDataplane = "prometheusDataplane"
// FlagLokiMetricDataplane // FlagLokiMetricDataplane
@ -260,7 +260,7 @@ const (
FlagRefactorVariablesTimeRange = "refactorVariablesTimeRange" FlagRefactorVariablesTimeRange = "refactorVariablesTimeRange"
// FlagUseCachingService // FlagUseCachingService
// When turned on, the new query and resource caching implementation using a wire service inject will be used in place of the previous middleware implementation // When active, the new query and resource caching implementation using a wire service inject replaces the previous middleware implementation.
FlagUseCachingService = "useCachingService" FlagUseCachingService = "useCachingService"
// FlagEnableElasticsearchBackendQuerying // FlagEnableElasticsearchBackendQuerying
@ -502,4 +502,8 @@ const (
// FlagAwsDatasourcesNewFormStyling // FlagAwsDatasourcesNewFormStyling
// Applies new form styling for configuration and query editors in AWS plugins // Applies new form styling for configuration and query editors in AWS plugins
FlagAwsDatasourcesNewFormStyling = "awsDatasourcesNewFormStyling" FlagAwsDatasourcesNewFormStyling = "awsDatasourcesNewFormStyling"
// FlagCachingOptimizeSerializationMemoryUsage
// If enabled, the caching backend gradually serializes query responses for the cache, comparing against the configured `[caching]max_value_mb` value as it goes. This can can help prevent Grafana from running out of memory while attempting to cache very large query responses.
FlagCachingOptimizeSerializationMemoryUsage = "cachingOptimizeSerializationMemoryUsage"
) )