grafana/docs/sources/setup-grafana/configure-security/export-logs.md

110 lines
5.8 KiB
Markdown
Raw Normal View History

---
aliases:
- ../../enterprise/usage-insights/export-logs/
description: Export logs of usage insights
keywords:
- grafana
- export
- usage-insights
- enterprise
Explicitly set all front matter labels in the source files (#71548) * Set every page to have defaults of 'Enterprise' and 'Open source' labels Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set administration pages to have of 'Cloud', 'Enterprise', and 'Open source' labels Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set administration/enterprise-licensing pages to have 'Enterprise' labels Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set administration/organization-management pages to have 'Enterprise' and 'Open source' labels Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set administration/provisioning pages to have 'Enterprise' and 'Open source' labels Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set administration/recorded-queries pages to have labels cloud,enterprise * Set administration/roles-and-permissions/access-control pages to have labels cloud,enterprise Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set administration/stats-and-license pages to have labels cloud,enterprise * Set alerting pages to have labels cloud,enterprise,oss * Set breaking-changes pages to have labels cloud,enterprise,oss * Set dashboards pages to have labels cloud,enterprise,oss * Set datasources pages to have labels cloud,enterprise,oss * Set explore pages to have labels cloud,enterprise,oss * Set fundamentals pages to have labels cloud,enterprise,oss * Set introduction/grafana-cloud pages to have labels cloud Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix introduction pages products Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set panels-visualizations pages to have labels cloud,enterprise,oss * Set release-notes pages to have labels cloud,enterprise,oss * Set search pages to have labels cloud,enterprise,oss * Set setup-grafana/configure-security/audit-grafana pages to have labels cloud,enterprise Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set setup-grafana/configure-security/configure-authentication pages to have labels cloud,enterprise,oss * Set setup-grafana/configure-security/configure-authentication/enhanced-ldap pages to have labels cloud,enterprise * Set setup-grafana/configure-security/configure-authentication/saml pages to have labels cloud,enterprise * Set setup-grafana/configure-security/configure-database-encryption/encrypt-secrets-using-hashicorp-key-vault pages to have labels cloud,enterprise * Set setup-grafana/configure-security/configure-request-security pages to have labels cloud,enterprise,oss Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set setup-grafana/configure-security/configure-team-sync pages to have labels cloud,enterprise Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set setup-grafana/configure-security/export-logs pages to have labels cloud,enterprise Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Set troubleshooting pages to have labels cloud,enterprise,oss * Set whatsnew pages to have labels cloud,enterprise,oss * Apply updated labels from review Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> Co-authored-by: Isabel <76437239+imatwawana@users.noreply.github.com> --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> Co-authored-by: Isabel <76437239+imatwawana@users.noreply.github.com>
2023-07-18 03:10:12 -05:00
labels:
products:
- cloud
- enterprise
title: Export logs of usage insights
weight: 900
---
# Export logs of usage insights
{{% admonition type="note" %}}
Available in [Grafana Enterprise]({{< relref "../../introduction/grafana-enterprise" >}}) version 7.4 and later, and [Grafana Cloud Pro and Advanced](/docs/grafana-cloud/).
{{% /admonition %}}
By exporting usage logs to Loki, you can directly query them and create dashboards of the information that matters to you most, such as dashboard errors, most active organizations, or your top-10 most-used queries. This configuration is done for you in Grafana Cloud, with provisioned dashboards. Read about them in the [Grafana Cloud documentation](/docs/grafana-cloud/usage-insights/).
## Usage insights logs
Usage insights logs are JSON objects that represent certain user activities, such as:
- A user opens a dashboard.
- A query is sent to a data source.
### Scope
A log is created every time a user opens a dashboard or when a query is sent to a data source in the dashboard view. A query that is performed via Explore does not generate a log.
### Format
Logs of usage insights contain the following fields, where the fields followed by \* are always available, and the others depend on the logged event:
| Field name | Type | Description |
| ---------- | ---- | ----------- |
| `eventName`\* | string | Type of the event, which can be either `data-request` or `dashboard-view`. |
| `folderName`\* | string | Name of the dashboard folder. |
| `dashboardName`\* | string | Name of the dashboard where the event happened. |
| `dashboardId`\* | number | ID of the dashboard where the event happened. |
| `datasourceName`| string | Name of the data source that was queried. |
| `datasourceType` | string | Type of the data source that was queried. For example, `prometheus`, `elasticsearch`, or `loki`. |
| `datasourceId` | number | ID of the data source that was queried. |
| `panelId` | number | ID of the panel of the query. |
| `panelName` | string | Name of the panel of the query. |
| `error` | string | Error returned by the query. |
| `duration` | number | Duration of the query. |
| `source` | string | Source of the query. For example, `dashboard` or `explore`. |
| `orgId`\* | number | ID of the users organization. |
| `orgName`\* | string | Name of the users organization. |
| `timestamp`\* | string | The date and time that the request was made, in Coordinated Universal Time (UTC) in [RFC3339](https://tools.ietf.org/html/rfc3339#section-5.6) format. |
| `tokenId`\* | number | ID of the users authentication token. |
| `username`\* | string | Name of the Grafana user that made the request. |
| `userId`\* | number | ID of the Grafana user that made the request. |
| `totalQueries`\* | number | Number of queries executed for the data request. |
| `cachedQueries`\* | number | Number of fetched queries that came from the cache. |
## Configuration
Enable doc-validator for specific directories (#63524) * Enable doc-validator for specific directories Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix one linting error to trigger CI Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update doc-validator to latest release Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update make-docs procedure Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use doc-validator version from CI in local make target Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Revert to 1.11.0 Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * adds missing descriptions * Fix titles and headings Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix link formats Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix easy to resolve anchors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Remove broken anchor link This anchor appears to have been broken for a long time. Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use doc-validator image with support for numbered anchors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update make-docs procedure to support doc-validator 2.0.x Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix a bunch of broken anchors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Ignore old whatsnew content Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update doc-validator to v2.0.x and use reviewdog to report errors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * removes broken links --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
2023-05-18 09:50:20 -05:00
To export your logs, enable the usage insights feature and [configure]({{< relref "../../setup-grafana/configure-grafana" >}}) an export location in the configuration file:
```ini
[usage_insights.export]
# Enable the usage insights export feature
enabled = true
# Storage type
storage = loki
```
The options for storage type are `loki` and `logger` (added in Grafana Enterprise 8.2).
If the storage type is set to `loki` you'll need to also configure Grafana
to export to a Loki ingestion server. To do this, you'll need Loki installed.
Refer to [Install Loki](/docs/loki/latest/installation/) for instructions
on how to install Loki.
```ini
[usage_insights.export.storage.loki]
# Set the communication protocol to use with Loki (can be grpc or http)
type = grpc
# Set the address for writing logs to Loki (format must be host:port)
url = localhost:9095
# Defaults to true. If true, it establishes a secure connection to Loki
tls = true
# Set the tenant ID for Loki communication, which is disabled by default.
# The tenant ID is required to interact with Loki running in multi-tenant mode.
tenant_id =
```
Enable doc-validator for specific directories (#63524) * Enable doc-validator for specific directories Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix one linting error to trigger CI Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update doc-validator to latest release Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update make-docs procedure Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use doc-validator version from CI in local make target Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Revert to 1.11.0 Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * adds missing descriptions * Fix titles and headings Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix link formats Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix easy to resolve anchors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Remove broken anchor link This anchor appears to have been broken for a long time. Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use doc-validator image with support for numbered anchors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update make-docs procedure to support doc-validator 2.0.x Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Fix a bunch of broken anchors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Ignore old whatsnew content Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update doc-validator to v2.0.x and use reviewdog to report errors Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * removes broken links --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Chris Moyer <chris.moyer@grafana.com>
2023-05-18 09:50:20 -05:00
Using `logger` will print usage insights to your [Grafana server log]({{< relref "../../setup-grafana/configure-grafana#log" >}}).
There is no option for configuring the `logger` storage type.
## Visualize Loki usage insights in Grafana
If you export logs into Loki, you can build Grafana dashboards to understand your Grafana instance usage.
1. Add Loki as a data source. Refer to [Grafana fundamentals tutorial](/tutorials/grafana-fundamentals/#6).
1. Import one of the following dashboards:
- [Usage insights](/grafana/dashboards/13785)
- [Usage insights datasource details](/grafana/dashboards/13786)
1. Play with usage insights to understand them:
- In Explore, you can use the query `{datasource="gdev-loki",kind="usage_insights"}` to retrieve all logs related to your `gdev-loki` data source.
- In a dashboard, you can build a table panel with the query `topk(10, sum by (error) (count_over_time({kind="usage_insights", datasource="gdev-prometheus"} | json | error != "" [$__interval])))` to display the 10 most common errors your users see using the `gdev-prometheus` data source.
- In a dashboard, you can build a graph panel with the queries `sum by(host) (count_over_time({kind="usage_insights"} | json | eventName="data-request" | error != "" [$__interval]))` and `sum by(host) (count_over_time({kind="usage_insights"} | json | eventName="data-request" | error = "" [$__interval]))` to show the evolution of the data request count over time. Using `by (host)` allows you to have more information for each Grafana server you have if you have set up Grafana for [high availability](<{{< relref "../../setup-grafana/set-up-for-high-availability/" >}}>).