Docs: Setup refactor (#49739)

* builds out refactored setup topics * Automatically fix some relrefs with mv-manager Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Use refs for tutorials content which is outside of this repository Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Manually fix complicated relrefs Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * consolidates team sync and db encryption topics * Fix relrefs Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * updates setup index file * Convert TOML to YAML Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Add current alias for new alerting content Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Add current aliases to new setup-grafana and configure-security pages Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Update docs/sources/setup-grafana/configure-grafana/_index.md Co-authored-by: Torkel Ödegaard <torkel@grafana.com> * moves saml docs, updates order in TOC * Manually fix relrefs Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * added usage insights topics, adjusted weights * corrected relrefs * Fix relrefs broken in rebase Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
2025-02-25 18:55:37 -06:00 · 2022-06-02 11:57:22 -05:00
parent 104c4a7630
commit a568d421f8
345 changed files with 1557 additions and 1733 deletions
--- a/docs/sources/setup-grafana/image-rendering/_index.md
+++ b/docs/sources/setup-grafana/image-rendering/_index.md
@@ -0,0 +1,434 @@
+---
+aliases:
+  - /docs/grafana/latest/administration/image_rendering/
+  - /docs/grafana/latest/image-rendering/
+  - /docs/grafana/latest/setup-grafana/image-rendering/
+description: Image rendering
+keywords:
+  - grafana
+  - image
+  - rendering
+  - plugin
+title: Set up image rendering
+weight: 1000
+---
+
+# Set up image rendering
+
+Grafana supports automatic rendering of panels as PNG images. This allows Grafana to automatically generate images of your panels to include in [alert notifications]({{< relref "../../alerting/notifications/" >}}), [PDF export]({{< relref "../../enterprise/export-pdf/" >}}), and [Reporting]({{< relref "../../enterprise/reporting/" >}}). PDF Export and Reporting are available only in [Grafana Enterprise]({{< relref "../../enterprise/" >}}).
+
+> **Note:** Image rendering of dashboards is not supported at this time.
+
+While an image is being rendered, the PNG image is temporarily written to the file system. When the image is rendered, the PNG image is temporarily written to the `png` folder in the Grafana `data` folder.
+
+A background job runs every 10 minutes and removes temporary images. You can configure how long an image should be stored before being removed by configuring the [temp_data_lifetime]({{< relref "../configure-grafana/#temp_data_lifetime" >}}) setting.
+
+You can also render a PNG by clicking the dropdown arrow next to a panel title, then clicking **Share > Direct link rendered image**.
+
+## Alerting and render limits
+
+Alert notifications can include images, but rendering many images at the same time can overload the server where the renderer is running. For instructions of how to configure this, see [concurrent_render_limit]({{< relref "../configure-grafana/#concurrent_render_limit" >}}).
+
+## Install Grafana Image Renderer plugin
+
+> **Note:** Starting from Grafana v7.0.0, all PhantomJS support has been removed. Please use the Grafana Image Renderer plugin or remote rendering service.
+
+To install the plugin, refer to the [Grafana Image Renderer Installation instructions](https://grafana.com/grafana/plugins/grafana-image-renderer#installation).
+
+## Configuration
+
+The Grafana Image Renderer plugin has a number of configuration options that are used in plugin or remote rendering modes.
+
+In plugin mode, you can specify them directly in the [Grafana configuration file]({{< relref "../configure-grafana/#plugingrafana-image-renderer" >}}).
+
+In remote rendering mode, you can specify them in a `.json` [configuration file](#configuration-file) or, for some of them, you can override the configuration defaults using environment variables.
+
+### Configuration file
+
+You can update your settings by using a configuration file, see [default.json](https://github.com/grafana/grafana-image-renderer/tree/master/default.json) for defaults. Note that any configured environment variable takes precedence over configuration file settings.
+
+You can volume mount your custom configuration file when starting the docker container:
+
+```bash
+docker run -d --name=renderer --network=host -v /some/path/config.json:/usr/src/app/config.json grafana/grafana-image-renderer:latest
+```
+
+You can see a docker-compose example using a custom configuration file [here](https://github.com/grafana/grafana-image-renderer/tree/master/devenv/docker/custom-config).
+
+### Rendering mode
+
+You can instruct how headless browser instances are created by configuring a rendering mode. Default is `default`, other supported values are `clustered` and `reusable`.
+
+#### Default
+
+Default mode will create a new browser instance on each request. When handling multiple concurrent requests, this mode increases memory usage as it will launch multiple browsers at the same time. If you want to set a maximum number of browser to open, you'll need to use the [clustered mode](#clustered).
+
+> **Note:** When using the `default` mode, it's recommended to not remove the default Chromium flag `--disable-gpu`. When receiving a lot of concurrent requests, not using this flag can cause Puppeteer `newPage` function to freeze, causing request timeouts and leaving browsers open.
+
+```bash
+RENDERING_MODE=default
+```
+
+```json
+{
+  "rendering": {
+    "mode": "default"
+  }
+}
+```
+
+#### Clustered
+
+With the `clustered` mode, you can configure how many browser instances or incognito pages can execute concurrently. Default is `browser` and will ensure a maximum amount of browser instances can execute concurrently. Mode `context` will ensure a maximum amount of incognito pages can execute concurrently. You can also configure the maximum concurrency allowed, which per default is `5`, and the maximum duration of a rendering request, which per default is `30` seconds.
+
+Using a cluster of incognito pages is more performant and consumes less CPU and memory than a cluster of browsers. However, if one page crashes it can bring down the entire browser with it (making all the rendering requests happening at the same time fail). Also, each page isn't guaranteed to be totally clean (cookies and storage might bleed-through as seen [here](https://bugs.chromium.org/p/chromium/issues/detail?id=754576)).
+
+```bash
+RENDERING_MODE=clustered
+RENDERING_CLUSTERING_MODE=browser
+RENDERING_CLUSTERING_MAX_CONCURRENCY=5
+RENDERING_CLUSTERING_TIMEOUT=30
+```
+
+```json
+{
+  "rendering": {
+    "mode": "clustered",
+    "clustering": {
+      "mode": "browser",
+      "maxConcurrency": 5,
+      "timeout": 30
+    }
+  }
+}
+```
+
+##### Cluster mode `contextPerRenderKey` (experimental)
+
+> **Note:** This feature is available in Image Renderer v3.4.0 and later versions.
+
+In `contextPerRenderKey` mode, the plugin will reuse the same [browser context](https://chromedevtools.github.io/devtools-protocol/tot/Target/#method-createBrowserContext) for all rendering requests sharing the same `renderKey` auth cookie and target domain within a short time window. Each new request will open a new page within the existing context. Contexts are closed automatically after 5s of inactivity.
+
+In the case of `contextPerRenderKey` mode, the `clustering.max_concurrency` option refers to the number of open contexts rather than the number of open pages. There is no way to limit the number of open pages in a context.
+
+`contextPerRenderKey` was designed to improve the performance of the [dashboard previews crawler]({{< relref "../../dashboards/previews/#about-the-dashboard-previews-crawler" >}}).
+
+```json
+{
+  "rendering": {
+    "mode": "clustered",
+    "clustering": {
+      "mode": "contextPerRenderKey",
+      "maxConcurrency": 5,
+      "timeout": 30
+    }
+  }
+}
+```
+
+#### Reusable (experimental)
+
+When using the rendering mode `reusable`, one browser instance will be created and reused. A new incognito page will be opened for each request. This mode is experimental since, if the browser instance crashes, it will not automatically be restarted. You can achieve a similar behavior using `clustered` mode with a high `maxConcurrency` setting.
+
+```bash
+RENDERING_MODE=reusable
+```
+
+```json
+{
+  "rendering": {
+    "mode": "reusable"
+  }
+}
+```
+
+#### Optimize the performance, CPU and memory usage of the image renderer
+
+The performance and resources consumption of the different modes depend a lot on the number of concurrent requests your service is handling. To understand how many concurrent requests your service is handling, [monitor your image renderer service]({{< relref "monitoring/" >}}).
+
+With no concurrent requests, the different modes show very similar performance and CPU / memory usage.
+
+When handling concurrent requests, we see the following trends:
+
+- To improve performance and reduce CPU and memory consumption, use [clustered](#clustered) mode with `RENDERING_CLUSTERING_MODE` set as `context`. This parallelizes incognito pages instead of browsers.
+- If you use the [clustered](#clustered) mode with a `maxConcurrency` setting below your average number of concurrent requests, performance will drop as the rendering requests will need to wait for the other to finish before getting access to an incognito page / browser.
+
+To achieve better performance, monitor the machine on which your service is running. If you don't have enough memory and / or CPU, every rendering step will be slower than usual, increasing the duration of every rendering request.
+
+### Other available settings
+
+> **Note:** Please note that not all settings are available using environment variables. If there is no example using environment variable below, it means that you need to update the configuration file.
+
+#### HTTP host
+
+Change the listening host of the HTTP server. Default is unset and will use the local host.
+
+```bash
+HTTP_HOST=localhost
+```
+
+```json
+{
+  "service": {
+    "host": "localhost"
+  }
+}
+```
+
+#### HTTP port
+
+Change the listening port of the HTTP server. Default is `8081`. Setting `0` will automatically assign a port not in use.
+
+```bash
+HTTP_PORT=0
+```
+
+```json
+{
+  "service": {
+    "port": 0
+  }
+}
+```
+
+#### Enable Prometheus metrics
+
+You can enable [Prometheus](https://prometheus.io/) metrics endpoint `/metrics` using the environment variable `ENABLE_METRICS`. Node.js and render request duration metrics are included, see [output example](./monitoring/#prometheus-metrics-endpoint-output-example) for details.
+
+Default is `false`.
+
+```bash
+ENABLE_METRICS=true
+```
+
+```json
+{
+  "service": {
+    "metrics": {
+      "enabled": true,
+      "collectDefaultMetrics": true,
+      "requestDurationBuckets": [1, 5, 7, 9, 11, 13, 15, 20, 30]
+    }
+  }
+}
+```
+
+#### Log level
+
+Change the log level. Default is `info` and will include log messages with level `error`, `warning` and `info`.
+
+```bash
+LOG_LEVEL=debug
+```
+
+```json
+{
+  "service": {
+    "logging": {
+      "level": "debug",
+      "console": {
+        "json": false,
+        "colorize": true
+      }
+    }
+  }
+}
+```
+
+#### Verbose logging
+
+Instruct headless browser instance whether to capture and log verbose information when rendering an image. Default is `false` and will only capture and log error messages. When enabled (`true`) debug messages are captured and logged as well.
+
+Note that you need to change log level to `debug`, see above, for the verbose information to be included in the logs.
+
+```bash
+RENDERING_VERBOSE_LOGGING=true
+```
+
+```json
+{
+  "rendering": {
+    "verboseLogging": true
+  }
+}
+```
+
+#### Capture browser output
+
+Instruct headless browser instance whether to output its debug and error messages into running process of remote rendering service. Default is `false`.
+This can be useful to enable (`true`) when troubleshooting.
+
+```bash
+RENDERING_DUMPIO=true
+```
+
+```json
+{
+  "rendering": {
+    "dumpio": true
+  }
+}
+```
+
+#### Custom Chrome/Chromium
+
+If you already have [Chrome](https://www.google.com/chrome/) or [Chromium](https://www.chromium.org/)
+installed on your system, then you can use this instead of the pre-packaged version of Chromium.
+
+> **Note:** Please note that this is not recommended, since you may encounter problems if the installed version of Chrome/Chromium is not compatible with the [Grafana Image renderer plugin](https://grafana.com/grafana/plugins/grafana-image-renderer).
+
+You need to make sure that the Chrome/Chromium executable is available for the Grafana/image rendering service process.
+
+```bash
+CHROME_BIN="/usr/bin/chromium-browser"
+```
+
+```json
+{
+  "rendering": {
+    "chromeBin": "/usr/bin/chromium-browser"
+  }
+}
+```
+
+#### Start browser with additional arguments
+
+Additional arguments to pass to the headless browser instance. Defaults are `--no-sandbox,--disable-gpu`. The list of Chromium flags can be found [here](https://peter.sh/experiments/chromium-command-line-switches/) and the list of flags used as defaults by Puppeteer can be found [there](https://github.com/puppeteer/puppeteer/blob/main/src/node/Launcher.ts#L172). Multiple arguments is separated with comma-character.
+
+```bash
+RENDERING_ARGS=--no-sandbox,--disable-setuid-sandbox,--disable-dev-shm-usage,--disable-accelerated-2d-canvas,--disable-gpu,--window-size=1280x758
+```
+
+```json
+{
+  "rendering": {
+    "args": [
+      "--no-sandbox",
+      "--disable-setuid-sandbox",
+      "--disable-dev-shm-usage",
+      "--disable-accelerated-2d-canvas",
+      "--disable-gpu",
+      "--window-size=1280x758"
+    ]
+  }
+}
+```
+
+#### Ignore HTTPS errors
+
+Instruct headless browser instance whether to ignore HTTPS errors during navigation. Per default HTTPS errors are not ignored.
+Due to the security risk it's not recommended to ignore HTTPS errors.
+
+```bash
+IGNORE_HTTPS_ERRORS=true
+```
+
+```json
+{
+  "rendering": {
+    "ignoresHttpsErrors": true
+  }
+}
+```
+
+#### Default timezone
+
+Instruct headless browser instance to use a default timezone when not provided by Grafana, .e.g. when rendering panel image of alert. See [ICU’s metaZones.txt](https://cs.chromium.org/chromium/src/third_party/icu/source/data/misc/metaZones.txt?rcl=faee8bc70570192d82d2978a71e2a615788597d1) for a list of supported timezone IDs. Fallbacks to `TZ` environment variable if not set.
+
+```bash
+BROWSER_TZ=Europe/Stockholm
+```
+
+```json
+{
+  "rendering": {
+    "timezone": "Europe/Stockholm"
+  }
+}
+```
+
+#### Default language
+
+Instruct headless browser instance to use a default language when not provided by Grafana, e.g. when rendering panel image of alert.
+Refer to the HTTP header Accept-Language to understand how to format this value.
+
+```json
+{
+  "rendering": {
+    "acceptLanguage": "fr-CH, fr;q=0.9, en;q=0.8, de;q=0.7, *;q=0.5"
+  }
+}
+```
+
+#### Viewport width
+
+Default viewport width when width is not specified in the rendering request. Default is `1000`.
+
+```json
+{
+  "rendering": {
+    "width": 1000
+  }
+}
+```
+
+#### Viewport height
+
+Default viewport height when height is not specified in the rendering request. Default is `500`.
+
+```json
+{
+  "rendering": {
+    "height": 500
+  }
+}
+```
+
+#### Viewport maximum width
+
+Limit the maximum viewport width that can be requested. Default is `3000`.
+
+```json
+{
+  "rendering": {
+    "maxWidth": 1000
+  }
+}
+```
+
+#### Viewport maximum height
+
+Limit the maximum viewport height that can be requested. Default is `3000`.
+
+```json
+{
+  "rendering": {
+    "maxHeight": 500
+  }
+}
+```
+
+#### Device scale factor
+
+Specify default device scale factor for rendering images. `2` is enough for monitor resolutions, `4` would be better for printed material. Setting a higher value affects performance and memory. Default is `1`.
+This can be overridden in the rendering request.
+
+```json
+{
+  "rendering": {
+    "deviceScaleFactor": 2
+  }
+}
+```
+
+#### Maximum device scale factor
+
+Limit the maximum device scale factor that can be requested. Default is `4`.
+
+```json
+{
+  "rendering": {
+    "maxDeviceScaleFactor": 4
+  }
+}
+```
--- a/docs/sources/setup-grafana/image-rendering/monitoring.md
+++ b/docs/sources/setup-grafana/image-rendering/monitoring.md
@@ -0,0 +1,218 @@
+---
+aliases:
+  - /docs/grafana/latest/image-rendering/monitoring/
+  - /docs/grafana/latest/setup-grafana/image-rendering/monitoring/
+description: Image rendering monitoring
+keywords:
+  - grafana
+  - image
+  - rendering
+  - plugin
+  - monitoring
+title: Monitoring the image renderer
+weight: 100
+---
+
+# Monitoring the image renderer
+
+Rendering images requires a lot of memory, mainly because Grafana creates browser instances in the background for the actual rendering. Monitoring your service can help you allocate the right amount of resources to your rendering service and set the right [rendering mode]({{< relref "/#rendering-mode" >}}).
+
+## Enable Prometheus metrics endpoint
+
+Configure this service to expose a Prometheus metrics endpoint. For information on how to configure and monitor this service using Prometheus as a data source, refer to [Grafana Image Rendering Service dashboard](https://grafana.com/grafana/dashboards/12203).
+
+**Metrics endpoint output example:**
+
+```
+# HELP process_cpu_user_seconds_total Total user CPU time spent in seconds.
+# TYPE process_cpu_user_seconds_total counter
+process_cpu_user_seconds_total 0.536 1579444523566
+
+# HELP process_cpu_system_seconds_total Total system CPU time spent in seconds.
+# TYPE process_cpu_system_seconds_total counter
+process_cpu_system_seconds_total 0.064 1579444523566
+
+# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
+# TYPE process_cpu_seconds_total counter
+process_cpu_seconds_total 0.6000000000000001 1579444523566
+
+# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
+# TYPE process_start_time_seconds gauge
+process_start_time_seconds 1579444433
+
+# HELP process_resident_memory_bytes Resident memory size in bytes.
+# TYPE process_resident_memory_bytes gauge
+process_resident_memory_bytes 52686848 1579444523568
+
+# HELP process_virtual_memory_bytes Virtual memory size in bytes.
+# TYPE process_virtual_memory_bytes gauge
+process_virtual_memory_bytes 2055344128 1579444523568
+
+# HELP process_heap_bytes Process heap size in bytes.
+# TYPE process_heap_bytes gauge
+process_heap_bytes 1996390400 1579444523568
+
+# HELP process_open_fds Number of open file descriptors.
+# TYPE process_open_fds gauge
+process_open_fds 31 1579444523567
+
+# HELP process_max_fds Maximum number of open file descriptors.
+# TYPE process_max_fds gauge
+process_max_fds 1573877
+
+# HELP nodejs_eventloop_lag_seconds Lag of event loop in seconds.
+# TYPE nodejs_eventloop_lag_seconds gauge
+nodejs_eventloop_lag_seconds 0.000915922 1579444523567
+
+# HELP nodejs_active_handles Number of active libuv handles grouped by handle type. Every handle type is C++ class name.
+# TYPE nodejs_active_handles gauge
+nodejs_active_handles{type="WriteStream"} 2 1579444523566
+nodejs_active_handles{type="Server"} 1 1579444523566
+nodejs_active_handles{type="Socket"} 9 1579444523566
+nodejs_active_handles{type="ChildProcess"} 2 1579444523566
+
+# HELP nodejs_active_handles_total Total number of active handles.
+# TYPE nodejs_active_handles_total gauge
+nodejs_active_handles_total 14 1579444523567
+
+# HELP nodejs_active_requests Number of active libuv requests grouped by request type. Every request type is C++ class name.
+# TYPE nodejs_active_requests gauge
+nodejs_active_requests{type="FSReqCallback"} 2
+
+# HELP nodejs_active_requests_total Total number of active requests.
+# TYPE nodejs_active_requests_total gauge
+nodejs_active_requests_total 2 1579444523567
+
+# HELP nodejs_heap_size_total_bytes Process heap size from node.js in bytes.
+# TYPE nodejs_heap_size_total_bytes gauge
+nodejs_heap_size_total_bytes 13725696 1579444523567
+
+# HELP nodejs_heap_size_used_bytes Process heap size used from node.js in bytes.
+# TYPE nodejs_heap_size_used_bytes gauge
+nodejs_heap_size_used_bytes 12068008 1579444523567
+
+# HELP nodejs_external_memory_bytes Nodejs external memory size in bytes.
+# TYPE nodejs_external_memory_bytes gauge
+nodejs_external_memory_bytes 1728962 1579444523567
+
+# HELP nodejs_heap_space_size_total_bytes Process heap space size total from node.js in bytes.
+# TYPE nodejs_heap_space_size_total_bytes gauge
+nodejs_heap_space_size_total_bytes{space="read_only"} 262144 1579444523567
+nodejs_heap_space_size_total_bytes{space="new"} 1048576 1579444523567
+nodejs_heap_space_size_total_bytes{space="old"} 9809920 1579444523567
+nodejs_heap_space_size_total_bytes{space="code"} 425984 1579444523567
+nodejs_heap_space_size_total_bytes{space="map"} 1052672 1579444523567
+nodejs_heap_space_size_total_bytes{space="large_object"} 1077248 1579444523567
+nodejs_heap_space_size_total_bytes{space="code_large_object"} 49152 1579444523567
+nodejs_heap_space_size_total_bytes{space="new_large_object"} 0 1579444523567
+
+# HELP nodejs_heap_space_size_used_bytes Process heap space size used from node.js in bytes.
+# TYPE nodejs_heap_space_size_used_bytes gauge
+nodejs_heap_space_size_used_bytes{space="read_only"} 32296 1579444523567
+nodejs_heap_space_size_used_bytes{space="new"} 601696 1579444523567
+nodejs_heap_space_size_used_bytes{space="old"} 9376600 1579444523567
+nodejs_heap_space_size_used_bytes{space="code"} 286688 1579444523567
+nodejs_heap_space_size_used_bytes{space="map"} 704320 1579444523567
+nodejs_heap_space_size_used_bytes{space="large_object"} 1064872 1579444523567
+nodejs_heap_space_size_used_bytes{space="code_large_object"} 3552 1579444523567
+nodejs_heap_space_size_used_bytes{space="new_large_object"} 0 1579444523567
+
+# HELP nodejs_heap_space_size_available_bytes Process heap space size available from node.js in bytes.
+# TYPE nodejs_heap_space_size_available_bytes gauge
+nodejs_heap_space_size_available_bytes{space="read_only"} 229576 1579444523567
+nodejs_heap_space_size_available_bytes{space="new"} 445792 1579444523567
+nodejs_heap_space_size_available_bytes{space="old"} 417712 1579444523567
+nodejs_heap_space_size_available_bytes{space="code"} 20576 1579444523567
+nodejs_heap_space_size_available_bytes{space="map"} 343632 1579444523567
+nodejs_heap_space_size_available_bytes{space="large_object"} 0 1579444523567
+nodejs_heap_space_size_available_bytes{space="code_large_object"} 0 1579444523567
+nodejs_heap_space_size_available_bytes{space="new_large_object"} 1047488 1579444523567
+
+# HELP nodejs_version_info Node.js version info.
+# TYPE nodejs_version_info gauge
+nodejs_version_info{version="v14.16.1",major="14",minor="16",patch="1"} 1
+
+# HELP grafana_image_renderer_service_http_request_duration_seconds duration histogram of http responses labeled with: status_code
+# TYPE grafana_image_renderer_service_http_request_duration_seconds histogram
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="1",status_code="200"} 0
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="5",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="7",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="9",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="11",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="13",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="15",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="20",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="30",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_bucket{le="+Inf",status_code="200"} 4
+grafana_image_renderer_service_http_request_duration_seconds_sum{status_code="200"} 10.492873834
+grafana_image_renderer_service_http_request_duration_seconds_count{status_code="200"} 4
+
+# HELP up 1 = up, 0 = not up
+# TYPE up gauge
+up 1
+
+# HELP grafana_image_renderer_http_request_in_flight A gauge of requests currently being served by the image renderer.
+# TYPE grafana_image_renderer_http_request_in_flight gauge
+grafana_image_renderer_http_request_in_flight 1
+
+# HELP grafana_image_renderer_step_duration_seconds duration histogram of browser steps for rendering an image labeled with: step
+# TYPE grafana_image_renderer_step_duration_seconds histogram
+grafana_image_renderer_step_duration_seconds_bucket{le="0.3",step="launch"} 0
+grafana_image_renderer_step_duration_seconds_bucket{le="0.5",step="launch"} 0
+grafana_image_renderer_step_duration_seconds_bucket{le="1",step="launch"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="2",step="launch"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="3",step="launch"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="5",step="launch"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="+Inf",step="launch"} 1
+grafana_image_renderer_step_duration_seconds_sum{step="launch"} 0.7914972
+grafana_image_renderer_step_duration_seconds_count{step="launch"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.3",step="newPage"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.5",step="newPage"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="1",step="newPage"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="2",step="newPage"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="3",step="newPage"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="5",step="newPage"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="+Inf",step="newPage"} 1
+grafana_image_renderer_step_duration_seconds_sum{step="newPage"} 0.2217868
+grafana_image_renderer_step_duration_seconds_count{step="newPage"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.3",step="prepare"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.5",step="prepare"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="1",step="prepare"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="2",step="prepare"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="3",step="prepare"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="5",step="prepare"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="+Inf",step="prepare"} 1
+grafana_image_renderer_step_duration_seconds_sum{step="prepare"} 0.0819274
+grafana_image_renderer_step_duration_seconds_count{step="prepare"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.3",step="navigate"} 0
+grafana_image_renderer_step_duration_seconds_bucket{le="0.5",step="navigate"} 0
+grafana_image_renderer_step_duration_seconds_bucket{le="1",step="navigate"} 0
+grafana_image_renderer_step_duration_seconds_bucket{le="2",step="navigate"} 0
+grafana_image_renderer_step_duration_seconds_bucket{le="3",step="navigate"} 0
+grafana_image_renderer_step_duration_seconds_bucket{le="5",step="navigate"} 0
+grafana_image_renderer_step_duration_seconds_bucket{le="+Inf",step="navigate"} 1
+grafana_image_renderer_step_duration_seconds_sum{step="navigate"} 15.3311258
+grafana_image_renderer_step_duration_seconds_count{step="navigate"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.3",step="panelsRendered"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.5",step="panelsRendered"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="1",step="panelsRendered"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="2",step="panelsRendered"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="3",step="panelsRendered"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="5",step="panelsRendered"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="+Inf",step="panelsRendered"} 1
+grafana_image_renderer_step_duration_seconds_sum{step="panelsRendered"} 0.0205577
+grafana_image_renderer_step_duration_seconds_count{step="panelsRendered"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.3",step="screenshot"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="0.5",step="screenshot"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="1",step="screenshot"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="2",step="screenshot"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="3",step="screenshot"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="5",step="screenshot"} 1
+grafana_image_renderer_step_duration_seconds_bucket{le="+Inf",step="screenshot"} 1
+grafana_image_renderer_step_duration_seconds_sum{step="screenshot"} 0.2866623
+grafana_image_renderer_step_duration_seconds_count{step="screenshot"} 1
+
+# HELP grafana_image_renderer_browser_info A metric with a constant '1 value labeled by version of the browser in use
+# TYPE grafana_image_renderer_browser_info gauge
+grafana_image_renderer_browser_info{version="HeadlessChrome/79.0.3945.0"} 1
+```
--- a/docs/sources/setup-grafana/image-rendering/troubleshooting.md
+++ b/docs/sources/setup-grafana/image-rendering/troubleshooting.md
@@ -0,0 +1,155 @@
+---
+aliases:
+  - /docs/grafana/latest/image-rendering/troubleshooting/
+  - /docs/grafana/latest/setup-grafana/image-rendering/troubleshooting/
+description: Image rendering troubleshooting
+keywords:
+  - grafana
+  - image
+  - rendering
+  - plugin
+  - troubleshooting
+title: Troubleshooting
+weight: 200
+---
+
+# Troubleshoot image rendering
+
+In this section, you'll learn how to enable logging for the image renderer and you'll find the most common issues.
+
+## Enable debug logging
+
+To troubleshoot the image renderer, different kind of logs are available.
+
+You can enable debug log messages for rendering in the Grafana configuration file and inspect the Grafana server logs.
+
+```bash
+[log]
+filters = rendering:debug
+```
+
+You can also enable more logs in image renderer service itself by:
+
+- Increasing the [log level]({{< relref "/#log-level" >}}).
+- Enabling [verbose logging]({{< relref "/#verbose-logging" >}}).
+- [Capturing headless browser output]({{< relref "/#capture-browser-output" >}}).
+
+## Missing libraries
+
+The plugin and rendering service uses [Chromium browser](https://www.chromium.org/) which depends on certain libraries.
+If you don't have all of those libraries installed in your system you may encounter errors when trying to render an image, e.g.
+
+```bash
+Rendering failed: Error: Failed to launch chrome!/var/lib/grafana/plugins/grafana-image-renderer/chrome-linux/chrome:
+error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory\n\n\nTROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md
+```
+
+In general you can use the [`ldd`](<https://en.wikipedia.org/wiki/Ldd_(Unix)>) utility to figure out what shared libraries
+are not installed in your system:
+
+```bash
+cd <grafana-image-render plugin directory>
+ldd chrome-linux/chrome
+      linux-vdso.so.1 (0x00007fff1bf65000)
+      libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2047945000)
+      libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2047924000)
+      librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f204791a000)
+      libX11.so.6 => not found
+      libX11-xcb.so.1 => not found
+      libxcb.so.1 => not found
+      libXcomposite.so.1 => not found
+        ...
+```
+
+**Ubuntu:**
+
+On Ubuntu 18.10 the following dependencies are required for the image rendering to function.
+
+```bash
+libx11-6 libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrender1 libxtst6 libglib2.0-0 libnss3 libcups2  libdbus-1-3 libxss1 libxrandr2 libgtk-3-0 libasound2 libxcb-dri3-0 libgbm1 libxshmfence1
+```
+
+**Debian:**
+
+On Debian 9 (Stretch) the following dependencies are required for the image rendering to function.
+
+```bash
+libx11 libcairo libcairo2 libxtst6 libxcomposite1 libx11-xcb1 libxcursor1 libxdamage1 libnss3 libcups libcups2 libxss libxss1 libxrandr2 libasound2 libatk1.0-0 libatk-bridge2.0-0 libpangocairo-1.0-0 libgtk-3-0 libgbm1 libxshmfence1
+```
+
+On Debian 10 (Buster) the following dependencies are required for the image rendering to function.
+
+```bash
+libxdamage1 libxext6 libxi6 libxtst6 libnss3 libcups2 libxss1 libxrandr2 libasound2 libatk1.0-0 libatk-bridge2.0-0 libpangocairo-1.0-0 libpango-1.0-0 libcairo2 libatspi2.0-0 libgtk3.0-cil libgdk3.0-cil libx11-xcb-dev libgbm1 libxshmfence1
+```
+
+**Centos:**
+
+On a minimal CentOS 7 installation, the following dependencies are required for the image rendering to function:
+
+```bash
+libXcomposite libXdamage libXtst cups libXScrnSaver pango atk adwaita-cursor-theme adwaita-icon-theme at at-spi2-atk at-spi2-core cairo-gobject colord-libs dconf desktop-file-utils ed emacs-filesystem gdk-pixbuf2 glib-networking gnutls gsettings-desktop-schemas gtk-update-icon-cache gtk3 hicolor-icon-theme jasper-libs json-glib libappindicator-gtk3 libdbusmenu libdbusmenu-gtk3 libepoxy liberation-fonts liberation-narrow-fonts liberation-sans-fonts liberation-serif-fonts libgusb libindicator-gtk3 libmodman libproxy libsoup libwayland-cursor libwayland-egl libxkbcommon m4 mailx nettle patch psmisc redhat-lsb-core redhat-lsb-submod-security rest spax time trousers xdg-utils xkeyboard-config alsa-lib
+```
+
+On a minimal CentOS 8 installation, the following dependencies are required for the image rendering to function:
+
+```bash
+libXcomposite libXdamage libXtst cups libXScrnSaver pango atk adwaita-cursor-theme adwaita-icon-theme at at-spi2-atk at-spi2-core cairo-gobject colord-libs dconf desktop-file-utils ed emacs-filesystem gdk-pixbuf2 glib-networking gnutls gsettings-desktop-schemas gtk-update-icon-cache gtk3 hicolor-icon-theme jasper-libs json-glib libappindicator-gtk3 libdbusmenu libdbusmenu-gtk3 libepoxy liberation-fonts liberation-narrow-fonts liberation-sans-fonts liberation-serif-fonts libgusb libindicator-gtk3 libmodman libproxy libsoup libwayland-cursor libwayland-egl libxkbcommon m4 mailx nettle patch psmisc redhat-lsb-core redhat-lsb-submod-security rest spax time trousers xdg-utils xkeyboard-config alsa-lib libX11-xcb
+```
+
+## Certificate signed by internal certificate authorities
+
+In many cases, Grafana runs on internal servers and uses certificates that have not been signed by a CA ([Certificate Authority](https://en.wikipedia.org/wiki/Certificate_authority)) known to Chrome, and therefore cannot be validated. Chrome internally uses NSS ([Network Security Services](https://en.wikipedia.org/wiki/Network_Security_Services)) for cryptographic operations such as the validation of certificates.
+
+If you are using the Grafana Image Renderer with a Grafana server that uses a certificate signed by such a custom CA (for example a company-internal CA), rendering images will fail and you will see messages like this in the Grafana log:
+
+```
+t=2019-12-04T12:39:22+0000 lvl=error msg="Render request failed" logger=rendering error=map[] url="https://192.168.106.101:3443/d-solo/zxDJxNaZk/graphite-metrics?orgId=1&refresh=1m&from=1575438321300&to=1575459921300&var-Host=master1&panelId=4&width=1000&height=500&tz=Europe%2FBerlin&render=1" timestamp=0001-01-01T00:00:00.000Z
+t=2019-12-04T12:39:22+0000 lvl=error msg="Rendering failed." logger=context userId=1 orgId=1 uname=admin error="Rendering failed: Error: net::ERR_CERT_AUTHORITY_INVALID at https://192.168.106.101:3443/d-solo/zxDJxNaZk/graphite-metrics?orgId=1&refresh=1m&from=1575438321300&to=1575459921300&var-Host=master1&panelId=4&width=1000&height=500&tz=Europe%2FBerlin&render=1"
+t=2019-12-04T12:39:22+0000 lvl=error msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=GET path=/render/d-solo/zxDJxNaZk/graphite-metrics status=500 remote_addr=192.168.106.101 time_ms=310 size=1722 referer="https://grafana.xxx-xxx/d/zxDJxNaZk/graphite-metrics?orgId=1&refresh=1m"
+```
+
+If this happens, then you have to add the certificate to the trust store. If you have the certificate file for the internal root CA in the file `internal-root-ca.crt.pem`, then use these commands to create a user specific NSS trust store for the Grafana user (`grafana` for the purpose of this example) and execute the following steps:
+
+**Linux:**
+
+```
+[root@server ~]# [ -d /usr/share/grafana/.pki/nssdb ] || mkdir -p /usr/share/grafana/.pki/nssdb
+[root@merver ~]# certutil -d sql:/usr/share/grafana/.pki/nssdb -A -n internal-root-ca -t C -i /etc/pki/tls/certs/internal-root-ca.crt.pem
+[root@server ~]# chown -R grafana: /usr/share/grafana/.pki/nssdb
+```
+
+**Windows:**
+
+```
+certutil –addstore "Root" <path>/internal-root-ca.crt.pem
+```
+
+## Custom Chrome/Chromium
+
+As a last resort, if you already have [Chrome](https://www.google.com/chrome/) or [Chromium](https://www.chromium.org/)
+installed on your system, then you can configure the [Grafana Image renderer plugin](../#custom-chromechromium) to use this
+instead of the pre-packaged version of Chromium.
+
+> **Note:** Please note that this is not recommended, since you may encounter problems if the installed version of Chrome/Chromium is not
+> compatible with the [Grafana Image renderer plugin](https://grafana.com/grafana/plugins/grafana-image-renderer).
+
+To override the path to the Chrome/Chromium executable in plugin mode, set an environment variable and make sure that it's available for the Grafana process. For example:
+
+```bash
+export GF_PLUGIN_RENDERING_CHROME_BIN="/usr/bin/chromium-browser"
+```
+
+In remote rendering mode, you need to set the environment variable or update the configuration file and make sure that it's available for the image rendering service process:
+
+```bash
+CHROME_BIN="/usr/bin/chromium-browser"
+```
+
+```json
+{
+  "rendering": {
+    "chromeBin": "/usr/bin/chromium-browser"
+  }
+}
+```