mirror of
https://github.com/grafana/grafana.git
synced 2024-11-26 10:50:37 -06:00
[DOC] Add profile-traces intro material; update Pyroscope data source info (#83739)
* Add profile-traces intro material; update Pyroscope data source info * Apply suggestions from code review Co-authored-by: Jack Baldry <jack.baldry@grafana.com> * Updates and file rename from review * Add PYROSCOPE_VERSION * Apply suggestions from code review * Format tables Signed-off-by: Jack Baldry <jack.baldry@grafana.com> * Apply suggestions from code review Co-authored-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Jennifer Villa <jvilla2013@gmail.com> * Apply suggestions from code review --------- Signed-off-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Jack Baldry <jack.baldry@grafana.com> Co-authored-by: Jennifer Villa <jvilla2013@gmail.com>
This commit is contained in:
Kim Nylander
GitHub
parent
89575f1df4
commit
57935250fd
@ -15,6 +15,7 @@ labels:
|
||||
- oss
|
||||
cascade:
|
||||
TEMPO_VERSION: latest
|
||||
PYROSCOPE_VERSION: latest
|
||||
title: Grafana open source documentation
|
||||
---
|
||||
|
||||
|
||||
@ -24,9 +24,13 @@ weight: 1150
|
||||
|
||||
Grafana Pyroscope is a horizontally scalable, highly available, multi-tenant, OSS, continuous profiling aggregation system. Add it as a data source, and you are ready to query your profiles in [Explore][explore].
|
||||
|
||||
To learn more about profiling and Pyroscope, refer to the [Introduction to Pyroscope](/docs/pyroscope/introduction/).
|
||||
Refer to [Introduction to Pyroscope](https://grafana.com/docs/pyroscope/<PYROSCOPE_VERSION>/introduction/) to understand profiling and Pyroscope.
|
||||
|
||||
For information on configuring the Pyroscope data source, refer to [Configure the Grafana Pyroscope data source](./configure-pyroscope-data-source).
|
||||
To use profiling data, you should:
|
||||
|
||||
- [Configure your application to send profiles](/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/)
|
||||
- [Configure the Grafana Pyroscope data source](./configure-pyroscope-data-source/).
|
||||
- [View and query profiling data in Explore](./query-profile-data/)
|
||||
|
||||
## Integrate profiles into dashboards
|
||||
|
||||
@ -38,12 +42,13 @@ In this case, the screenshot shows memory profiles alongside panels for logs and
|
||||
## Visualize traces and profiles data using Traces to profiles
|
||||
|
||||
You can link profile and tracing data using your Pyroscope data source with the Tempo data source.
|
||||
To learn more about how profiles and tracing can work together, refer to [Profiling and tracing synergies](./profiling-and-tracing/).
|
||||
|
||||
Combined traces and profiles let you see granular line-level detail when available for a trace span. This allows you pinpoint the exact function that's causing a bottleneck in your application as well as a specific request.
|
||||
|
||||
|
||||
|
||||
For more information, refer to the [Traces to profile section][configure-tempo-data-source] of the Tempo data source documentation.
|
||||
For more information, refer to the [Traces to profile section][configure-tempo-data-source] and [Link tracing and profiling with span profiles](https://grafana.com/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/trace-span-profiles/).
|
||||
|
||||
{{< youtube id="AG8VzfFMLxo" >}}
|
||||
|
||||
|
||||
16
docs/sources/datasources/pyroscope/profiling-and-tracing.md
Normal file
16
docs/sources/datasources/pyroscope/profiling-and-tracing.md
Normal file
@ -0,0 +1,16 @@
|
||||
---
|
||||
title: How profiling and tracing work together
|
||||
menuTitle: How profiling and tracing work together
|
||||
description: Learn about how profiling and tracing work together.
|
||||
weight: 250
|
||||
keywords:
|
||||
- pyroscope data source
|
||||
- continuous profiling
|
||||
- tracing
|
||||
---
|
||||
|
||||
# How profiling and tracing work together
|
||||
|
||||
[//]: # 'Shared content for Trace to profiles in the Pyroscope data source'
|
||||
|
||||
{{< docs/shared source="grafana" lookup="datasources/pyroscope-profile-tracing-intro.md" version="<GRAFANA_VERSION>" >}}
|
||||
@ -0,0 +1,123 @@
|
||||
---
|
||||
headless: true
|
||||
labels:
|
||||
products:
|
||||
- enterprise
|
||||
- oss
|
||||
---
|
||||
|
||||
[//]: # 'This file documents the introductory material for traces to profiling for the Pyroscope data source.'
|
||||
[//]: # 'This shared file is included in these locations:'
|
||||
[//]: # '/grafana/docs/sources/datasources/pyroscope/profiling-and-tracing.md'
|
||||
[//]: # '/website/docs/grafana-cloud/data-configuration/traces/traces-query-editor.md'
|
||||
[//]: # '/docs/sources/view-and-analyze-profile-data/profile-tracing/_index.md'
|
||||
[//]: #
|
||||
[//]: # 'If you make changes to this file, verify that the meaning and content are not changed in any place where the file is included.'
|
||||
[//]: # 'Any links should be fully qualified and not relative: /docs/grafana/ instead of ../grafana/.'
|
||||
|
||||
<!-- Profiling and tracing integration -->
|
||||
|
||||
Profiles, continuous profiling, and distributed traces are all tools that can be used to improve the performance and reliability of applications.
|
||||
However, each tool has its own strengths and weaknesses, and it is important to choose the right tool for the job as well as understand when to use both.
|
||||
|
||||
## Profiling
|
||||
|
||||
Profiling offers a deep-dive into an application's performance at the code level, highlighting resource usage and performance hotspots.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th scope="row">Usage</th>
|
||||
<td>During development, major releases, or upon noticing performance quirks.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Benefits</th>
|
||||
<td>
|
||||
<ul>
|
||||
<li>Business: Boosts user experience through enhanced application performance.</li>
|
||||
<li>Technical: Gives clear insights into code performance and areas of refinement.</li>
|
||||
</ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Example</th>
|
||||
<td>A developer uses profiling upon noting slow app performance, identifies a CPU-heavy function, and optimizes it.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Continuous profiling
|
||||
|
||||
Continuous profiling provides ongoing performance insights, capturing long-term trends and intermittent issues.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th scope="row">Usage</th>
|
||||
<td>Mainly in production, especially for high-priority applications.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Benefits</th>
|
||||
<td>
|
||||
<ul>
|
||||
<li>Business: Preemptively addresses inefficiencies, potentially saving costs.</li>
|
||||
<li>Technical: Highlights performance trends and issues like potential memory leaks over time.</li>
|
||||
</ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Example</th>
|
||||
<td>A month-long data from continuous profiling suggests increasing memory consumption, hinting at a memory leak.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Distributed tracing
|
||||
|
||||
Traces requests as they cross multiple services, revealing interactions and service dependencies.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th scope="row">Usage</th>
|
||||
<td>Essential for systems like microservices where requests touch multiple services.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Benefits</th>
|
||||
<td>
|
||||
<ul>
|
||||
<li>Business: Faster issue resolution, reduced downtimes, and strengthened customer trust.</li>
|
||||
<li>Technical: A broad view of the system's structure, revealing bottlenecks and inter-service dependencies.</li>
|
||||
</ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Example</th>
|
||||
<td>In e-commerce, a user's checkout request might involve various services. Tracing depicts this route, pinpointing where time is most spent.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Combined power of tracing and profiling
|
||||
|
||||
When used together, tracing and profiling provide a powerful tool for understanding system and application performance.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th scope="row">Usage</th>
|
||||
<td>For comprehensive system-to-code insights, especially when diagnosing complex issues spread across services and codebases.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Benefits</th>
|
||||
<td>
|
||||
<ul>
|
||||
<li>Business: Reduces downtime, optimizes user experience, and safeguards revenues.</li>
|
||||
<li>Technical:
|
||||
<ul>
|
||||
<li>Holistic view: Tracing pinpoints bottle-necked services, while profiling delves into the responsible code segments.</li>
|
||||
<li>End-to-end insight: Visualizes a request's full journey and the performance of individual code parts.</li>
|
||||
<li>Efficient diagnosis: Tracing identifies service latency; profiling zeroes in on its cause, be it database queries, API calls, or specific code inefficiencies.</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th scope="row">Example</th>
|
||||
<td>Tracing reveals latency in a payment service. Combined with profiling, it's found that a particular function, making third-party validation calls, is the culprit. This insight guides optimization, refining system efficiency.</td>
|
||||
</tr>
|
||||
</table>
|
||||
@ -95,7 +95,7 @@ Spans with the same color belong to the same service. The grey text to the right
|
||||
The Tempo data source supports streaming responses to TraceQL queries so you can see partial query results as they come in without waiting for the whole query to finish.
|
||||
|
||||
{{% admonition type="note" %}}
|
||||
To use this experimental feature, enable the `traceQLStreaming` feature toggle. If you’re using Grafana Cloud and would like to enable this feature, please contact customer support.
|
||||
To use this feature in Grafana OSS v10.1 and later, enable the `traceQLStreaming` feature toggle. This capability is enabled by default in Grafana Cloud.
|
||||
{{% /admonition %}}
|
||||
|
||||
Streaming is available for both the **Search** and **TraceQL** query types, and you'll get immediate visibility of incoming traces on the results table.
|
||||
|
||||
@ -29,14 +29,16 @@ There are two ways to configure the trace to profiles feature:
|
||||
- Configure a custom query where you can use a template language to interpolate variables from the trace or span.
|
||||
|
||||
{{< admonition type="note">}}
|
||||
Traces to profile requires a Tempo data source with Traces to profiles configured and a Pyroscope data source. This integration supports profile data generated using Go, Ruby, and Java instrumentation SDKs.
|
||||
Traces to profile requires a Tempo data source with Traces to profiles configured and a Pyroscope data source. This integration supports profile data generated using [Go](/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/trace-span-profiles/go-span-profiles/), [Ruby](/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/trace-span-profiles/ruby-span-profiles/), and [Java](/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/trace-span-profiles/java-span-profiles/) instrumentation SDKs.
|
||||
|
||||
As with traces, your application needs to be instrumented to emit profiling data. For more information, refer to [Linking tracing and profiling with span profiles](/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/trace-span-profiles/).
|
||||
{{< /admonition >}}
|
||||
|
||||
To use trace to profiles, navigate to **Explore** and query a trace. Each span now links to your queries. Clicking a link runs the query in a split panel. If tags are configured, Grafana dynamically inserts the span attribute values into the query. The query runs over the time range of the (span start time - 60) to (span end time + 60 seconds).
|
||||
|
||||
|
||||
|
||||
To use trace to profiles, you must have a configured Grafana Pyroscope data source. For more information, refer to the [Grafana Pyroscope data source](/docs/grafana/latest/datasources/grafana-pyroscope/) documentation.
|
||||
To use trace to profiles, you must have a configured Grafana Pyroscope data source. For more information, refer to the [Grafana Pyroscope data source](/docs/grafana/<GRAFANA_VERSION>/datasources/grafana-pyroscope/) documentation.
|
||||
|
||||
**Embedded flame graphs** are also inserted into each span details section that has a linked profile (requires a configured Grafana Pyroscope data source).
|
||||
This lets you see resource consumption in a flame graph visualization for each span without having to navigate away from the current view.
|
||||
@ -88,7 +90,7 @@ To use a custom query with the configuration, follow these steps:
|
||||
1. Select a Pyroscope data source in the **Data source** drop-down.
|
||||
1. Optional: Choose any tags to use in the query. If left blank, the default values of `service.name` and `service.namespace` are used.
|
||||
|
||||
These tags can be used in the custom query with `${__tags}` variable. This variable interpolates the mapped tags as list in an appropriate syntax for the data source. Only the tags that were present in the span are included; tags that aren't present are omitted. You can also configure a new name for the tag. This is useful in cases where the tag has dots in the name and the target data source doesn't allow using dots in labels. For example, you can remap `service.name` to `service_name`. If you don’t map any tags here, you can still use any tag in the query, for example: `method="${__span.tags.method}"`. You can learn more about custom query variables [here](/docs/grafana/latest/datasources/tempo/configure-tempo-data-source/#custom-query-variables).
|
||||
These tags can be used in the custom query with `${__tags}` variable. This variable interpolates the mapped tags as list in an appropriate syntax for the data source. Only the tags that were present in the span are included; tags that aren't present are omitted. You can also configure a new name for the tag. This is useful in cases where the tag has dots in the name and the target data source doesn't allow using dots in labels. For example, you can remap `service.name` to `service_name`. If you don’t map any tags here, you can still use any tag in the query, for example: `method="${__span.tags.method}"`. You can learn more about custom query variables [here](/docs/grafana/<GRAFANA_VERSION>/datasources/tempo/configure-tempo-data-source/#custom-query-variables).
|
||||
|
||||
1. Select one or more profile types to use in the query. Select the drop-down and choose options from the menu.
|
||||
1. Switch on **Use custom query** to enter a custom query.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user