Live debugging #999

wildum · 2024-06-07T08:06:58Z

PR Description

This PR brings the live debugging feature to main.

Which issue(s) this PR fixes

Fixes #986

Notes to the Reviewer

This PR is made from three parts that have been reviewed already:

back-end: Live debugging service with prometheus.relabel #797
front-end: Live debugging front #977
doc: Live debugging doc #987

I added these two screenshots to the media folder:

PR Checklist

* create xray service to manage the debug streams * add xray support to the prometheus.relabel component * add xray endpoint to the api * rename handler to manager * defer cleanup func * rework to add multi-streams support * more tests * additional multi delete test * add todo comment for buffer size * add error checks * improve naming and readability, split interfaces and add a check to avoid expensive string computation * move register and isregistered to a new debugRegistry interface

* add live debugging page in UI * change button size * move icon in front of text in buttons * add warning if the number of incoming lines is greater than the buffer size

* add live debugging doc * Update docs/sources/tasks/debug.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/tasks/debug.md Co-authored-by: Clayton Cornell <[email protected]> * Update docs/sources/tasks/debug.md Co-authored-by: Clayton Cornell <[email protected]> * improve doc * Update docs/sources/tasks/debug.md Co-authored-by: Clayton Cornell <[email protected]> --------- Co-authored-by: Clayton Cornell <[email protected]>

ptodev

Looks great! I can't wait to try it!

ptodev · 2024-06-07T10:47:09Z

docs/sources/tasks/debug.md

@@ -48,6 +48,7 @@ Clicking a component in the graph navigates to the [Component detail page](#comp

 ### Component detail page

+<!-- TODO: update this screenshot once the branch is ready to go to main because it contains an additional button now -->


It'd be nice to just upload a second screenshot under a different name. That way if we rollback this change, the old markdown will fallback to using the old screenshot.

docs/sources/tasks/debug.md

internal/service/livedebugging/livedebugging.go

rfratto

Nice work! I have a few small comments and suggestions for future improvements. I'm really looking forward to this expanding into more components :)

docs/sources/tasks/debug.md

internal/service/livedebugging/livedebugging.go

internal/component/prometheus/relabel/relabel.go

internal/service/livedebugging/service.go

internal/web/api/api.go

CHANGELOG.md

thampiotr · 2024-06-07T14:46:33Z

Good stuff! I have a few comments from testing it out, not necessarily blockers, but I'd like to raise them here.

I may remembering incorrectly, but I thought this was meant to be explicitly enabled in a config file somehow? The reason is that an attacker with HTTP access to Alloy will be able to snoop the data which was previously not exposed or encrypted on the way out. We should give users an option to have this disabled IMO.
There is a warning in dev console on Chrome, maybe unrelated to your changes, but sharing just in case:
I find that the button to enable Auto Scroll is tiny and hidden at the bottom, why not make it a normal button like others, next to Clear, Copy, etc?
When I filter the results and click Copy, I get all the data instead of just the filtered data. Is this intentional? I'd expect to only get what I have in the buffer.
Nit: I am not sure about using => as a way to designate the mutation in prometheus.relabel. It may work for this component, but I think that other components may have more kinds of events (e.g. 'connected' or 'reloading targets' etc) and may end up with more logs-like data stream.... which now made me wonder if we could have implemented all of this using loggers, where you can peek into debug-level logs of any component on-demand. Would work with services too... kind a liking this idea now 😂

Overall, this is great and looking forward to having this on more components.

wildum · 2024-06-07T15:03:38Z

I find that the button to enable Auto Scroll is tiny and hidden at the bottom, why not make it a normal button like others, next to Clear, Copy, etc?

It's part of an imported react autoscroll component, I could not find a good way to bring it up and style it like the other buttons :(

wildum · 2024-06-07T15:12:00Z

Nit: I am not sure about using => as a way to designate the mutation in prometheus.relabel. It may work for this component, but I think that other components may have more kinds of events (e.g. 'connected' or 'reloading targets' etc) and may end up with more logs-like data stream.... which now made me wonder if we could have implemented all of this using loggers, where you can peek into debug-level logs of any component on-demand. Would work with services too... kind a liking this idea now

I think that these are two different things. The original idea of the x-ray feature is to see the data/the flow through the pipelines. Events data that you see in logs is not part of the plan. It does not replace logs but gives another tool that can be used alongside the others to troubleshoot Alloy.

internal/web/ui/src/pages/LiveDebugging.module.css

wildum · 2024-06-07T15:47:35Z

I may remembering incorrectly, but I thought this was meant to be explicitly enabled in a config file somehow? The reason is that an attacker with HTTP access to Alloy will be able to snoop the data which was previously not exposed or encrypted on the way out. We should give users an option to have this disabled IMO.

Would it be ok if I create an issue for this and it comes in a follow up PR? I would create a block for the service and would add also other options to customize it (e.g. size of the buffer). I agree that it is important but not a blocker because having access to the Alloy's API but not its config file is not a very popular scenario I guess?

wildum · 2024-06-10T11:18:39Z

When I filter the results and click Copy, I get all the data instead of just the filtered data. Is this intentional? I'd expect to only get what I have in the buffer.

Fixed in 17aaf44

rfratto

🎉 LGTM!

… The stability is also updated to experimental

docs/sources/reference/config-blocks/livedebugging.md

Co-authored-by: Clayton Cornell <[email protected]>

rfratto

LGTM % one comment (screenshots look good too). Thanks!

docs/sources/reference/config-blocks/livedebugging.md

clayton-cornell

Some suggestions to remove the possessive "Alloy's"

docs/sources/reference/config-blocks/livedebugging.md

Co-authored-by: Clayton Cornell <[email protected]>

wildum · 2024-06-11T16:56:40Z

I find that the button to enable Auto Scroll is tiny and hidden at the bottom, why not make it a normal button like others, next to Clear, Copy, etc?

FYI I created an issue for this: #1021

wildum and others added 4 commits June 7, 2024 09:56

Live debugging front (#977)

dcddffa

* add live debugging page in UI * change button size * move icon in front of text in buttons * add warning if the number of incoming lines is greater than the buffer size

changelog

4e7320e

wildum requested a review from clayton-cornell as a code owner June 7, 2024 08:06

wildum requested a review from rfratto June 7, 2024 08:07

ptodev reviewed Jun 7, 2024

View reviewed changes

rfratto reviewed Jun 7, 2024

View reviewed changes

internal/web/ui/src/pages/LiveDebugging.module.css Outdated Show resolved Hide resolved

wildum added 6 commits June 10, 2024 09:59

add list of supported components in debug.md

6e4f5f8

switch to pull based registration

1b7c260

switch stability to GA

0ed6956

update changelog

f1d355e

use monospace font for log lines

5f5178c

copy to clipboard now copies the filtered data

17aaf44

wildum requested a review from rfratto June 10, 2024 12:03

rfratto approved these changes Jun 10, 2024

View reviewed changes

wildum force-pushed the xray branch from addabb1 to 6cbe077 Compare June 10, 2024 16:16

disable livedebugging by default and add a config block to enable it.…

a809977

… The stability is also updated to experimental

wildum force-pushed the xray branch from 6cbe077 to a809977 Compare June 10, 2024 16:34

clayton-cornell reviewed Jun 10, 2024

View reviewed changes

docs/sources/reference/config-blocks/livedebugging.md Outdated Show resolved Hide resolved

docs/sources/reference/config-blocks/livedebugging.md Outdated Show resolved Hide resolved

clayton-cornell added the type/docs Docs Squad label across all Grafana Labs repos label Jun 10, 2024

wildum and others added 3 commits June 11, 2024 09:36

Update docs/sources/reference/config-blocks/livedebugging.md

d228914

Co-authored-by: Clayton Cornell <[email protected]>

Update docs/sources/reference/config-blocks/livedebugging.md

960e31d

Co-authored-by: Clayton Cornell <[email protected]>

replace debugging data by live debugging data in the doc

f114a78

wildum requested a review from rfratto June 11, 2024 13:07

update screenshots

3ffa72d

rfratto approved these changes Jun 11, 2024

View reviewed changes

docs/sources/reference/config-blocks/livedebugging.md Outdated Show resolved Hide resolved

clayton-cornell reviewed Jun 11, 2024

View reviewed changes

docs/sources/reference/config-blocks/livedebugging.md Outdated Show resolved Hide resolved

docs/sources/reference/config-blocks/livedebugging.md Outdated Show resolved Hide resolved

docs/sources/reference/config-blocks/livedebugging.md Outdated Show resolved Hide resolved

wildum and others added 2 commits June 11, 2024 18:48

Update docs/sources/reference/config-blocks/livedebugging.md

718f319

Co-authored-by: Clayton Cornell <[email protected]>

Update docs/sources/reference/config-blocks/livedebugging.md

7434a5b

Co-authored-by: Clayton Cornell <[email protected]>

clayton-cornell approved these changes Jun 11, 2024

View reviewed changes

small doc fix

41f50a8

wildum merged commit 7aca045 into main Jun 11, 2024
18 checks passed

wildum deleted the xray branch June 11, 2024 17:06

ptodev mentioned this pull request Jun 14, 2024

feat. X-Ray - Real time stream debugging for Flow components grafana/agent#6045

Closed

3 tasks

github-actions bot added the frozen-due-to-age label Jul 12, 2024

github-actions bot locked as resolved and limited conversation to collaborators Jul 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Live debugging #999

Live debugging #999

wildum commented Jun 7, 2024 •

edited

Loading

ptodev left a comment

ptodev Jun 7, 2024

rfratto left a comment

thampiotr commented Jun 7, 2024

wildum commented Jun 7, 2024

wildum commented Jun 7, 2024

wildum commented Jun 7, 2024

wildum commented Jun 10, 2024

rfratto left a comment

rfratto left a comment

clayton-cornell left a comment

wildum commented Jun 11, 2024

		@@ -48,6 +48,7 @@ Clicking a component in the graph navigates to the [Component detail page](#comp

		### Component detail page

		<!-- TODO: update this screenshot once the branch is ready to go to main because it contains an additional button now -->

Live debugging #999

Live debugging #999

Conversation

wildum commented Jun 7, 2024 • edited Loading

PR Description

Which issue(s) this PR fixes

Notes to the Reviewer

PR Checklist

ptodev left a comment

Choose a reason for hiding this comment

ptodev Jun 7, 2024

Choose a reason for hiding this comment

rfratto left a comment

Choose a reason for hiding this comment

thampiotr commented Jun 7, 2024

wildum commented Jun 7, 2024

wildum commented Jun 7, 2024

wildum commented Jun 7, 2024

wildum commented Jun 10, 2024

rfratto left a comment

Choose a reason for hiding this comment

rfratto left a comment

Choose a reason for hiding this comment

clayton-cornell left a comment

Choose a reason for hiding this comment

wildum commented Jun 11, 2024

wildum commented Jun 7, 2024 •

edited

Loading