DOC-964 New bloblang error functions #151

asimms41 · 2025-01-21T09:55:46Z

Description

Resolves DOC-964 also includes refactoring of error-handling page.
Review deadline: 5 February

Page previews

Bloblang functions
Error-handing
What's new

Checks

New feature
Content gap
Support Follow-up
Small fix (typos, links, copyedits, etc)

netlify · 2025-01-21T09:56:03Z

✅ Deploy Preview for redpanda-connect ready!

Name	Link
🔨 Latest commit
🔍 Latest deploy log	https://app.netlify.com/sites/redpanda-connect/deploys/67a483caa25771264f1f49fa
😎 Deploy Preview	https://deploy-preview-151--redpanda-connect.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

mihaitodor

LGTM, thank you!

JakeSCahill

Do we need to add these functions with more examples of real outputs to xref:configuration:error_handling.adoc?

mihaitodor · 2025-01-27T02:30:16Z

@JakeSCahill Good point! I took a stab at it here:

diff --git a/modules/configuration/pages/error_handling.adoc b/modules/configuration/pages/error_handling.adoc
index 73907a5..84e279d 100644
--- a/modules/configuration/pages/error_handling.adoc
+++ b/modules/configuration/pages/error_handling.adoc
@@ -49,18 +49,21 @@ pipeline:
 
 == Logging errors
 
-When an error occurs there will occasionally be useful information stored within the error flag that can be exposed with the interpolation function xref:configuration:interpolation.adoc#bloblang-queries[`error`]. This allows you to expose the information with processors.
+When an error occurs there will occasionally be useful information stored within the error flag that can be exposed with the interpolation function xref:configuration:interpolation.adoc#bloblang-queries[`error`]. Additionally, you can use the xref:guides:bloblang/methods.adoc#error_source_label[`error_source_label`], xref:guides:bloblang/methods.adoc#error_source_name[`error_source_name`] and / or xref:guides:bloblang/methods.adoc#error_source_path[`error_source_path`] bloblang functions to get extra information about the processor which failed.
 
-For example, when catching failed processors you can xref:components:processors/log.adoc[`log`] the messages:
+For example, when xref:components:processors/catch.adoc[`catch`]ing failed processors, you can xref:components:processors/log.adoc[`log`] the error message along with the label of the processor which failed:
 
 [source,yaml]
 ----
 pipeline:
   processors:
-    - resource: foo # Processor that might fail
+    - try:
+      - resource: foo # Processor that might fail
+      - resource: bar # Processor that might fail
+      - resource: baz # Processor that might fail
     - catch:
       - log:
-          message: "Processing failed due to: ${!error()}"
+          message: "Processor ${!error_source_label()} failed due to: ${!error()}"
 ----
 
 Or perhaps augment the message payload with the error message:

You can experiment with this:

res.yaml

processor_resources:
  - label: foo
    mapping: |
      root = content().uppercase()

  - label: bar
    mapping: |
      root = throw("kaboom!")

  - label: baz
    mapping: |
      root = content() + "blobfish"

conf.yaml

pipeline:
  processors:
    - try:
      - resource: foo # Processor that might fail
      - resource: bar # Processor that might fail
      - resource: baz # Processor that might fail

    - catch:
      - log:
          message: "Processor ${!error_source_label()} failed due to: ${!error()}"

$ rpk connect run -r res.yaml conf.yaml

Type something and hit enter.

WDYT? (cc @asimms41)

Feediver1

LGTM

mihaitodor

Nice job! Just saw a few issues which need to be fixed

modules/configuration/pages/error_handling.adoc

mihaitodor

LGTM, thank you for the updates! Just a small nit & feel free to

modules/configuration/pages/error_handling.adoc

JakeSCahill · 2025-02-05T16:19:37Z

@JakeSCahill Good point! I took a stab at it here:

diff --git a/modules/configuration/pages/error_handling.adoc b/modules/configuration/pages/error_handling.adoc
index 73907a5..84e279d 100644
--- a/modules/configuration/pages/error_handling.adoc
+++ b/modules/configuration/pages/error_handling.adoc
@@ -49,18 +49,21 @@ pipeline:
 
 == Logging errors
 
-When an error occurs there will occasionally be useful information stored within the error flag that can be exposed with the interpolation function xref:configuration:interpolation.adoc#bloblang-queries[`error`]. This allows you to expose the information with processors.
+When an error occurs there will occasionally be useful information stored within the error flag that can be exposed with the interpolation function xref:configuration:interpolation.adoc#bloblang-queries[`error`]. Additionally, you can use the xref:guides:bloblang/methods.adoc#error_source_label[`error_source_label`], xref:guides:bloblang/methods.adoc#error_source_name[`error_source_name`] and / or xref:guides:bloblang/methods.adoc#error_source_path[`error_source_path`] bloblang functions to get extra information about the processor which failed.
 
-For example, when catching failed processors you can xref:components:processors/log.adoc[`log`] the messages:
+For example, when xref:components:processors/catch.adoc[`catch`]ing failed processors, you can xref:components:processors/log.adoc[`log`] the error message along with the label of the processor which failed:
 
 [source,yaml]
 ----
 pipeline:
   processors:
-    - resource: foo # Processor that might fail
+    - try:
+      - resource: foo # Processor that might fail
+      - resource: bar # Processor that might fail
+      - resource: baz # Processor that might fail
     - catch:
       - log:
-          message: "Processing failed due to: ${!error()}"
+          message: "Processor ${!error_source_label()} failed due to: ${!error()}"
 ----
 
 Or perhaps augment the message payload with the error message:

You can experiment with this:

res.yaml

processor_resources:
  - label: foo
    mapping: |
      root = content().uppercase()

  - label: bar
    mapping: |
      root = throw("kaboom!")

  - label: baz
    mapping: |
      root = content() + "blobfish"

conf.yaml

pipeline:
  processors:
    - try:
      - resource: foo # Processor that might fail
      - resource: bar # Processor that might fail
      - resource: baz # Processor that might fail

    - catch:
      - log:
          message: "Processor ${!error_source_label()} failed due to: ${!error()}"

$ rpk connect run -r res.yaml conf.yaml

Type something and hit enter.

WDYT? (cc @asimms41)

Nice, thanks 😄

JakeSCahill

Love the additions to the error handling guide.

new bloblang error functions

1fbfb52

asimms41 added 3 commits January 21, 2025 10:17

minor rewording

29c16dc

minor rewording

3cff561

style and consistency changes

563fc4e

asimms41 requested a review from mihaitodor January 21, 2025 10:34

asimms41 marked this pull request as ready for review January 21, 2025 10:34

asimms41 requested a review from a team as a code owner January 21, 2025 10:34

mihaitodor approved these changes Jan 21, 2025

View reviewed changes

asimms41 requested a review from JakeSCahill January 21, 2025 15:30

JakeSCahill reviewed Jan 21, 2025

View reviewed changes

Feediver1 approved these changes Jan 29, 2025

View reviewed changes

asimms41 marked this pull request as draft January 31, 2025 15:06

asimms41 added 2 commits February 3, 2025 11:55

refactoring of error-handling topic

378785e

Merge branch 'main' into DOC-964_error_bloblang_functions

a326617

asimms41 marked this pull request as ready for review February 3, 2025 12:19

asimms41 requested a review from mihaitodor February 3, 2025 12:20

mihaitodor suggested changes Feb 3, 2025

View reviewed changes

asimms41 added 2 commits February 5, 2025 10:16

dev review comments

8e554bf

add change to what's new

2380a20

asimms41 requested a review from mihaitodor February 5, 2025 10:37

mihaitodor approved these changes Feb 5, 2025

View reviewed changes

modules/configuration/pages/error_handling.adoc Outdated Show resolved Hide resolved

code review comment

476fd3c

asimms41 requested a review from JakeSCahill February 5, 2025 15:26

JakeSCahill approved these changes Feb 5, 2025

View reviewed changes

asimms41 merged commit 776db97 into main Feb 6, 2025
5 checks passed

asimms41 deleted the DOC-964_error_bloblang_functions branch February 6, 2025 09:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC-964 New bloblang error functions #151

DOC-964 New bloblang error functions #151

asimms41 commented Jan 21, 2025 •

edited

Loading

netlify bot commented Jan 21, 2025 •

edited

Loading

mihaitodor left a comment

JakeSCahill left a comment

mihaitodor commented Jan 27, 2025

Feediver1 left a comment

mihaitodor left a comment

mihaitodor left a comment

JakeSCahill commented Feb 5, 2025

JakeSCahill left a comment

DOC-964 New bloblang error functions #151

DOC-964 New bloblang error functions #151

Conversation

asimms41 commented Jan 21, 2025 • edited Loading

Description

Page previews

Checks

netlify bot commented Jan 21, 2025 • edited Loading

✅ Deploy Preview for redpanda-connect ready!

mihaitodor left a comment

Choose a reason for hiding this comment

JakeSCahill left a comment

Choose a reason for hiding this comment

mihaitodor commented Jan 27, 2025

Feediver1 left a comment

Choose a reason for hiding this comment

mihaitodor left a comment

Choose a reason for hiding this comment

mihaitodor left a comment

Choose a reason for hiding this comment

JakeSCahill commented Feb 5, 2025

JakeSCahill left a comment

Choose a reason for hiding this comment

asimms41 commented Jan 21, 2025 •

edited

Loading

netlify bot commented Jan 21, 2025 •

edited

Loading