Skip to content

Commit

Permalink
DOCSP-19689: fix sink connector error tolerance description
Browse files Browse the repository at this point in the history
  • Loading branch information
Chris Cho committed May 16, 2022
1 parent c1b9752 commit 214e216
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 24 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ meta.yaml
fabfile
giza.log
backups

*.swp
48 changes: 25 additions & 23 deletions source/sink-connector/configuration-properties/error-handling.txt
Original file line number Diff line number Diff line change
Expand Up @@ -41,28 +41,30 @@ Settings
|
| **Description:**
| Whether to continue processing messages if the connector encounters
an error.
an error. Allows the connector to override the ``errors.tolerance``
Kafka cluster setting.
| When set to ``none``, the connector reports any error and
blocks further processing of the rest of the messages.
| When set to ``all``, the connector ignores any problematic messages.
| To learn more about error handling strategies, see the
| To learn more about error handling strategies, see the
:ref:`<kafka-sink-handle-errors>` page.

.. note::

This property overrides the `errors.tolerance <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-tolerance>`__
property of the Connect Framework.

| **Default:** ``"none"``
| **Default:** Inherits the value from the ``errors.tolerance``
setting.
| **Accepted Values**: ``"none"`` or ``"all"``

* - | **mongo.errors.log.enable**
- | **Type:** boolean
|
| **Description:**
| Whether the connector should write details of errors including
| Whether the connector should write details of errors including
failed operations to the log file. The connector classifies
errors as "tolerated" or "not tolerated" using the
errors as "tolerated" or "not tolerated" using the
``errors.tolerance`` or ``mongo.errors.tolerance`` settings.

| When set to ``true``, the connector logs both "tolerated" and
Expand Down Expand Up @@ -95,7 +97,7 @@ Settings
| Name of topic to use as the dead letter queue. If blank, the
connector does not send any invalid messages to the dead letter
queue.
| For more information about the dead letter queue, see the
| For more information about the dead letter queue, see the
:ref:`Dead Letter Queue Configuration Example <sink-dead-letter-queue-configuration-example>`.
|
| **Default:** ``""``
Expand All @@ -107,7 +109,7 @@ Settings
| **Description:**
| Whether the connector should include context headers when it
writes messages to the dead letter queue.
| To learn more about the dead letter queue, see the
| To learn more about the dead letter queue, see the
:ref:`Dead Letter Queue Configuration Example <sink-dead-letter-queue-configuration-example>`.
| To learn about the exceptions the connector defines and
reports through context headers, see the
Expand Down Expand Up @@ -145,7 +147,7 @@ as context headers when performing bulk writes:

* - Name
- Description

* - | ``WriteException``
- | **Description:**
| Contains details of a
Expand All @@ -155,26 +157,26 @@ as context headers when performing bulk writes:
| **Message Format:**

This class outputs the error in the following format:

.. code-block:: none

v=%d, code=%d, message=%s, details=%s

The fields in the preceding message contain the following information:

- ``v``: The version of the ``WriteException`` message format.
This field helps parse the messages produced by this exception.
This field helps parse the messages produced by this exception.
For version {+connector_version+} of the connector, the version of the
message format is {+write-exception-version+}.
- ``code``: The code associated with the error. To learn more see the
- ``code``: The code associated with the error. To learn more see the
`getCode() <{+connector_driver_api_doc_url_base+}apidocs/mongodb-driver-core/com/mongodb/WriteError.html#getCode()>`__
method documentation.
method documentation.
- ``message``: The message associated with the error. To learn more, see the
`getMessage() <{+connector_driver_api_doc_url_base+}apidocs/mongodb-driver-core/com/mongodb/WriteError.html#getMessage()>`__
method documentation.
method documentation.
- ``details``: The details associated with the error in JSON format. To
learn more, see the following method documentation:

- `getDetails() <{+connector_driver_api_doc_url_base+}apidocs/mongodb-driver-core/com/mongodb/WriteError.html#getDetails()>`__
- `toJson() <{+connector_driver_api_doc_url_base+}apidocs/bson/org/bson/BsonDocument.html#toJson()>`__

Expand All @@ -187,27 +189,27 @@ as context headers when performing bulk writes:
| **Message Format:**

This class outputs the error in the following format:

.. code-block:: none

v=%d, code=%d, codeName=%d, message=%s, details=%s

The fields in the preceding message contain the following information:

- ``v``: The version of the ``WriteConcernException`` message format.
This field helps parse the messages produced by this exception.
This field helps parse the messages produced by this exception.
For version {+connector_version+} of the
connector, the version of the message format is
{+write-concern-exception-version+}.
- ``code``: The code associated with the error. To learn more see the
`getCode() <{+connector_driver_api_doc_url_base+}apidocs/mongodb-driver-core/com/mongodb/bulk/WriteConcernError.html#getCode()>`__
method documentation.
- ``codeName``: The code name associated with the error. To learn more, see the
method documentation.
- ``codeName``: The code name associated with the error. To learn more, see the
`getCodeName() <{+connector_driver_api_doc_url_base+}apidocs/mongodb-driver-core/com/mongodb/bulk/WriteConcernError.html#getCode()>`__
method documentation.
method documentation.
- ``message``: The message associated with the error. To learn more, see the
`getMessage() <{+connector_driver_api_doc_url_base+}apidocs/mongodb-driver-core/com/mongodb/bulk/WriteConcernError.html#getMessage()>`__
method documentation.
method documentation.
- ``details``: The details associated with the error in JSON format. To
learn more, see the following method documentation:

Expand All @@ -227,7 +229,7 @@ as context headers when performing bulk writes:
write operations, see the :ref:`<sink-configuration-message-processing>` page.
| **Message Format:**

This exception produces no message.
This exception produces no message.

To enable bulk write exception reporting to the dead letter queue, use the
following connector configuration:
Expand All @@ -244,8 +246,8 @@ following connector configuration:
Dead Letter Queue Configuration Example
---------------------------------------

Apache Kafka version 2.6 added support for handling errant records. The
Kafka connector automatically sends messages that it cannot process to the
Apache Kafka version 2.6 added support for handling errant records. The
Kafka connector automatically sends messages that it cannot process to the
**dead letter queue**. Once on the dead letter queue, you can inspect the
errant records, update them, and resubmit them for processing.

Expand Down

0 comments on commit 214e216

Please sign in to comment.