From d02300a64d37ef875de26b8e2678a782e87b2b44 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 10:37:18 -0400 Subject: [PATCH 01/34] (DOCSP-43962) Unnested 1 example in tip. --- .../recover-from-invalid-resume-token.txt | 20 +++++++++---------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/source/troubleshooting/recover-from-invalid-resume-token.txt b/source/troubleshooting/recover-from-invalid-resume-token.txt index 807d590a..f6cdaccd 100644 --- a/source/troubleshooting/recover-from-invalid-resume-token.txt +++ b/source/troubleshooting/recover-from-invalid-resume-token.txt @@ -161,22 +161,20 @@ property like this: - Records the number of times you reset your connector - Documents to which connector an offset partition belongs - .. example:: + For example, if you named your source connector ``"source-values"`` and you are + setting the ``offset.partition.name`` property for the first time, you + should configure your connector as follows: - Assume you named your source connector ``"source-values"`` and you are - setting the ``offset.partition.name`` property for the first time. - You would configure your connector as follows: - - .. code-block:: properties + .. code-block:: properties - offset.partition.name=source-values.1 + offset.partition.name=source-values.1 - The next time you reset your connector's offset data, configure - your connector as follows: + The next time you reset your connector's offset data, configure + your connector as follows: - .. code-block:: properties + .. code-block:: properties - offset.partition.name=source-values.2 + offset.partition.name=source-values.2 To learn more about the ``offset.partition.name`` configuration property, see the :ref:`` page. From 95fc90b9ee5c48aaff4c020ccf0db6420965c0ac Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 11:21:54 -0400 Subject: [PATCH 02/34] (DOCSP-43962) Removing the subjunctive should --- source/troubleshooting/recover-from-invalid-resume-token.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/troubleshooting/recover-from-invalid-resume-token.txt b/source/troubleshooting/recover-from-invalid-resume-token.txt index f6cdaccd..d13f836c 100644 --- a/source/troubleshooting/recover-from-invalid-resume-token.txt +++ b/source/troubleshooting/recover-from-invalid-resume-token.txt @@ -162,8 +162,8 @@ property like this: - Documents to which connector an offset partition belongs For example, if you named your source connector ``"source-values"`` and you are - setting the ``offset.partition.name`` property for the first time, you - should configure your connector as follows: + setting the ``offset.partition.name`` property for the first time, + configure your connector as follows: .. code-block:: properties From 2f7fb1522ee76be25c148413cd646df4a7fba716 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 11:58:10 -0400 Subject: [PATCH 03/34] (DOCSP-42962) Unnested 1 important admonition in table. --- .../configuration-properties/connector-message.txt | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/source/sink-connector/configuration-properties/connector-message.txt b/source/sink-connector/configuration-properties/connector-message.txt index 06a988f5..8e2e9f28 100644 --- a/source/sink-connector/configuration-properties/connector-message.txt +++ b/source/sink-connector/configuration-properties/connector-message.txt @@ -112,13 +112,11 @@ Settings connector may create fewer than the maximum tasks specified if it cannot handle the level of parallelism you specify. - .. important:: Multiple Tasks May Process Messages Out of Order - - If you specify a value greater than ``1``, the connector enables - parallel processing of the tasks. If your topic has multiple - partition logs, which enables the connector to read from the - topic in parallel, the tasks may process the messages out of - order. + :gold:`IMPORTANT:`If you specify a value greater than ``1``, + the connector enables parallel processing of the tasks. + If your topic has multiple partition logs, which enables + the connector to read from the topic in parallel, + the tasks may process the messages out of order. | **Default**: ``1`` | **Accepted Values**: An integer From 8ff73bfb0e580140228e6162c06d9e28bc3199cf Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 12:24:07 -0400 Subject: [PATCH 04/34] (DOCSP-42962) Typo fix. --- .../configuration-properties/connector-message.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/sink-connector/configuration-properties/connector-message.txt b/source/sink-connector/configuration-properties/connector-message.txt index 8e2e9f28..7bf19abf 100644 --- a/source/sink-connector/configuration-properties/connector-message.txt +++ b/source/sink-connector/configuration-properties/connector-message.txt @@ -112,7 +112,7 @@ Settings connector may create fewer than the maximum tasks specified if it cannot handle the level of parallelism you specify. - :gold:`IMPORTANT:`If you specify a value greater than ``1``, + :gold:`IMPORTANT:` If you specify a value greater than ``1``, the connector enables parallel processing of the tasks. If your topic has multiple partition logs, which enables the connector to read from the topic in parallel, From d59c83447b9d3e3c62989b77d23eb568bbe8af4e Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 12:30:51 -0400 Subject: [PATCH 05/34] (DOCSP-42962) Unnested 1 important admonition in table. --- source/includes/externalize-secrets.rst | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/source/includes/externalize-secrets.rst b/source/includes/externalize-secrets.rst index 3d8054a5..749a60b8 100644 --- a/source/includes/externalize-secrets.rst +++ b/source/includes/externalize-secrets.rst @@ -1,7 +1,5 @@ -.. important:: Avoid Exposing Your Authentication Credentials - - To avoid exposing your authentication credentials in your - ``connection.uri`` setting, use a - `ConfigProvider `__ - and set the appropriate configuration parameters. +:gold:`IMPORTANT:` To avoid exposing your authentication credentials in your +``connection.uri`` setting, use a +`ConfigProvider `__ +and set the appropriate configuration parameters. From d997a0eb9cfdc20904d6234c3b85911c076ae5ce Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 12:36:35 -0400 Subject: [PATCH 06/34] (DOCSP-42962) Unnested 1 tip in table. --- .../configuration-properties/mongodb-namespace.txt | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/source/sink-connector/configuration-properties/mongodb-namespace.txt b/source/sink-connector/configuration-properties/mongodb-namespace.txt index 47fb7509..cbc14787 100644 --- a/source/sink-connector/configuration-properties/mongodb-namespace.txt +++ b/source/sink-connector/configuration-properties/mongodb-namespace.txt @@ -44,12 +44,10 @@ Settings ``DefaultNamespaceMapper`` uses values specified in the ``database`` and ``collection`` properties. - .. seealso:: - - The connector includes an alternative class for specifying the - database and collection called ``FieldPathNamespaceMapper``. See - the :ref:`FieldPathNamespaceMapper settings ` - for more information. + The connector includes an alternative class for specifying the + database and collection called ``FieldPathNamespaceMapper``. See + the :ref:`FieldPathNamespaceMapper settings ` + for more information. | **Default**: From 2785c1769fe79646b218b88ff40cecd41b1c7b35 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 12:49:46 -0400 Subject: [PATCH 07/34] (DOCSP-42962) Fixing indentation error and unnesting 1 important admonition in table. --- .../configuration-properties/connector-message.txt | 10 +++++----- .../configuration-properties/mongodb-namespace.txt | 8 ++++---- .../configuration-properties/time-series.txt | 7 +++---- 3 files changed, 12 insertions(+), 13 deletions(-) diff --git a/source/sink-connector/configuration-properties/connector-message.txt b/source/sink-connector/configuration-properties/connector-message.txt index 7bf19abf..1a521a92 100644 --- a/source/sink-connector/configuration-properties/connector-message.txt +++ b/source/sink-connector/configuration-properties/connector-message.txt @@ -112,11 +112,11 @@ Settings connector may create fewer than the maximum tasks specified if it cannot handle the level of parallelism you specify. - :gold:`IMPORTANT:` If you specify a value greater than ``1``, - the connector enables parallel processing of the tasks. - If your topic has multiple partition logs, which enables - the connector to read from the topic in parallel, - the tasks may process the messages out of order. + :gold:`IMPORTANT:` If you specify a value greater than ``1``, + the connector enables parallel processing of the tasks. + If your topic has multiple partition logs, which enables + the connector to read from the topic in parallel, + the tasks may process the messages out of order. | **Default**: ``1`` | **Accepted Values**: An integer diff --git a/source/sink-connector/configuration-properties/mongodb-namespace.txt b/source/sink-connector/configuration-properties/mongodb-namespace.txt index cbc14787..8fb672ff 100644 --- a/source/sink-connector/configuration-properties/mongodb-namespace.txt +++ b/source/sink-connector/configuration-properties/mongodb-namespace.txt @@ -44,10 +44,10 @@ Settings ``DefaultNamespaceMapper`` uses values specified in the ``database`` and ``collection`` properties. - The connector includes an alternative class for specifying the - database and collection called ``FieldPathNamespaceMapper``. See - the :ref:`FieldPathNamespaceMapper settings ` - for more information. + The connector includes an alternative class for specifying the + database and collection called ``FieldPathNamespaceMapper``. See + the :ref:`FieldPathNamespaceMapper settings ` + for more information. | **Default**: diff --git a/source/sink-connector/configuration-properties/time-series.txt b/source/sink-connector/configuration-properties/time-series.txt index e34506c6..5850ba0f 100644 --- a/source/sink-connector/configuration-properties/time-series.txt +++ b/source/sink-connector/configuration-properties/time-series.txt @@ -60,6 +60,7 @@ Settings The connector passes the date format pattern to the Java `DateTimeFormatter.ofPattern(pattern, locale) `__ method to perform date and time conversions on the time field. + | If the date value from the source data only contains date information, the connector sets the time information to the start of the specified day. If the date value does not contain the timezone offset, the @@ -112,10 +113,8 @@ Settings | Which top-level field to read from the source data to describe a group of related time series documents. - .. important:: - - This field must not be the ``_id`` field nor the field you specified - in the ``timeseries.timefield`` setting. + :gold:`IMPORTANT:` This field must not be the ``_id`` field nor the field you specified + in the ``timeseries.timefield`` setting. | **Default**: ``""`` | **Accepted Values**: An empty string or the name of a field From 219faf88524bbe7f00c6c6a83e92a65b7dfe6880 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 12:55:37 -0400 Subject: [PATCH 08/34] (DOCSP-42962) Spacing. --- .../configuration-properties/mongodb-namespace.txt | 8 ++++---- .../configuration-properties/time-series.txt | 1 + 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/source/sink-connector/configuration-properties/mongodb-namespace.txt b/source/sink-connector/configuration-properties/mongodb-namespace.txt index 8fb672ff..cbc14787 100644 --- a/source/sink-connector/configuration-properties/mongodb-namespace.txt +++ b/source/sink-connector/configuration-properties/mongodb-namespace.txt @@ -44,10 +44,10 @@ Settings ``DefaultNamespaceMapper`` uses values specified in the ``database`` and ``collection`` properties. - The connector includes an alternative class for specifying the - database and collection called ``FieldPathNamespaceMapper``. See - the :ref:`FieldPathNamespaceMapper settings ` - for more information. + The connector includes an alternative class for specifying the + database and collection called ``FieldPathNamespaceMapper``. See + the :ref:`FieldPathNamespaceMapper settings ` + for more information. | **Default**: diff --git a/source/sink-connector/configuration-properties/time-series.txt b/source/sink-connector/configuration-properties/time-series.txt index 5850ba0f..d8d6c44c 100644 --- a/source/sink-connector/configuration-properties/time-series.txt +++ b/source/sink-connector/configuration-properties/time-series.txt @@ -80,6 +80,7 @@ Settings | **Description:** | Whether to convert the data in the field into the BSON ``Date`` format. + | When set to ``true``, the connector uses the milliseconds after epoch and discards fractional parts if the value is a number. If the value is a string, the connector uses the From ee9456d9818ef22a16c91906d99d8e367663ce9c Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 13:00:39 -0400 Subject: [PATCH 09/34] (DOCSP-42962) Build errors. --- .../configuration-properties/mongodb-namespace.txt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/sink-connector/configuration-properties/mongodb-namespace.txt b/source/sink-connector/configuration-properties/mongodb-namespace.txt index cbc14787..8fb672ff 100644 --- a/source/sink-connector/configuration-properties/mongodb-namespace.txt +++ b/source/sink-connector/configuration-properties/mongodb-namespace.txt @@ -44,10 +44,10 @@ Settings ``DefaultNamespaceMapper`` uses values specified in the ``database`` and ``collection`` properties. - The connector includes an alternative class for specifying the - database and collection called ``FieldPathNamespaceMapper``. See - the :ref:`FieldPathNamespaceMapper settings ` - for more information. + The connector includes an alternative class for specifying the + database and collection called ``FieldPathNamespaceMapper``. See + the :ref:`FieldPathNamespaceMapper settings ` + for more information. | **Default**: From 40fc989731eeae1212013018196515efe67bf946 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 13:03:15 -0400 Subject: [PATCH 10/34] (DOCSP-42962) Unnested 1 example and 1 note in table. --- .../configuration-properties/topic-override.txt | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/source/sink-connector/configuration-properties/topic-override.txt b/source/sink-connector/configuration-properties/topic-override.txt index a592776e..fd87edd0 100644 --- a/source/sink-connector/configuration-properties/topic-override.txt +++ b/source/sink-connector/configuration-properties/topic-override.txt @@ -41,17 +41,13 @@ Settings | Specify a topic and property name to override the corresponding global or default property setting. - .. example:: + For example, the ``topic.override.foo.collection=bar`` setting instructs + the sink connector to store data from the ``foo`` topic in the ``bar`` + collection. - The ``topic.override.foo.collection=bar`` setting instructs the - sink connector to store data from the ``foo`` topic in the ``bar`` - collection. - - .. note:: - - You can specify any valid configuration setting in the - ```` segment on a per-topic basis except - ``connection.uri`` and ``topics``. + You can specify any valid configuration setting in the + ```` segment on a per-topic basis except + ``connection.uri`` and ``topics``. | **Default**: ``""`` | **Accepted Values**: Accepted values specific to the overridden property From b3e5c3219ed379d9800a03292fbfb88b410076cc Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 13:21:51 -0400 Subject: [PATCH 11/34] (DOCSP-42962) Unnested 2 examples and 1 note in include. --- source/includes/copy-existing-admonition.rst | 12 ++++---- .../configuration-properties/startup.txt | 30 ++++++++----------- .../usage-examples/copy-existing-data.txt | 4 ++- 3 files changed, 21 insertions(+), 25 deletions(-) diff --git a/source/includes/copy-existing-admonition.rst b/source/includes/copy-existing-admonition.rst index 835ac270..1c51d141 100644 --- a/source/includes/copy-existing-admonition.rst +++ b/source/includes/copy-existing-admonition.rst @@ -1,7 +1,5 @@ -.. note:: Data Copy Can Produce Duplicate Events - - If any system changes the data in the database while the source connector - converts existing data from it, MongoDB may produce duplicate change - stream events to reflect the latest changes. Since the change stream - events on which the data copy relies are idempotent, the copied data is - eventually consistent. +If any system changes the data in the database while the source connector +converts existing data from it, MongoDB may produce duplicate change +stream events to reflect the latest changes. Since the change stream +events on which the data copy relies are idempotent, the copied data is +eventually consistent. diff --git a/source/source-connector/configuration-properties/startup.txt b/source/source-connector/configuration-properties/startup.txt index b5cef31f..09ff8f4a 100644 --- a/source/source-connector/configuration-properties/startup.txt +++ b/source/source-connector/configuration-properties/startup.txt @@ -95,19 +95,17 @@ Settings which to copy data. A namespace describes the MongoDB database name and collection separated by a period (for example, ``databaseName.collectionName``). - .. example:: + For example, the following regular-expression setting matches + collections that start with "page" in the ``stats`` database: - In the following example, the regular-expression setting matches - collections that start with "page" in the ``stats`` database. + .. code-block:: none - .. code-block:: none + startup.mode.copy.existing.namespace.regex=stats\.page.* - startup.mode.copy.existing.namespace.regex=stats\.page.* - - The "\" character in the example above escapes the "." character - that follows it in the regular expression. For more information on - how to build regular expressions, see the Java API documentation on - `Patterns `__. + The ``\`` character in the example above escapes the "." character + that follows it in the regular expression. For more information on + how to build regular expressions, see the Java API documentation on + `Patterns `__. | **Default**: ``""`` | **Accepted Values**: A valid regular expression @@ -121,15 +119,13 @@ Settings setting to filter the source collection and improve the use of indexes in the copying process. - .. example:: - - The following example shows how you can use the :manual:`$match ` - aggregation operator to instruct the connector to copy only - documents that contain a ``closed`` field with a value of ``false``. + For example, the following setting uses the :manual:`$match ` + aggregation operator to instruct the connector to copy only + documents that contain a ``closed`` field with a value of ``false``. - .. code-block:: none + .. code-block:: none - startup.mode.copy.existing.pipeline=[ { "$match": { "closed": "false" } } ] + startup.mode.copy.existing.pipeline=[ { "$match": { "closed": "false" } } ] | **Default**: ``""`` | **Accepted Values**: Valid aggregation pipeline stages diff --git a/source/source-connector/usage-examples/copy-existing-data.txt b/source/source-connector/usage-examples/copy-existing-data.txt index dcdac3ca..6894320f 100644 --- a/source/source-connector/usage-examples/copy-existing-data.txt +++ b/source/source-connector/usage-examples/copy-existing-data.txt @@ -65,7 +65,9 @@ specifying the following configuration options in your source connector: Your source connector copies your collection by creating change event documents that describe inserting each document into your collection. -.. include:: /includes/copy-existing-admonition.rst +.. note:: Data Copy Can Produce Duplicate Events + + .. include:: /includes/copy-existing-admonition.rst To learn more about change event documents, see the :ref:`Change Streams ` guide. From 9541a1d68bd74b3a208431be42f10a9de26bfc65 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 13:26:19 -0400 Subject: [PATCH 12/34] (DOCSP-42962) Replace quotation marks with monospace. --- source/source-connector/configuration-properties/startup.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/source-connector/configuration-properties/startup.txt b/source/source-connector/configuration-properties/startup.txt index 09ff8f4a..e2a00346 100644 --- a/source/source-connector/configuration-properties/startup.txt +++ b/source/source-connector/configuration-properties/startup.txt @@ -102,7 +102,7 @@ Settings startup.mode.copy.existing.namespace.regex=stats\.page.* - The ``\`` character in the example above escapes the "." character + The ``\`` character in the example above escapes the ``.`` character that follows it in the regular expression. For more information on how to build regular expressions, see the Java API documentation on `Patterns `__. From 0d992678e9c3f8bc53953879732144bcac54a3a3 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 13:38:31 -0400 Subject: [PATCH 13/34] (DOCSP-42962) Unnested 2 notes in table. --- .../configuration-properties/error-handling.txt | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/source/sink-connector/configuration-properties/error-handling.txt b/source/sink-connector/configuration-properties/error-handling.txt index 4fbf2b97..e762322c 100644 --- a/source/sink-connector/configuration-properties/error-handling.txt +++ b/source/sink-connector/configuration-properties/error-handling.txt @@ -56,10 +56,8 @@ Settings | To learn more about error handling strategies, see the :ref:`` page. - .. note:: - - This property overrides the `errors.tolerance `__ - property of the Connect Framework. + This property overrides the `errors.tolerance `__ + property of the Connect Framework. | **Default:** Inherits the value from the ``errors.tolerance`` setting. @@ -78,10 +76,8 @@ Settings "not tolerated" errors. | When set to ``false``, the connector logs "not tolerated" errors. - .. note:: - - This property overrides the `errors.log.enable `__ - property of the Connect Framework. + This property overrides the `errors.log.enable `__ + property of the Connect Framework. | **Default:** ``false`` | **Accepted Values**: ``true`` or ``false`` From 0a599958a869fa1de005cb37622021b511f524e2 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 13:44:49 -0400 Subject: [PATCH 14/34] (DOCSP-42962) consistency. --- .../configuration-properties/error-handling.txt | 12 ++++++++---- .../configuration-properties/post-processors.txt | 8 +++----- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/source/sink-connector/configuration-properties/error-handling.txt b/source/sink-connector/configuration-properties/error-handling.txt index e762322c..a80ceb51 100644 --- a/source/sink-connector/configuration-properties/error-handling.txt +++ b/source/sink-connector/configuration-properties/error-handling.txt @@ -100,7 +100,8 @@ Settings | Name of topic to use as the dead letter queue. If blank, the connector does not send any invalid messages to the dead letter queue. - | For more information about the dead letter queue, see the + + | To learn more about the dead letter queue, see the :ref:`Dead Letter Queue Configuration Example `. | | **Default:** ``""`` @@ -112,11 +113,13 @@ Settings | **Description:** | Whether the connector should include context headers when it writes messages to the dead letter queue. + | To learn more about the dead letter queue, see the :ref:`Dead Letter Queue Configuration Example `. + | To learn about the exceptions the connector defines and - reports through context headers, see the - :ref:`` section. + reports through context headers, see + :ref:``. | | **Default:** ``false`` | **Accepted Values**: ``true`` or ``false`` @@ -128,7 +131,8 @@ Settings | The number of nodes on which to replicate the dead letter queue topic. If you are running a single-node Kafka cluster, you must set this to ``1``. - | For more information about the dead letter queue, see the + + | To learn more about the dead letter queue, see the :ref:`Dead Letter Queue Configuration Example `. | | **Default:** ``3`` diff --git a/source/sink-connector/configuration-properties/post-processors.txt b/source/sink-connector/configuration-properties/post-processors.txt index f5c512a3..b7bf06f9 100644 --- a/source/sink-connector/configuration-properties/post-processors.txt +++ b/source/sink-connector/configuration-properties/post-processors.txt @@ -41,11 +41,9 @@ Settings | A list of post-processor classes the connector should apply to process the data before saving it to MongoDB. - .. seealso:: - - For more information on post-processors and examples of - their usage, see the section on - :doc:`Post-processors `. + For more information on post-processors and examples of + their usage, see the section on + :doc:`Post-processors `. | | **Default**: From 1422bbe91da09889368ae16845beb4869b3b94a8 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 13:50:17 -0400 Subject: [PATCH 15/34] (DOCSP-42962) Edited for consistency and unnested 2 tips. --- .../configuration-properties/post-processors.txt | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/source/sink-connector/configuration-properties/post-processors.txt b/source/sink-connector/configuration-properties/post-processors.txt index b7bf06f9..3fe4a301 100644 --- a/source/sink-connector/configuration-properties/post-processors.txt +++ b/source/sink-connector/configuration-properties/post-processors.txt @@ -41,9 +41,9 @@ Settings | A list of post-processor classes the connector should apply to process the data before saving it to MongoDB. - For more information on post-processors and examples of - their usage, see the section on - :doc:`Post-processors `. + To learn more about post-processors and see examples of + their usage, see + :doc:`Sink Connector Post Processors `. | | **Default**: @@ -128,10 +128,8 @@ Settings | The class that specifies the ``WriteModelStrategy`` the connector should use for :manual:`Bulk Writes `. - .. seealso:: - - For information on how to create your own strategy, see - :ref:``. + To learn more about how to create your own strategy, see + :ref:``. | | **Default**: From e27b4692f53f2980fd69acb5ff59569d81ad8cc5 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 13:54:37 -0400 Subject: [PATCH 16/34] (DOCSP-42962) Removing extra line breaks. --- .../sink-connector/configuration-properties/post-processors.txt | 2 -- 1 file changed, 2 deletions(-) diff --git a/source/sink-connector/configuration-properties/post-processors.txt b/source/sink-connector/configuration-properties/post-processors.txt index 3fe4a301..b005fe1d 100644 --- a/source/sink-connector/configuration-properties/post-processors.txt +++ b/source/sink-connector/configuration-properties/post-processors.txt @@ -44,7 +44,6 @@ Settings To learn more about post-processors and see examples of their usage, see :doc:`Sink Connector Post Processors `. - | | **Default**: @@ -130,7 +129,6 @@ Settings To learn more about how to create your own strategy, see :ref:``. - | | **Default**: From 2c49319424ca25b3ac57f36fda78610f33083a0e Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 16:19:21 -0400 Subject: [PATCH 17/34] (DOCSP-42962) Unnested 2 important admonitions in table. --- .../configuration-properties/mongodb-connection.txt | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/source/source-connector/configuration-properties/mongodb-connection.txt b/source/source-connector/configuration-properties/mongodb-connection.txt index e84032a2..e75aba1c 100644 --- a/source/source-connector/configuration-properties/mongodb-connection.txt +++ b/source/source-connector/configuration-properties/mongodb-connection.txt @@ -73,11 +73,8 @@ Settings | Name of the collection in the database to watch for changes. If not set, the connector watches all collections for changes. - .. important:: - - If your ``database`` configuration is set to ``""``, the connector - ignores the ``collection`` setting. - + :gold:`IMPORTANT:` If your ``database`` configuration is set to ``""``, the connector + ignores the ``collection`` setting. | | **Default**: ``""`` | **Accepted Values**: A single collection name From 12c83479eac587b100255c5ddf7adcb283765ed4 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 16:21:36 -0400 Subject: [PATCH 18/34] (DOCSP-42962) Fix rendering error. --- .../sink-connector/configuration-properties/post-processors.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/sink-connector/configuration-properties/post-processors.txt b/source/sink-connector/configuration-properties/post-processors.txt index b005fe1d..6a5c0b1c 100644 --- a/source/sink-connector/configuration-properties/post-processors.txt +++ b/source/sink-connector/configuration-properties/post-processors.txt @@ -44,6 +44,7 @@ Settings To learn more about post-processors and see examples of their usage, see :doc:`Sink Connector Post Processors `. + | | **Default**: @@ -129,6 +130,7 @@ Settings To learn more about how to create your own strategy, see :ref:``. + | | **Default**: From 58123ae017c4b1da64776984016677130db53bae Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 16:23:57 -0400 Subject: [PATCH 19/34] (DOCSP-42962) Fixing rendering issues. --- .../configuration-properties/mongodb-connection.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/source/source-connector/configuration-properties/mongodb-connection.txt b/source/source-connector/configuration-properties/mongodb-connection.txt index e75aba1c..7b391225 100644 --- a/source/source-connector/configuration-properties/mongodb-connection.txt +++ b/source/source-connector/configuration-properties/mongodb-connection.txt @@ -75,6 +75,7 @@ Settings :gold:`IMPORTANT:` If your ``database`` configuration is set to ``""``, the connector ignores the ``collection`` setting. + | | **Default**: ``""`` | **Accepted Values**: A single collection name From bb6e26f4d6ccf027423d51af3c53451bd0f5c746 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 16:29:24 -0400 Subject: [PATCH 20/34] (DOCSP-42962) Unnested 2 notes and 1 example in table. --- .../configuration-properties/kafka-topic.txt | 24 +++++++------------ 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/source/sink-connector/configuration-properties/kafka-topic.txt b/source/sink-connector/configuration-properties/kafka-topic.txt index ef4777ff..ee05b892 100644 --- a/source/sink-connector/configuration-properties/kafka-topic.txt +++ b/source/sink-connector/configuration-properties/kafka-topic.txt @@ -42,10 +42,8 @@ Settings | **Description:** | A list of Kafka topics that the sink connector watches. - .. note:: - - You can define either the ``topics`` or the ``topics.regex`` - setting, but not both. + You can define either the ``topics`` or the ``topics.regex`` + setting, but not both. | **Accepted Values**: A comma-separated list of valid Kafka topics @@ -58,20 +56,16 @@ Settings | A regular expression that matches the Kafka topics that the sink connector watches. - .. example:: - - .. code-block:: properties - - topics.regex=activity\\.\\w+\\.clicks$ + For example, the following regex matches topic names such as + "activity.landing.clicks" and "activity.support.clicks". + It does not match the topic names "activity.landing.views" and "activity.clicks". - This regex matches topic names such as "activity.landing.clicks" - and "activity.support.clicks". It does not match the topic names - "activity.landing.views" and "activity.clicks". + .. code-block:: properties - .. note:: + topics.regex=activity\\.\\w+\\.clicks$ - You can define either the ``topics`` or the ``topics.regex`` - setting, but not both. + You can define either the ``topics`` or the ``topics.regex`` + setting, but not both. | **Accepted Values**: A valid regular expression pattern using ``java.util.regex.Pattern``. From 92ab6d9d265bf1d1811aede3d75b467ee3d22a8b Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 16:35:17 -0400 Subject: [PATCH 21/34] (DOCSP-42962) Unnested 3 important admonitions --- .../error-handling.txt | 20 ++++++++----------- 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/source/source-connector/configuration-properties/error-handling.txt b/source/source-connector/configuration-properties/error-handling.txt index 1a5b0f1b..96a8cd30 100644 --- a/source/source-connector/configuration-properties/error-handling.txt +++ b/source/source-connector/configuration-properties/error-handling.txt @@ -45,10 +45,9 @@ Settings | Set this to ``"all"`` if you want the connector to continue processing messages and ignore any errors it encounters. - .. important:: - - This property overrides the `errors.tolerance `__ - Connect Framework property. + :gold:`IMPORTANT:` This property overrides the + `errors.tolerance `__ + Connect Framework property. | | **Default:** ``"none"`` @@ -65,10 +64,9 @@ Settings tolerate using the ``errors.tolerance`` or ``mongo.errors.tolerance`` setting. - .. important:: - - This property overrides the `errors.log.enable `__ - Connect Framework property. + :gold:`IMPORTANT:` This property overrides the + `errors.log.enable `__ + Connect Framework property. | | **Default:** ``false`` @@ -84,10 +82,8 @@ Settings | If you leave this setting blank, the connector does not write invalid messages to any topic. - .. important:: - - You must set ``errors.tolerance`` or ``mongo.errors.tolerance`` - setting to ``"all"`` to enable this property. + :gold:`IMPORTANT:` You must set ``errors.tolerance`` or ``mongo.errors.tolerance`` + setting to ``"all"`` to enable this property. | **Default:** ``""`` | **Accepted Values**: A valid Kafka topic name From a42377185a0893c16c55e952cfd34936588ad764 Mon Sep 17 00:00:00 2001 From: your proper name Date: Thu, 3 Oct 2024 16:45:17 -0400 Subject: [PATCH 22/34] (DOCSP-42962) Spacing fix. --- .../configuration-properties/error-handling.txt | 2 -- 1 file changed, 2 deletions(-) diff --git a/source/source-connector/configuration-properties/error-handling.txt b/source/source-connector/configuration-properties/error-handling.txt index 96a8cd30..327d24a4 100644 --- a/source/source-connector/configuration-properties/error-handling.txt +++ b/source/source-connector/configuration-properties/error-handling.txt @@ -49,7 +49,6 @@ Settings `errors.tolerance `__ Connect Framework property. - | | **Default:** ``"none"`` | **Accepted Values**: ``"none"`` or ``"all"`` @@ -68,7 +67,6 @@ Settings `errors.log.enable `__ Connect Framework property. - | | **Default:** ``false`` | **Accepted Values**: ``true`` or ``false`` From b44b8f532f939c3eebab5cbead11dfbb0c73daf7 Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 10:32:33 -0400 Subject: [PATCH 23/34] (DOCSP-42962) Unnested 1 example and 3 tips. --- .../change-stream.txt | 29 +++++++++---------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/source/source-connector/configuration-properties/change-stream.txt b/source/source-connector/configuration-properties/change-stream.txt index 925de1ca..f6aade60 100644 --- a/source/source-connector/configuration-properties/change-stream.txt +++ b/source/source-connector/configuration-properties/change-stream.txt @@ -45,16 +45,16 @@ Settings You must configure this setting for the change stream event document, not the ``fullDocument`` field. - .. example:: + For example: - .. code-block:: none + .. code-block:: none - [{"$match": { "$and": [{"operationType": "insert"}, {"fullDocument.eventId": 1404 }] } }] + [{"$match": { "$and": [{"operationType": "insert"}, {"fullDocument.eventId": 1404 }] } }] - .. tip:: Additional Examples + For more examples, see: - - :ref:`` - - :ref:`` + - :ref:`` + - :ref:`` | **Default**: ``"[]"`` | **Accepted Values**: Valid aggregation pipeline stage @@ -75,11 +75,10 @@ Settings | The ``required`` setting returns the updated document and raises an error if it is not available. - .. tip:: - - For more information on how this change stream option works, see - the MongoDB server manual guide on :manual:`Lookup Full Document for - Update Operations `. + For more information on how this change stream option works, see + :manual:`Lookup Full Document for Update Operations + ` + in the MongoDB manual. | | **Default**: ``""`` @@ -94,11 +93,9 @@ Settings published while copying existing data, and the pre-image configuration has no effect on copying. - .. tip:: - - To learn how to configure a collection to enable - pre-images, see the :manual:`Server manual entry on pre- and - post-images `. + To learn how to configure a collection to enable + pre-images, see the :manual:`Change Streams with Document Pre- and Post-Images + ` in the MongoDB manual. | The default setting suppresses the document pre-image. | The ``whenAvailable`` setting returns the document pre-image if From 8a87fdabbe01593d0665a9690d75baccf2492288 Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 10:46:17 -0400 Subject: [PATCH 24/34] (DOCSP-42962) Unnested 1 note, 2 tips, 1 important admonition. --- .../change-stream.txt | 8 ++--- .../output-format.txt | 36 ++++++++----------- 2 files changed, 17 insertions(+), 27 deletions(-) diff --git a/source/source-connector/configuration-properties/change-stream.txt b/source/source-connector/configuration-properties/change-stream.txt index f6aade60..99fcad76 100644 --- a/source/source-connector/configuration-properties/change-stream.txt +++ b/source/source-connector/configuration-properties/change-stream.txt @@ -92,11 +92,9 @@ Settings operations. The pre-image is not available for source records published while copying existing data, and the pre-image configuration has no effect on copying. - - To learn how to configure a collection to enable - pre-images, see the :manual:`Change Streams with Document Pre- and Post-Images - ` in the MongoDB manual. - + | To learn how to configure a collection to enable + pre-images, see the :manual:`Change Streams with Document Pre- and Post-Images + ` in the MongoDB manual. | The default setting suppresses the document pre-image. | The ``whenAvailable`` setting returns the document pre-image if it's available, before it was replaced, updated, or diff --git a/source/source-connector/configuration-properties/output-format.txt b/source/source-connector/configuration-properties/output-format.txt index a2a6361d..e50feaed 100644 --- a/source/source-connector/configuration-properties/output-format.txt +++ b/source/source-connector/configuration-properties/output-format.txt @@ -54,13 +54,11 @@ Settings | **Default**: ``json`` | **Accepted Values**: ``bson``, ``json``, ``schema`` - .. note:: Protobuf Output Format Support - - The connector supports Protobuf as an - output data format. You can enable this format by specifying the - ``schema`` value and installing and :ref:`configuring - ` the `Kafka Connect Protobuf - Converter `__. + | The connector supports Protobuf as an + output data format. You can enable this format by specifying the + ``schema`` value and installing and :ref:`configuring + ` the `Kafka Connect Protobuf + Converter `__. * - | **output.json.formatter** - | **Type:** string @@ -91,13 +89,11 @@ Settings - | **Type:** string | | **Description:** - | Specifies an AVRO schema definition for the key document of the + | Specifies an Avro schema definition for the key document of the `SourceRecord <{+kafka_api_docs_base+}javadoc/org/apache/kafka/connect/source/SourceRecord.html>`__. - .. seealso:: - - For more information on AVRO schema, see the - :ref:`Data Formats ` guide. + To learn more about Avro schema, see :ref:`Avro ` in the + :ref:`Data Formats guide `. | **Default**: @@ -109,19 +105,17 @@ Settings "fields" : [ { "name": "_id", "type": "string" } ]" } - | **Accepted Values**: A valid AVRO schema + | **Accepted Values**: A valid Avro schema * - | **output.schema.value** - | **Type:** string | | **Description:** - | Specifies an AVRO schema definition for the value document of the + | Specifies an Avro schema definition for the value document of the `SourceRecord <{+kafka_api_docs_base+}javadoc/org/apache/kafka/connect/source/SourceRecord.html>`__. - .. seealso:: - - For more information on AVRO schema, see the - :ref:`Data Formats ` guide. + | To learn more about Avro schema, see :ref:`Avro ` in the + :ref:`Data Formats guide `. | **Default**: @@ -170,10 +164,8 @@ Settings Since the connector processes each document in isolation, the connector may generate many schemas. - .. important:: - - The connector only reads this setting when you set your - ``output.format.value`` setting to ``schema``. + | :gold:`IMPORTANT:` The connector only reads this setting when you set your + ``output.format.value`` setting to ``schema``. | **Default**: ``false`` | **Accepted Values**: ``true`` or ``false`` From b15946b82be19428208663039a1daa8f8255809a Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 10:54:07 -0400 Subject: [PATCH 25/34] (DOCSP-42962) Spacing. --- .../change-stream.txt | 21 +++++++++++-------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/source/source-connector/configuration-properties/change-stream.txt b/source/source-connector/configuration-properties/change-stream.txt index 99fcad76..b9bc48c6 100644 --- a/source/source-connector/configuration-properties/change-stream.txt +++ b/source/source-connector/configuration-properties/change-stream.txt @@ -44,14 +44,14 @@ Settings | An array of aggregation pipelines to run in your change stream. You must configure this setting for the change stream event document, not the ``fullDocument`` field. - - For example: + | + | For example: .. code-block:: none [{"$match": { "$and": [{"operationType": "insert"}, {"fullDocument.eventId": 1404 }] } }] - For more examples, see: + | For more examples, see: - :ref:`` - :ref:`` @@ -65,21 +65,24 @@ Settings | **Description:** | Determines what values your change stream returns on update operations. + | | The default setting returns the differences between the original document and the updated document. + | | The ``updateLookup`` setting returns the differences between the original document and updated document as well as a copy of the entire updated document at a *point in time* after the update. + | | The ``whenAvailable`` setting returns the updated document, if available. + | | The ``required`` setting returns the updated document and raises an error if it is not available. - - For more information on how this change stream option works, see - :manual:`Lookup Full Document for Update Operations - ` - in the MongoDB manual. - + | + | For more information on how this change stream option works, see + :manual:`Lookup Full Document for Update Operations + ` + in the MongoDB manual. | | **Default**: ``""`` | **Accepted Values**: ``""``, ``"updateLookup"``, ``"whenAvailable"``, or ``"required"`` From ea5dc5e42381375b2b9d04f6b576e5d0924aa125 Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 11:00:24 -0400 Subject: [PATCH 26/34] (DOCSP-42962) Spacing. --- .../change-stream.txt | 20 +++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/source/source-connector/configuration-properties/change-stream.txt b/source/source-connector/configuration-properties/change-stream.txt index b9bc48c6..7e70b6d7 100644 --- a/source/source-connector/configuration-properties/change-stream.txt +++ b/source/source-connector/configuration-properties/change-stream.txt @@ -65,20 +65,20 @@ Settings | **Description:** | Determines what values your change stream returns on update operations. - | + | The default setting returns the differences between the original document and the updated document. - | + | The ``updateLookup`` setting returns the differences between the original document and updated document as well as a copy of the entire updated document at a *point in time* after the update. - | + | The ``whenAvailable`` setting returns the updated document, if available. - | + | The ``required`` setting returns the updated document and raises an error if it is not available. - | + | For more information on how this change stream option works, see :manual:`Lookup Full Document for Update Operations ` @@ -95,13 +95,17 @@ Settings operations. The pre-image is not available for source records published while copying existing data, and the pre-image configuration has no effect on copying. + | | To learn how to configure a collection to enable - pre-images, see the :manual:`Change Streams with Document Pre- and Post-Images + pre-images, see :manual:`Change Streams with Document Pre- and Post-Images ` in the MongoDB manual. + | | The default setting suppresses the document pre-image. + | | The ``whenAvailable`` setting returns the document pre-image if it's available, before it was replaced, updated, or deleted. + | | The ``required`` setting returns the document pre-image and raises an error if it is not available. | @@ -118,12 +122,12 @@ Settings updated document. To learn more about the ``fullDocument`` field, see the :manual:`update Event ` in the Server manual. - + | | When set to ``true``, the connector overrides the ``change.stream.full.document`` setting and sets it to ``updateLookup`` so that the ``fullDocument`` field contains updated documents. - + | | **Default**: ``false`` | **Accepted Values**: ``true`` or ``false`` From 6fa0ae512d042a9babd5e4c9314c2fd1f0c6857f Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 11:12:12 -0400 Subject: [PATCH 27/34] (DOCSP-42962) Normalizing spacing and adding reference to the Data Formats page. --- source/introduction/data-formats.txt | 2 ++ .../configuration-properties/change-stream.txt | 18 +++++++++--------- .../configuration-properties/output-format.txt | 18 +++++++++--------- 3 files changed, 20 insertions(+), 18 deletions(-) diff --git a/source/introduction/data-formats.txt b/source/introduction/data-formats.txt index e6baf41f..fd447f1e 100644 --- a/source/introduction/data-formats.txt +++ b/source/introduction/data-formats.txt @@ -1,3 +1,5 @@ +.. _kafka-data-formats: + ============ Data Formats ============ diff --git a/source/source-connector/configuration-properties/change-stream.txt b/source/source-connector/configuration-properties/change-stream.txt index 7e70b6d7..ecbbe08b 100644 --- a/source/source-connector/configuration-properties/change-stream.txt +++ b/source/source-connector/configuration-properties/change-stream.txt @@ -55,7 +55,7 @@ Settings - :ref:`` - :ref:`` - + | | **Default**: ``"[]"`` | **Accepted Values**: Valid aggregation pipeline stage @@ -65,20 +65,20 @@ Settings | **Description:** | Determines what values your change stream returns on update operations. - + | | The default setting returns the differences between the original document and the updated document. - + | | The ``updateLookup`` setting returns the differences between the original document and updated document as well as a copy of the entire updated document at a *point in time* after the update. - + | | The ``whenAvailable`` setting returns the updated document, if available. - + | | The ``required`` setting returns the updated document and raises an error if it is not available. - + | | For more information on how this change stream option works, see :manual:`Lookup Full Document for Update Operations ` @@ -139,7 +139,7 @@ Settings Tombstone events contain the keys of deleted documents with ``null`` values. This setting applies only when ``publish.full.document.only`` is ``true``. - + | | **Default**: ``false`` | **Accepted Values**: ``true`` or ``false`` @@ -149,12 +149,12 @@ Settings | **Description:** | Whether to use the document key for the source record key if the document key is present. - + | | When set to ``true``, the connector adds keys of the deleted documents to the tombstone events. When set to ``false``, the connector uses the resume token as the source key for the tombstone events. - + | | **Default**: ``true`` | **Accepted Values**: ``true`` or ``false`` diff --git a/source/source-connector/configuration-properties/output-format.txt b/source/source-connector/configuration-properties/output-format.txt index e50feaed..3e401d12 100644 --- a/source/source-connector/configuration-properties/output-format.txt +++ b/source/source-connector/configuration-properties/output-format.txt @@ -91,10 +91,10 @@ Settings | **Description:** | Specifies an Avro schema definition for the key document of the `SourceRecord <{+kafka_api_docs_base+}javadoc/org/apache/kafka/connect/source/SourceRecord.html>`__. - - To learn more about Avro schema, see :ref:`Avro ` in the - :ref:`Data Formats guide `. - + | + | To learn more about Avro schema, see :ref:`Avro ` in the + :ref:`Data Formats guide `. + | | **Default**: .. code-block:: json @@ -113,10 +113,10 @@ Settings | **Description:** | Specifies an Avro schema definition for the value document of the `SourceRecord <{+kafka_api_docs_base+}javadoc/org/apache/kafka/connect/source/SourceRecord.html>`__. - + | | To learn more about Avro schema, see :ref:`Avro ` in the - :ref:`Data Formats guide `. - + :ref:`Data Formats guide `. + | | **Default**: .. code-block:: json @@ -163,10 +163,10 @@ Settings document of the `SourceRecord <{+kafka_api_docs_base+}javadoc/org/apache/kafka/connect/source/SourceRecord.html>`__. Since the connector processes each document in isolation, the connector may generate many schemas. - + | | :gold:`IMPORTANT:` The connector only reads this setting when you set your ``output.format.value`` setting to ``schema``. - + | | **Default**: ``false`` | **Accepted Values**: ``true`` or ``false`` From 077ac479c3fe490079a9b51373cfb38d537b69ae Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 13:25:09 -0400 Subject: [PATCH 28/34] (DOCSP-42962) Unnested 3 tips, 1 example and 1 important admonition in table. --- .../configuration-properties/kafka-topic.txt | 71 ++++++++----------- 1 file changed, 29 insertions(+), 42 deletions(-) diff --git a/source/source-connector/configuration-properties/kafka-topic.txt b/source/source-connector/configuration-properties/kafka-topic.txt index 61b9d38b..947fc6bd 100644 --- a/source/source-connector/configuration-properties/kafka-topic.txt +++ b/source/source-connector/configuration-properties/kafka-topic.txt @@ -50,12 +50,9 @@ Settings The destination topic name is composed of the ``topic.prefix`` value followed by the database and collection names, separated by the value specified in the ``topic.separator`` property. - - .. seealso:: - - :ref:`Topic Naming Prefix ` - usage example - + | + | To learn more, see the example in :ref:`Topic Naming Prefix `. + | | **Default**: ``""`` | **Accepted Values**: A string composed of ASCII alphanumeric characters including ".", "-", and "_" @@ -69,12 +66,9 @@ Settings The destination topic name is composed of the database and collection names followed by the ``topic.suffix`` value, separated by the value specified in the ``topic.separator`` property. - - .. seealso:: - - :ref:`Topic Naming Suffix ` - usage example - + | + | To learn more, see the example in :ref:`Topic Naming Suffix `. + | | **Default**: ``""`` | **Accepted Values**: A string composed of ASCII alphanumeric characters including ".", "-", and "_" @@ -86,16 +80,14 @@ Settings | Specifies a JSON mapping between change stream document :manual:`namespaces ` and topic names. - - .. tip:: Namespace Mapping Behavior - - You can specify complex mappings by using the - ``topic.namespace.map`` property. This property supports regex - and wildcard matching. To learn more about these behaviors and - view examples, see the :ref:`Topic Namespace Map - ` section of the Topic Naming - page. - + | + | You can specify complex mappings by using the + ``topic.namespace.map`` property. This property supports regex + and wildcard matching. To learn more about these behaviors and + view examples, see the :ref:`Topic Namespace Map + ` section of the Topic Naming + page. + | | **Default**: ``""`` | **Accepted Values**: A valid JSON object @@ -113,29 +105,24 @@ Settings #. ``collection`` #. ``topic.suffix`` - .. example:: - - The following configuration instructs the connector to publish - change stream documents from the ``coll`` collection of the - ``db`` database to the ``prefix-db-coll`` - topic: + | For example, the following configuration instructs the connector to publish + change stream documents from the ``coll`` collection of the + ``db`` database to the ``prefix-db-coll`` topic: - .. code-block:: properties - :copyable: false + .. code-block:: properties + :copyable: false - topic.prefix=prefix - database=db - collection=coll - topic.separator=- + topic.prefix=prefix + database=db + collection=coll + topic.separator=- - .. important:: Topic Separator and Topic Namespace Map - - When you use the ``topic.separator`` property, keep in mind that it - does not affect how you define the ``topic.namespace.map`` property. - The ``topic.namespace.map`` property uses MongoDB - :manual:`namespaces ` - which you must always specify with a ``"."`` character to separate - the database and collection name. + | :gold:`IMPORTANT:` When you use the ``topic.separator`` property, keep in mind that it + does not affect how you define the ``topic.namespace.map`` property. + The ``topic.namespace.map`` property uses MongoDB + :manual:`namespaces ` + which you must always specify with a ``.`` character to separate + the database and collection name. | | **Default**: ``"."`` From 40392a582badc744e302cbb8d8dfd22f639eac79 Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 13:38:52 -0400 Subject: [PATCH 29/34] (DOCSP-42962) Regularizing usage of pipeline operator line breaks. --- .../connector-message.txt | 14 ++++----- .../error-handling.txt | 30 +++++++++++-------- .../post-processors.txt | 17 +++++------ .../topic-override.txt | 18 +++++------ .../change-stream.txt | 2 +- 5 files changed, 42 insertions(+), 39 deletions(-) diff --git a/source/sink-connector/configuration-properties/connector-message.txt b/source/sink-connector/configuration-properties/connector-message.txt index 1a521a92..6773ecdf 100644 --- a/source/sink-connector/configuration-properties/connector-message.txt +++ b/source/sink-connector/configuration-properties/connector-message.txt @@ -111,13 +111,13 @@ Settings | The maximum number of tasks to create for this connector. The connector may create fewer than the maximum tasks specified if it cannot handle the level of parallelism you specify. - - :gold:`IMPORTANT:` If you specify a value greater than ``1``, - the connector enables parallel processing of the tasks. - If your topic has multiple partition logs, which enables - the connector to read from the topic in parallel, - the tasks may process the messages out of order. - + | + | :gold:`IMPORTANT:` If you specify a value greater than ``1``, + | the connector enables parallel processing of the tasks. + | If your topic has multiple partition logs, which enables + | the connector to read from the topic in parallel, + | the tasks may process the messages out of order. + | | **Default**: ``1`` | **Accepted Values**: An integer diff --git a/source/sink-connector/configuration-properties/error-handling.txt b/source/sink-connector/configuration-properties/error-handling.txt index a80ceb51..c29450ca 100644 --- a/source/sink-connector/configuration-properties/error-handling.txt +++ b/source/sink-connector/configuration-properties/error-handling.txt @@ -48,17 +48,21 @@ Settings | Whether to continue processing messages if the connector encounters an error. Allows the connector to override the ``errors.tolerance`` Kafka cluster setting. + | | When set to ``none``, the connector reports any error and blocks further processing of the rest of the messages. + | | When set to ``all``, the connector ignores any problematic messages. + | | When set to ``data``, the connector tolerates only data errors and fails on all other errors. + | | To learn more about error handling strategies, see the :ref:`` page. - - This property overrides the `errors.tolerance `__ - property of the Connect Framework. - + | + | This property overrides the `errors.tolerance `__ + | property of the Connect Framework. + | | **Default:** Inherits the value from the ``errors.tolerance`` setting. | **Accepted Values**: ``"none"`` or ``"all"`` @@ -71,14 +75,14 @@ Settings failed operations to the log file. The connector classifies errors as "tolerated" or "not tolerated" using the ``errors.tolerance`` or ``mongo.errors.tolerance`` settings. - + | | When set to ``true``, the connector logs both "tolerated" and "not tolerated" errors. | When set to ``false``, the connector logs "not tolerated" errors. - - This property overrides the `errors.log.enable `__ - property of the Connect Framework. - + | + | This property overrides the `errors.log.enable `__ + | property of the Connect Framework. + | | **Default:** ``false`` | **Accepted Values**: ``true`` or ``false`` @@ -100,7 +104,7 @@ Settings | Name of topic to use as the dead letter queue. If blank, the connector does not send any invalid messages to the dead letter queue. - + | | To learn more about the dead letter queue, see the :ref:`Dead Letter Queue Configuration Example `. | @@ -113,10 +117,10 @@ Settings | **Description:** | Whether the connector should include context headers when it writes messages to the dead letter queue. - + | | To learn more about the dead letter queue, see the :ref:`Dead Letter Queue Configuration Example `. - + | | To learn about the exceptions the connector defines and reports through context headers, see :ref:``. @@ -131,7 +135,7 @@ Settings | The number of nodes on which to replicate the dead letter queue topic. If you are running a single-node Kafka cluster, you must set this to ``1``. - + | | To learn more about the dead letter queue, see the :ref:`Dead Letter Queue Configuration Example `. | diff --git a/source/sink-connector/configuration-properties/post-processors.txt b/source/sink-connector/configuration-properties/post-processors.txt index 6a5c0b1c..59f1ca4f 100644 --- a/source/sink-connector/configuration-properties/post-processors.txt +++ b/source/sink-connector/configuration-properties/post-processors.txt @@ -40,11 +40,10 @@ Settings | **Description:** | A list of post-processor classes the connector should apply to process the data before saving it to MongoDB. - - To learn more about post-processors and see examples of - their usage, see - :doc:`Sink Connector Post Processors `. - + | + | To learn more about post-processors and see examples of + | their usage, see + | :doc:`Sink Connector Post Processors `. | | **Default**: @@ -127,10 +126,10 @@ Settings | **Description:** | The class that specifies the ``WriteModelStrategy`` the connector should use for :manual:`Bulk Writes `. - - To learn more about how to create your own strategy, see - :ref:``. - + | + | To learn more about how to create your own strategy, see + | :ref:``. + | | | **Default**: diff --git a/source/sink-connector/configuration-properties/topic-override.txt b/source/sink-connector/configuration-properties/topic-override.txt index fd87edd0..d6d39f96 100644 --- a/source/sink-connector/configuration-properties/topic-override.txt +++ b/source/sink-connector/configuration-properties/topic-override.txt @@ -40,15 +40,15 @@ Settings | **Description:** | Specify a topic and property name to override the corresponding global or default property setting. - - For example, the ``topic.override.foo.collection=bar`` setting instructs - the sink connector to store data from the ``foo`` topic in the ``bar`` - collection. - - You can specify any valid configuration setting in the - ```` segment on a per-topic basis except - ``connection.uri`` and ``topics``. - + | + | For example, the ``topic.override.foo.collection=bar`` setting instructs + | the sink connector to store data from the ``foo`` topic in the ``bar`` + | collection. + | + | You can specify any valid configuration setting in the + | ```` segment on a per-topic basis except + | ``connection.uri`` and ``topics``. + | | **Default**: ``""`` | **Accepted Values**: Accepted values specific to the overridden property diff --git a/source/source-connector/configuration-properties/change-stream.txt b/source/source-connector/configuration-properties/change-stream.txt index ecbbe08b..2c4b8105 100644 --- a/source/source-connector/configuration-properties/change-stream.txt +++ b/source/source-connector/configuration-properties/change-stream.txt @@ -55,7 +55,7 @@ Settings - :ref:`` - :ref:`` - | + | **Default**: ``"[]"`` | **Accepted Values**: Valid aggregation pipeline stage From 848dfed97cd80a2a327f9fdc7ec917da0180e60b Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 13:57:56 -0400 Subject: [PATCH 30/34] (DOCSP-42962) Spacing. --- .../connector-message.txt | 13 ++++---- .../configuration-properties/id-strategy.txt | 5 +-- .../mongodb-connection.txt | 2 ++ .../mongodb-namespace.txt | 14 ++++---- .../post-processors.txt | 1 - .../configuration-properties/time-series.txt | 32 +++++++++++-------- 6 files changed, 38 insertions(+), 29 deletions(-) diff --git a/source/sink-connector/configuration-properties/connector-message.txt b/source/sink-connector/configuration-properties/connector-message.txt index 6773ecdf..93db2e3b 100644 --- a/source/sink-connector/configuration-properties/connector-message.txt +++ b/source/sink-connector/configuration-properties/connector-message.txt @@ -75,8 +75,9 @@ Settings When set to ``true``, the default value, the connector writes a batch of records as an ordered bulk write operation. - | To learn more about bulk write operations, see the :ref:`Write - Model Strategies page `. + | + | To learn more about bulk write operations, see + :ref:`Bulk Write Operations `. | | **Default**: ``true`` | **Accepted Values**: ``true`` or ``false`` @@ -113,10 +114,10 @@ Settings cannot handle the level of parallelism you specify. | | :gold:`IMPORTANT:` If you specify a value greater than ``1``, - | the connector enables parallel processing of the tasks. - | If your topic has multiple partition logs, which enables - | the connector to read from the topic in parallel, - | the tasks may process the messages out of order. + the connector enables parallel processing of the tasks. + If your topic has multiple partition logs, which enables + the connector to read from the topic in parallel, + the tasks may process the messages out of order. | | **Default**: ``1`` | **Accepted Values**: An integer diff --git a/source/sink-connector/configuration-properties/id-strategy.txt b/source/sink-connector/configuration-properties/id-strategy.txt index 5b0fba46..1a684b69 100644 --- a/source/sink-connector/configuration-properties/id-strategy.txt +++ b/source/sink-connector/configuration-properties/id-strategy.txt @@ -76,8 +76,9 @@ Settings | | **Description:** | Whether the connector should delete documents when the key value - matches a document in MongoDB and the value field is null. This - setting applies when you specify an id generation strategy that + matches a document in MongoDB and the value field is null. + | + | This setting applies when you specify an id generation strategy that operates on the key document such as ``FullKeyStrategy``, ``PartialKeyStrategy``, and ``ProvidedInKeyStrategy``. | diff --git a/source/sink-connector/configuration-properties/mongodb-connection.txt b/source/sink-connector/configuration-properties/mongodb-connection.txt index 73a412f0..4b8dcf94 100644 --- a/source/sink-connector/configuration-properties/mongodb-connection.txt +++ b/source/sink-connector/configuration-properties/mongodb-connection.txt @@ -68,6 +68,7 @@ Settings | When set to ``true``, if the connector calls a command on your MongoDB instance that's deprecated in the declared {+stable-api+} version, it raises an exception. + | | You can set the API version with the ``server.api.version`` configuration option. For more information on the {+stable-api+}, see the MongoDB manual entry on the @@ -83,6 +84,7 @@ Settings | When set to ``true``, if the connector calls a command on your MongoDB instance that's not covered in the declared {+stable-api+} version, it raises an exception. + | | You can set the API version with the ``server.api.version`` configuration option. For more information on the {+stable-api+}, see the MongoDB manual entry on the diff --git a/source/sink-connector/configuration-properties/mongodb-namespace.txt b/source/sink-connector/configuration-properties/mongodb-namespace.txt index 8fb672ff..964053e1 100644 --- a/source/sink-connector/configuration-properties/mongodb-namespace.txt +++ b/source/sink-connector/configuration-properties/mongodb-namespace.txt @@ -43,12 +43,12 @@ Settings database or collection in which to sink the data. The default ``DefaultNamespaceMapper`` uses values specified in the ``database`` and ``collection`` properties. - - The connector includes an alternative class for specifying the - database and collection called ``FieldPathNamespaceMapper``. See - the :ref:`FieldPathNamespaceMapper settings ` - for more information. - + | + | The connector includes an alternative class for specifying the + | database and collection called ``FieldPathNamespaceMapper``. See + | :ref:`FieldPathNamespaceMapper Settings ` + | for more information. + | | **Default**: .. code-block:: none @@ -147,10 +147,12 @@ You can use the following settings to customize the behavior of the | **Description**: | Whether to throw an exception when either the document is missing the mapped field or it has an invalid BSON type. + | | When set to ``true``, the connector does not process documents missing the mapped field or that contain an invalid BSON type. The connector may halt or skip processing depending on the related error-handling configuration settings. + | | When set to ``false``, if a document is missing the mapped field or if it has an invalid BSON type, the connector defaults to writing to the specified ``database`` and ``collection`` settings. diff --git a/source/sink-connector/configuration-properties/post-processors.txt b/source/sink-connector/configuration-properties/post-processors.txt index 59f1ca4f..0d89b216 100644 --- a/source/sink-connector/configuration-properties/post-processors.txt +++ b/source/sink-connector/configuration-properties/post-processors.txt @@ -130,7 +130,6 @@ Settings | To learn more about how to create your own strategy, see | :ref:``. | - | | **Default**: .. code-block:: none diff --git a/source/sink-connector/configuration-properties/time-series.txt b/source/sink-connector/configuration-properties/time-series.txt index d8d6c44c..ae6c9ff3 100644 --- a/source/sink-connector/configuration-properties/time-series.txt +++ b/source/sink-connector/configuration-properties/time-series.txt @@ -57,10 +57,11 @@ Settings | The date format pattern the connector should use to convert the source data contained in the field specified by the ``timeseries.timefield`` setting. - The connector passes the date format pattern to the Java + | + | The connector passes the date format pattern to the Java `DateTimeFormatter.ofPattern(pattern, locale) `__ method to perform date and time conversions on the time field. - + | | If the date value from the source data only contains date information, the connector sets the time information to the start of the specified day. If the date value does not contain the timezone offset, the @@ -80,7 +81,7 @@ Settings | **Description:** | Whether to convert the data in the field into the BSON ``Date`` format. - + | | When set to ``true``, the connector uses the milliseconds after epoch and discards fractional parts if the value is a number. If the value is a string, the connector uses the @@ -101,8 +102,9 @@ Settings | | **Description:** | Which ``DateTimeFormatter`` locale language tag to use with the date - format pattern (e.g. ``"en-US"``). For more information on - locales, see the Java SE documentation of `Locale `__. + format pattern (e.g. ``"en-US"``). + | + | To learn more about locales, see the Java SE documentation of `Locale `__. | | **Default**: ``ROOT`` | **Accepted Values**: A valid ``Locale`` language tag format @@ -113,10 +115,10 @@ Settings | **Description:** | Which top-level field to read from the source data to describe a group of related time series documents. - - :gold:`IMPORTANT:` This field must not be the ``_id`` field nor the field you specified - in the ``timeseries.timefield`` setting. - + | + | :gold:`IMPORTANT:` This field must not be the ``_id`` field nor the field you specified + in the ``timeseries.timefield`` setting. + | | **Default**: ``""`` | **Accepted Values**: An empty string or the name of a field that contains any BSON type except ``BsonArray``. @@ -128,8 +130,9 @@ Settings | The number of seconds MongoDB should wait before automatically removing the time series collection data. The connector disables timed expiry when the setting value is less than ``1``. - For more information on this collection setting, see the MongoDB - Server Manual page on :manual:`Automatic Removal for Time Series Collections `. + | + | To learn more, see :manual:`Set up Automatic Removal for Time Series Collections ` + in the MongoDB manual. | | **Default**: ``0`` | **Accepted Values**: An integer @@ -140,9 +143,10 @@ Settings | | **Description:** | The expected interval between subsequent measurements of your - source data. For more information on this setting, see the - MongoDB server manual page on :manual:`Granularity for Time - Series Data `. + source data. + | + | To learn more, see :manual:`Set Granularity for Time Series Data + ` in the MongoDB manual. | | *Optional* | **Default**: ``""`` From 1de3bc74efe469c150320c48e46fd3271d25dae2 Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 14:07:28 -0400 Subject: [PATCH 31/34] (DOCSP-42962) Spacing. --- .../configuration-properties/kafka-topic.txt | 16 ++++++++-------- .../mongodb-connection.txt | 12 +++++++----- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/source/source-connector/configuration-properties/kafka-topic.txt b/source/source-connector/configuration-properties/kafka-topic.txt index 947fc6bd..b0e8b5fa 100644 --- a/source/source-connector/configuration-properties/kafka-topic.txt +++ b/source/source-connector/configuration-properties/kafka-topic.txt @@ -81,12 +81,13 @@ Settings :manual:`namespaces ` and topic names. | - | You can specify complex mappings by using the - ``topic.namespace.map`` property. This property supports regex - and wildcard matching. To learn more about these behaviors and - view examples, see the :ref:`Topic Namespace Map - ` section of the Topic Naming - page. + | You can use to ``topic.namespace.map`` property to + specify complex mappings. This property supports regex + and wildcard matching. + | + | To learn more about these behaviors and + view examples, see :ref:`Topic Namespace Map + `. | | **Default**: ``""`` | **Accepted Values**: A valid JSON object @@ -115,7 +116,7 @@ Settings topic.prefix=prefix database=db collection=coll - topic.separator=- + topic.separator=- | :gold:`IMPORTANT:` When you use the ``topic.separator`` property, keep in mind that it does not affect how you define the ``topic.namespace.map`` property. @@ -123,7 +124,6 @@ Settings :manual:`namespaces ` which you must always specify with a ``.`` character to separate the database and collection name. - | | **Default**: ``"."`` | **Accepted Values**: A string diff --git a/source/source-connector/configuration-properties/mongodb-connection.txt b/source/source-connector/configuration-properties/mongodb-connection.txt index 7b391225..9d01c04c 100644 --- a/source/source-connector/configuration-properties/mongodb-connection.txt +++ b/source/source-connector/configuration-properties/mongodb-connection.txt @@ -49,7 +49,8 @@ Settings | **Description:** | The :manual:`URI connection string ` to connect to your MongoDB instance or cluster. - | For more information, see the :ref:`` guide + | + | To learn more, see :ref:``. .. include:: /includes/externalize-secrets.rst @@ -72,10 +73,9 @@ Settings | **Description:** | Name of the collection in the database to watch for changes. If not set, the connector watches all collections for changes. - - :gold:`IMPORTANT:` If your ``database`` configuration is set to ``""``, the connector - ignores the ``collection`` setting. - + | + | :gold:`IMPORTANT:` If your ``database`` configuration is set to ``""``, the connector + ignores the ``collection`` setting. | | **Default**: ``""`` | **Accepted Values**: A single collection name @@ -99,6 +99,7 @@ Settings | When set to ``true``, if the connector calls a command on your MongoDB instance that's deprecated in the declared {+stable-api+} version, it raises an exception. + | | You can set the API version with the ``server.api.version`` configuration option. For more information on the {+stable-api+}, see the MongoDB manual entry on the @@ -114,6 +115,7 @@ Settings | When set to ``true``, if the connector calls a command on your MongoDB instance that's not covered in the declared {+stable-api+} version, it raises an exception. + | | You can set the API version with the ``server.api.version`` configuration option. For more information on the {+stable-api+}, see the MongoDB manual entry on the From 9e4aecbd60d97247c6d5fbb5ce48decb631ca0f2 Mon Sep 17 00:00:00 2001 From: your proper name Date: Fri, 4 Oct 2024 14:25:03 -0400 Subject: [PATCH 32/34] (DOCSP-42962) Normalizing spacing across pages. --- .../error-handling.txt | 60 +++++++++++-------- .../output-format.txt | 6 +- .../configuration-properties/startup.txt | 39 +++++++----- 3 files changed, 62 insertions(+), 43 deletions(-) diff --git a/source/source-connector/configuration-properties/error-handling.txt b/source/source-connector/configuration-properties/error-handling.txt index 327d24a4..9a8172ee 100644 --- a/source/source-connector/configuration-properties/error-handling.txt +++ b/source/source-connector/configuration-properties/error-handling.txt @@ -39,16 +39,18 @@ Settings | **Description:** | Whether to continue processing messages when the connector encounters an error. + | | Set this to ``"none"`` if you want the connector to stop processing messages and report the issue if it encounters an error. + | | Set this to ``"all"`` if you want the connector to continue processing messages and ignore any errors it encounters. - - :gold:`IMPORTANT:` This property overrides the - `errors.tolerance `__ - Connect Framework property. - + | + | :gold:`IMPORTANT:` This property overrides the + `errors.tolerance `__ + Connect Framework property. + | | **Default:** ``"none"`` | **Accepted Values**: ``"none"`` or ``"all"`` @@ -57,16 +59,18 @@ Settings | | **Description:** | Whether the connector should report errors in the log file. + | | Set this to ``true`` to log all errors the connector encounters. + | | Set this to ``false`` to log errors that are not tolerated by the connector. You can specify which errors the connector should tolerate using the ``errors.tolerance`` or ``mongo.errors.tolerance`` setting. - - :gold:`IMPORTANT:` This property overrides the - `errors.log.enable `__ - Connect Framework property. - + | + | :gold:`IMPORTANT:` This property overrides the + `errors.log.enable `__ + Connect Framework property. + | | **Default:** ``false`` | **Accepted Values**: ``true`` or ``false`` @@ -75,14 +79,16 @@ Settings | | **Description:** | The name of topic to use as the dead letter queue. + | | If you specify a value, the connector writes invalid messages to the dead letter queue topic as :manual:`extended JSON strings `. + | | If you leave this setting blank, the connector does not write invalid messages to any topic. - :gold:`IMPORTANT:` You must set ``errors.tolerance`` or ``mongo.errors.tolerance`` - setting to ``"all"`` to enable this property. - + | :gold:`IMPORTANT:` You must set ``errors.tolerance`` or ``mongo.errors.tolerance`` + setting to ``"all"`` to enable this property. + | | **Default:** ``""`` | **Accepted Values**: A valid Kafka topic name @@ -92,12 +98,14 @@ Settings | **Description:** | The custom offset partition name to use. You can use this option to instruct the connector to start a new change stream when an - existing offset contains an invalid resume token. If you leave - this setting blank, the connector uses the default partition name - based on the connection details. To view a strategy for naming - offset partitions, see the :ref:`` - guide. - + existing offset contains an invalid resume token. + | + | If you leave this setting blank, the connector uses the default partition name + based on the connection details. + | + | To view a strategy for naming + offset partitions, see :ref:``. + | | **Default:** ``""`` | **Accepted Values**: A string. To learn more about naming a partition, see @@ -110,15 +118,19 @@ Settings | **Description:** | The number of milliseconds the connector waits between sending heartbeat messages. The connector sends heartbeat messages when - source records are not published in the specified interval. + source records are not published in the specified interval. This mechanism improves + resumability of the connector for low volume namespaces. + | | Heartbeat messages contain a ``postBatchResumeToken`` data field. The value of this field contains the MongoDB server oplog entry that the connector last read from the change stream. - | This mechanism improves resumability of the connector for low volume - namespaces. See the :ref:`Invalid Resume Token ` - page for more information on this feature. - + | | Set this to ``0`` to disable heartbeat messages. + | + | To learn more, see :ref:`Prevention ` + in the :ref:`Invalid Resume Token ` + page. + | | **Default**: ``0`` | **Accepted Values**: An integer diff --git a/source/source-connector/configuration-properties/output-format.txt b/source/source-connector/configuration-properties/output-format.txt index 3e401d12..14129667 100644 --- a/source/source-connector/configuration-properties/output-format.txt +++ b/source/source-connector/configuration-properties/output-format.txt @@ -51,14 +51,14 @@ Settings | Specifies which data format the source connector outputs the value document. | - | **Default**: ``json`` - | **Accepted Values**: ``bson``, ``json``, ``schema`` - | The connector supports Protobuf as an output data format. You can enable this format by specifying the ``schema`` value and installing and :ref:`configuring ` the `Kafka Connect Protobuf Converter `__. + | + | **Default**: ``json`` + | **Accepted Values**: ``bson``, ``json``, ``schema`` * - | **output.json.formatter** - | **Type:** string diff --git a/source/source-connector/configuration-properties/startup.txt b/source/source-connector/configuration-properties/startup.txt index e2a00346..359fa07b 100644 --- a/source/source-connector/configuration-properties/startup.txt +++ b/source/source-connector/configuration-properties/startup.txt @@ -65,7 +65,7 @@ Settings setting is equivalent to the deprecated setting ``copy.existing=true``. .. include:: /includes/copy-existing-admonition.rst - + | **Default**:``latest`` | **Accepted Values**: ``latest``, ``timestamp``, ``copy_existing`` @@ -74,9 +74,12 @@ Settings | | **Description:** | Actuated only if ``startup.mode=timestamp``. Specifies the - starting point for the change stream. To learn more about - Change Stream parameters, see the :manual:`Server manual entry - `. + starting point for the change stream. + | + | To learn more about Change Stream parameters, see + :manual:`$changeStream (aggregation) ` + in the MongoDB manual. + | | **Default**: ``""`` | **Accepted Values**: @@ -94,19 +97,20 @@ Settings | Regular expression the connector uses to match namespaces from which to copy data. A namespace describes the MongoDB database name and collection separated by a period (for example, ``databaseName.collectionName``). - - For example, the following regular-expression setting matches - collections that start with "page" in the ``stats`` database: + | + | For example, the following regular-expression setting matches + collections that start with "page" in the ``stats`` database: .. code-block:: none startup.mode.copy.existing.namespace.regex=stats\.page.* - The ``\`` character in the example above escapes the ``.`` character - that follows it in the regular expression. For more information on - how to build regular expressions, see the Java API documentation on - `Patterns `__. - + | The ``\`` character in the example above escapes the ``.`` character + that follows it in the regular expression. For more information on + how to build regular expressions, see + `Patterns `__ + in the Java API documentation. + | | **Default**: ``""`` | **Accepted Values**: A valid regular expression @@ -118,10 +122,10 @@ Settings the connector runs when copying existing data. You can use this setting to filter the source collection and improve the use of indexes in the copying process. - - For example, the following setting uses the :manual:`$match ` - aggregation operator to instruct the connector to copy only - documents that contain a ``closed`` field with a value of ``false``. + | + | For example, the following setting uses the :manual:`$match ` + aggregation operator to instruct the connector to copy only + documents that contain a ``closed`` field with a value of ``false``. .. code-block:: none @@ -135,6 +139,7 @@ Settings | | **Description:** | The maximum number of threads the connector can use to copy data. + | | **Default**: number of processors available in the environment | **Accepted Values**: An integer @@ -143,6 +148,7 @@ Settings | | **Description:** | The size of the queue the connector can use when copying data. + | | **Default**: ``16000`` | **Accepted Values**: An integer @@ -152,6 +158,7 @@ Settings | **Description:** | When set to ``true``, the connector uses temporary disk storage for the copy existing aggregation. + | | **Default**: ``true`` | **Accepted Values**: ``true`` or ``false`` From 212683a39dba72fb72c69b39b086c90203ab45e0 Mon Sep 17 00:00:00 2001 From: Elyse Foreman Date: Fri, 4 Oct 2024 15:03:57 -0400 Subject: [PATCH 33/34] Update source/source-connector/configuration-properties/kafka-topic.txt Co-authored-by: carriecwk <107137496+carriecwk@users.noreply.github.com> --- .../source-connector/configuration-properties/kafka-topic.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/source-connector/configuration-properties/kafka-topic.txt b/source/source-connector/configuration-properties/kafka-topic.txt index b0e8b5fa..b2aa4fcf 100644 --- a/source/source-connector/configuration-properties/kafka-topic.txt +++ b/source/source-connector/configuration-properties/kafka-topic.txt @@ -119,7 +119,7 @@ Settings topic.separator=- | :gold:`IMPORTANT:` When you use the ``topic.separator`` property, keep in mind that it - does not affect how you define the ``topic.namespace.map`` property. + doesn't affect how you define the ``topic.namespace.map`` property. The ``topic.namespace.map`` property uses MongoDB :manual:`namespaces ` which you must always specify with a ``.`` character to separate From 1eaa52716604babc82014408c44fa7d07cf31c17 Mon Sep 17 00:00:00 2001 From: Elyse Foreman Date: Fri, 4 Oct 2024 15:04:05 -0400 Subject: [PATCH 34/34] Update source/source-connector/configuration-properties/kafka-topic.txt Co-authored-by: carriecwk <107137496+carriecwk@users.noreply.github.com> --- .../source-connector/configuration-properties/kafka-topic.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/source-connector/configuration-properties/kafka-topic.txt b/source/source-connector/configuration-properties/kafka-topic.txt index b2aa4fcf..e95444dd 100644 --- a/source/source-connector/configuration-properties/kafka-topic.txt +++ b/source/source-connector/configuration-properties/kafka-topic.txt @@ -118,7 +118,7 @@ Settings collection=coll topic.separator=- - | :gold:`IMPORTANT:` When you use the ``topic.separator`` property, keep in mind that it + | :gold:`IMPORTANT:` When you use the ``topic.separator`` property, note that it doesn't affect how you define the ``topic.namespace.map`` property. The ``topic.namespace.map`` property uses MongoDB :manual:`namespaces `