Skip to content

Commit

Permalink
EDU:1551 Adds tables to explain settings
Browse files Browse the repository at this point in the history
  • Loading branch information
@franrob-projects
franrob-projects committed Jan 15, 2025
1 parent 7998fe2 commit f692061
Show file tree
Hide file tree
Showing 6 changed files with 167 additions and 15 deletions.
31 changes: 31 additions & 0 deletions content/integrations/continuous-streaming/amqp-rule.textile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,37 @@ To create a rule in your "dashboard":https://ably.com/dashboard#58:
* Configure the settings applicable to your use case and your AMQP installation.
* Click *Create* to create the rule.

h4(#header). APMQ header and authentication settings:

|_. Section |_. Purpose |
|URL |Specifies the AMQP connection URL in the format `amqps://username:[email protected]/vhost`. |
|Headers |Allows the inclusion of additional information in key-value format. |
|Another header button |Adds additional headers for the message. |


h4(#general). APMQ general rule settings

The following explains the APMQ general rule settings:

|_. Section |_. Purpose |
|URL |Specifies the HTTPS URL for the SQS queue, including credentials, region, and stream name. |
|AWS region |Defines the AWS region associated with the SQS queue. |
|AWS authentication scheme |Allows selection of the authentication method: AWS credentials or ARN of an assumable role. |
|AWS credentials |Enter AWS credentials in `key:value` format for authentication. |

h4(#specific). APMQ-specific settings

The following explains the APMQ-specific settings:

|_. Section |_. Purpose |
|Routing key |Specifies the routing key used by the AMQP exchange to route messages to a physical queue. Supports interpolation. |
|Route mandatory |Ensures delivery is rejected if the route does not exist. |
|Route persistent |Marks messages as persistent, instructing the broker to write them to disk if the queue is durable. |
|Optional TTL (minutes)|Allows overriding the default queue TTL for messages to be persisted. |
|Create button |Click to save and provision the AMQP settings. |
|Cancel button |Click to cancel the configuration and return to the previous screen. |


h3(#creating-rule-control-api). Creating a AMQP rule using Control API

See some examples of creating rules in the "rules section":/account/control-api#examples-rules of the Control API documentation. You can also refer to the "rule section":/api/control-api#tag/rules/paths/~1apps~1{app_id}~1rules/post of the Control API Reference for information on creating a AMQP rule. You will need to select a @ruleType@ of @amqp@.
23 changes: 23 additions & 0 deletions content/integrations/continuous-streaming/kafka-rule.textile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,29 @@ To create a rule in your "dashboard":https://ably.com/dashboard#58:
* Choose Kafka.
* Configure the settings applicable to your use case and your Kafka installation.

h4(#general). Kafka general rule settings

The following explains the Kafka general rule settings:

|_. Section |_. Purpose |
|Source |Select the type of event source. |
|Channel filter |Allows filtering of the rule based on a regular expression matching the channel name. |
|Encoding |Selects a setting for encoding the payload. |
|Enveloped |Wraps the payload with additional metadata when checked. Uncheck to receive raw payloads.|

h4(#specific). Kafka-specific settings

The following explains the Kafka-specific settings:

|_. Section |_. Purpose |
|Routing key |Used to route messages to Kafka topics. |
|Mechanism |Dropdown to select the SASL/SCRAM mechanism used for Kafka connection. |
|Username |Enter the username required to authenticate with the Kafka server. |
|Password |Enter the password required to authenticate with the Kafka server. |
|Brokers |List of Kafka broker endpoints in the format `<host>:<port>`. |
|Another broker |Option to add additional Kafka broker endpoints. |
|Create button |Click to save and provision the Kafka settings for the integration rule. |

In this section you need to set up your Authentication for Kafka by selecting your preferred mechanism for authentication and providing credentials.

The supported Authentication Mechanisms are:
Expand Down
21 changes: 21 additions & 0 deletions content/integrations/continuous-streaming/kinesis-rule.textile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,27 @@ To create a rule in your "dashboard":https://ably.com/dashboard#58:
* Configure the settings applicable to your use case and your Kinesis installation.
* Click *Create* to create the rule.

h4(#aws). AWS Kinesis rule settings

The following explains the AWS Kinesis generalrule settings:

|_. Section |_. Purpose |
|AWS Region |Specifies the AWS region for the Kinesis Stream. |
|Stream Name |Defines the name of the Kinesis Stream to connect to. |
|AWS authentication scheme |Choose the authentication method: AWS credentials or ARN of an assumable role. |
|AWS Credentials |Enter your AWS credentials in `key:value` format. |

h4(#general). Kinesis general rule settings

The following explains the Kinesis general rule settings:

|_. Section |_. Purpose |
|Source |Defines the type of event to deliver . |
|Channel filter |Allows filtering of the rule using a regular expression matching the channel name. |
|Encoding |Selects a setting for encoding the payload. |
|Enveloped |When checked, wraps payloads with additional metadata. Uncheck for raw payloads. |


h3(#creating-rule-control-api). Creating a Kinesis rule using Control API

See some examples of creating rules in the "rules section":/account/control-api#examples of the Control API documentation. You can also refer to the "rule section":/api/control-api#tag/rules/paths/~1apps~1{app_id}~1rules/post of the Control API Reference for information on creating a Kinesis rule. You will need to select a @ruleType@ of @aws/kinesis@.
24 changes: 24 additions & 0 deletions content/integrations/continuous-streaming/pulsar-rule.textile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,30 @@ To create a rule in your "dashboard":https://ably.com/dashboard#58:
* Configure the settings applicable to your use case and your Pulsar installation.
* Click *Create* to create the rule.

h4(#general). Pulsar general rule settings

The following explains the Pulsar general rule settings:

|_. Section |_. Purpose |
|Source |Defines the type of event to deliver. |
|Channel Filter |Allows filtering of the rule based on a regular expression matching the channel name. |
|Encoding |Selects a setting for encoding the payload. |
|Enveloped |When checked, wraps payloads with additional metadata. Uncheck for raw payloads. |

h4(#specific). Pulsar-specific settings

The following explains the Pulsar-specific settings:

|_. Section |_. Purpose |
|Routing key |An optional value to use as the `partitionKey` on messages published to Pulsar. |
|Topic |Defines the Pulsar topic to publish messages to. Must be in the format `tenant/namespace/topic_name`. |
|Service URL |Specifies the Pulsar cluster URL in the format `pulsar://host:port` or `pulsar+ssl://host:port`. |
|JWT Token |Used for authentication with JWT tokens. Only JWT token authentication is supported. |
|TLS trust certificates |Allows for specifying a list of trusted CA certificates to verify TLS certificates presented by Pulsar. |
|Another TLS trust cert |Option to add additional trusted certificates for verification. |
|Create button |Click to finalize and provision the Pulsar settings for the integration rule. |
|Cancel button |Click to cancel the configuration and return to the previous screen. |

h3(#creating-rule-control-api). Creating a Pulsar rule using Control API

See some examples of creating rules in the "rules section":/account/control-api#examples-rules of the Control API documentation. You can also refer to the "rules section":/api/control-api#tag/rules/paths/~1apps~1{app_id}~1rules/post of the Control API Reference for information on creating a Pulsar rule. You will need to select a @ruleType@ of @pulsar@.
24 changes: 24 additions & 0 deletions content/integrations/continuous-streaming/sqs-rule.textile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,30 @@ To create a rule in your "dashboard":https://ably.com/dashboard#58:
* Configure the settings applicable to your use case and your SQS installation.
* Click *Create* to create the rule.

h4(#general). SQS general rule settings

The following explains the SQS general rule settings:

|_. Section |_. Purpose |
|URL |Specifies the HTTPS URL for the SQS queue, including credentials, region, and stream name. |
|AWS region |Defines the AWS region associated with the SQS queue. |
|AWS authentication scheme |Allows selection of the authentication method: AWS credentials or ARN of an assumable role. |
|AWS credentials |Enter AWS credentials in `key:value` format for authentication. |

h4(#specific). SQS-specific settings

The following explains the SQS-specific settings:

|_. Section |_. Purpose |
|Source |Defines the type of event to deliver, such as Message, Presence, Channel Lifecycle, or Channel Occupancy. |
|Channel filter |Allows filtering of the rule based on a regular expression matching the channel name. |
|Encoding |Selects a setting for encoding the payload. |
|Enveloped |When checked, wraps payloads with additional metadata. Uncheck for raw payloads. |
|Create button |Click to save and apply the rule settings. |
|Cancel button |Click to cancel the configuration and return to the previous screen. |



h3(#creating-rule-control-api). Creating a SQS rule using Control API

See some examples of creating rules in the "rules section":/account/control-api#examples of the Control API documentation. You can also refer to the "rule section":/api/control-api#tag/rules/paths/~1apps~1{app_id}~1rules/post of the Control API Reference for information on creating a SQS rule. You will need to select a @ruleType@ of @aws/sqs@.
59 changes: 44 additions & 15 deletions content/integrations/queues.textile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,7 @@ A message queueing pattern is a better fit here as multiple worker servers can s
The message queue ensures that work is distributed evenly to the pool of servers, work is not duplicated (resulting for example in more than one email notification being sent) and the system is resilient to crashes or spikes in load (messages are stored until a consumer is ready to retrieve them).

Ably combines both pub/sub and queueing functionality in its "platform":https://ably.com/platform as illustrated in the diagram below:
<Question ID="1" Shortcut="chapter 1" Order="" ElementType="chapter" >
<LongCaption></LongCaption>
</Question>

<a href="@content/diagrams/ably-queues-overview.png" target="_blank">
<img src="@content/diagrams/ably-queues-overview.png" style="width: 100%" alt="Ably Queues overview">
</a>
Expand All @@ -55,12 +53,15 @@ Ably Queues are setup within your app "dashboard":https://ably.com/dashboard and

Please note that the total number of queues, TTL and max length for each queue is limited based on your account type. Find out more about account and package "limits":/pricing/limits.

The procedure to provision a queue is as follows:
The following explains how to set up a new queue:


* In your Ably "dashboard":https://ably.com/dashboard click on the **Queues** tab.
* Click the **Provision a new Queue** button.
* Fill out information for your queue in the **New Queue** dialog, and click **Create**:
* Confirm your queue has been provisioned. Your new queue will appear in the **Queues** tab along with realtime metrics for your queue.
|_. Section |_. Purpose |_. How to Use |
|Name |Allows you to specify a unique name for the queue. |Enter a descriptive name for the queue.|
|Region |Defines the physical data center location for the queue.|Choose a region from the dropdown. |
|TTL (minutes) |Sets the time-to-live for each message in the queue. |Enter a value between 1 and 60. Default is 60 minutes. |
|Max length |Limits the maximum number of messages in the queue. |Specify a value up to 10,000. Default is 10,000. |
|Create |Provisions the queue with the specified settings. |Click *Create* to finalize and provision the queue.|

You will see that a "dead letter queue":#deadletter has automatically been provisioned for you. A dead letter queue is used to store messages that failed to be consumed, have passed their TTL, or were discarded due to a queue being full.

Expand All @@ -87,9 +88,41 @@ The procedure to create a queue rule is as follows:

h3(#dashboard-stats). Queue dashboards and stats

Provisioned queues are visible in your app "dashboard":https://ably.com/dashboard and provide near-realtime stats for the current state of each queue. See an example screenshot below.

Whilst the queue dashboard stats show the current state of your queue, your app and account dashboard provide up-to-date live and historical stats for all messages published to your queues. See an example screenshot from an "app dashboard":https://ably.com/dashboard below.
Provisioned queues are visible in your app "dashboard":https://ably.com/dashboard and provide near-realtime stats for the current state of each queue:

|_. Section |_. Purpose |
|Name |Displays the unique identifier for the queue. |
|State |Indicates the current operational state of the queue. |
|Region |Specifies the physical data center location of the queue. |
|Server endpoint |Provides the connection endpoint for the queue. |
|Vhost |Specifies the virtual host associated with the queue. |
|Messages ready |Shows the number of messages waiting to be consumed. |
|Messages unacked |Displays the number of messages acknowledged by consumers. |
|Total messages |Indicates the total number of messages currently in the queue. |
|Incoming message rates |Shows the rate at which messages are added to the queue (per second).|
|Deliver/get message rates|Shows the rate at which messages are delivered or fetched. |
|Ack message rates |Displays the rate of message acknowledgments. |
|TTL (minutes) |Indicates how long messages remain in the queue before expiring. |
|Max length |Defines the maximum number of messages allowed in the queue. |
|Action |Allows for deletion of the queue. |

The queue dashboard stats show the current state of your queue. Your app and account dashboard provide up-to-date live and historical stats for all messages published to your queues:

|_. Section | Purpose |
|Messages total | Shows the total number of messages processed during the specified time period.|
| Messages published (REST & Realtime) | Displays the total number of messages sent to Ably via REST or Realtime APIs. |
| Messages received (Realtime) | Shows the number of messages delivered to subscribers through Realtime channels. |
| Messages retrieved (history) | Indicates how many historical messages were retrieved from storage. |
| Messages persisted (history) |Tracks the number of messages saved in Ably's message persistence system. |
|Presence events (REST & Realtime) |Displays the number of presence events (e.g., users joining/leaving channels). |
|Reactor WebHook / Function | Shows the total number of WebHooks or serverless functions triggered by Ably events. |
|Reactor Queue | Displays the number of messages published to queues for asynchronous processing. |
|Reactor Firehose |Tracks the number of messages streamed via the Firehose integration to external systems. |
|Push & Smart Notifications | Indicates the number of push notifications or smart notifications sent. |
|Peak connections | Shows the maximum number of simultaneous connections to the Ably platform. |
|Peak channels |Tracks the highest number of active channels during the specified period. |
|Data transferred |Displays the amount of data transferred through the Ably system during the specified period. |
| Avg. message size |Indicates the average size of messages processed during the specified time frame.|

h3(#testing-rules). Testing your queue rules

Expand Down Expand Up @@ -151,8 +184,6 @@ The AMQP protocol provides a rich set of functionality to amongst other things b

However, unlike our pub/sub channels, queues are pre-provisioned via our queue dashboards and all routing is handled by the queue rules. As such, when subscribing to messages from the provisioned queues, you must not attempt to bind to an exchange or declare a queue as these requests will be rejected. Instead, you should subscribe directly to the queue you wish to consume messages from.

Take the following queue as an example:

In order to subscribe to messages from this queue you will need:

- The queue name := @UATwBQ:example-queue@ which is made up of your app ID and the name you assigned to your queue
Expand Down Expand Up @@ -206,8 +237,6 @@ h4(#consume-stomp). Using STOMP

The STOMP protocol is a simple text-based protocol designed for working with message-oriented middleware. It provides an interoperable wire format that allows STOMP clients to talk with any message broker that supports the STOMP protocol and as such is a good fit for use with Ably Queues.

Assuming the following queue has been set up, we'll show you a simple example of subscribing to a STOMP queue.

In order to subscribe to messages from this queue you will need:

- The queue name := @UATwBQ:example-queue@ which is made up of your app ID and the name you assigned to your queue
Expand Down

0 comments on commit f692061

Please sign in to comment.