From b335e5b7f116c8b0c2176f18ccf61726c2af540e Mon Sep 17 00:00:00 2001 From: Zixuan Liu Date: Tue, 18 Feb 2025 18:46:41 +0800 Subject: [PATCH] Compatible with existing designs as much as possible Signed-off-by: Zixuan Liu --- pip/pip-398.md | 61 ++++++++++---------------------------------------- 1 file changed, 12 insertions(+), 49 deletions(-) diff --git a/pip/pip-398.md b/pip/pip-398.md index 400221e56fdff..59c9c1b1a4adb 100644 --- a/pip/pip-398.md +++ b/pip/pip-398.md @@ -27,15 +27,11 @@ to the broker, namespace and topic levels, the system reduces the need for expli The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI and API. -# Out scope - -This feature depends on https://github.com/apache/pulsar/pull/23757, which makes the subscription replication nullable -on the consumer level. Otherwise, the broker, namespace and topic levels will be ignored. - # High Level Design Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when -enabled, all consumers under the topic will automatically replicate their subscription states to remote clusters. +enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote +clusters. The priority for the subscription replication configuration is as follows: @@ -108,39 +104,25 @@ public class ServiceConfiguration implements PulsarConfiguration { ### Consumer level -We currently support the `replicateSubscriptionState` flag for new subscription at the consumer level, which can be set -to `true` or `false`. If `true`, we persist `pulsar.replicated.subscription=1L` to the cursor properties, and if -`false`, we remove that. - -To keep the consumer-level configuration, we need to consider the `false` case, so propose using -`pulsar.replicated.subscription=0L` instead of removing the property. - -The consumer-level configuration takes precedence over both the topic, broker and namespace levels, ensuring that -consumer-specific configurations can still override the more general settings. +No changes. When the consumer with `replicateSubscriptionState=true`, the old/new subscription will be snapshot. If `false`, no operation will +be performed. ### Subscription replication applied When the subscription replication is changed on the broker, namespace or topic level, the subscription replication will -be applied to all consumers under the topic. +be applied to all consumers under the broker/namespace/topic. -Please notice that the `org.apache.pulsar.client.admin.Topics.setReplicatedSubscriptionStatus()` can override the -subscription replication configuration for the special subscription, and equals the consumer level. +We exclusively consider the `true` case for replication. If the consumer level is set to `false`, the +replication configuration defined at the topic, namespace, or broker level will be applied to the subscription. + +There is a special case here, if the user intentionally sets `false` at the consumer level, and then set `true` at the +topic/ns/broker level, this may disrupt the user's behavior. This way, we can minimize changes to the Pulsar public API +as much as possible. ## Public-facing Changes ### Public API -### Client API - -Change the set replicated subscription status method in the `org.apache.pulsar.client.admin.Topics` interface to accept the `Boolean` type: - -```java -void setReplicatedSubscriptionStatus(String topic, String subName, Boolean enabled) throws PulsarAdminException; -CompletableFuture setReplicatedSubscriptionStatusAsync(String topic, String subName, Boolean enabled); -``` - -#### REST API - ##### Namespace level - `/{tenant}/{namespace}/replicateSubscriptionState`: enable/disable/remove the subscription replication on the @@ -181,8 +163,6 @@ CompletableFuture setReplicatedSubscriptionStatusAsync(String topic, Strin - true - false - null -- `/{tenant}/{namespace}/{topic}/subscription/{subName}/replicatedSubscriptionStatus`: Change the body type from boolean - to Boolean. ### CLI @@ -203,10 +183,6 @@ CompletableFuture setReplicatedSubscriptionStatusAsync(String topic, Strin subscription replication configuration on the topic level. - `pulsar-admin topicPolicies remove-replicate-subscription-state //` to remove the subscription replication configuration on the topic level. -- `pulsar-admin topicPolicies remove-replicate-subscription-state //` to remove the - subscription replication configuration on the topic level. -- `pulsar-admin topics remove-replicated-subscription-status //` to remove the - subscription replication configuration on the consumer(subscription) level. # Security Considerations @@ -230,20 +206,7 @@ Both write and read operations require the necessary permissions, which already ## Upgrade -Because the consumer's `replicateSubscriptionSate=false` does not persist to the cursor properties, it leads to -compatibility issues: - -- When the existing subscription with `replicateSubscriptionSate=false` on the consumer level, which uses the policy - from the topic, namespace, or broker level. If you want to use the initial configuration, please use the pulsar-admin - API orCLI: - - CLI - ```shell - bin/pulsar-admin topics set-replicated-subscription-status // -s --disable - ``` - - API - ```java - admin.topics().setReplicatedSubscriptionStatus(topic, subName, false); - ``` +None. ## Downgrade / Rollback