Skip to content

Commit

Permalink
Compatible with existing designs as much as possible
Browse files Browse the repository at this point in the history
Signed-off-by: Zixuan Liu <[email protected]>
  • Loading branch information
nodece committed Feb 18, 2025
1 parent 5935c9b commit b335e5b
Showing 1 changed file with 12 additions and 49 deletions.
61 changes: 12 additions & 49 deletions pip/pip-398.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,11 @@ to the broker, namespace and topic levels, the system reduces the need for expli
The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
Pulsar Admin CLI and API.

# Out scope

This feature depends on https://github.com/apache/pulsar/pull/23757, which makes the subscription replication nullable
on the consumer level. Otherwise, the broker, namespace and topic levels will be ignored.

# High Level Design

Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when
enabled, all consumers under the topic will automatically replicate their subscription states to remote clusters.
enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote
clusters.

The priority for the subscription replication configuration is as follows:

Expand Down Expand Up @@ -108,39 +104,25 @@ public class ServiceConfiguration implements PulsarConfiguration {

### Consumer level

We currently support the `replicateSubscriptionState` flag for new subscription at the consumer level, which can be set
to `true` or `false`. If `true`, we persist `pulsar.replicated.subscription=1L` to the cursor properties, and if
`false`, we remove that.

To keep the consumer-level configuration, we need to consider the `false` case, so propose using
`pulsar.replicated.subscription=0L` instead of removing the property.

The consumer-level configuration takes precedence over both the topic, broker and namespace levels, ensuring that
consumer-specific configurations can still override the more general settings.
No changes. When the consumer with `replicateSubscriptionState=true`, the old/new subscription will be snapshot. If `false`, no operation will
be performed.

### Subscription replication applied

When the subscription replication is changed on the broker, namespace or topic level, the subscription replication will
be applied to all consumers under the topic.
be applied to all consumers under the broker/namespace/topic.

Please notice that the `org.apache.pulsar.client.admin.Topics.setReplicatedSubscriptionStatus()` can override the
subscription replication configuration for the special subscription, and equals the consumer level.
We exclusively consider the `true` case for replication. If the consumer level is set to `false`, the
replication configuration defined at the topic, namespace, or broker level will be applied to the subscription.

There is a special case here, if the user intentionally sets `false` at the consumer level, and then set `true` at the
topic/ns/broker level, this may disrupt the user's behavior. This way, we can minimize changes to the Pulsar public API
as much as possible.

## Public-facing Changes

### Public API

### Client API

Change the set replicated subscription status method in the `org.apache.pulsar.client.admin.Topics` interface to accept the `Boolean` type:

```java
void setReplicatedSubscriptionStatus(String topic, String subName, Boolean enabled) throws PulsarAdminException;
CompletableFuture<Void> setReplicatedSubscriptionStatusAsync(String topic, String subName, Boolean enabled);
```

#### REST API

##### Namespace level

- `/{tenant}/{namespace}/replicateSubscriptionState`: enable/disable/remove the subscription replication on the
Expand Down Expand Up @@ -181,8 +163,6 @@ CompletableFuture<Void> setReplicatedSubscriptionStatusAsync(String topic, Strin
- true
- false
- null
- `/{tenant}/{namespace}/{topic}/subscription/{subName}/replicatedSubscriptionStatus`: Change the body type from boolean
to Boolean.

### CLI

Expand All @@ -203,10 +183,6 @@ CompletableFuture<Void> setReplicatedSubscriptionStatusAsync(String topic, Strin
subscription replication configuration on the topic level.
- `pulsar-admin topicPolicies remove-replicate-subscription-state <tenant>/<namespace>/<topic>` to remove the
subscription replication configuration on the topic level.
- `pulsar-admin topicPolicies remove-replicate-subscription-state <tenant>/<namespace>/<topic>` to remove the
subscription replication configuration on the topic level.
- `pulsar-admin topics remove-replicated-subscription-status <tenant>/<namespace>/<topic>` to remove the
subscription replication configuration on the consumer(subscription) level.

# Security Considerations

Expand All @@ -230,20 +206,7 @@ Both write and read operations require the necessary permissions, which already

## Upgrade

Because the consumer's `replicateSubscriptionSate=false` does not persist to the cursor properties, it leads to
compatibility issues:

- When the existing subscription with `replicateSubscriptionSate=false` on the consumer level, which uses the policy
from the topic, namespace, or broker level. If you want to use the initial configuration, please use the pulsar-admin
API orCLI:
- CLI
```shell
bin/pulsar-admin topics set-replicated-subscription-status <tenant>/<namespace>/<topic> -s <subName> --disable
```
- API
```java
admin.topics().setReplicatedSubscriptionStatus(topic, subName, false);
```
None.

## Downgrade / Rollback

Expand Down

0 comments on commit b335e5b

Please sign in to comment.