Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DSIP-9][Feature][Server] Add Raft consensus algorithm registry, remove zookeeper dependency #11144

Closed
wants to merge 9 commits into from
Closed

[DSIP-9][Feature][Server] Add Raft consensus algorithm registry, remove zookeeper dependency #11144

wants to merge 9 commits into from

Conversation

zhuxt2015
Copy link
Contributor

Purpose of the pull request

fix #10874

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

@codecov-commenter
Copy link

codecov-commenter commented Jul 26, 2022 •
edited
Loading

Codecov Report

Merging #11144 (f86f4a4) into dev (d76d6da) will decrease coverage by 0.31%.
The diff coverage is 16.56%.

@@             Coverage Diff              @@
##                dev   #11144      +/-   ##
============================================
- Coverage     40.49%   40.17%   -0.32%     
- Complexity     4887     4951      +64     
============================================
  Files           951      988      +37     
  Lines         37201    37712     +511     
  Branches       4080     4145      +65     
============================================
+ Hits          15065    15152      +87     
- Misses        20613    21021     +408     
- Partials       1523     1539      +16
Impacted Files Coverage Δ
...erver/master/event/TaskRetryStateEventHandler.java 0.00% <0.00%> (ø)
...ler/server/master/event/TaskStateEventHandler.java 0.00% <0.00%> (ø)
...ver/master/event/TaskTimeoutStateEventHandler.java 0.00% <0.00%> (ø)
...server/master/event/WorkflowStartEventHandler.java 0.00% <0.00%> (ø)
...server/master/event/WorkflowStateEventHandler.java 0.00% <0.00%> (ø)
...master/event/WorkflowTimeoutStateEventHandler.java 0.00% <0.00%> (ø)
.../server/master/runner/WorkflowExecuteRunnable.java 7.79% <0.00%> (ø)
...r/server/master/service/MasterFailoverService.java 54.47% <0.00%> (-2.31%) ⬇️
...dolphinscheduler/plugin/task/api/AbstractTask.java 0.00% <ø> (ø)
...olphinscheduler/plugin/task/api/TaskConstants.java 0.00% <0.00%> (ø)
... and 59 more

Help us with your feedback. Take ten seconds to tell us how you rate us.

@davidzollo
Copy link
Contributor

please add related license, refer to https://dolphinscheduler.apache.org/en-us/community/development/DS-License.html

leo-lao
leo-lao reviewed Jul 27, 2022
View reviewed changes
Copy link

@leo-lao leo-lao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally good job!

...aft/src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/EphemeralNodeManager.java Outdated Show resolved Hide resolved
...aft/src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/EphemeralNodeManager.java Outdated Show resolved Hide resolved
...aft/src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/EphemeralNodeManager.java Outdated Show resolved Hide resolved
...gistry-raft/src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/RaftRegistry.java Show resolved Hide resolved
@zhuxt2015
Copy link
Contributor Author

please add related license, refer to https://dolphinscheduler.apache.org/en-us/community/development/DS-License.html

Done.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Aug 1, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

ruanwenjun
ruanwenjun reviewed Aug 1, 2022
View reviewed changes
dolphinscheduler-dist/release-docs/LICENSE Show resolved Hide resolved
dolphinscheduler-master/pom.xml
@@ -225,6 +229,10 @@
<artifactId>jersey-core</artifactId>
<groupId>com.sun.jersey</groupId>
</exclusion>
<exclusion>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to define the protobuf-java version in bom. Then you don't need to exclude it in everywhere.

...cheduler-registry/dolphinscheduler-registry-plugins/dolphinscheduler-registry-raft/README.md Show resolved Hide resolved
...cheduler-registry/dolphinscheduler-registry-plugins/dolphinscheduler-registry-raft/README.md Show resolved Hide resolved
...gistry-raft/src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/RaftRegistry.java Show resolved Hide resolved
...gistry-raft/src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/RaftRegistry.java Show resolved Hide resolved
...t/src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/RaftRegistryProperties.java Show resolved Hide resolved
...t/src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/RaftRegistryProperties.java
@ConfigurationProperties(prefix = "registry")
public class RaftRegistryProperties {
private String clusterName;
private String serverAddressList;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to add validator for this class?

...src/main/java/org/apache/dolphinscheduler/plugin/registry/raft/SubscribeListenerManager.java Show resolved Hide resolved
@caishunfeng
Copy link
Contributor

Hi @zhuxt2015 please check the review comment and resolve conflicts.

@zhuxt2015
Copy link
Contributor Author

Hi @zhuxt2015 please check the review comment and resolve conflicts.

OK, testing in local

@EricGao888 EricGao888 self-requested a review December 5, 2022 05:11
@EricGao888 EricGao888 added 3.2.0 for 3.2.0 version feature new feature labels Dec 5, 2022
@EricGao888 EricGao888 added this to the 3.2.0 milestone Dec 5, 2022
@EricGao888
Copy link
Member

@zhuxt2015 This is a significant improvement. By using jRaft as registry plugin, it seems registry center will no more need to rely on external third-party components such as mysql, etcd, or zookeeper. May I ask whether you have time to follow up with this PR? Thanks : )

@EricGao888 EricGao888 mentioned this pull request Jan 13, 2023
Closed
1 task
@EricGao888 EricGao888 mentioned this pull request Feb 9, 2023
Open
12 tasks
@EricGao888
Copy link
Member

@zhuxt2015 Hi, would u like to continue with this work? If you are no more interested or do not have time for it, do u mind me helping with the rest?

@zhongjiajie zhongjiajie modified the milestones: 3.2.0, 3.3.0 Aug 30, 2023
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 9, 2024

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

@github-actions github-actions bot added the Stale label Feb 9, 2024
@github-actions GitHub Actions
Copy link

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.

@github-actions github-actions bot closed this Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ruanwenjun ruanwenjun ruanwenjun left review comments

@leo-lao leo-lao leo-lao left review comments

@caishunfeng caishunfeng caishunfeng left review comments

@kezhenxu94 kezhenxu94 Awaiting requested review from kezhenxu94

@EricGao888 EricGao888 Awaiting requested review from EricGao888

Assignees

@zhuxt2015 zhuxt2015

Labels
3.2.0 for 3.2.0 version backend feature new feature Stale
Projects
None yet
Milestone
3.3.0
Development

Successfully merging this pull request may close these issues.

[DSIP-9][Feature][Server] Add new registry plugin based on raft
8 participants
@zhuxt2015 @codecov-commenter @davidzollo @caishunfeng @EricGao888 @leo-lao @ruanwenjun @zhongjiajie