Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use OriginalRunID instead of RunID to generate child workflow IDs #1803

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

gow
Copy link
Contributor

@gow gow commented Feb 5, 2025

What was changed

We now use workflowInfo.OriginalRunID in parent when it is generating a child workflow ID.

Why?

We need this change to enable resetting of workflows that have pending children. What this means is Go SDK should be able to successfully replay StartChildWorkflowExecutionInitiated event in a parent that was reset.
Without this change Go SDK will run into non-determinism error since the child ID it generates (based on new run ID) doesn't match with the ID available in the StartChildWorkflowExecutionInitiated event. Since OriginalRunID stays the same across resets, with this change the SDK can successfully replay StartChildWorkflowExecutionInitiated events.

Checklist

  1. Closes
    N/A

  2. How was this tested:
    Manually tested it (unit tests coming)

  3. Any docs updates needed?
    N/A

@gow gow requested a review from a team as a code owner February 5, 2025 07:04
@cretz
Copy link
Member

cretz commented Feb 5, 2025

Questions here:

  1. Can we confirm what Java does here?
  2. IIUC Core-based languages make child workflow IDs with a seeded random that changes seed on reset midway through execution, meaning post-reset child workflow IDs would be different than they were on original run but pre-reset ones are the same. This was done very much by intention. Should we consider a similar approach in Go/Java that has a "current run ID" that can change midway through execution?
  3. Can we assess the impact of behavior changes like this on existing users with existing expectations? I am guessing there is no impact because you could not reset to before child workflows?
  4. Can you confirm OriginalRunID always matches RunID at this point of code today? Meaning since resets were never allowed over child workflows, it is impossible to have these two values different at child workflow creation time today?

@gow
Copy link
Contributor Author

gow commented Feb 5, 2025

Questions here:

  1. Can we confirm what Java does here?

Java seems to be generating UUIDs when an ID is not provided in ChildWorkflow options. Additionally when I replayed an existing StartChildWorkflowExecutionInitiated event (by resetting to a point after the child completion) it didn't run into non-determinism error. Go on the other hand ran into non-det error complaining about workflowID. ("Error":"[TMPRL1100] unknown command CommandType: ChildWorkflow, ID: 0174afb2-0213-4644-9214-8b977310cc9f_14, possible causes are nondeterministic workflow definition code or incompatible change in the workflow definition")

  1. IIUC Core-based languages make child workflow IDs with a seeded random that changes seed on reset midway through execution, meaning post-reset child workflow IDs would be different than they were on original run but pre-reset ones are the same. This was done very much by intention. Should we consider a similar approach in Go/Java that has a "current run ID" that can change midway through execution?

Absolutely yes. Java seems to be already doing this (at least in my testing). Looks like it changes the seed at reset-point?
If this is an easier change in Go-SDK, I'd rather do it. Otherwise, we still need this fix since Go-SDK is currently broken and the upcoming reset changes will increase the exposure to this bug (See below)

  1. Can we assess the impact of behavior changes like this on existing users with existing expectations? I am guessing there is no impact because you could not reset to before child workflows?

Yes. So far all server versions have prevented resetting of workflows when the reset point falls in between child Init & Completed events (i.e within the lifespan of child). So no potential user impact or backward compatibility issues there.
However the server has always allowed workflows to be reset when the reset-point chosen is outside the child init & completed events. In cases where reset-point is after child complete event, workers with Go SDK run into non-det error while other SDKs seems to simply reapply events (child Init, Start & complete) without any issues. In that regard this PR is more of a bug fix in Go-SDK. And now with the upcoming server changes to allow reset-points to be in between Init-Completed, it's important that Go-SDK successfully replay a StartChildWorkflowExecutionInitiated event.

  1. Can you confirm OriginalRunID always matches RunID at this point of code today? Meaning since resets were never allowed over child workflows, it is impossible to have these two values different at child workflow creation time today?

Yes it is possible to have these two values different at child creation time. When the reset-point is outside the child's lifespan (Init - Completed) then the server has always allowed resets. So in cases when the reset-point is before the child-init, the parent will try to create a child again and this time OriginalRunID is different from RunID.

@cretz
Copy link
Member

cretz commented Feb 5, 2025

@gow - thanks for this info! So reading all of that I am seeing two things:

  1. We need to fix Go to be at least a little like other SDKs. We have two options:
    1. Start offering a re-seeded random to users and we use it for child IDs in Go with an SDK flag to only apply to new workflows
    2. Make available (maybe only to us) a "current run ID" that is updated when reset that is used for child workflow IDs (no SDK flag needed for compatibility)
  2. We are not concerned with backwards compatibility of only changing Go since it never worked anyways after reset, so long as the non-reset logic remains working

For 1, I will start an internal discussion and invite you. Regardless, I am not sure the desired outcome is what is in this PR.

@gow
Copy link
Contributor Author

gow commented Feb 5, 2025

The desired outcome of this PR is for Go-SDK to be able to replay StartChildWorkflowExecutionInitiated events like other SDKs. Currently it fails. Using OriginalRunID will fix this issue and unblock reset feature development. But I understand it doesn't bring Go-SDK to be fully be at parity with other SDKs.

@gow
Copy link
Contributor Author

gow commented Feb 7, 2025

Putting this on hold. Will discuss internally and work on a better fix.

@gow gow marked this pull request as draft February 7, 2025 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants