Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 17 additions & 18 deletions references/core/versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,20 @@ else:

**Phase 1: Patch In**

- Add both old and new code paths
- New workflows take new path, old workflows take old path
- Add both old and new code paths.
- New executions take the new path; histories that predate the patch replay the old path.

**Phase 2: Deprecate**

- After all old workflows complete, remove old code
- Keep deprecation marker for history compatibility
- Wait until executions that can return the retired version have left retention before removing that branch or raising the minimum supported version.
- Keep the Workflow's deprecation/version marker at the same deterministic location for history compatibility.
Comment on lines +49 to +50

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth mentioning you only need to wait until they've left retention if you query them after they've completed, which is somewhat rare. If you know you don't do that, then you can remove the code when they've all completed, but before they've been archived/deleted.


**Phase 3: Remove**

- After all deprecated workflows complete
- Remove patch entirely, only new code remains
- Removing the final marker is a separate operation with patching-API-specific rules beyond branch retirement.
- Retire the patch/change identifier permanently when the patching API requires it; do not reuse an identifier whose first marker call was removed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems unhelpful. It's basically saying "you can remove the marker when you should", whereas the old text tried to tell you when "should" is. Some combination of old/new would be appropriate here.


See the language-specific versioning reference for exact marker, visibility, retention, and replay rules.

### When to Use

Expand Down Expand Up @@ -192,31 +194,28 @@ For long-running Workflows that cannot use Continue-as-New (e.g., compliance aud

## Best Practices

1. **Check for open executions** before removing old code
1. **Wait for affected executions to leave retention** before removing old code

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment about the difference between open & retention

2. **Use descriptive patch IDs** (e.g., "add-fraud-check" not "patch-1")
3. **Deploy incrementally**: patch → deprecate → remove
4. **Test replay compatibility** before deploying changes
5. **Monitor old workflow counts** during migration

## Finding Workflows by Version

```bash
# Find workflows with specific patch
temporal workflow list --query \
'WorkflowType = "OrderWorkflow" AND TemporalChangeVersion = "add-fraud-check"'
Different patching APIs encode `TemporalChangeVersion` values differently. A pre-patch execution has no marker for that patch, but marker absence can also mean an optional patch site has not executed. Follow the language-specific guide to query and classify the old-version population rather than guessing a marker value.

# Find pre-patch workflows
temporal workflow list --query \
'WorkflowType = "OrderWorkflow" AND TemporalChangeVersion IS NULL'
Worker Deployment versions use a separate Search Attribute:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to have dropped a bunch of context about how to do queries with TemporalChangeVersion?

The previous paragraph refers to it without explaining what it is


# Find workflows on specific worker version
```bash
temporal workflow list --query \
'TemporalWorkerDeploymentVersion = "my-service:v1.0.0"'
```

## Common Mistakes

1. **Removing old code too early** - Breaks replaying workflows
2. **Not testing with replay** - Catches issues before production
3. **Patching non-Command changes** - Unnecessary complexity
4. **Forgetting to deprecate** - Accumulates dead code
2. **Treating branch removal and final marker removal as identical changes** - Both can have retention gates, while final marker removal can also retire the identifier permanently
3. **Treating marker absence as proof of an old branch** - Optional patch sites and other markers make absence ambiguous
4. **Not testing with replay** - Replay catches compatibility failures before production
5. **Patching non-Command changes** - Unnecessary complexity
6. **Forgetting to deprecate** - Accumulates dead code
10 changes: 6 additions & 4 deletions references/dotnet/versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ public class OrderWorkflow

**Step 2: Deprecate the Patch**

Once all pre-patch Workflow Executions have completed:
After all pre-patch Workflow Executions have left retention:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same on open/retention


```csharp
[Workflow]
Expand All @@ -92,7 +92,7 @@ public class OrderWorkflow

**Step 3: Remove the Patch**

After all workflows with the deprecated patch marker have completed, remove the `DeprecatePatch()` call entirely:
In a later deployment, after the deprecation rollout is complete and the pre-patch Workflow Executions have left retention, remove the `DeprecatePatch()` call entirely. A deprecated marker is intentionally safe to omit during replay, so marker-bearing executions do not require another retention wait:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This last sentence is hard to parse. I think it means something like "If a workflow's history contains a deprecated marker, it can safely be replayed by code without any of the old code or the patch call."


```csharp
[Workflow]
Expand Down Expand Up @@ -121,11 +121,13 @@ Use List Filters to find workflows with specific patch versions:
temporal workflow list --query \
'WorkflowType = "OrderWorkflow" AND ExecutionStatus = "Running" AND TemporalChangeVersion = "add-fraud-check"'

# Find running workflows without any patch (pre-patch versions)
# Find retained workflows without any patch marker (simple pre-patch case)
temporal workflow list --query \
'WorkflowType = "OrderWorkflow" AND ExecutionStatus = "Running" AND TemporalChangeVersion IS NULL'
'WorkflowType = "OrderWorkflow" AND TemporalChangeVersion IS NULL'
```

`TemporalChangeVersion IS NULL` is reliable only when no other patch marker could make the attribute non-null. For multiple or optional patch sites, inspect all retained executions of the Workflow Type and classify the exact marker set and Event History. A zero running count identifies no immediate live blocker, but it does not prove that pre-patch histories have left retention. Replay selected histories to test compatibility; sampled replay does not prove the retained population is empty.

Comment on lines +129 to +130

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`TemporalChangeVersion IS NULL` is reliable only when no other patch marker could make the attribute non-null. For multiple or optional patch sites, inspect all retained executions of the Workflow Type and classify the exact marker set and Event History. A zero running count identifies no immediate live blocker, but it does not prove that pre-patch histories have left retention. Replay selected histories to test compatibility; sampled replay does not prove the retained population is empty.
`TemporalChangeVersion IS NULL` is reliable only when no other same-id patch marker could make the attribute non-null. For multiple or optional patch sites, inspect all retained executions of the Workflow Type and classify the exact marker set and Event History. A zero running count identifies no immediate live blocker, but it does not prove that pre-patch histories have left retention. Replay selected histories to test compatibility; sampled replay does not prove the retained population is empty.

## Workflow Type Versioning

For incompatible changes, create a new Workflow Type instead of using patches:
Expand Down
109 changes: 90 additions & 19 deletions references/go/versioning.md
Comment thread
robzienert marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ For conceptual overview and guidance on choosing an approach, see `references/co

## GetVersion API

`workflow.GetVersion` safely performs backwards-incompatible changes to Workflow Definitions. It returns the version to branch on, recording the result as a marker in the Event History.
[`workflow.GetVersion`](https://pkg.go.dev/go.temporal.io/sdk/workflow#GetVersion) safely performs backwards-incompatible changes to Workflow Definitions. It returns the version to branch on, recording the result as a marker in the Event History.

```go
v := workflow.GetVersion(ctx, "changeID", workflow.DefaultVersion, maxSupported)
Expand All @@ -15,9 +15,11 @@ v := workflow.GetVersion(ctx, "changeID", workflow.DefaultVersion, maxSupported)
- `maxSupported`: current/newest version
- Returns `maxSupported` for new executions; returns the recorded version on replay

### Three-Step Lifecycle
### Retirement Lifecycle

**Step 1: Add GetVersion with both code paths**
Treat `GetVersion` cleanup as replay-safety work, not ordinary dead-code removal. Removing an old branch and deleting the first marker call are separate changes with different gates.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Treat `GetVersion` cleanup as replay-safety work, not ordinary dead-code removal. Removing an old branch and deleting the first marker call are separate changes with different gates.
`GetVersion` requires care to remain replay safe. Removing an old branch and deleting the first marker call are separate changes with different gates.


#### Step 1: Add `GetVersion` with both code paths

Original code calls `ActivityA`. You want to replace it with `ActivityC`:

Expand All @@ -32,26 +34,82 @@ if v == workflow.DefaultVersion {
}
```

For new executions, `GetVersion` returns `1` and records a marker. For replay of pre-change workflows (no marker), it returns `DefaultVersion` (`-1`).
For new executions, `GetVersion` returns `1` and records a marker. For replay of pre-change workflows with no marker for `Step1`, it returns `workflow.DefaultVersion` (`-1`). **`DefaultVersion` is not version `0`**; do not search for `Step1-0` unless the code actually recorded version `0`.

#### Step 2: Inventory and prove the old version has left retention

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section got really big in Go for some reason, but not the others (the smaller versions seem preferable. There's a lot of redundancy with the non-language-specific file).


Before raising `minSupported`, record enough context to review each change independently:

| Field | Example |
|---|---|
| Workflow type | `OrderWorkflow` |
| Change ID | `Step1` |
| Old version | `workflow.DefaultVersion`, `0`, or `1` |
| Retained version | `1` |
| Branch effect | Activity, child workflow, timer, signal/update flow |
| Call location | Startup, loop, selector, or handler |

For a recorded integer version, query the exact old `changeID-version` value across open and closed executions still in retention. For example, when retiring version `1` after a later `1 → 2` migration:

```bash
temporal workflow list --query \
'WorkflowType = "OrderWorkflow" AND TemporalChangeVersion = "Step1-1"'
```

A nonzero result means version `1` has not left retention. Keep that branch. In the initial `workflow.DefaultVersion → 1` example above, the old branch has no recorded integer value, so use the marker-absence procedure instead.

Histories from before the first `GetVersion` call have no `TemporalChangeVersion` entry for that change ID. If this is the workflow's only patch marker, find the simple pre-marker population still in retention with:

```bash
temporal workflow list --query \
'WorkflowType = "OrderWorkflow" AND TemporalChangeVersion IS NULL'
```

For workflows with multiple patch markers, query all running executions of the workflow type and inspect their `TemporalChangeVersion` values for the absence of the retained marker. Do the same when the `GetVersion` call is on an optional path: absence can mean either a pre-marker history or an execution that has not reached that call.

Treat marker absence as an unverified candidate set. Before raising `minSupported`, classify every candidate using its history or establish another authoritative population bound that proves every execution which could return the old version has left retention. Representative replay is still required for compatibility evidence, but it cannot classify an unexamined population.

A zero count of running old-version executions is necessary operational evidence, but it is not the retirement gate: closed histories can still be replayed or reset while retained. Do not raise `minSupported` until executions on the old version have left retention. A replay test can show whether selected histories are compatible; it does not prove that the old-version population has left retention.

**Step 2: Remove old branch (increase minSupported)**
#### Step 3: Remove the old branch but keep the first marker call

After all `DefaultVersion` Workflow Executions have completed:
After executions on the old version have left retention, raise `minSupported` and collapse the branch:

```go
v := workflow.GetVersion(ctx, "Step1", 1, 1)
// Only the new code path remains
_ = workflow.GetVersion(ctx, "Step1", 1, 1)
err = workflow.ExecuteActivity(ctx, ActivityC, data).Get(ctx, &result1)
```

Keep the `GetVersion` call even with a single branch. This ensures:
Keep the first `GetVersion` call for the change ID at the same deterministic point in the Workflow. Moving it across other Commands can itself break replay. The pinned call causes an older unsupported history to fail at the version boundary instead of continuing with the wrong behavior, and it leaves a safe place to add a later version.

1. If an older execution replays on this code, it fails fast instead of proceeding incorrectly
2. If you need further changes, you just bump `maxSupported`
Only the first call for a change ID needs to be retained. Subsequent calls with that same change ID return the recorded value and can be removed once their surrounding branches are gone.

**Step 3: Further changes (bump maxSupported)**
Deploy and replay representative open and closed histories before making another retirement change.

Later, replace `ActivityC` with `ActivityD`:
#### Step 4: Optionally remove the first marker call

Deleting the first `GetVersion` call is stricter than removing an old branch. Do it only when:

1. All executions with older versions have left retention.
2. Replay verification covers the retained behavior.
3. You will permanently retire that change ID.

After deleting the call, you **must never reuse** `Step1`. A future change at the same code location needs a new change ID and must start again from `workflow.DefaultVersion`:

```go
v := workflow.GetVersion(ctx, "Step1-activity-d", workflow.DefaultVersion, 1)
if v == workflow.DefaultVersion {
err = workflow.ExecuteActivity(ctx, ActivityC, data).Get(ctx, &result1)
} else {
err = workflow.ExecuteActivity(ctx, ActivityD, data).Get(ctx, &result1)
}
```

When in doubt, leave the pinned marker call in place. It is small, explicit replay-safety state.

#### Further changes before final marker removal

To replace `ActivityC` with `ActivityD` while retaining `Step1`, bump `maxSupported`:

```go
v := workflow.GetVersion(ctx, "Step1", 1, 2)
Expand All @@ -62,13 +120,25 @@ if v == 1 {
}
```

After all version-1 executions complete, collapse again:
After executions on version `1` have left retention, collapse again while preserving the first call:

```go
_ = workflow.GetVersion(ctx, "Step1", 2, 2)
err = workflow.ExecuteActivity(ctx, ActivityD, data).Get(ctx, &result1)
```

#### Replay verification checklist

Before raising `minSupported` or deleting a marker call:

- Replay saved histories for every known version, including `DefaultVersion` when applicable.
- Replay both open and closed histories, especially long-running and error-path executions.
- Disable test caching so the evidence comes from the current code.
- Inspect targeted histories when marker absence is ambiguous.
- Record the visibility query, result count, replay command, and code version in the change review.

Replay and visibility answer different questions: replay tests compatibility for the histories selected, while visibility helps estimate whether an old-version population still exists. Use both; neither substitutes for the other.

### Using GetVersion in Loops

The return value for a given `changeID` is immutable once recorded. In loops, append the iteration number to the `changeID`:
Expand Down Expand Up @@ -269,11 +339,12 @@ if workflow.GetInfo(ctx).GetTargetWorkerDeploymentVersionChanged() {

## Best Practices

1. **Keep GetVersion calls** even when only a single branch remains -- it guards against stale replays and simplifies future changes
2. **Use `TemporalChangeVersion` search attribute** to find Workflows running on old versions:
1. **Keep the first `GetVersion` call** when collapsing to a single branch unless the final retention and change-ID retirement gates are satisfied
2. **Query the exact recorded value** using `changeID-version`:
```bash
temporal workflow list --query \
'WorkflowType = "MyWorkflow" AND ExecutionStatus = "Running" AND TemporalChangeVersion = "Step1"'
'WorkflowType = "MyWorkflow" AND TemporalChangeVersion = "Step1-1"'
```
3. **Test with replay** before removing old branches to verify determinism is preserved
4. **Prefer Worker Versioning** for large-scale deployments to avoid accumulating patching branches
3. **Treat `DefaultVersion` as marker absence, not version `0`**, and inspect histories when absence is ambiguous
4. **Test with replay** before removing old branches or marker calls, while remembering that sampled replay does not prove a version has left retention
5. **Prefer Worker Versioning** for large-scale deployments to avoid accumulating patching branches
23 changes: 14 additions & 9 deletions references/java/versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ public class OrderWorkflowImpl implements OrderWorkflow {

**Step 2: Remove Old Code Path**

Once all pre-patch Workflow Executions have completed, remove the old branch and set `minSupported` to `1`:
After all pre-patch Workflow Executions have left retention, remove the old branch and set `minSupported` to `1`:

```java
public class OrderWorkflowImpl implements OrderWorkflow {
Expand All @@ -85,7 +85,7 @@ public class OrderWorkflowImpl implements OrderWorkflow {

**Step 3: Remove the Patch**

After all workflows with the patch marker have completed, remove the `getVersion` call entirely:
In a later deployment, after all executions with older versions have left retention, you may remove the first `getVersion` call entirely. Once removed, permanently retire that change ID; a future change at the same location must use a new ID starting from `Workflow.DEFAULT_VERSION`:

```java
public class OrderWorkflowImpl implements OrderWorkflow {
Expand All @@ -99,7 +99,7 @@ public class OrderWorkflowImpl implements OrderWorkflow {

### Recording TemporalChangeVersion Search Attribute

Unlike the Python and TypeScript SDKs, the Java SDK does **not** automatically record the `TemporalChangeVersion` search attribute. You must manually upsert it:
Java `getVersion` records a history marker, but `TemporalChangeVersion` visibility requires either the Java SDK 1.30+ `enableUpsertVersionSearchAttributes` option or a manual upsert. The following example uses a manual upsert:

```java
import io.temporal.workflow.Workflow;
Expand All @@ -114,9 +114,12 @@ public class OrderWorkflowImpl implements OrderWorkflow {
public String run(Order order) {
int version = Workflow.getVersion("add-fraud-check", Workflow.DEFAULT_VERSION, 1);

// Manually record for query filtering
Workflow.upsertTypedSearchAttributes(
TEMPORAL_CHANGE_VERSION.valueSet(List.of("add-fraud-check-1")));
// Pre-patch histories return DEFAULT_VERSION and have no version marker.
if (version != Workflow.DEFAULT_VERSION) {
Workflow.upsertTypedSearchAttributes(
TEMPORAL_CHANGE_VERSION.valueSet(
List.of("add-fraud-check-" + version)));
}

if (version >= 1) {
activities.checkFraud(order);
Expand All @@ -126,13 +129,15 @@ public class OrderWorkflowImpl implements OrderWorkflow {
}
```

Query with:
When retiring a recorded integer version, query its exact `changeID-version` value across open and closed executions still in retention:

```bash
temporal workflow list --query \
'TemporalChangeVersion = "add-fraud-check-1" AND ExecutionStatus = "Running"'
'WorkflowType = "OrderWorkflow" AND TemporalChangeVersion = "add-fraud-check-1"'
```

`Workflow.DEFAULT_VERSION` is marker absence, not version `0`. If this is the only version marker, find the simple pre-marker population with `TemporalChangeVersion IS NULL`. For multiple or optional version sites, inspect all retained executions of the Workflow Type and classify the exact marker set and Event History. A zero running count identifies no immediate live blocker, but it does not prove that older histories have left retention. Replay selected histories to test compatibility; sampled replay does not prove the retained population is empty.

## Workflow Type Versioning

For incompatible changes, create a new Workflow Type:
Expand Down Expand Up @@ -318,7 +323,7 @@ if (Workflow.getInfo().isTargetWorkerDeploymentVersionChanged()) {
1. **Check for open executions** before removing old code paths
2. **Use descriptive change IDs** that explain the change (e.g., `"add-fraud-check"` not `"patch-1"`)
3. **Deploy patches incrementally**: patch, remove old path, remove `getVersion`
4. **Manually upsert `TemporalChangeVersion`** search attribute when using `getVersion` if you need query filtering
4. **Enable automatic `TemporalChangeVersion` upserts on Java 1.30+** when query filtering is needed; otherwise upsert the returned non-default version manually
5. **Use PINNED for short workflows** to simplify version management
6. **Use AUTO_UPGRADE with patching** for long-running workflows that need updates
7. **Generate Build IDs from code** (git hash) to ensure changes produce new versions
Loading