Let each workflow carry its own version instead of one DefaultVersion for the whole graph

[mgoelz]

Description

A sub-workflow cannot carry its own version. Every sub-orchestration in a workflow graph falls back to
the worker's single Versioning.DefaultVersion, because the extension never supplies one; the root's
version comes from the client at schedule time. So there is exactly one version stamp per worker for the
whole graph, and no way to distinguish the workflow that changed from the ones that didn't.

A topology change to one leaf workflow therefore invalidates every in-flight run of the graph. For an
application that has opted into worker versioning, the generated-orchestrator model pushes it to
MatchStrategy.Strict (see below), and the previous worker deployment must then stay alive until the
whole population drains. Our runs park on RequestPort gates
waiting for humans, for days or weeks, so a one-line change to a leaf workflow pins an old worker
deployment in every environment for as long as the longest outstanding human wait.

The execution model already has the isolation — each sub-workflow runs as its own sub-orchestration.
Only the version stamp is shared.

The runtime already supports this

Every piece of this exists in Microsoft.DurableTask; none of it is reachable from the extension.

1. Per-sub-orchestration versions — SubOrchestrationOptions.Version, present since at least 1.18.0.
   A sub-orchestration's version is independent of its parent's, falling back to the worker's
   DefaultVersion only when the caller passes nothing (v1.25.0 TaskOrchestrationContextWrapper):

   string version = options?.Version is { } v ? v.Version : this.GetDefaultVersion();

2. Several versions of one orchestration name in one worker — AddOrchestratorFunc(name, version, factory),
   added in 1.25.0 (durabletask-dotnet#695).
   Dispatch routes each work item to its matching version, falling back to an unversioned registration
   when the name has none. UseWorkItemFilters() follows suit — per #695 it emits the concrete distinct
   version set actually registered for each name (Strict overrides it with the single worker version) —
   so the filters cooperate with multi-version registration rather than fighting it.

Together these take the drain off the deployment path: keep the old and new implementations of the changed
workflow registered side by side under CurrentOrOlder, and every in-flight instance keeps replaying
against the code it started on. No second worker deployment, and nothing to wait for before shipping. (The
old graph does have to stay registered until its instances drain — removing it early yields
OrchestratorTaskNotFound under #695's dispatch rule — but that is a later cleanup step, not a gate on
the deploy.)

Activities need the same treatment, and this is the easy part to miss. They inherit the parent
instance's version, since they execute in the parent's history — so two versions of one workflow dispatch
the same activity name at two different versions. With today's name-only activity registration, #695's
fallback rule resolves both to the single unversioned implementation, and a 1.2.0 child would execute
1.3.0's executor code: exactly the nondeterminism this issue is trying to prevent. #695 provides
AddActivityFunc(name, version, factory) for precisely this case.

Where the extension closes the door

Two call sites, both on main @ ad941ef:

1. DurableExecutorDispatcher.cs#L203
— the two-argument overload, so no version can be supplied:

DurableWorkflowResult? workflowResult = await context.CallSubOrchestratorAsync<DurableWorkflowResult?>(
    orchestrationName,
    workflowInput).ConfigureAwait(true);

2. ServiceCollectionExtensions.cs#L255
— name-only registration, so the multi-version registry is never engaged:

registry.AddOrchestratorFunc<DurableWorkflowInput<object>, DurableWorkflowResult>(
    registration.OrchestrationName,
    (context, input) => RunWorkflowOrchestrationAsync(context, input, durableOptions));

Applications can't work around it either: AddWorkflow takes no version, and DurableWorkflowRunner is
internal sealed with an internal RunWorkflowOrchestrationAsync.

Why we're on Strict, and why this ask doesn't depend on it

Worth separating two things the word "versioning" covers: per-task registration versioning (which
implementation runs for a given (name, version)) and worker admission policy (UseVersioning's
MatchStrategy, which instances a worker will accept at all). This issue is about the first.

They really are independent — MatchStrategy defaults to VersionMatchStrategy.None
(DurableTaskWorkerOptions.cs), and the SDK's own ActivityVersioningSample at v1.25.0 registers
versioned orchestrators and activities and schedules across them without calling UseVersioning at
all. So per-workflow versions would be useful to the extension regardless of what admission policy an
application picks; nothing here forces one on anyone.

For context on why we ended up at Strict + Reject specifically: CurrentOrOlder assumes the
orchestrator author can write a context.Version branch to keep an older instance's replay
deterministic. Here the orchestrator body is generated from the graph, so there is nowhere to put that
branch, and a newer worker accepting an older instance faults it on replay. Strict avoids the fault;
the price is that the version stamp is all-or-nothing across the graph. Per-workflow versions remove the
need for the branch entirely — each version gets its own registered graph — which is what would make
CurrentOrOlder viable for a generated-orchestrator model.

Proposal

Scope: standalone workers. The Azure Functions hosting surface can't take this as-is and is called out
separately below.

Prerequisite: the extension pins Microsoft.DurableTask.* to 1.18.0
(dotnet/Directory.Packages.props), and the version-aware AddOrchestratorFunc(name, version, factory) /
AddActivityFunc(name, version, factory) overloads only exist from 1.25.0. So none of the below
compiles until that pin moves; the bump and its compatibility validation are part of the work, not a
precondition someone else handles. (For what it's worth, we moved our own app from 1.18.0 to 1.25.0 with
no code changes and no test failures.)

• accept an optional version with the workflow — AddWorkflow(workflow, version), or a Version property
  on Workflow if that composes better with the declarative and Python surfaces;
• register orchestrations via AddOrchestratorFunc(name, version, factory), and their activities via
  AddActivityFunc(name, version, factory), when a version is present;
• schedule the child at its registered version from ExecuteSubWorkflowAsync (see the sample below);
• carry the version through every name-keyed structure in registration, not just the one. At ad941ef
  there are four, and missing any single one silently drops the second version rather than failing loudly:
  • DurableWorkflowOptions._workflows, keyed on workflow.Name;
  • BuildWorkflowRegistrationRecursive's registeredOrchestrations —
    if (!registeredOrchestrations.Add(orchestrationName)) { return; } skips a duplicate name by early
    return, so the second version is never registered and its sub-workflow subtree is never walked;
  • BuildWorkflowRegistration's registeredActivities (a HashSet<string> shared across the whole
    graph walk) together with the name-only AddActivityFunc above;
  • ExecutorRegistry.Register — _executors.TryAdd(executorName, ...), first-writer-wins and never
    evicted. This is the registry AddWorkflow silently overwrites an existing workflow registered under the same name #50 identifies as the source of "B's topology while executing A's code";
    two versions of one workflow share executor names and would hit it by construction.
• version the top-level start paths too, not just child dispatch. DurableWorkflowClient.RunAsync and
  StreamAsync call ScheduleNewOrchestrationInstanceAsync with StartOrchestrationOptions carrying only
  an instance id, so a root's version comes solely from the client's single UseDefaultVersion. Once
  top-level workflows carry their own versions that default is wrong for all but one of them — schedule
  workflow A (v1.1) while the client default is v1.4 and exact (name, version) dispatch finds no
  implementation. These surfaces should resolve and pass the registered version, and the design needs an
  answer for what a name-only start does when a name has several versions (newest wins? require an
  explicit version? configurable?).

Backward compatible: an unversioned workflow behaves exactly as today, which is also #695's
unversioned-fallback rule.

Azure Functions hosting needs its own answer

Same-name multi-version registration is explicitly unsupported in Azure Functions per #695, which emits
DURABLE3004 for it — function names derive from the durable task name, so two same-named tasks generate
colliding triggers. The extension's own DurableWorkflowsFunctionMetadataTransformer has exactly that
shape: it walks workflowOptions.Workflows, derives WorkflowNamingHelper.ToOrchestrationFunctionName(workflow.Key)
and the workflows/{workflow.Key}/run HTTP route from the workflow name alone, and dedupes through a
registeredFunctions HashSet<string>. Two versions of one workflow would collapse to a single set of
triggers.

So this proposal is scoped to standalone workers. The Functions surface needs either its own design or an
explicit, early failure — rejecting a versioned workflow under Functions hosting with a message pointing
at DURABLE3004 would be far better than the silent collapse the transformer would produce today.

Worth settling against #50 / PR #66 first

Open PR #66 makes AddWorkflow
throw on a duplicate name, fixing #50.
Right fix — but it hardens name uniqueness, the invariant this proposal needs widened, since hosting v1
and v2 of a changed workflow means two registrations sharing a name.

Keying on (name, version) reconciles both: duplicate unversioned names still throw (#50 stays fixed),
distinct versions under one name become legal, and a true (name, version) collision still throws. The
same widening applies to the other three name-keyed structures listed above, ExecutorRegistry in
particular — it is the one #50 actually blames, and a name-only key there defeats the purpose even if
AddWorkflow is fixed. Cheap to align while #66 is open; a breaking change afterwards.

Narrower fallback

Exposing a hook to supply SubOrchestrationOptions on sub-workflow dispatch would, on its own, let
applications version their children independently — leaving them to manage registration by hand.

Alternatives considered

• CurrentOrOlder with a single version — unsafe, per above.
• Application-side versioned registration — blocked by DurableWorkflowRunner being internal.
• ContinueAsNewOptions.NewVersion (durabletask-dotnet 1.23.1) — not exposed by the extension either,
  and wouldn't preserve an in-flight run's position mid-graph.
• Never changing a workflow's topology while runs are parked — what we do today; it makes deployment
  cadence a function of human response time.

Code sample

// Only the workflow that actually changed gets a new version.
// Both implementations stay registered, so nothing needs to drain.
services.ConfigureDurableWorkflows(
    workflows =>
    {
        workflows.AddWorkflow(rootWorkflow, version: "1.4.0");          // unchanged
        workflows.AddWorkflow(decompositionWorkflow, version: "1.1.0"); // unchanged
        workflows.AddWorkflow(framingWorkflowV1, version: "1.2.0");     // retained for in-flight
        workflows.AddWorkflow(framingWorkflowV2, version: "1.3.0");     // new topology
    },
    workerBuilder: worker =>
    {
        worker.UseDurableTaskScheduler(connectionString, credential);
        // UseVersioning is optional — per-task registration versioning works without it
        // (MatchStrategy defaults to None). This is our admission posture, not a requirement
        // of the feature.
        worker.UseVersioning(new DurableTaskWorkerOptions.VersioningOptions
        {
            // Must be >= the highest per-workflow version above, or CurrentOrOlder
            // rejects those instances and they re-queue forever.
            Version = "1.4.0",
            DefaultVersion = "1.4.0",
            MatchStrategy = DurableTaskWorkerOptions.VersionMatchStrategy.CurrentOrOlder,
            FailureStrategy = DurableTaskWorkerOptions.VersionFailureStrategy.Reject,
        });
    });

// ExecuteSubWorkflowAsync would resolve the child's registered version and schedule with it:
//
//   await context.CallSubOrchestratorAsync<DurableWorkflowResult?>(
//       orchestrationName,
//       workflowInput,
//       childVersion is null ? null : new TaskOptions { Version = new TaskVersion(childVersion) });
//
// (TaskOptions.Version, not SubOrchestrationOptions.Version — the latter is a shadowing property
//  kept for binary compatibility, and 1.25.0's own docs point new code at the base.)
//
// A run parked on a gate under root 1.4.0 resumes on the new worker. A framing child already in
// flight at 1.2.0 keeps replaying against framingWorkflowV1. New framing children get 1.3.0.

Acceptance criteria

• Microsoft.DurableTask.* moved from 1.18.0 to 1.25.0 in dotnet/Directory.Packages.props, solution
  building and existing tests passing against it.
• AddWorkflow accepts an optional version, and registration keys on (name, version) rather than name
  across all four structures listed above — _workflows, registeredOrchestrations,
  registeredActivities, and ExecutorRegistry.
• Orchestrations and activities registered through the version-aware AddOrchestratorFunc /
  AddActivityFunc overloads when a version is present; unversioned workflows register exactly as today.
• ExecuteSubWorkflowAsync schedules each child at its registered version.
• DurableWorkflowClient.RunAsync / StreamAsync resolve and pass the registered version, with a
  documented, tested answer for what a name-only start does when a name has several versions.
• Azure Functions hosting either gains a working design or rejects a versioned workflow at startup with an
  explicit error referencing DURABLE3004 — not the silent trigger collapse the metadata transformer
  would produce today.
• Tests covering two versions of one workflow co-hosted in a single worker: an in-flight v1 instance keeps
  replaying v1's graph and v1's executors, while newly started instances get v2.