feat(engine): resolve registry references in sub-workflow workflow: field#188
Merged
Conversation
…ield Extends type: workflow agents to accept registry references (workflow[@registry][#ref] syntax) in the workflow: field, not just local file paths. Resolves issue #172. Resolution order in _resolve_subworkflow_path(): 1. If agent_workflow resolves to an existing file relative to the parent workflow directory, return that path (backward compat — preserves extensionless local file references like 'analysis'). 2. Parse via registry/resolver.resolve_ref() to detect file vs registry syntax. 3. If file-kind, return the (non-existent) path so the caller emits a clear "file not found" error. 4. If registry-kind, call cache.fetch_workflow() via asyncio.to_thread() to fetch and cache the workflow, returning the local cached path. Both _execute_subworkflow and _execute_subworkflow_with_inputs use the new helper. RegistryError from resolve_ref() and fetch_workflow() are wrapped in ExecutionError with agent context. conductor validate now resolves registry refs for all type: workflow agents (including for_each inline agents) when workflow_path is provided. Validation is recursive with cycle detection and a depth cap of 10. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tion The previous implementation built a `_visited` set in `_validate_subworkflow_refs` but called `validate_workflow_config` recursively without threading the set through. Each recursive level started with a fresh empty `_visited`, so cycles like A → B → A were not detected and would either recurse to the depth cap silently or loop until an unrelated failure (e.g. fetch error). Adds two internal-only kwargs to `validate_workflow_config` (`_visited_subworkflows`, `_subworkflow_depth`) that are threaded through recursive validation so the visited set accumulates across the full sub-workflow chain. Adds a test exercising A → B → A. Also adds a docstring note to `_resolve_subworkflow_path` explaining that mutable registry refs (e.g. `name@registry#main`, or no `#ref`) may resolve to a different commit on `conductor resume` if the upstream branch has moved. Pinned tags or commit SHAs guarantee deterministic resume. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a full round-trip test for the documented resume + registry ref behavior: fail mid-sub-workflow → checkpoint → resume → success. The test verifies that: 1. fetch_workflow is called again on resume (resolution is not memoized on the engine instance), and 2. for a stable cached ref, the resolved path is identical between the original run and the resumed run. This is the determinism guarantee for cached/SHA-pinned registry sub-workflow refs across resume. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Code-review findings: * **Cycle detection now uses inode identity** (`(st_dev, st_ino)`) instead of `str(Path.resolve())`. The previous string-based approach still caught cycles eventually but treated case-variants of the same file as distinct entries on case-insensitive filesystems (macOS, Windows). Inode-based identity catches the cycle on the first revisit and correctly handles symlinks too. Adds a test that exercises the case-variant scenario (skipped on case-sensitive FS). * **Depth-limit silent return now emits a warning.** Previously `_validate_subworkflow_refs` returned empty errors/warnings when `_MAX_SUBWORKFLOW_VALIDATION_DEPTH` was hit, so the user couldn't tell whether the workflow was clean or whether validation was truncated. Adds a regression test that builds a chain longer than the limit and asserts the warning surfaces. * **Bare `except Exception` narrowed to `except ConfigurationError`** for the recursive `validate_workflow_config` call (the only exception that recursive call documents raising). The `load_config` catch stays broad — load can fail many ways (YAML parse errors, schema errors, file IO). Test-quality findings: * `test_registry_ref_resolved_and_executed` no longer mocks `_resolve_subworkflow_path` — it now mocks only the registry config loader and `fetch_workflow`, so the engine's real resolution flow runs end-to-end (real `resolve_ref` parses `analysis@team-a#v1.0.0`, real precedence check confirms no local file shadows it, real registry branch is taken). * `test_registry_ref_validates_fetched_workflow` no longer mocks `resolve_ref` — it captures `fetch_workflow` arguments and asserts the parsed workflow name, registry name, and ref are correct, exercising real parser logic. * `test_local_file_takes_precedence_over_registry` now patches `resolve_ref` and asserts it is not called when a local file exists, guaranteeing the precedence short-circuit is preserved. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Merged
jrob5756
added a commit
that referenced
this pull request
May 14, 2026
- feat(engine): registry references in sub-workflow workflow: field (#188) - feat: Claude Code plugin marketplace (#186) - docs(skill): refresh conductor skill with latest features (#187) - docs: rewrite 'Why Conductor?' README section (#185) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #172.
What
Extends
type: workflowagents to accept registry references (name@registry#refsyntax) in theworkflow:field, in addition to local file paths. Connects the existing sub-workflow composition feature (#79) and the workflow registry, enabling cross-team workflow composition by reference.How
Engine (
src/conductor/engine/workflow.py)New
_resolve_subworkflow_path()async helper, used by both_execute_subworkflowand_execute_subworkflow_with_inputs. Resolution order:analysisreferring to a sibling file without a.yamlextension.registry/resolver.resolve_ref()to detect file vs registry syntax.cache.fetch_workflow()viaasyncio.to_thread()(synchronous HTTP) and return the local cached path.RegistryErrorfrom bothresolve_ref()andfetch_workflow()is wrapped inExecutionErrorwith agent context.Validator (
src/conductor/config/validator.py)conductor validatenow resolves and recursively validates sub-workflow references for alltype: workflowagents (includingfor_eachinline agents). For registry refs, this fetches the workflow into the local cache during validation. This is a behavioral change:validatenow makes network requests when registry refs are present (the issue calls for this explicitly to validate the full composition tree).Validation is recursive with cycle detection (visited-set threaded through
validate_workflow_configvia internal kwargs) and a depth cap of 10.Resume / replay compatibility
The resolver is called on every sub-workflow execution, including after
conductor resume. For SHA-pinned refs (name@registry#v1.2.3orname@registry#<sha>) the cache is content-addressable so resume is fully deterministic. For mutable refs (#mainor omitted) resume may pick up a newer commit if the upstream branch has moved — documented in the helper's docstring with a recommendation to pin in production.Tests
13 new tests covering:
ExecutionError; extensionless local file takes precedence over registry; malformed ref →ExecutionError; full round-trip resume verifying the same cached path is used on both runs.for_eachinline workflow agent validated; A → B → A cycle detection.All 2445 tests pass;
make check(ruff + ty) clean.Files
Design note
The registry design doc states "Keep the registry mechanism out of the engine: it is a CLI/loader concern." This PR crosses that line by adding registry resolution to the engine's sub-workflow path-resolution step. Per the issue's analysis, the alternative (pre-fetch and reference by local path) defeats the purpose of composable workflows. The change is contained to a single helper called immediately before
load_config().