Skip to content

feat(flows): support runtime agent fan-out - #34

Open
antoinegg1 wants to merge 2 commits into
mainfrom
feat/runtime-dynamic-agents
Open

feat(flows): support runtime agent fan-out#34
antoinegg1 wants to merge 2 commits into
mainfrom
feat/runtime-dynamic-agents

Conversation

@antoinegg1

@antoinegg1 antoinegg1 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Background

Humanize flows traditionally declare a fixed agent tuple before a run starts. That is correct for validating backend capabilities and preparing the run, but it does not cover workflows whose fan-out is only known after an initial planning/triage turn. For example, a triage agent may select a variable number of specialist departments, and the workflow should then run one independent specialist agent per selected department.

Agent.clone() and asynchronous turns already provide the low-level building blocks, but a clone created after Runner.run() starts was previously outside the run lifecycle: it was absent from Runner.agents, cycle records, TUI monitoring, and Runner.stop(). This change adds the missing lifecycle registration without changing the existing static flow contract or backend/session implementations.

Implementation

  • Add hmz.flows.spawn(template, names) as the opt-in public API. It validates non-empty, unique names, clones the template configuration/skills, and returns the new agents in input order.
  • Add a run-local, thread-safe agent registry to Cycle. Registration is atomic across the whole root run, including called flows, so name collisions cannot partially admit a fan-out.
  • Attach each admitted agent to the cycle and record an additive spawned journal event with parent and agent configuration. cycle.read() reconstructs these agents alongside the declared agents.
  • Share the root registry through nested Sub cycles. A child created by a called flow is recorded in its own flow record and in the root run inventory, and is stopped with the rest of the run.
  • Extend Runner with a dynamic agents snapshot, an agent-join watcher, and run-wide stop handling. A stop request racing with spawn() stops the newly admitted agent before it can take a turn. The stop flag is reset after a run so a Runner can be reused.
  • Wire the same join callback into the TUI and Tally so dynamic agents receive event watchers, prompt/ask handlers, progress monitoring, and cost accounting. SDK Run.stop() delegates to the runner registry.
  • Keep the existing fixed flow declaration checks, CLI agent selection, backend process handling, container logic, and formal fixed-agent workflows unchanged. The new behavior is activated only by calling spawn().

Runtime Behavior

A typical run now behaves as follows:

@flow
async def run(agents: Agents, task: str) -> None:
    plan = await agents.planner.aturn(task, schema=Plan)
    workers = spawn(
        agents.worker,
        (f"worker-{index:03d}" for index in range(1, len(plan.items) + 1)),
    )
    results = await asyncio.gather(
        *(worker.aturn(item) for worker, item in zip(workers, plan.items, strict=True))
    )
  1. Runner performs the normal pre-run validation and starts one root cycle containing the declared agents.
  2. After the planner/triage result is available, spawn() validates all requested names and creates independent clones. The original template is not reconfigured.
  3. The complete group is registered under the cycle lock before spawn() returns. Runner.agents, SDK Run.agents, the TUI, and cycle tracing can see the new agents before their first turn.
  4. The flow may run the returned agents concurrently with asyncio.gather() or another scheduling policy. Each clone has its own sessions, identity, configuration, workspace selection, and trace entries.
  5. If the run is stopped concurrently, the registry and stop path coordinate through the same lifecycle boundary. Existing agents are stopped immediately, and an agent joining after the stop request is stopped before it can continue.
  6. On normal completion, failure, cancellation, or nested-flow return, spawned agents are stopped and detached. The final cycle contains their spawned and opened records, while old cycles without spawned events remain readable.
  7. Calling spawn() outside a running cycle still works as a named group of clone() calls; in that case the caller owns cleanup because there is no Humanize run to manage them.

Compatibility and Scope

This is an additive change. Existing flows that do not call spawn() retain their previous behavior and fixed startup validation. No Flowverse changes are included in this PR; the companion reference flow remains local until the core API is reviewed and merged.

Validation

  • PYTHONPATH=src /usr/local/bin/pytest -c /tmp/hmz-pytest.ini tests/test_dynamic_agents.py -q -> 7 passed
  • Dynamic + cycle targeted tests -> 19 passed, 1 unrelated provider-subprocess environment failure (/usr/bin/python3 cannot import local hmz)
  • Ruff, formatting, and Python compilation checks passed
  • GitHub Actions build/check jobs passed; the Python 3.12/3.13/3.14 test jobs are running in CI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant