Skip to content

Isolate and type fan-out payloads, check input at the front door, trace a Ctrl-C'd node - #76

Merged
Shashankss1205 merged 2 commits into
mainfrom
fix/kernel-isolation-and-input-validation
Aug 3, 2026
Merged

Isolate and type fan-out payloads, check input at the front door, trace a Ctrl-C'd node#76
Shashankss1205 merged 2 commits into
mainfrom
fix/kernel-isolation-and-input-validation

Conversation

@Shashankss1205

Copy link
Copy Markdown
Collaborator

Three defects in grapharc/runtime/graph.py, all of the same family: a contract the module documents, enforced everywhere except at one boundary. They share a file, so they share a PR.

#67 — fan-out payloads bypassed isolation and were never typed

_enter's own comment states the rule: "Nodes get a deep copy: the returned dict is the only write channel." The copy was guarded by isinstance(state, BaseModel), and a Send payload can be anything. Two Sends built from one dict therefore handed both parallel workers the same live object:

before  workers reported: ['dict hits=1 id=139338015848512',
                           'dict hits=2 id=139338015848512']   # same id
        shared dict after run: {'who': 'orig', 'hits': [1, 1]}

after   workers reported: ['hits=1', 'hits=1']                 # distinct copies
        shared dict after run: {'who': 'orig', 'hits': []}

That is a data race between nodes that are meant to be isolated, and the mutations travel through a channel no node declared a write to and no trace event records — so the write-permission model and the audit trail both miss it, and which worker got there first decides the answer. Fan-out is where the isolation matters most; it was the one path that skipped it.

The other half of the same boundary: _check_goto_target validated Send.node and left Send.arg alone, so input_schema — whose docstring says it "types a fan-out worker's Send payload" — enforced nothing.

before  input_schema=P, dict payload  -> ran; the worker got a dict
        input_schema=P, Q() payload   -> AttributeError: 'Q' object has no attribute 'who'

after   input_schema=P, dict payload  -> StateTypeError: the fan-out dispatcher on node 's'
                                         sent node 'w' a payload its input_schema rejects:
                                         expected P, got dict ({'who': 'orig', 'hits': []}); …
        input_schema=P, Q() payload   -> StateTypeError: … expected P, got Q (Q(other=1)); …

Every payload is deep-copied now, whatever its type, and Send.arg gets the treatment Send.node already got. Declaring no input_schema stays legal — a worker that names no schema is claiming nothing — but the copy is unconditional, and add_node's docstring now says so rather than leaving it to be inferred. A BaseModel payload was already isolated correctly and still is, with its own regression test.

#68 — the front door was the one door the state contract did not hold

state.py opens by promising "a typo'd update key fails loudly at the edge instead of silently polluting downstream nodes." update_state kept that promise; the entry points handed input to LangGraph, which filters a dict down to known channels before the state model is ever constructed, so extra="forbid" never got a chance.

before  invoke({'quesiton': 'typo'})   -> {'out': "saw:''"}          # ran on defaults
        stream(…)                      -> [{'n': {'out': "saw:''"}}]
        ainvoke(…) / astream(…)        -> same
        update_state({'quesiton': …})  -> WritePermissionError: update_state targets unknown
                                          state fields: ['quesiton']

after   invoke({'quesiton': 'typo'})   -> WritePermissionError: invoke() targets unknown
                                          state fields: ['quesiton']
        stream / ainvoke / astream     -> the same sentence, named for the entry point

Misspell the field carrying the question and you got a complete, plausible-looking run against an empty question, with nothing said to the caller — the quietest failure in the runtime, on the door every user goes through first. All four entry points plus astream_events now refuse an unknown key, from the same helper update_state uses so the wording cannot drift. Unchanged on purpose: a wrongly typed input value still raises Pydantic's ValidationError (invoke({"out": 5})), and a state model or None passes through untouched.

#70 — a Ctrl-C'd sync node left no ending in the trace

The sync wrapper caught Exception; the async twin catches BaseException, and its comment gives the reason — "a stop with no trace line is a stop nobody can audit afterwards."

before  sync   trace phases: [topology, start]
        async  trace phases: [topology, start, error]

after   sync   trace phases: [topology, start, error]   # KeyboardInterrupt and SystemExit
        async  trace phases: [topology, start, error]

metrics.summarize then reported errors: 0 for the sync run, and an audit reads it as having simply stopped between nodes. Ctrl-C is not an exotic ending; it is the commonest way a human stops a long run. The wrapper now matches its twin and re-raises the exception untouched — only the trace write is new.

Tests

Twelve, each verified failing before its fix:

  • Fan-out (tests/test_runtime_discipline.py) — two workers sharing one dict cannot observe each other's mutations; a payload contradicting input_schema is refused at dispatch; a wrong model class likewise; an untyped payload stays legal; a BaseModel payload is still isolated (regression guard, passes before and after).
  • Front door (tests/test_runtime_discipline.py, tests/test_async_kernel.py) — invoke and stream reject the typo verbatim, ainvoke/astream in an async twin; the type-error and valid-input paths are pinned unchanged.
  • Ctrl-C (tests/test_async_kernel.py) — one parametrised test asserts both wrappers write an error event for KeyboardInterrupt and SystemExit, so the two paths cannot drift again. Driven with asyncio.run rather than @pytest.mark.asyncio, because asyncio re-raises these two out of the task step and into the loop.

Scope

Deliberately untouched: budget metering, deadline guards, write permissions, and trace event shapes beyond the one missing error event.

Performance

Deep-copying every payload is free at graph scale — 200 fan-out workers, dict payloads, best of five:

with the change     41.1 ms
without             41.6 ms

Run-to-run noise dominates. In isolation, 200 copy.deepcopy calls on a realistic shard cost 0.5 ms and 200 model_copy(deep=True) cost 0.7 ms, against ~41 ms of graph execution.

Verification

ruff check . clean. Full suite: 1811 passed, 1 failed. The single failure is test_cookbook_agents.py::test_snippet_runs_as_printed[11-sandbox-refusals.py], which is environmental and fails identically on untouched main in this worktree — the snippet normalises a <site-packages> path and the worktree's symlinked .venv resolves elsewhere. None of the SIGALRM timing tests flaked.

Fixes #67
Fixes #68
Fixes #70

🤖 Generated with Claude Code

Shashankss1205 and others added 2 commits August 4, 2026 00:49
…ce a Ctrl-C'd node

Three defects in `grapharc/runtime/graph.py`, all of the same family: a
contract the module documents, enforced everywhere except at one boundary.

**Fan-out handed every worker the same object, and never typed it.** `_enter`'s
comment states the rule — "Nodes get a deep copy: the returned dict is the
*only* write channel" — but the copy was guarded by
`isinstance(state, BaseModel)`, and a `Send` payload can be anything. Two Sends
built from one dict therefore gave both parallel workers the same live dict:

    workers reported: ['dict hits=1 id=139338015848512',
                       'dict hits=2 id=139338015848512']   # same id
    shared dict after run: {'who': 'orig', 'hits': [1, 1]}

That is a data race between nodes that are supposed to be isolated, and the
mutations travel through a channel no node declared a write to and no trace
event records — so the write-permission model and the audit trail both miss it,
and which worker ran first decides the answer. Fan-out is the one place the
isolation matters most; it was the one place that skipped it.

The other half of the same boundary: `_check_goto_target` validated `Send.node`
and left `Send.arg` alone, so `input_schema` — whose docstring says it "types a
fan-out worker's Send payload" — checked nothing. A dict where a model was
declared reached the worker and failed as a bare `AttributeError` several frames
from the dispatcher that produced it; a wrong model class sharing a field name
did not fail at all. That `Send.node` is checked and `Send.arg` is not reads as
an oversight rather than a decision, so `Send.arg` now gets the same treatment,
with `StateTypeError` naming the node, the schema and what arrived.

Every payload is deep-copied now, whatever its type. Declaring no `input_schema`
stays legal — a worker that names no schema is claiming nothing, so there is
nothing to check — but the copy is unconditional, and `add_node`'s docstring now
says so instead of leaving it to be inferred. A `BaseModel` payload was already
isolated correctly and still is, asserted by its own test.

**The front door was the one door the state contract did not hold.**
`state.py` opens by promising "a typo'd update key fails loudly at the edge
instead of silently polluting downstream nodes". `update_state` keeps that
promise and so does constructing the model; the entry points did not. They
handed `input` to LangGraph, which filters a dict down to known channels
*before* the state model is ever constructed, so `extra="forbid"` never got a
chance:

    invoke({'quesiton': 'typo'})       -> {'out': "saw:''"}     # ran on defaults
    update_state({'quesiton': 'typo'}) -> WritePermissionError: unknown state fields

Misspell the field carrying the question and you get a complete, plausible run
against an empty question and are told nothing — the quietest failure in the
runtime, on the door every user goes through first. `invoke`, `stream`,
`ainvoke`, `astream` and `astream_events` now refuse an unknown input key, in
the same words `update_state` uses, from the same helper so the two cannot
drift. A wrongly *typed* input value was already loud and still raises
Pydantic's `ValidationError`; a state model or `None` is passed through
untouched.

**A Ctrl-C'd sync node left no ending in the trace.** The sync wrapper caught
`Exception`; the async twin catches `BaseException`, and its comment gives the
reason — "a stop with no trace line is a stop nobody can audit afterwards". So
the same node body, run each way:

    sync   trace phases: [topology, start]
    async  trace phases: [topology, start, error]

`metrics.summarize` then reported `errors: 0` for the sync run, and an audit
reads it as having simply stopped between nodes. Ctrl-C is not an exotic
ending; it is the commonest way a human stops a long run. The sync wrapper now
matches its twin. The exception is re-raised untouched — only the trace write
is new.

Tests: twelve, each failing on main. Fan-out — two workers sharing one dict
cannot see each other's mutations, a payload contradicting `input_schema` is
refused at dispatch, a wrong model class likewise, an untyped payload stays
legal, and a `BaseModel` payload is still isolated. Front door — `invoke` and
`stream` reject the typo verbatim, `ainvoke`/`astream` in a parametrised twin,
with the type-error and valid-input paths pinned unchanged. Ctrl-C — one test
asserts both wrappers write the `error` event for `KeyboardInterrupt` and
`SystemExit`, so the two paths cannot drift again.

Deep-copying every payload costs nothing measurable: 200 fan-out workers run in
41.1ms with the change and 41.6ms without (best of five, dict payloads, well
inside run-to-run noise), because 200 `copy.deepcopy` calls on a realistic
shard total 0.5ms against ~41ms of graph execution.

Fixes #67
Fixes #68
Fixes #70

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Shashankss1205
Shashankss1205 merged commit ce9789d into main Aug 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant