feat(langgraph): record which node ran, and the ones that call no model

[xizhuomengcontin]

Orca-Code-Review — push 1

Severity	Count
P0	0
P1	0
P2	0
P3	0

✅ no blocking findings

A LangGraph run's shape is not on the wire. Searching a recorded three-node run for the node names, the graph's name, langgraph_node and langgraph_step finds none of them in any request. orcareplay-langgraph is the second framework adapter, and it puts that shape in the trace without anyone editing their graph.

graph.node.start   plan
model.request / model.response
graph.node.end     plan
graph.node.start   validate      ← makes no request of any kind
graph.node.end     validate
graph.node.start   answer
graph.node.end     answer

validate is the point rather than a curiosity. Validators, state reducers, routers and writers call no model, so without this a proxy sees a graph with those nodes and a graph without them as the same run. The same goes for a parallel fan-out, which reaches the wire as two ordinary consecutive turns, and for a node that raised — graph.node.end carries the exception's class, which no request can.

the trace can answer	without	with
which nodes ran, and in which superstep	no	yes
that a node ran at all, when it called no model	no	yes
that two calls were one parallel superstep, not two turns	no	yes
when each node began and ended	no	yes

How a node is recognised, which is the whole design

on_chain_start fires for everything a graph runs, and at that moment a node, a runnable inside a node, a subgraph and a conditional edge all look alike: same inherited langgraph_node, same langgraph_step, an arbitrary name. Measured on one graph: name='INNER-RUNNABLE' carrying langgraph_node='first', name='router' carrying langgraph_node='validate_only', and name='the_graph' carrying no langgraph_node at all. Reporting on name reports all four.

So a node is reported only when name equals metadata['langgraph_node'] — compared, never copied. That rule is also what keeps the file safe: langgraph_node is the name the author wrote in add_node(...), while name is whatever the caller supplied, including .with_config({"run_name": f"lookup for {customer.email}"}). The two are equal only for an author-defined node, so a runtime string cannot reach the transport through this path.

Two records per node and nothing else — node, step, the run ids that pair them, the instants, and error. No state, no inputs, no outputs, no messages, and no exception message. The model exchanges stay with the proxy, which already has them byte for byte.

Lazy, because the bootstrap is in front of every Python process

Importing the module that holds langchain-core's hook registry costs 617 ms against an interpreter that starts in about 50, and orca record loads this before python --version. So install() arms a sys.meta_path finder and returns; registration happens the first time anything under langgraph is imported, in a process already paying a second for it. Measured: arming costs under 100 ms and leaks no langchain module into a process that never touches a graph.

Waiting for langchain_core.tracers.context to appear on its own was the first design and does not work — measured across the 164 find_spec calls of one from langgraph.graph import StateGraph, it is loaded for none of them, because _configure imports it at the first invoke, already too late to register in.

The trigger is the langgraph root only, and that is a correctness constraint rather than a preference. The finder runs inside _find_and_load_unlocked, past its sys.modules check, so importing langchain-core from a langchain_core.* lookup re-enters that load — and for a top-level name CPython does not re-check sys.modules before executing. Measured against a finder widened to watch both: langchain_core executed twice, leaving one stale module object. A test reproduces it from the only direction that reaches it.

A bug this found in the drain

The drain stamped events from span.started_at alone. A record that closes something carries only ended_at, so every one of them got the drain's own clock — which runs after the agent has exited. Measured: all three graph.node.end events landed within a millisecond of each other, 600 ms after the last node finished, so the first node appeared to close after the second had opened. occurredAt is also what turnAt reads, so the same mistake filed them under the wrong turn.

The bound in readAgentSpans is widened to match. It exists to keep an instant a date-time cannot write down out of TraceWriter.append, where the trace is being sealed and the transport has already been deleted; a fallback field the bound did not cover would have been a hole the size of the one it closed. Both directions are tested.

Reporting a missing adapter: used, not installed

With a second adapter the old rule broke. It asked importlib.metadata at startup whether the framework was installed, and on an ordinary machine that is yes for frameworks the run never touches — one venv here has both langgraph and openai-agents, so every recorded run in it was told to pip install two adapters it had no relationship to, and the count grows with every adapter shipped. A warning that fires when nothing is wrong is how warnings stop being read.

The bootstrap now reports only once the agent actually imports the framework, still confirmed by importlib.metadata — so the false positive #83 was about, a project's own agents.py, stays excluded. Eight behaviour cases were checked against a real interpreter: not recording, imports nothing, imports langgraph, imports langgraph.graph only, imports agents, imports both, many submodules reported once, and a same-named module with no metadata.

PYTHON_ADAPTERS is now one declaration that generates the bootstrap's Python and the operator's warning prose, so a package can no longer be reported with another package's explanation — which is exactly what happened the moment a second adapter existed and that text was still a constant.

Format

graph.node.start / graph.node.end, so SCHEMA_VERSION goes 0.2.0 → 0.3.0 in both the TypeScript constants and the Python reader. The spec gains the two rows and a normative sentence the agent.* family also needed: these types are handed to orca by code inside the agent's interpreter rather than observed, so an implementation MUST keep them to structure the harness names for itself and MUST NOT carry prompts, model output, tool arguments, tool results or exception messages.

The example trace gains one of each, because scripts/conformance.mjs reports every declared type no shipped trace exercises. Checkpoint and turn-span goldens were recomputed by running deriveCheckpoints and turnsOf over the new events rather than adjusted by hand.

Proof

• 46 tests in the package. The integration half builds real graphs and invokes them, because the unit half only asserts what langchain-core is believed to pass.
• 26 mutants, 26 killed, 0 survived — including the inverted discriminator, a peeking closing(), run_inline off, a dropped ignore_chain, the widened finder trigger, and an eager install.
• langgraph-nodes integration check: 2 exchanges, replayed exact with the origin down, asserting plan/validate/answer present and INNER-RUNNABLE/route/the_graph absent. The runner grows an expectNodes option, because the event types alone do not make that claim.
• scripts/conformance.mjs: 63 events, 0 failures, every declared type exercised.
• Full suite 2431 passed, with the 10 known Windows failures unchanged; openai-agents-handoff still passes against the rewritten bootstrap.
• Wheel and sdist build; twine check PASSED on both.

ignore_chain is worth naming. langchain-core reads every ignore_* with a bare getattr, and a missing one raises per callback, which it logs and swallows. Omitting it gave a graph that ran to completion, exit 0, four Error in ... callback lines, and nothing recorded — a silent miss arriving through the one attribute that had to be False. A test now compares the whole set against BaseCallbackHandler, so a release that adds an eighth is caught by a test rather than by an empty trace.

Not fixed here

A model.request is stamped when orca persists it — after the response and after a workspace snapshot, measured 95 ms late on one call and 31 ms on the next — while a node record carries the instant the callback fired, exact against the agent's own clock. So node boundaries are trustworthy, and a model call within about a tenth of a second of one can sit on the wrong side of it. Fixing that means giving RecordedExchange its own timestamps and threading them through the proxy, which is its own change. The README and docs/integrations.md say so rather than overclaiming attribution this does not deliver.

Publishing needs a one-time PyPI pending publisher for orcareplay-langgraph (release-langgraph.yml, environment pypi); until it exists the upload step fails and nothing else is affected.

🤖 Generated with Claude Code

[orcacode-review]

🐳 OrcaCode Review

✅ No findings — nothing to flag in this PR. Great work!

_{OrcaCode Review — Route Smarter. Ship Safer. Spend Less.
Engine-reported: 1141 calls · 170.8M tokens · 99% cached}

❤️ Share · Install OrcaCode Review

Free on GitHub — the review runs on your own OrcaRouter key. If it helped, a shout-out goes a long way.

Share: X · Reddit · LinkedIn
Follow: X · Discord · LinkedIn · OrcaRouter