Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .github/ISSUE_TEMPLATE/task.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
name: Task or defect
about: The standard format for every issue in this repository
title: "area: what is wrong or missing, stated plainly"
labels: ''
assignees: ''
---

<!--
Keep these seven sections, in this order, for every issue. The title prefix is
the subsystem: runtime, planner, harness, observability, policy, session,
server, cli, docs, packaging.

The rule this repo runs on: a claim needs evidence. Quote the code or paste the
command output that shows the problem, rather than describing it from memory.
-->

## Summary

What is wrong or missing, in one short paragraph. Quote the offending code or the
real command output rather than paraphrasing it.

## Why this matters

What breaks, or what a user cannot do, as a consequence. Prefer a concrete
failure over an adjective.

## Where in the code

- `path/to/file.py:LINE` — what is there and why it is relevant

Include a command a reader can run to confirm the problem for themselves:

```bash
```

## What to change

1. …
2. …

State anything deliberately out of scope, so a pull request does not grow past
what was agreed here.

## How to verify

```bash
uv run pytest -q
uv run ruff check .
```

Note which new test proves the fix. This repo's convention is that a change
arrives with a test that **fails without it** — check that by reverting your
source edit and watching the new test go red.

## Acceptance criteria

- [ ] …
- [ ] `uv run pytest` stays green and `uv run ruff check .` is clean
- [ ] Any README or cookbook sentence this changes is updated in the same pull
request (several are byte-compared against real output by the test suite)

## Skill level

Pick one and delete the other.

**good first issue** — say why it is well bounded, point at a sibling file that
shows the pattern to copy, and invite questions on the issue.

**experience required** — say which subsystems the change spans, what could
silently break, and ask for a design comment on the issue before any code is
written.
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -423,7 +423,7 @@ Three things that phrase over-promises if left alone. **An interrupt does not st

**The HTTP API is FastAPI plus SSE** — create a session, list, get, post an event, stream the trace, fetch it as NDJSON, healthz. A request may name a registered graph and supply input and a budget; it may not *describe* a graph, because topology comes from a registry the operator fills in Python. But note the seam: **it does not use the session layer above.** It ships its own in-process runtime whose sessions die with the process, never evict, and record `message` and `approval` events without delivering them into a running graph. Two session layers that have not been joined ([ROADMAP.md](ROADMAP.md) §12.3).

**Policy is a TOML document** over nodes, edges, tools and spend, with tiered evaluation — every `deny` before every `ask` before every `allow`, so a broad deny beats a narrow allow including one scoped to a single tenant. Every decision lands in an audit record naming the rule id, the reason, the policy version and a digest of the document, so a decision can be tied to the exact text that made it. And the seam again, because it is the biggest one in the repo: **nothing in the package imports `grapharc.policy`.** There is no bridge from a policy document to the `EdgePolicy` the admission checker consults, and no call from the agent or the CLI to the `PermissionPolicy` it can produce. The governance a run is actually subject to today is what an operator wrote in Python.
**Policy is a TOML document** over nodes, edges, tools and spend, with tiered evaluation — every `deny` before every `ask` before every `allow`, so a broad deny beats a narrow allow including one scoped to a single tenant. Every decision lands in an audit record naming the rule id, the reason, the policy version and a digest of the document, so a decision can be tied to the exact text that made it. And the seam, now narrowed to exactly half: **the edge half is wired and the tool half is not.** `PolicyEngine.edge_policy()` compiles the document into the `EdgePolicy` the admission checker consults, and `grapharc plan --policy` is a real caller — so what may connect to what *is* governed by a document you can read. But `permission_policy()`, `check_tool()` and `approval_router()` have no caller outside `grapharc/policy/`, so `grapharc agent` still assembles its tool gating from `--allow` / `--deny` / `--ask` globs. The most dangerous surface in the package is the one the document cannot reach yet; [issue #6](https://github.com/CodeGraphContext/GraphARC/issues/6) is that work, and the precedence question it has to settle is what happens when a flag `allow` meets a document `deny`.

## Reading a run afterwards

Expand Down Expand Up @@ -480,15 +480,21 @@ Re-derived on 2026-07-28 by running each item, not by reading the commit log.

- **The HTTP API does not use the durable session layer.** It has its own `InProcessRuntime`, whose sessions die with the process and whose approvals are recorded without being delivered. [ROADMAP.md](ROADMAP.md) §12.3.
- *Closed:* `grapharc plan` drives the governed loop; `PolicyEngine.edge_policy()` compiles the TOML document into the gate `AdmissionChecker` consults, and `grapharc plan --policy` is the caller; `grapharc demo --memory PATH` hands the shipped graphs the durable SQLite store.
- *Closed:* the shipped registry withheld the trace recorder from its `PlannerNode` and `Materializer`, so `grapharc plan` wrote a file with no `plan` event and **no `start`/`end` pair for any node it executed** — the paragraph above claiming otherwise was true of a hand-wired loop and false of the one the command drives. Both now get the recorder, and a test asserts the phase counts.

**Real limits of things that do work**

- **Admission authorises a kind, not its arguments.** A proposal carrying `args={"path": "/etc/passwd"}` is admitted on the strength of its kind alone.
- **The audit-hook sandbox is in-process confinement, not a kernel boundary.** `os.stat` outside the workspace is not blocked, because CPython raises no event for it. `ContainerExecutor` is the boundary where one is needed.
- **`run_command` is not confined.** Argv-only and never a shell, but the child is an ordinary process with your privileges.
- *Closed:* a `max_seconds` past the platform's `time_t` — `float("inf")`, or a plausible "effectively unlimited" like `1e10` — used to **disable the deadline guard for the rest of the process**. `setitimer` raised *after* the SIGALRM handler was installed and the process-wide slot taken, leaking both, so every later run silently fell back to the mechanism that cannot unwind a blocking syscall: a 0.3s ceiling then took a 5s sleep to notice. Arming is undone on failure now, and the armed delay is clamped to what both mechanisms accept.
- *Closed:* every `async def` node **double-charged** its token re-reports. The re-report ledger was keyed by thread ident, but `on_llm_end` is sync — under `ainvoke` LangChain dispatches it to a worker thread while the body stays on the event loop — so the automatic charge found no ledger and the node's named re-report was charged again. Any node using the shipped `charge_usage`, `AgentNode._charge_tokens` or `planner.proposal._charge` reported double its real spend and hit `max_tokens` at half its declared allowance. The ledger is a `contextvars` scope now, which also fixes an inner scope discarding the enclosing node's.
- *Closed:* a bracket anywhere in a model's **prose** hijacked JSON extraction, because only the first `{`/`[` was ever tried. `Based on the context [lines 3-5]: {…}` was rejected as unparseable, and — worse — `Analysis (note [1]): {"supported": false}` returned a perfectly valid `[1]`, substituting a fabricated value for the verifier's actual answer. Every opener is tried now, longest parse wins; junk still returns `None`, so fail-closed is unchanged.
- **`interrupt()` suspends but cannot be resumed.** LangGraph's native interrupt stops the graph and shows on `get_state`, and there is no supported resume path — resuming means passing a `Command` as *input*, which is closed by design. Use the session layer's approval gate for human-in-the-loop.
- **Still unwrapped from LangGraph:** `retry_policy`, `cache_policy`, `durability`, subgraphs. `.inner` reaches them, but execution entry points there fail closed, so `.inner` is an inspection escape hatch and not a way to run the graph.
- **Cost is recorded when a backend reports one, estimated when it does not.** Both gateways publish the provider's `cost_usd` through the same `llm_output` envelope, the runtime's usage callback writes it onto the node's `end` event, and an agent's `model` events carry the per-call breakdown. A backend that reports no price still falls back to a `RateCard` estimate, and the two figures stay apart — `recorded_cost_usd` is never a guess. Still missing: no tenant on a trace event, so per-tenant attribution is not offered.
- **A node's tokens are its own, not the run's movement while it ran.** Worth stating because it was the other way round: an `end` event carried the difference between two readings of the run's *shared* meter, so under fan-out the workers' windows overlapped and each was credited with its siblings' concurrent spend. Three workers costing 8 tokens each traced as 24/16/8, and `metrics` and `cost` agreed on 48 for 24 tokens of real work — doubling the estimated bill purely because the work ran in parallel. Attribution now comes from a per-node scope on the meter, so the same work costs the same serially and in parallel; a hand charge the usage callback never saw still lands on the node that made it.
- **A planning round is an envelope, not a measurement.** A `round` event used to carry the planner's `tokens` and the round's `duration_ms`, both of which `metrics`, `cost` and `replay` add on top of node totals — and the planner's spend was already reported by its own `plan` event, so it was counted twice, and a round's duration encloses the plan plus every node it ran. Neither is on the event now; both are on its `state_delta` as `round_tokens` / `round_iterations` / `round_duration_ms`, where no reader sums them. `RoundRecord.iterations` also holds a figure now rather than always `0`.
- **The Claude CLI backend is completion-only.** Tool calling and structured output need one of the OpenAI-wire backends: `openrouter`, `openai`, or a local `ollama`.
- **A session turn is synchronous**, and a runner claim is a claim rather than a lease — nothing reclaims a session whose runner died holding it.
- **A bare model spec resolves to the paid `claude-cli` backend.** `--model mock` does not reach the scripted double; it becomes the model name `mock` on the subscription backend. Only the slash form (`mock/anything`) reaches the double. A mistyped backend *with* a slash is rejected properly, exit 2.
Expand Down
66 changes: 52 additions & 14 deletions grapharc/cli/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
from pathlib import Path
from typing import Any

from grapharc.cli import optional
from grapharc.cli import optional, style
from grapharc.cli.output import EXIT_FAILED, EXIT_OK, emit, fail

# Entry points accepted from `grapharc.tools`, in preference order: a registrar
Expand Down Expand Up @@ -241,24 +241,62 @@ def run_agent(
"refused": len(result.refused),
}

width = style.LABEL_WIDTH
note = f" {style.dim(f'({result.note})')}" if result.note else ""

def count(number: int) -> str:
"""A count, red once it is not zero.

Zero refusals is not news; one is the reason to read the tool-call rows
underneath it. The digits are the same either way when colour is off.
"""
return style.err(str(number)) if number else str(number)

lines = [
f"task : {task}",
f"model : {model_spec}",
f"workspace : {workspace}",
f"tools : {', '.join(visible) or '(none visible under this policy)'}",
f"policy : allow={allow} ask={ask} deny={deny}",
style.kv("task", task, width=width),
style.kv("model", model_spec, width=width, tint=style.accent),
style.kv("workspace", str(workspace), width=width, tint=style.accent),
style.kv(
"tools",
", ".join(visible) or "(none visible under this policy)",
width=width,
),
style.kv(
"policy",
f"{style.dim('allow=')}{allow} {style.dim('ask=')}{ask} {style.dim('deny=')}{deny}",
width=width,
),
"",
f"stopped : {reason}{f' ({result.note})' if result.note else ''}",
f"turns : {result.iterations} tool calls: {len(result.tool_calls)} "
f"denied: {len(result.denied)} refused: {len(result.refused)}",
f"tokens : {meter.tokens:,}",
style.kv(
"stopped",
f"{(style.ok if met else style.warn)(reason)}{note}",
width=width,
),
style.kv(
"turns",
f"{result.iterations} {style.dim('tool calls:')} {len(result.tool_calls)} "
f"{style.dim('denied:')} {count(len(result.denied))} "
f"{style.dim('refused:')} {count(len(result.refused))}",
width=width,
),
style.kv("tokens", f"{meter.tokens:,}", width=width),
]
for call in result.tool_calls:
suffix = f" [{call.refused_by}]" if call.refused_by else ""
lines.append(f" {call.status.value:<8} {call.tool}{suffix}")
suffix = f" {style.dim(f'[{call.refused_by}]')}" if call.refused_by else ""
# `ToolCallStatus` is ok / denied / error; anything a later version adds
# lands on amber rather than being quietly called a success.
verdict = {"ok": True, "denied": False, "error": False}.get(call.status.value)
lines.append(
f" {style.cell(call.status.value, 8, tint=style.tint_for(verdict))} "
f"{style.accent(call.tool)}{suffix}"
)
lines.append("")
lines.append(f"answer : {result.output}" if met else f"partial : {result.partial_output}")
lines.append(f"trace : {trace_path}")
lines.append(
style.kv("answer", str(result.output), width=width)
if met
else style.kv("partial", str(result.partial_output), width=width, tint=style.dim)
)
lines.append(style.kv("trace", str(trace_path), width=width, tint=style.accent))

emit(payload, lines, as_json=as_json)
return EXIT_OK if met else EXIT_FAILED
Expand Down
57 changes: 37 additions & 20 deletions grapharc/cli/graphrun.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
from pathlib import Path
from typing import Any

from grapharc.cli import style
from grapharc.cli.config import ConfigError
from grapharc.cli.config import load as load_settings
from grapharc.cli.generate import resolve_or_generate_policy
Expand Down Expand Up @@ -197,15 +198,32 @@ def run_graph(
**settings.provenance(policy_source=policy_source),
}

# `REFUSED` and `ADMITTED` are the whole answer, so on a terminal they carry
# the only two colours that matter here. The words, the widths and the order
# are untouched: `--check-only` is what CI runs, and CI reads text.
width = style.LABEL_WIDTH
header = [
style.kv("graph", graph_path, width=width, tint=style.accent),
style.kv("policy", policy_description, width=width),
]
trace_line = style.kv("trace", str(trace_path), width=width, tint=style.accent)

if not verdict.admitted:
lines = [
f"graph : {graph_path}",
f"policy : {policy_description}",
*header,
"",
f"REFUSED : {len(verdict.rejections)} objection(s)",
style.kv(
"REFUSED",
f"{len(verdict.rejections)} objection(s)",
width=width,
key_tint=style.err,
tint=style.err,
),
]
lines += [
f" {style.cell(r.code, 18, tint=style.err)} {r.detail}" for r in verdict.rejections
]
lines += [f" {r.code:<18} {r.detail}" for r in verdict.rejections]
lines += ["", f"trace : {trace_path}"]
lines += ["", trace_line]
emit({"ok": False, **common}, lines, as_json=as_json)
return EXIT_FAILED

Expand All @@ -224,13 +242,12 @@ def run_graph(
compiled = materializer.materialize(verdict, proposal)
except MaterializationError as exc:
lines = [
f"graph : {graph_path}",
f"policy : {policy_description}",
*header,
"",
"ADMITTED, BUT CANNOT BE BUILT",
style.err("ADMITTED, BUT CANNOT BE BUILT"),
f" {exc}",
"",
f"trace : {trace_path}",
trace_line,
]
emit(
{"ok": False, "buildable": False, "error": str(exc), **common},
Expand All @@ -241,12 +258,13 @@ def run_graph(

if check_only:
lines = [
f"graph : {graph_path}",
f"policy : {policy_description}",
f"nodes : {proposal.node_count()}",
*header,
style.kv("nodes", str(proposal.node_count()), width=width),
"",
"ADMITTED and buildable. Nothing was run.",
f"fingerprint: {verdict.fingerprint}",
style.ok("ADMITTED") + style.dim(" and buildable. Nothing was run."),
# Wider than the label column on purpose, and always has been: the
# fingerprint is what a later run is compared against.
style.kv("fingerprint", verdict.fingerprint, width=width, tint=style.accent),
]
emit(
{"ok": True, "checked_only": True, "buildable": True, **common},
Expand All @@ -259,13 +277,12 @@ def run_graph(

payload = {"ok": True, "checked_only": False, **common, "state": state}
lines = [
f"graph : {graph_path}",
f"policy : {policy_description}",
f"nodes : {proposal.node_count()}",
*header,
style.kv("nodes", str(proposal.node_count()), width=width),
"",
"ADMITTED and executed.",
f"state : {state}",
f"trace : {trace_path}",
style.ok("ADMITTED") + style.dim(" and executed."),
style.kv("state", str(state), width=width),
trace_line,
]
emit(payload, lines, as_json=as_json)
return EXIT_OK
Expand Down
Loading
Loading