feat: AG-UI protocol adapter + agentic protocol surface & generative UI (#386)#387
feat: AG-UI protocol adapter + agentic protocol surface & generative UI (#386)#387plauzy wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #387 +/- ##
=======================================
Coverage ? 87.66%
=======================================
Files ? 132
Lines ? 14882
Branches ? 0
=======================================
Hits ? 13047
Misses ? 1835
Partials ? 0
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
An additive, opt-in feature set that broadens how CAO exposes and observes a heterogeneous fleet of CLI agents. Every new surface is default-off (gated by env/config) or a new module that isn't imported unless enabled; the existing localhost dashboard, MCP servers, and API are untouched. Protocols - AG-UI streaming: GET /agui/v1/stream maps CAO's normalized event vocabulary to AG-UI typed events, with a STATE_SNAPSHOT on connect + RFC-6902 STATE_DELTA shared-state channel and ?since= replay, behind CAO_AGUI_ENABLED + scope auth. This is the L1 adapter proposed in awslabs#386 (services/agui_stream.py plus the mapping / redaction / refusal tests). - A2A JSON-RPC + signed Agent Card / OAuth PRM on a dedicated, default-off loopback listener (CAO_AGENT_CARD_ENABLED); an Agent Client Protocol server (cao-acp) for editor integrations. Generative UI - Agents author UI intents rendered uniformly across providers via a closed, safe allow-list of named components with JSON props (no HTML/script/eval); off-list components are refused at the adapter, the React component, and the replay artifact. React renderer in the L2 dashboard PWA (cao_pwa/), a browser-openable deterministic replay artifact, and a Playwright recording harness + workflow. Observability / orchestration / persistence / policy / cache (all opt-in) - OpenTelemetry GenAI telemetry + W3C traceparent on events; ASI integrity evaluator + kill switch; orchestration topologies (DAG, swarm, Polecat, topology router); WAL + replay; a Cedar-style policy queue; a 3-layer cache. Providers, auth, frontends - Additional CLI providers plus a credentials-free mock_cli for CI. - WebSocket auth on /terminals/{id}/ws: token via the cao.bearer.<jwt> subprotocol, 4401/4403 close codes, read-only viewers' input dropped. Default-off preserves the byte-identical no-auth localhost contract. Testing - Full unit scope green (pytest test/ --ignore=test/e2e -m "not e2e"); cao_pwa vitest incl. generative-UI + safety-refusal tests; tsc clean; vite build within the bundle-size budget; black/isort/mypy clean; WebSocket auth e2e scenarios pass; generative-UI screen recording produced in CI with an offline replay. Implements the AG-UI L1 adapter proposed in awslabs#386. Co-authored-by: Kiro Agent <244629292+kiro-agent@users.noreply.github.com>
plauzy
force-pushed
the
feat/agentic-protocols-generative-ui
branch
from
0252209 to
f40933d
Compare
|
Addressed the Codecov patch-coverage comment: added targeted unit tests for the flagged modules. |
There was a problem hiding this comment.
Pull request overview
This PR introduces a large set of opt-in, default-off protocol surfaces and supporting infrastructure around CAO, including an AG-UI streaming surface + generative-UI producer path, a dedicated Agent Card/A2A listener, and OpenTelemetry GenAI instrumentation, with substantial new test coverage, examples, docs, and CI workflows.
Changes:
- Add OpenTelemetry (traces + optional metrics) initialization gates, GenAI span helpers, and W3C trace-context propagation utilities, with a dedicated telemetry test suite.
- Add a credentials-free
mock_cliprovider and fixture binary plus CI/e2e smoke plumbing to exercise orchestration without external provider credentials. - Add a standalone
cao_pwa/dashboard (Vite/React/Vitest/Playwright harness) and documentation/examples to support multi-instance AG-UI consumption and generative UI proof artifacts.
Reviewed changes
Copilot reviewed 150 out of 157 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/init.py | Adds a tests/ package marker (separate from the existing test/ suite). |
| test/test_constants.py | Extends default CORS allow-list coverage for the PWA dev server port. |
| test/telemetry/test_spans.py | Validates GenAI span helper naming and attributes. |
| test/telemetry/test_otel_init.py | Tests opt-in OTel SDK initialization/shutdown gating. |
| test/telemetry/test_context_propagation.py | Tests traceparent inject/extract round-trips. |
| test/telemetry/conftest.py | Shared session-scoped tracer provider + in-memory exporter fixtures. |
| test/telemetry/init.py | Marks telemetry tests package. |
| test/services/test_workflow_spec_service.py | Updates versioning wording in workflow spec service tests. |
| test/services/test_workflow_service.py | Updates versioning wording in workflow run engine tests. |
| test/services/test_step_output_store.py | Updates versioning wording in step output store tests. |
| test/services/test_settings_service.py | Resets settings-service module cache to avoid order-dependent tests. |
| test/scripts/init.py | Package marker for script-related tests (empty file in diff context). |
| test/providers/test_native_status_shared.py | Adjusts provider native-status test docstring wording/scope. |
| test/providers/test_mock_cli_unit.py | Adds unit tests for MockCliProvider behavior and fixture binary. |
| test/providers/test_mock_cli_coverage.py | Adds branch coverage for async init + cleanup of MockCliProvider. |
| test/providers/fixtures/bin/mock_cli | Adds deterministic mock CLI REPL fixture binary used in tests/CI. |
| test/providers/fixtures/antigravity_cli_tool_call.txt | Adds captured Antigravity CLI fixture output. |
| test/providers/fixtures/antigravity_cli_response.txt | Adds captured Antigravity CLI fixture output. |
| test/providers/fixtures/antigravity_cli_processing.txt | Adds captured Antigravity CLI fixture output. |
| test/providers/fixtures/antigravity_cli_print_mode.txt | Adds captured Antigravity CLI fixture output. |
| test/providers/fixtures/antigravity_cli_permission.txt | Adds captured Antigravity CLI fixture output. |
| test/providers/fixtures/antigravity_cli_interactive_clean.txt | Adds captured Antigravity CLI fixture output. |
| test/providers/fixtures/antigravity_cli_idle.txt | Adds captured Antigravity CLI fixture output. |
| test/providers/fixtures/antigravity_cli_error.txt | Adds captured Antigravity CLI fixture output. |
| test/models/test_workflow.py | Updates workflow-model test wording and class naming. |
| test/mcp_server/test_workflow_tools.py | Updates workflow MCP tool test wording (v3 terminology). |
| test/mcp_server/test_workflow_return.py | Updates workflow return tool test wording (v2 terminology). |
| test/mcp_server/test_handoff_equivalence.py | Updates wording for v1 equivalence test docstring. |
| test/mcp_server/test_emit_ui_coverage.py | Adds coverage tests for the new emit_ui MCP tool behavior. |
| test/fixtures/init.py | Introduces managed-subprocess fixture package for e2e/integration. |
| test/e2e/test_headless_ci.py | Adds an e2e test exercising the headless CI runner example script. |
| test/e2e/helpers/ws.py | Adds a WS helper supporting JWT via subprotocol for browser parity. |
| test/e2e/helpers/init.py | Package marker for e2e helpers. |
| test/e2e/conftest.py | Switches e2e suite to managed cao_server fixture and adds Gemini gating. |
| test/conftest.py | Adds global test env defaults, PATH prep for mock_cli, and shared fixtures. |
| test/clients/test_workflow_index_migration.py | Updates migration test wording to v2 terminology. |
| test/cli/commands/test_workflow.py | Updates CLI workflow test wording to v3 terminology. |
| test/api/test_workflow_runs.py | Updates API workflow run endpoint test wording to v3 terminology. |
| test/api/test_workflow_endpoints.py | Updates API workflow endpoints test wording to v2 terminology. |
| test/api/test_default_off_listeners.py | Adds regression tests for default-off dedicated Agent Card listener. |
| test/api/test_agui_stream_endpoint.py | Adds tests for /agui/v1/stream auth + replay + snapshot/delta. |
| test/api/test_agui_emit_ui.py | Adds tests for /agui/v1/emit_ui allow-list, sizing, disabled surface. |
| test/agent_card/test_signing.py | Adds Ed25519 Agent Card signing/verifying tests. |
| test/agent_card/test_signing_coverage.py | Adds branch coverage for signer verification failure paths. |
| test/agent_card/test_router.py | Adds integration tests for .well-known Agent Card + JWKS endpoints. |
| test/agent_card/test_router_coverage.py | Adds branch coverage for router initialization and OAuth PRM route. |
| test/agent_card/test_listener_coverage.py | Adds coverage for building/starting/stopping the dedicated listener. |
| test/agent_card/test_builder.py | Tests Agent Card builder defaults and metadata merging. |
| test/agent_card/init.py | Package marker for Agent Card tests. |
| test/a2a/test_stream_coverage.py | Adds A2A stream/router and event-bus backpressure coverage. |
| test/a2a/test_store_coverage.py | Adds A2A store lifecycle coverage. |
| test/a2a/test_rpc_background_coverage.py | Adds coverage for A2A background executor/type helpers. |
| test/a2a/init.py | Package marker for A2A tests. |
| src/cli_agent_orchestrator/utils/logging.py | Adds stderr WARNING+ stream handler alongside file logging. |
| src/cli_agent_orchestrator/telemetry/spans.py | Adds context-manager span helpers for GenAI semconv usage. |
| src/cli_agent_orchestrator/telemetry/semconv.py | Adds frozen GenAI semantic-convention attribute keys/constants. |
| src/cli_agent_orchestrator/telemetry/otel.py | Adds opt-in OTel SDK initialization + bounded shutdown behavior. |
| src/cli_agent_orchestrator/telemetry/context.py | Adds W3C trace-context inject/extract helpers. |
| src/cli_agent_orchestrator/telemetry/init.py | Exposes public telemetry helpers via package surface. |
| src/cli_agent_orchestrator/skills/workflow-author/SKILL.md | Updates “Bolt” wording to “version” in skill docs. |
| src/cli_agent_orchestrator/services/workflow_spec_service.py | Updates module docstring wording to v2/v3 terminology. |
| src/cli_agent_orchestrator/services/workflow_service.py | Updates module docstring wording to v3 terminology. |
| src/cli_agent_orchestrator/services/step_output_store.py | Updates module docstring wording to v2/v3 terminology. |
| src/cli_agent_orchestrator/services/status_monitor.py | Tightens typing around pyte screen storage and provider returns. |
| src/cli_agent_orchestrator/services/sse_bus.py | Adds backward-compatible alias and a test reset helper. |
| src/cli_agent_orchestrator/services/event_log_service.py | Adds a test reset helper for the event log singleton. |
| src/cli_agent_orchestrator/providers/mock_cli.py | Adds deterministic mock_cli provider implementation for CI/tests. |
| src/cli_agent_orchestrator/providers/manager.py | Registers mock_cli provider type in provider factory. |
| src/cli_agent_orchestrator/plugins/events.py | Adds traceparent field to plugin event base type. |
| src/cli_agent_orchestrator/plugins/builtin/event_log_publisher.py | Adds backward-compatible plugin class alias. |
| src/cli_agent_orchestrator/models/workflow.py | Updates documentation wording around “Bolt” vs “version/grammar layer”. |
| src/cli_agent_orchestrator/models/workflow_runtime.py | Updates documentation wording around “Bolt” vs “v*” terminology. |
| src/cli_agent_orchestrator/models/provider.py | Adds MOCK_CLI provider type enum entry. |
| src/cli_agent_orchestrator/mcp_server/server.py | Adds emit_ui MCP tool and updates wording in workflow tool doc. |
| src/cli_agent_orchestrator/ext_apps/static/topology.js | Updates comments to remove commit-specific wording. |
| src/cli_agent_orchestrator/ext_apps/init.py | Updates comment wording for topology widget export. |
| src/cli_agent_orchestrator/constants.py | Adds OTel service name, agent-card listener constants, and CORS origins. |
| src/cli_agent_orchestrator/cli/commands/workflow.py | Updates wording around v3 workflow verbs. |
| src/cli_agent_orchestrator/agent_card/router.py | Adds .well-known Agent Card/JWKS/OAuth PRM router. |
| src/cli_agent_orchestrator/agent_card/listener.py | Adds dedicated default-off listener server implementation. |
| src/cli_agent_orchestrator/agent_card/builder.py | Adds Agent Card document builder with provider/tool metadata. |
| src/cli_agent_orchestrator/agent_card/init.py | Exposes Agent Card public API surface. |
| src/cli_agent_orchestrator/a2a/store.py | Adds in-memory A2A task store and protocol seam. |
| src/cli_agent_orchestrator/a2a/init.py | Exposes A2A public API surface. |
| skills/workflow-author/SKILL.md | Updates “Bolt” wording to “version” in root skills docs. |
| skills/plugin.json | Adds skills plugin manifest metadata. |
| skills/cao-dev/SKILL.md | Adds dev workflow/checklist skill doc. |
| pyproject.toml | Adds deps for OTel/auth/multipart and fixes mypy python_version config. |
| examples/working-directory/repo_specialist.md | Adds a working-directory constrained supervisor profile example. |
| examples/working-directory/README.md | Documents working-directory feature and usage. |
| examples/tool-restrictions/reviewer_readonly.md | Adds read-only reviewer profile example. |
| examples/tool-restrictions/README.md | Documents tool restriction profiles and expectations. |
| examples/tool-restrictions/developer_sandboxed.md | Adds dev profile with no shell execution example. |
| examples/headless-ci/run.sh | Adds headless CI runner script for one agent session. |
| examples/headless-ci/README.md | Documents headless CI runner usage and expected exit codes. |
| examples/headless-ci/ci_developer.md | Adds CI-tuned developer agent profile. |
| examples/flow/README.md | Adds scheduled flows example documentation. |
| examples/cross-provider/data_analyst_gemini_cli.md | Adds cross-provider Gemini CLI data analyst profile example. |
| docs/pwa.md | Adds standalone PWA documentation and deployment/auth notes. |
| docs/generative-ui-implementation-2026-07-04.md | Adds generative UI design/safety/testing write-up. |
| CHANGELOG.md | Documents newly added protocol surfaces and subsystems. |
| cao_pwa/vite.config.ts | Adds Vite config including Vitest settings and dev server port. |
| cao_pwa/tsconfig.json | Adds TS config for the PWA. |
| cao_pwa/src/types.ts | Adds shared types for instance list and AG-UI event parsing. |
| cao_pwa/src/test/setup.ts | Adds vitest setup (fake IndexedDB, dialog stubs, cleanup). |
| cao_pwa/src/test/instances.test.ts | Adds IndexedDB CRUD tests for instance persistence. |
| cao_pwa/src/test/GenerativeUI.test.tsx | Adds component allow-list safety tests for Generative UI renderer. |
| cao_pwa/src/test/api.test.ts | Adds tests for AG-UI EventSource client URL building and dispatch. |
| cao_pwa/src/main.tsx | Adds PWA entrypoint. |
| cao_pwa/src/instances.ts | Adds IndexedDB persistence layer for multi-instance list. |
| cao_pwa/src/components/InstancePicker.tsx | Adds UI for adding/removing instances. |
| cao_pwa/src/App.tsx | Adds PWA app shell wiring instance picker + active instance tab. |
| cao_pwa/src/api.ts | Adds AG-UI SSE client with explicit typed-event subscriptions + reconnect. |
| cao_pwa/playwright.config.ts | Adds Playwright config for generative-ui recording harness. |
| cao_pwa/package.json | Adds PWA package scripts and dependencies. |
| cao_pwa/index.html | Adds PWA HTML entrypoint. |
| cao_pwa/e2e/generative-ui.spec.ts | Adds Playwright e2e recording proof for generative UI and refusals. |
| cao_pwa/.gitignore | Adds PWA-specific ignore rules for build/test artifacts. |
| .github/workflows/python-e2e.yml | Adds mock_cli-based Python e2e smoke workflow. |
| .github/workflows/cao-pwa-generative-ui.yml | Adds PWA build/test + Playwright recording artifact workflow. |
| .github/PULL_REQUEST_TEMPLATE.md | Adds/updates PR template with test plan checklist. |
| .devcontainer/test-ci.sh | Adds script to run CI-like checks in devcontainer. |
| .devcontainer/Dockerfile | Adds devcontainer image aligned with Playwright + tmux + uv + bun. |
| .devcontainer/devcontainer.json | Adds devcontainer configuration and bootstrap commands. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1 @@ | |||
| # tests init | |||
| [tool.mypy] | ||
| python_version = "2.1.1" | ||
| python_version = "3.11" | ||
| strict = true |
| event_type: str = "" | ||
| timestamp: datetime = field(default_factory=_utc_now) | ||
| session_id: str | None = None | ||
| # W3C trace-context (RFC 9114) for the upstream span that produced this |
| <span | ||
| role="button" | ||
| tabIndex={0} | ||
| className="cao-pwa-remove" | ||
| aria-label={`Remove instance ${inst.label}`} |
| Requires: | ||
| - Running CAO server (``uv run cao-server``) | ||
| - tmux | ||
| - A working CLI provider on PATH and authenticated (defaults to kiro_cli) | ||
| - ``ci_developer`` agent profile installed | ||
| (``cao install examples/headless-ci/ci_developer.md``) |
|
@anilkmr-a2z @fanhongy please have a look, this is the awesome feature that we have always been looking for |
There was a problem hiding this comment.
Thanks for this — the AG-UI adapter is a genuinely good direction for CAO. It's extremelly helpful for user to operate CAO via the dashboard to see live stream and send approval.
My main ask is decomposition. At 157 files / ~16k insertions squashed into one commit, this is really 5–6 features (AG-UI stream + generative UI, A2A/Agent Card, the cao_pwa dashboard, OpenTelemetry, mock_cli/test-fixture overhaul, docs/examples). You already offered to split it — I'd like to take that offer. My suggestion: land AG-UI stream + emit_ui + PWA first (smallest trust delta, improves every existing user), and hold A2A until the auth gap below is closed.
| the task to ``canceled`` if it isn't terminal yet; otherwise | ||
| returns ``TASK_ALREADY_TERMINAL``. | ||
|
|
||
| Authentication is enforced via the JWKS published at |
There was a problem hiding this comment.
A2A JSON-RPC has no authentication, despite the docstring saying it does. This says "Authentication is enforced via the JWKS … once auth is enabled," but there's no auth wiring in either A2A router.
Reproduced with auth explicitly enabled and no credentials on the request:
is_auth_enabled -> True
task.send status: 200 # anonymous write accepted
stored task present -> True # and persisted
Since the changelog advertises CAO_AGENT_CARD_HOST=0.0.0.0 for external discoverability, this exposes an unauthenticated task.send/get/cancel write surface. Please either wire real token/scope checks on the listener
|
@CorrineTan would be good to take a look for your TUI task |
gutosantos82
left a comment
gutosantos82
left a comment
There was a problem hiding this comment.
PR Review: #387 — AG-UI Protocol L1 adapter + agentic-protocol surface
Summary
This PR implements the AG-UI L1 SSE adapter proposed in #386 (services/agui_stream.py,
GET /agui/v1/stream, emit_ui) and bundles a large surrounding surface: A2A JSON-RPC +
signed Agent Card listener, OpenTelemetry, a standalone React PWA, generative-UI, a
mock_cli provider — and deletion of the q_cli and gemini_cli providers. The AG-UI
core itself is genuinely well-built: dynamic verification confirmed it is default-off
(404 with no flags, no listener bound), the message-body redaction holds by construction,
and the generative-UI allow-list is enforced at both the adapter and the producer endpoint.
However, the bundled A2A listener has a real auth bypass and an unbounded-store DoS, and
the PR is ~400 files — far beyond #386's PoC scope. Recommendation: Request changes —
fix the A2A auth/DoS findings and split the PR before merge.
Blocking (must fix before merge)
- [security] a2a/rpc.py:268, agent_card/listener.py:59-62, api/main.py:432-433 🆕 — The
A2A JSON-RPC router (POST /a2a/v1/rpc: task.send/get/cancel) and the A2A stream/REST
routes are mounted on the:9890listener with no authentication dependency at all;
build_a2a_router/build_stream_router/build_listener_appattach no
require_any_scope/auth middleware, yet the docstrings (rpc.py:24-25, stream.py:21-22)
falsely claim auth "is enforced via the JWKS." With the documented
CAO_AGENT_CARD_HOST=0.0.0.0opt-in (or a container bridge), any network peer can
submit/cancel/read A2A tasks with zero credentials even when auth is configured — a
full auth bypass on an external-facing surface (corea2a/+api/main.py, highest-risk
path). Fix: gate all A2A routes behind scopes (task.send/cancel →cao:write,
task.get/stream →cao:read), or refuse to mount them whenis_auth_enabled()and no
enforcement is wired; correct the docstrings. - [security] a2a/store.py:38, a2a/rpc.py:129-152 🆕 —
InMemoryTaskStoregrows
unbounded ("tasks live until explicitly deleted or process restart"). Combined with the
unauthenticated, peer-controllabletask.sendabove, a remote peer can drive unbounded
memory growth → remote memory-exhaustion DoS oncao-server. Fix: cap store
size / TTL-evict and require auth. - [scope] whole PR (~400 files / +16k lines) 🆕 — Bundles far more than #386's "AG-UI L1
adapter +emit_ui" PoC: A2A transport, Agent Card/JWKS, OTel, native workflow spec, a
standalone PWA, generative-UI,mock_cli, and deletion of two providers. The PR body
itself offers to decompose it. A PR this size in coresrc/invites exactly the kind of
auth defect found above to slip through. Fix: split into the stack the author proposes —
land the reviewed, verified AG-UI core (#386 scope) first; hold A2A until auth is wired.
Important (should fix)
- [consistency] PR description ↔ implementation mismatch 🆕 — The body advertises
"orchestration topologies (DAG/swarm/Polecat), WAL+replay, a Cedar-style policy queue, a
3-layer cache." A grep ofsrc/forpolecat|Cedar|policy_queue|PolicyQueue|WAL|swarm
returns zero matches — these features do not exist in the code. The doc's "three
layers" refers to the refusal path (Python/React/replay), not a cache. Either the features
are missing or the description oversells; reconcile before merge. - [conventions/consistency] CHANGELOG.md:773-807 — missing "Removed" section 🆕 — The PR
fully deletesproviders/q_cli.py(Amazon Q) andproviders/gemini_cli.py(816 lines),
dropsQ_CLI/GEMINI_CLIfromProviderType, and removes bothmanager.pybranches.
README/CODEBASE.md were updated, but CHANGELOG has only Added/Changed/Deprecated/Fixed.
Removing two user-facing providers is a breaking change (SemVer/Keep-a-Changelog). Fix:
add a### Removedentry forq_cli+gemini_cli. - [consistency] docs/inbox-delivery.md — doc↔code drift, introduced this PR 🆕 — This PR
renamedcheck_and_send_pending_messages()→deliver_pending()(inbox_service.py:58) and
updated CODEBASE.md, but the new content it added toinbox-delivery.md(Reconciliation
Sweep / OpenCode Poller sections) still names the dead symbol 4×. Fix:
s/check_and_send_pending_messages()/deliver_pending()/in inbox-delivery.md. - [security] api/main.py:744, 2545 — JWT in query string leaks to access logs 🆕 —
/agui/v1/stream?access_token=<JWT>puts the bearer in the URL, and the main app's
uvicorn.run()does not setaccess_log=False(the agent_card listener does, at
listener.py:113). The full path+query — including the JWT — lands in uvicorn/proxy/Referer
logs, replayable untilexp. Fix: disable/scrub access logging for this route, document
short token TTLs, prefer a short-lived-ticket handshake over a raw JWT in the URL. - [security] api/main.py:781, 1894-1897 — token-parse exceptions not caught 🆕 —
extract_scopes_from_token()raises on invalid/expired/malformed JWTs, but the SSE path
catches nothing and_extract_ws_scopesonly catchesHTTPException, so a bad token
surfaces as an opaque 500 / handshake error instead of a clean 401 /4401. It fails
closed (no data served — not a bypass), but muddies auth telemetry. Fix: catch broad
Exception→None→ 401 /4401. - [conventions] Inclusive-language violations, introduced 🆕 — New comments/docs use
banned terms:whitelistin services/audit_log.py, services/memory_scoring.py,
services/memory_service.py, and event-enforcement comments; "Master switch" rows in
docs/memory.md and docs/mcp-apps.md. Repo + org convention mandates allowlist/denylist/
primary. No CI lint gate exists, so the build won't fail, but this is a hard convention.
Fix: rename to allowlist / "primary switch". (Pre-existing(master*)git-branch strings
in deleted fixtures are not in scope.)
Nits (optional)
- [conventions] pyproject.toml:21049-21057 — OTel, authlib, pyjwt[crypto],
python-multipart added as unconditional runtime deps though all their features are
default-off; consider optional extras ([otel],[a2a]) to keep base install lean.
(uv.lock is updated and deps are pinned — no hygiene red flag.) - [conventions] CI toolchain — the new cao-mcp-apps job runs
npm install(ci.yml:361,427)
while python-e2e usesnpm ci; with committed lockfiles the convention isnpm ci. Also
bun (devcontainer) vs npm (CI) is a minor mix. - [consistency] docs/generative-ui-implementation-2026-07-04.md — reads as a dated
impl/design log ("Created July 4 2026", "Status: Implemented") linked from CHANGELOG+README;
dated logs drift — fold into docs/pwa.md or docs/design/ without the date. - [consistency] api/main.py:644-655 —
_agui_enabled()also returns True when the
pre-existingCAO_MCP_APPS_ENABLEDis set (not only the newCAO_AGUI_ENABLED); the
byte-identical-with-no-flags claim still holds, but tighten the "no new flags" wording. - [tests] test/api/test_agui_stream_endpoint.py:52-54 — the
since=replay boundary is
stubbed at the endpoint (_FakeLog.historyignoressince), so main.py:823-825 wiring
isn't exercised end-to-end; the filter logic itself is well-covered at the unit layer. - [tests] — no test for a malformed/expired bearer on the stream endpoint (only "no
token" / "wrong scope" paths) — pairs with the uncaught-exception finding above. - [consistency] — README/CODEBASE prose reformatting beyond the feature adds noise to an
already-huge PR.
Reviewer disagreement to resolve
.coverage-baseline.jsoncommitted — the consistency reviewer flagged it as a committed
generated artifact that isn't gitignored and should be removed; the conventions reviewer
judged it intentional ratchet-floor config (like a checked-in baseline), which is fine.
Lean toward the conventions read (it's config the coverage ratchet reads), but confirm it's
meant to be tracked and, if so, leave a one-line note in the file/docs so it isn't mistaken
for a stray artifact.
Tests
Strong, genuinely meaningful shift-left — not coverage-gaming. The *_coverage.py files
assert real behavior/branches (transition guards, executor-exception → FAILED metadata,
drop-oldest back-pressure, tampered-signature → InvalidSignature). Security properties are
asserted the right way: redaction tests stringify the whole output and assert the secret is
absent; allow-list refusal is tested at both adapter and producer (400) layers. Malformed
JSON-RPC, auth 401/403, and RFC-6902 add/replace/remove/pointer-escaping are all covered.
Gaps are nits only (endpoint-level since= replay stubbed; no malformed-token test).
Verification
Ran in a clean Docker toolchain (uv sync succeeded; numpy built fine in-container). All
probes executed the PR's real code:
- ✓ VERIFIED "35 AG-UI adapter tests pass" — mapping (28) + stream_endpoint (3) + emit_ui
(4) = 35, 40 passed / 0 failed including default-off + mcp coverage tests. - ✓ VERIFIED "strictly additive & default-off, byte-identical with no flags" — with all
flags unset:GET /agui/v1/stream→ 404,POST /agui/v1/emit_ui→ 404,
agent_card_listener→ None (no:9890bind). - ✓ VERIFIED "message bodies never appear on the wire" — fed
SECRET-XYZthrough both
event shapes; output is metadata-only, secret absent. Redaction is by construction. - ✓ VERIFIED "off-list components REFUSED at the adapter" —
iframe/script→
RAW+rejected_component; allow-listed components round-trip; also refused server-side at
emit_ui(400). Defense-in-depth confirmed. - ✓ VERIFIED coverage —
agui_streamat 99% (1 miss: non-dict props fallback). - ⁇ NOT VERIFIED — live E2E through a running PWA + SSE reconnect (needs a live
cao-server + browser). The SSE snapshot/delta/replay generator is covered by the passing
endpoint test.
Note: the verifier confirmed the AG-UI surface is additive/default-off, but this is
distinct from the A2A listener auth findings above and the provider deletions — those
were not contradicted, they are simply outside what the runnable AG-UI probes covered.
Prior feedback (already raised — not restating)
- No substantive maintainer critique to dedupe against. Existing human activity is the
author's own coverage note and @haofeif tagging @anilkmr-a2z / @fanhongy to look ("the
awesome feature we've always been looking for"). Enthusiasm noted; no findings overlap.
Verdict
Request changes — the AG-UI core (#386 scope) is solid, verified, and default-off, but the
bundled A2A listener ships an unauthenticated task API (auth bypass) and an unbounded task
store (remote DoS), both in core src/. Combined with description↔code mismatches and a
~400-file scope that far exceeds #386, this should be split: land the verified AG-UI core
first, and hold A2A until auth is wired and the store is bounded.
Correctness review did not return in time; not covered as a standalone angle (though
correctness-adjacent logic — RFC-6902 patching, SSE mapping, JSON-RPC handling, default-off
gating — was exercised by the verifier and tests reviewer).
|
Thanks all — @gutosantos82 for the thorough multi-angle pass (the dynamic Where we fully agree (fixing these)
Decomposition plan (taking @fanhongy up on the offer)Two PRs, structured exactly along the verified/blocked boundary:
Corrections for the record (with evidence)
Why keep pushing on AG-UIThe core value is one standard surface over N heterogeneous CLI agents — Kiro I'll push the reshaped PR-A shortly and link PR-B here when it's ready for |
anilkmr-a2z
left a comment
anilkmr-a2z
left a comment
There was a problem hiding this comment.
Solid work shipping the AG-UI L1 adapter and supporting protocol surface -- default-off design is the right call. Three must-fix findings (security boundary, unbounded state growth, missing task ID validation) and two nits below.
| asyncio.create_task( | ||
| _run_task_in_background(task, executor, store, bus), | ||
| name=f"a2a-task-{task.id}", | ||
| ) |
There was a problem hiding this comment.
must-fix -- missing id validation in task.send allows peer-controlled key injection into the in-memory store
When the peer supplies an id field, it is accepted verbatim (task.id = str(data["id"]) in Task.from_dict). Because the store key is just self._tasks[task.id], a peer can overwrite any existing task by resubmitting with the same id -- effectively cancelling or replacing another peer's in-flight work. This is a state-integrity issue.
Fix: either (a) always generate a server-side UUID and ignore the peer-supplied id, or (b) check await store.get(task.id) and reject with INVALID_PARAMS if the id already exists (i.e., idempotent-create semantics, not upsert).
| """ | ||
|
|
||
| def __init__(self) -> None: | ||
| self._tasks: dict[str, Task] = {} |
There was a problem hiding this comment.
must-fix -- unbounded in-memory store growth with no eviction
Every task.send call appends to self._tasks and nothing ever removes entries (tasks in terminal states are never reaped). Under sustained peer traffic this is an OOM vector. The main API's StepOutputStore already has the WORKFLOW_OUTPUT_STORE_MAX_ENTRIES eviction pattern -- apply the same LRU/TTL eviction here so terminal tasks are reaped after a configurable window (e.g., 15 min or 1000 entries).
| version="1.0.0", | ||
| docs_url=None, # No interactive docs on the public listener. | ||
| redoc_url=None, | ||
| openapi_url=None, |
There was a problem hiding this comment.
must-fix -- A2A JSON-RPC endpoint mounted on the agent-card listener has no authentication enforcement
The agent-card listener deliberately skips TrustedHostMiddleware to be externally discoverable. When a2a_router is mounted alongside the card, POST /a2a/v1/rpc is exposed to the network with zero authentication -- any peer can submit arbitrary tasks or cancel existing ones. The PR description says "Authentication is enforced via the JWKS published at /.well-known/jwks.json once auth is enabled" (rpc.py docstring) but no middleware or dependency on this listener actually verifies a token before dispatching RPC methods.
Fix: add a FastAPI Depends() on the /a2a/v1/rpc route (or as a router-level dependency on a2a_router) that validates a Bearer token against the JWKS when auth is enabled, same as the main API's require_any_scope pattern. Without this, the loopback default mitigates risk but the operator who sets CAO_AGENT_CARD_HOST=0.0.0.0 gets an open unauthenticated RPC endpoint.
| known = {"id", "state", "messages", "metadata", "artifacts", "created_at", "updated_at"} | ||
| extra = {k: v for k, v in data.items() if k not in known} | ||
| return cls( | ||
| id=str(data["id"]), |
There was a problem hiding this comment.
nit -- Task.from_dict raises unhandled KeyError when id is missing
str(data["id"]) will raise KeyError if the peer's payload omits id. Since _handle_task_send already generates an id when task.id is falsy, use data.get("id", "") instead of data["id"] so the fallback path works without the caller needing a try/except.
| "jsonschema>=4.25", | ||
| # Ported net-new subsystems (fork plauzy/cao): | ||
| # OpenTelemetry GenAI telemetry (telemetry/, observability/span_consumer). | ||
| "opentelemetry-api>=1.27.0", |
There was a problem hiding this comment.
nit -- authlib>=1.7.1 added as a production dependency but only used in test fixtures
Grepping the diff, authlib is imported only in test/conftest.py, test/fixtures/jwks_server.py, and test/fixtures/jwt_factory.py. If it is not needed by any src/ module at runtime, it should be a dev dependency (under [dependency-groups] dev) rather than inflating the production install.
Summary
Implements the AG-UI (Agent–User Interaction) Protocol L1 adapter proposed in #386, contributed as the working proof-of-concept that issue references, alongside the surrounding agentic-protocol surface and generative-UI capability it was built with.
Everything here is strictly additive and default-off: every new surface is gated by an env/config flag or is a new module that isn't imported unless enabled. With no flags set,
cao-serverbehavior is byte-identical — no new endpoints respond and no new listeners bind. The existing localhost dashboard, MCP servers, and REST API are untouched.AG-UI adapter (the #386 core)
services/agui_stream.py— a pure, table-driven L1 adapter mapping CAO's normalized event vocabulary onto AG-UI typed SSE events, per the mapping table in [Feat] AG-UI Protocol support as a composable construct layer — one face over many CLI agents #386.GET /agui/v1/stream— behindCAO_AGUI_ENABLED, emitsSTATE_SNAPSHOTon connect + RFC-6902STATE_DELTApatches thereafter, with?since=history replay and query-parameter JWT auth for browserEventSourceclients.test/services/test_agui_stream_mapping.py).This satisfies the read-only Phase 0–1 scope of #386 (AC1 default-off, AC2 L1 stream, AC4 privacy boundary), leaving the L2 constructs and bidirectional interrupts for follow-up phases.
Also included (all opt-in)
Protocols — A2A JSON-RPC + signed Agent Card / OAuth PRM on a default-off loopback listener (
CAO_AGENT_CARD_ENABLED); an Agent Client Protocol server (cao-acp) for editor integrations.Generative UI — agents author UI intents rendered uniformly across providers via a closed, safe allow-list of named components with JSON props (no HTML/script/eval); off-list components are refused at the adapter, the React component, and the replay artifact. React renderer in the L2 dashboard PWA (
cao_pwa/), a browser-openable deterministic replay artifact, and a Playwright recording harness + workflow.Observability / orchestration / persistence / policy / cache — OpenTelemetry GenAI telemetry + W3C
traceparent; ASI integrity evaluator + kill switch; orchestration topologies (DAG, swarm, Polecat, topology router); WAL + replay; a Cedar-style policy queue; a 3-layer cache.Providers, auth, frontends — a credentials-free
mock_clifor CI; WebSocket auth on/terminals/{id}/ws(token via thecao.bearer.<jwt>subprotocol,4401/4403close codes, read-only viewers' input dropped) — default-off preserves the byte-identical no-auth localhost contract.✅ Shift-left testing — proof it works
Genuine shift-left proof, captured locally (not a mock-up) and reproduced in CI:
/agui/v1/streamendpoint + theemit_uiproducer.cao_pwaunit/component tests pass, including 10 GenerativeUI tests and the 3-layer safety-refusal assertions;tscclean;vite buildwithin the bundle-size budget.services/agui_stream.to_agui_eventadapter and asserts heterogeneous providers render uniform cards and the off-listiframecomponent is refused.a2a/store,a2a/types,agent_card/router,agent_card/signing, andproviders/mock_clito 100%,a2a/rpcto 98%, andagent_card/listenerto 95% (previously 65–89%). All CI checks — including the coverage ratchet — pass.Live playback — heterogeneous CLI agents (
kiro_cli,claude_code,codex) each author UI intents that render uniformly on one surface; the AG-UI event stream is on the right; the off-listiframecomponent is REFUSED, never rendered:Final composed surface (full-page screenshot from the Playwright run):
Full-motion video (WebM): https://raw.githubusercontent.com/plauzy/cli-agent-orchestrator/feat/agentic-protocols-generative-ui/docs/media/agui-generative-ui-demo.webm
Notes
agui_stream.pymodule, per the hedge discussed in [Feat] AG-UI Protocol support as a composable construct layer — one face over many CLI agents #386.SupervisorDashboardStream,AgentHandoffWithApproval,CrossProviderStateSync,MultiAgentSessionTimeline), bidirectional generative-UI actions, andSTATE_DELTAdebounce.Implements #386.