Add isolated operator workload fixture - #933
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@codex review |
There was a problem hiding this comment.
🟡 Changes recommended
There are a few correctness/documentation issues (Markdown table rendering, potential guardian shutdown race, and validator buffer sizing) that should be addressed before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds an opt-in “isolated operator workload fixture” under bench/ that runs the real relay + pairing/auth boundary while serving deterministic, synthetic read + stream responses for observing unchanged operator clients, and documents the resulting qualification/evidence boundary across the docs set.
Changes:
- Introduces a local fixture runner (
bench/operator_workload_fixture.py) that provisions a pinned relay binary, generates fresh authority state + pairing artifacts, and serves a synthetic FastAPI surface behind the real operator gateway authorization. - Adds a source-pinned schema validator (
bench/validate_operator_fixture.cjs) plus a small loopback protocol test client and watchdog lease process, with focused pytest coverage. - Documents the fixture contract, lifecycle, and evidence limits; cross-links the boundary in architecture and contributor docs.
File summaries
| File | Description |
|---|---|
| website/docs/operator-workload-fixture.md | New contract doc for the fixture’s lifecycle, API subset, and evidence limits. |
| website/docs/architecture.md | Adds an explicit architecture boundary section for the fixture (not a Node component). |
| website/docs/architecture-reference.md | Adds a dense reference entry describing the fixture entrypoints and scope. |
| website/docs/api-guide.md | Notes that the fixture serves a synthetic subset of API paths for local qualification. |
| CONTRIBUTING.md | References the fixture as a deterministic synthetic observation tool (not capacity evidence). |
| CLAUDE.md | Updates agent-facing architecture notes to include the fixture and its limits. |
| bench/operator_workload_fixture.py | Implements the expiring local fixture runner (relay provisioning, TLS gateway, synthetic app). |
| bench/operator_fixture_app.py | Implements the synthetic canonical-read + SSE/PCM endpoints behind real operator auth. |
| bench/operator_fixture_lease.py | Adds an independent watchdog that reaps the relay on expiry/parent EOF. |
| bench/operator_fixture_client.py | Adds a loopback-only, no-retry TLS-over-WebSocket test client for the fixture contract. |
| bench/validate_operator_fixture.cjs | Adds source-pinned schema validation against an external app checkout + commit. |
| bench/tests/test_operator_workload_fixture.py | Adds opt-in integration coverage for real relay + pairing + synthetic reads. |
| bench/tests/test_operator_fixture_lease.py | Tests watchdog expiry/parent-EOF behavior and unexpected child exit handling. |
| bench/tests/test_operator_fixture_app.py | Tests generated bodies, real token lifecycle, mutation unavailability, and bounds. |
Review details
Suppressed comments (1)
bench/operator_workload_fixture.py:241
- With
_watch_guardian()updated to account for shutdown, pass the shutdown event into the monitor task so normal teardown/expiry can be distinguished from an early guardian exit.
async with asyncio.TaskGroup() as group:
monitor = group.create_task(_watch_guardian(guardian))
listener = group.create_task(
- Files reviewed: 14/14 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a397d561fe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
🟡 Changes recommended
The fixture’s QR writer likely fails at runtime because it saves to an fd-backed file object without specifying an explicit image format.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 14/14 changed files
- Comments generated: 1
- Review effort level: Lite
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 754cfaf2f0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Summary
Validation
Boundary
Targets the isolated relay initiative, not dev or a release. This is not physical-device capture, an observed workload profile, hosted qualification, or a supported-capacity claim. No production changes, provider resources, or app protocol changes.