[avabbbb]

Why

The current validation work mixes three different things:

1. Playwright-driven frontend regression
2. deterministic CLI / pipeline smoke
3. real Coding-Agent operation of OfferU

The first two are useful, but they do not prove that OMP/SWE-2 (or another real Agent) received a natural-language goal, discovered OfferU Skills, selected Operations, issued model-facing tool calls, stopped at Proposal/HITL, and continued from the resulting Career Truth.

The current scripted omp_executor.py is especially easy to over-interpret: it watches a request file and then Python control flow chooses a known Operation sequence. That proves plumbing, not SWE-2 autonomy.

This PR makes that distinction explicit before more Eval work builds on the wrong success definition.

What this PR adds

docs/evals/AGENT_NATIVE_ACCEPTANCE.md

A detailed acceptance contract covering:

• frontend Playwright regression vs deterministic CLI smoke vs Agent-native tool-use vs computer-use
• why scripted executors are not model evidence
• the canonical OfferU Golden Path:
  natural language → real Agent → Skill → model-issued bash/tool → CLI/Bridge → Operation Registry → Proposal/HITL → human UI approval → Career Truth → Agent continuation
• runtime/model identity requirements
• model-issued tool-call evidence
• trusted Operation execution evidence
• outcome vs trajectory
• multi-turn continuity
• cancellation / late-result handling
• private-data isolation
• visible HITL acceptance
• pass^3 reliability
• report template and status vocabulary
• guidance for separating Connector Registry work from Eval/Harness experimentation

AGENTS.md

Adds two hard guardrails:

1. Playwright cannot stand in for Agent-native acceptance.
2. A scripted executor cannot be labelled OMP/SWE-2/Codex Agent E2E unless the actual model/harness is verifiably involved.

Intended effect on PR #14

This does not reject the ApplicationActionConnector Registry work.

The recommendation is to keep Connector Registry/capability-matrix work reviewable as product infrastructure, while reclassifying or splitting the Eval-specific pieces:

• omp_executor.py
• resume eval suite
• runner/grader handoff changes
• E2E-EVAL-REPORT.md

If those files remain in #14, the report should state:

DETERMINISTIC_PIPELINE_SMOKE = PASS
FRONTEND_PLAYWRIGHT_FLOW     = PASS

AGENT_NATIVE_E2E             = NOT_RUN
MODEL_IDENTITY               = UNVERIFIED
MODEL_TOOL_SELECTION         = NOT_VERIFIED

until a real Agent-native run exists.

External references

• OMP distinguishes the model-facing bash tool-call surface from user bang/RPC shell helpers:
  https://github.com/can1357/oh-my-pi/blob/main/docs/bash-tool-runtime.md
• OMP model-facing bash tool:
  https://github.com/can1357/oh-my-pi/blob/main/docs/tools/bash.md
• Anthropic's Agent Eval guidance distinguishes trajectory/transcript from final environment outcome:
  https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

Non-goals

• no product/runtime code changes
• no deletion of Playwright tests
• no deletion of deterministic smoke tests
• no new Eval framework
• no claim that visible headless=false automation is required

The goal is simply to make sure future reports prove the product claim they say they prove.

[avabbbb]

Concrete implementation of this acceptance contract is now in stacked PR #16: real OMP RPC process, model/thinking identity snapshots, model-issued tool_execution_* capture, no scripted Operation chooser, and explicit denial of app.cli confirm at the OMP eval layer.

[avabbbb]

The acceptance intent from this PR has now been distilled into the current Eval path rather than merged as a parallel 792-line authority.

Current location:

• PR feat(eval): run real OMP agent through RPC on current main #16 / docs/evals/LIVE_EVAL.md: validation classes + Agent-native PASS gates
• PR feat(eval): run real OMP agent through RPC on current main #16 / docs/evals/GRADING.md: trusted execution vs trajectory boundary
• AGENTS.md on main: Playwright/scripted-executor guardrails

The real implementation path is now #16, rebuilt directly on current main. Keeping this PR open would create a second Eval authority, so I’m closing it as superseded by the distilled contract + #16 implementation.