Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ release tags and PEP 440 for the Python package.

## Unreleased

- Coordinated Tasks can opt into pinned primary-deliverable acceptance: required fields,
typed JSON equality, and unit-aware rate thresholds with optional result-claim verification.
Missing evidence needs review; deterministic acceptance is separate from execution completion.
Evidence-bound, audited human decisions preserve failed checks; an accepted-result endpoint
gates primary output export. The bilingual Console exposes configuration, results and controls.
Unaccepted opted-in Tasks skip automatic memory capture; completion cards omit result excerpts.
- Organizational Memory no longer equates different content in one Memory Type with contradiction.
Optional normalized subject keys enable scoped competing-record checks, exposed as UNKNOWN,
NO_COMPETING_RECORDS, or REVIEW_REQUIRED rather than a false confirmed conflict. Assessments use
Expand Down
58 changes: 58 additions & 0 deletions alembic/versions/20260930_0058_deliverable_acceptance.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
"""Add audited deliverable acceptance decisions without changing task execution status.

Revision ID: 20260930_0058
Revises: 20260930_0057
"""

import sqlalchemy as sa

from alembic import op

revision = "20260930_0058"
down_revision = "20260930_0057"
branch_labels = None
depends_on = None

_OLD_ACTIONS = (
"ACCEPT_CANDIDATE",
"REJECT_TASK",
"INCREASE_BUDGET_AND_RESUME",
"RECONCILE_MCP_SUCCEEDED",
"RECONCILE_MCP_FAILED",
"BIND_A2A_REMOTE_TASK",
"RECONCILE_A2A_NOT_DELIVERED",
"RECONCILE_RUNTIME_SUCCEEDED",
"RECONCILE_RUNTIME_FAILED",
"RECONCILE_RUNTIME_CANCELED",
"RECONCILE_RUNTIME_TIMED_OUT",
)
_NEW_ACTIONS = ("ACCEPT_DELIVERABLE", "REJECT_DELIVERABLE")


def _replace_constraint(actions: tuple[str, ...]) -> None:
op.drop_constraint("ck_task_resolutions_action", "task_resolutions", type_="check")
op.create_check_constraint(
"ck_task_resolutions_action",
"task_resolutions",
"action IN (" + ", ".join(repr(action) for action in actions) + ")",
)


def upgrade() -> None:
_replace_constraint(_OLD_ACTIONS + _NEW_ACTIONS)


def downgrade() -> None:
count = (
op.get_bind()
.execute(
sa.text(
"SELECT count(*) FROM task_resolutions "
"WHERE action IN ('ACCEPT_DELIVERABLE', 'REJECT_DELIVERABLE')"
)
)
.scalar_one()
)
if count:
raise RuntimeError("Cannot downgrade while deliverable acceptance audit records exist")
_replace_constraint(_OLD_ACTIONS)
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ For the exact shipped boundary, use [implementation status](implementation-statu
| Try AgentMesh without a key | [Five-minute guide](getting-started.md) | [Customer-feedback task with real-model screenshots](scenarios/customer-feedback-deepseek.md) |
| Configure employees, models, or memory | [Model and memory setup](model-and-memory-setup.md) | [SSH-only credential administration](operations/model-connections-ssh.md) |
| Notify a Feishu group | [Feishu notifications](integrations/feishu-notifications.md) | [Feature gates](architecture/modules/feature-gates.md) |
| Check a collaborative result before delivery | [Deliverable acceptance](deliverable-acceptance.md) | [Customer-feedback walkthrough](scenarios/customer-feedback-deepseek.md) |
| Use a business scenario | [Customer-feedback walkthrough](scenarios/customer-feedback-deepseek.md) | [Market-research scenario](scenarios/market-research.md) |
| Deploy or operate AgentMesh | [Administrator best practices](best-practices.md) | [SLO and restore](operations/slo-and-restore.md), [runtime rollback](operations/runtime-direct-cutover-rollback.md) |
| Build an extension | [Runtime Extension Protocol](architecture/modules/runtime-extension-protocol.md) | [Extension Starter](https://github.com/0YHR0/AgentMesh-Extension-Starter), [formal module contracts](architecture/modules/formal/README.md) |
Expand Down
1 change: 1 addition & 0 deletions docs/README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
| 不用 Key 试用 AgentMesh | [5 分钟上手](getting-started.zh-CN.md) | [真实模型客户反馈教程](scenarios/customer-feedback-deepseek.zh-CN.md) |
| 配置员工、模型或记忆 | [模型与记忆配置](model-and-memory-setup.zh-CN.md) | [仅通过 SSH 管理凭据](operations/model-connections-ssh.md) |
| 通知飞书群 | [飞书通知配置](integrations/feishu-notifications.zh-CN.md) | [Feature Gate](architecture/modules/feature-gates.md) |
| 交付前检查协作结果 | [交付验收](deliverable-acceptance.zh-CN.md) | [客户反馈协作教程](scenarios/customer-feedback-deepseek.zh-CN.md) |
| 跑一个业务场景 | [客户反馈协作教程](scenarios/customer-feedback-deepseek.zh-CN.md) | [市场研究场景](scenarios/market-research.zh-CN.md) |
| 部署或维护 AgentMesh | [管理员最佳实践](best-practices.zh-CN.md) | [SLO 与恢复](operations/slo-and-restore.md)、[Runtime 回退](operations/runtime-direct-cutover-rollback.md) |
| 开发一个扩展 | [Runtime 扩展协议](architecture/modules/runtime-extension-protocol.md) | [Extension Starter](https://github.com/0YHR0/AgentMesh-Extension-Starter)、[正式版模块契约](architecture/modules/formal/README.md) |
Expand Down
109 changes: 109 additions & 0 deletions docs/deliverable-acceptance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Deliverable acceptance

[English](deliverable-acceptance.md) | [简体中文](deliverable-acceptance.zh-CN.md)

**An employee finishing work is not the same as the result being acceptable.**
Optional acceptance checks make that distinction visible for collaborative Tasks.
Simple Tasks do not need this configuration and keep their existing behavior.

## Example: evaluating a pilot launch

A Researcher collects pilot observations, an Analyst compares them, and an Editor
produces a recommendation. You require a `summary` field and a support-ticket rate
of no more than 12 tickets per 100 buyers. The denominator is **buyers**, not orders:
18 tickets and 100 orders cannot establish this rate if buyer count is missing.

In the Console:

1. Create a collaborative Task and bind the three published employees. Set Research
as the Analyst's dependency, and both as the Editor's dependencies.
2. Open the optional delivery restrictions, select the Editor as primary if the plan
has multiple terminal work items, and enable deliverable acceptance.
3. Require the result path `summary`. Optionally enable the rate check: enter the
numerator and denominator values with their exact units, scale `100`, operator
`LTE` (less than or equal), and threshold `12`. Leave an unavailable value blank;
do not substitute an unrelated quantity.
4. If you need a person to approve the result even after checks pass, enable human
review. Run the Task and inspect its independent acceptance panel.

You can also check an optional structured rate claim in the primary output. For
example, a result `support_rate: {value: 10, unit: "ticket/buyer", scale: 100}` is
checked against the configured source quantities. Checking the input facts alone
does **not** prove that every statement in the generated text is correct.

## Results and control

| Acceptance state | Meaning | Accepted-result export |
| --- | --- | --- |
| Not ready | Execution has not produced the pinned primary result | Blocked |
| Passed | All required configured checks pass | Allowed |
| Failed | At least one required check fails | Blocked |
| Needs review | Required evidence is missing, or human review is mandatory | Blocked |
| Human accepted | An authorized person explicitly accepts the exact result/evidence | Allowed; failed/unknown checks remain visible |
| Human rejected | An authorized person rejects the exact result/evidence | Blocked |

Human acceptance is an **audited override**, not proof that a failed check passed.
It requires the `human_resolution` feature and Task resolution permission. The
decision records the actor, reason, and digests of the policy, output, and evidence.
Refresh before deciding if the output snapshot has changed. Execution stays
`COMPLETED`; it is not confused with the execution-time `WAITING_APPROVAL` barrier.

The accepted-result download only returns the pinned primary deliverable. Other
attachments are not implicitly validated. Raw results remain available for audit;
they are not a substitute for the accepted-result endpoint.

## API example

Include this optional property in a Coordinated Task creation request. Input
quantities are explicit facts supplied by the caller, not an independent data audit.

```json
{
"acceptance_policy": {
"require_human_review": false,
"checks": [
{"key": "summary", "description": "Summary is present", "kind": "OUTPUT_PATH_EXISTS", "path": ["summary"]},
{
"key": "support-rate", "description": "At most 12 tickets per 100 buyers",
"kind": "RATE_THRESHOLD",
"numerator": {"source": "TASK_INPUT", "path": ["acceptance_facts", "tickets"], "unit": "ticket"},
"denominator": {"source": "TASK_INPUT", "path": ["acceptance_facts", "buyers"], "unit": "buyer"},
"scale": 100, "operator": "LTE", "threshold": 12,
"claim": {"source": "DELIVERABLE", "path": ["support_rate"], "unit": "ticket/buyer"}
}
]
},
"input": {"acceptance_facts": {"tickets": {"value": 18, "unit": "ticket"}}}
}
```

This fragment intentionally omits buyers: the required rate check is `UNKNOWN`,
so the result needs review instead of receiving a fabricated pass. Supply the rest
of the normal Task fields, plan, and published Agent bindings when creating it.

Supported checks are `OUTPUT_PATH_EXISTS` (present and non-null),
`OUTPUT_PATH_EQUALS` (typed JSON equality), and `RATE_THRESHOLD` (`LTE`/`GTE`).
Quantity sources are `TASK_INPUT` and `DELIVERABLE`; units must match exactly.
Negative numerators, nonpositive denominators, nonfinite numbers, invalid units,
and inconsistent claims cannot pass. Policies are size bounded and pinned at
creation; a plan patch cannot remove or change the primary selection.

`GET /api/v1/tasks/{id}` includes `deliverable_acceptance`.
`GET /api/v1/tasks/{id}/accepted-deliverable` rejects an unaccepted or unconfigured
result. Human decisions use `POST /api/v1/tasks/{id}/deliverable-acceptance/decision`
with `decision`, `reason`, both expected policy/output digests, and preferably an
`Idempotency-Key`. The server derives the actor from the authenticated principal.

## Boundaries

- This is a deterministic contract checker, not a general truth detector, semantic
reviewer, or automatic judge of arbitrary prose. A model's self-reported verdict
does not grant acceptance.
- It does not undo external tool actions already performed during execution.
Irreversible actions still require their own governance and approvals.
- Configured results not yet accepted do not enter automatic completed-Task memory
extraction. Human acceptance does not automatically re-trigger extraction.
- Completion notifications still report execution completion, but omit the result
summary for opted-in Tasks and direct readers to the independent acceptance state.
- Automatic revision loops and a dedicated external publishing connector remain
future work. Existing raw audit APIs are intentionally retained.
49 changes: 49 additions & 0 deletions docs/deliverable-acceptance.zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# 交付验收

[English](deliverable-acceptance.md) | [简体中文](deliverable-acceptance.zh-CN.md)

**员工完成执行,不代表结果满足你的要求。** 协作任务可以按需启用交付验收;
简单任务不用配置,原有行为不变。

## 场景:是否启动试点

研究员收集材料,分析员比较方案,编辑员给出建议。你要求结果包含 `summary`,
并要求「每 100 位买家的客服工单不超过 12 个」。这里的分母是**买家**,不是订单。
如果材料只有 18 个工单、100 笔订单,没有买家数量,就不能判断这个条件通过。

在创建协作任务时,展开可选交付限制,开启交付验收:

1. 指定主交付物(多个最终工作项时必须选择),填写必填结果路径,例如 `summary`。
2. 可选开启比例检查:输入分子、分母及单位,倍率 `100`、运算符 `LTE`(小于等于)、
阈值 `12`。没有的数据留空,不要用不同口径的数据替代。
3. 如需最终人工批准,勾选人工审核。运行后分别查看「执行状态」和「验收状态」。

可指定主结果中结构化比例声明的路径,如 `support_rate`。对应内容需为
`{"value":10,"unit":"ticket/buyer","scale":100}`,平台会重算并检查声明是否一致。
如果只检查输入事实的比例,则**不代表生成文本里的所有结论都已被验证**。

## 状态意味着什么

- **通过**:所有必需的已配置检查通过,可以下载验收后的主交付物。
- **未通过**:存在失败的必需检查;不能导出验收结果。
- **需要审核**:缺少必需数据,或开启了人工审批;不能导出验收结果。
- **人工接受/拒绝**:具有权限的人说明理由后,对固定结果作出决定。
人工接受是有记录的覆盖决定,原来失败或缺失的检查仍保留,并非变成通过。

人工决定需要开启 `human_resolution`,并具备任务处置权限。系统保存操作人、理由、
验收规则和结果摘要及证据摘要;数据改变后旧决定不会自动沿用。任务执行状态仍为
`COMPLETED`,不会与执行中的审批暂停混淆。导出只覆盖主交付物,不自动覆盖附件。
原始输出仍可查看,用于排错和审计。

## 能力边界

支持字段存在、字段严格相等、带单位的比例阈值检查。数据源可以是任务输入或主结果。
单位必须完全一致;分母不能为零或负数;非有限数、错误单位和错误比例声明不能通过。
配置在任务创建时固定,不会因重试或改计划偷偷换验收对象。

这不是万能的事实判断器,也不会撤销运行中已经调用工具产生的外部动作。输入事实由用户
提供,平台不保证其来源真实。模型自称「验收通过」没有决定权。
尚未接受的配置任务不会自动进入完成任务记忆提取;人工接受也不会自动补触发提取。
飞书完成通知只说明执行完成,配置了验收的任务不自动发送结果摘要。

完整请求例子、接口和限制见 [英文参考](deliverable-acceptance.md)。
5 changes: 3 additions & 2 deletions docs/implementation-status.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ and non-critical single-team use. It is **not** a production HA or multi-tenant
| Area | Current capability | Important limit |
| --- | --- | --- |
| Tasks and execution | Direct, independently Reviewed, and Coordinated Subtask DAGs; durable Runs/Attempts, retry, pause/resume, cancellation, Handoffs, Plan Patches, and Artifacts | General active-Run replanning and compensation remain future work |
| Deliverable acceptance | Optional pinned Coordinated primary checks, typed/unit-aware rates, audited human decisions, and accepted-result export | Checks cover only configured fields/facts, not arbitrary prose truth or already executed tool actions; see [guide](deliverable-acceptance.md) |
| Agents and models | Published immutable Agent Versions, role-bound execution, deterministic keyless demo, encrypted OpenAI/DeepSeek model connections in the authenticated Console | A private DeepSeek Direct and three-employee Coordinated run passed; OpenAI has fixture qualification, not a live-provider acceptance claim |
| Collaboration and tools | Governed MCP registry/read tools/idempotent writes; trusted A2A peer registration, delegation, polling, cancellation, and unknown-outcome reconciliation | External providers/peers require configuration; irreversible writes and A2A streaming/push are not supported |
| External notifications | Optional Feishu bot cards for Task completion, failure, pending human review, and governed approval requests, backed by durable PostgreSQL delivery jobs | Gate is off by default; app credentials and a test group are required; inbound commands/approvals are not yet supported |
Expand All @@ -37,8 +38,8 @@ See the [machine-readable report](qualification/a4-2-parity.json) and

## What remains

- Configurable deterministic acceptance for Coordinated business deliverables, distinct from
execution completion ([#187](https://github.com/0YHR0/AgentMesh/issues/187)).
- Automated revision loops, semantic business review, and dedicated post-acceptance publishing
remain future work; [configured deterministic acceptance](deliverable-acceptance.md) is available.
- Durable non-LangGraph reattach/restart and production admission ([#136](https://github.com/0YHR0/AgentMesh/issues/136)).
- Shared Governed Action SDK beyond MCP/A2A ([#137](https://github.com/0YHR0/AgentMesh/issues/137)).
- Repeatable chaos and correctness qualification ([#138](https://github.com/0YHR0/AgentMesh/issues/138)).
Expand Down
2 changes: 1 addition & 1 deletion docs/qualification/a4-2-coordinated-activation.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"schema": "agentmesh.qualification.a4-2-coordinated-activation.v1",
"milestone": "A4.2c.2f8",
"status": "pending_activation",
"migration_head": "20260930_0057",
"migration_head": "20260930_0058",
"downgrade_floor": "20260915_0053",
"previous_migration_head": "20260915_0053",
"feature_gate": {
Expand Down
8 changes: 4 additions & 4 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ only; it is not a claim that a proposal has shipped.

## Next product validation

- Next priority: opt-in acceptance checks for Coordinated deliverables
([#187](https://github.com/0YHR0/AgentMesh/issues/187)). Reuse the existing Reviewed/checking
primitives where possible; make missing inputs, denominator/unit mismatches and human review
visible without complicating simple tasks. Complete provider output is not business acceptance.
- Validate [opt-in Coordinated deliverable acceptance](deliverable-acceptance.md) on repeatable
real-model workflows. It distinguishes missing inputs, unit mismatches and human review without
complicating simple tasks. Semantic review, automated revision and external post-acceptance
publishing are separate follow-ups, not guaranteed by the current deterministic checker.
- Make one real-model customer-feedback workflow easy to repeat from the Console; use the
[screenshot walkthrough](scenarios/customer-feedback-deepseek.md) as the current acceptance
example. A private DeepSeek run passed; the public demo remains keyless.
Expand Down
Loading
Loading