Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion packages/pi-agent-team/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@ pi -e npm:@geminixiang/pi-agent-team

`pi remove npm:@geminixiang/pi-agent-team` uninstalls it. Use `pi install -l` to write to project settings (`.pi/settings.json`) instead of user settings.

## Lifecycle API migration

Existing callers do not need to change the handle they store: the tool parameter remains named `runId` for compatibility, but its value now identifies the retained team (`teamId`), not an individual execution. Snapshots expose team lifecycle through both `teamStatus` and its `lifecycle` alias, while `latestRound`/`currentRound` explicitly identify the current or latest `RoundRun`; the existing flat `roundId`, `roundIndex`, objective, result, and round `status` fields remain compatibility aliases. Terminal `rounds` are bounded to the latest 16 summaries, and restricted message bodies are never included.

`team_cancel` is idempotent for the latest cancelled round. Callers that retry asynchronously may pass the observed optional `roundId`; if a newer round has started, the stale cancellation is a no-op. Once cancellation settles, `team_prompt` starts a fresh round on the same member `sessionId`/`sessionRef` values. Events and late runtime callbacks are round-scoped so an older completion cannot settle or modify a newer round.

## What ships

The bundled `pi-agent-team` skill teaches the parent agent when to use foreground or detached mode, how to observe and intervene without polling or unsolicited guidance, and how to continue a settled retained team. It is withheld from member sessions, which lack `team_start`, while their other installed skills remain available.
Expand Down Expand Up @@ -107,7 +113,8 @@ These tests prove the generic runtime routes isolated adapters correctly. They d
## Limits

- A `completed` settlement means only that all members called `team_finish`; it is not a scenario-specific correctness verdict. A `quiescent` settlement means no members are runnable while unfinished members remain — `errored-members-remain` is terminal failure, `blocked-members-remain` is an explicit requester wait (for non-detached runs), and `no-runnable-members` may be a genuine deadlock. An `exhausted` settlement means the turn budget ran out with runnable members remaining, enforced as a real per-wave hard cap rather than only checked before a wave starts; the partial result is still returned. Inspect public speech and causal evidence before claiming that the objective succeeded.
- A synchronous `team_start` remains foreground until its first round settles. Its manifest's `team` id is then accepted by `team_get` and `team_prompt`; prompting a settled member continues the retained team in a new background round. Retained teams and their controls are process/session-local: no daemon, socket API, or restart persistence. A `TeamRuntime` instance still represents exactly one round; continuation creates a clean runtime over the retained agent sessions rather than reusing settled coordination state.
- **Stable team identity, explicit round identity.** Control tools continue to accept the original `runId` parameter as a stable `teamId` handle. Every objective execution has a unique `roundId` and monotonic `roundIndex`; snapshots expose `lifecycle`, the current/latest round, cancellation metadata, and capped terminal summaries.
- A synchronous `team_start` remains foreground until its first round settles. Its manifest's stable `team` id is then accepted by `team_get` and `team_prompt`; prompting an available team continues the retained team in a new background round. Retained teams and their controls are process/session-local: no daemon, socket API, or restart persistence. A `TeamRuntime` instance still represents exactly one round; continuation creates a clean runtime over the retained agent sessions rather than reusing settled coordination state. Cancellation settles only the current round; the retained member sessions remain available.
- A quiescent team may stop before every member calls `team_finish`; this is reported as `quiescent`, never silently called success.
- The parent supplies the initial objective/message. Runtime-generated follow-up hints and scenario-specific fallback decisions do not exist; runtime messages are limited to task-agnostic control notices (claim results, bounces, budget and error alerts).
- A command batch returned by one turn first passes a hard freshness fence: if an authorized observation arrived for that member during `act()`, none of the stale batch commits and the member rethinks from all new observations. Otherwise it is applied in order and fail-stop, not as a transaction: commands that already committed are never rolled back, but the first rejected command halts the batch and everything after it is discarded. The rejection bounces back (`COMMAND_FAILED`, plus `COMMAND_BATCH_HALTED` when later commands were discarded) and wakes the member to replan next turn — so a rejected send can no longer be sealed off by a `finish` queued behind it in the same batch. A genuine cross-command dependency still needs a cross-turn fence (a claim, or waiting for a confirmation message), not same-batch ordering.
Expand Down
138 changes: 138 additions & 0 deletions packages/pi-agent-team/docs/lifecycle-improvement-evaluation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Agent Team Lifecycle Improvement Evaluation

## Decision under evaluation

Separate a retained team identity from each objective execution:

- `TeamHandle`: stable roster and retained member sessions.
- `RoundRun`: one objective, one execution lifecycle, and one outcome.
- Cancelling a round must not destroy the retained team; a later prompt can start a new round.

This evaluation is intentionally domain-neutral. Werewolf is one stress scenario, not part of the production model.

## Why this is first

Today one run identifier and one status surface conflate the retained team, its current round, and the latest outcome. That makes cancellation and continuation ambiguous. The change is successful only if it makes those states explicit without weakening channel isolation, bounded snapshots, or existing foreground/detached behavior.

## Frozen success criteria

### Gate A — Identity and history correctness (must pass)

1. One stable `teamId` survives at least three rounds.
2. Every round has a unique `roundId` and monotonic `roundIndex`.
3. A round snapshot identifies both `teamId` and `roundId`.
4. The retained team exposes bounded summaries for completed/cancelled rounds; a new round does not overwrite the identity of an older one.
5. Existing member `sessionId` and `sessionRef` values remain stable across continuation rounds.

### Gate B — Cancellation semantics (must pass)

1. Cancelling a running round reaches a terminal round outcome within the existing cancellation timeout.
2. After cancellation, the team becomes available rather than permanently terminal.
3. `team_prompt` after cancellation starts a new round over the same member sessions.
4. Messages or completion from the cancelled round cannot mutate the new round.
5. Repeated cancel is deterministic and does not create an extra round.

### Gate C — Backward compatibility (must pass)

1. `team_start`, `team_get`, `team_wait`, `team_prompt`, and `team_cancel` remain usable through the existing retained handle.
2. The foreground manifest still provides one handle that can be passed to continuation tools.
3. Existing settlement meanings remain unchanged: `completed`, `quiescent`, and `exhausted` describe a round outcome, not objective correctness.
4. Existing public/direct/restricted-group routing and redaction tests remain green.
5. Existing bounded-output limits remain enforced.

### Gate D — Observability (must pass)

A snapshot must let an operator answer, without reading member transcripts:

- Is the team available, running, closing, or closed?
- Which round is current/latest?
- What was that round's objective and terminal outcome?
- Was cancellation requested, and why?
- Which member sessions are retained?

No restricted message body may be added to snapshots, round summaries, or events.

### Gate E — Efficiency guardrails (must not regress)

Use deterministic scripted agents; do not use live-model token cost as a CI gate.

1. Starting a continuation creates no replacement member sessions.
2. Cancellation plus continuation adds no polling loop.
3. `team_wait` remains event-driven through monotonic `stateChangeSeq`.
4. Event history remains capped at the existing limit.
5. Round summaries must have an explicit fixed cap; exceeding it evicts oldest summaries without invalidating the stable team handle.

## Required automated scenarios

Create a focused lifecycle evaluation test suite covering:

| Scenario | Required assertions |
|---|---|
| Three completed rounds | Stable `teamId`; three unique `roundId`s; indices 1, 2, 3; stable member sessions |
| Cancel then continue | Round 1 cancelled; team available; round 2 runs and settles normally |
| Cancel race | Late round-1 completion cannot settle or write into round 2 |
| Wait across transitions | `stateChangeSeq` strictly increases for cancel request, round settlement, and next-round start |
| History cap | Oldest round summary evicted at cap; latest/current identity remains correct |
| Privacy regression | Round metadata contains no direct/group plaintext canary |
| Existing channels | Public, DM, and restricted group tests remain unchanged and green |
| Foreground manifest | Stable team handle plus explicit latest round identity/outcome |

## Baseline to record before implementation

Run from repository root:

```sh
npm test --workspace @geminixiang/pi-agent-team
npm run check --workspace @geminixiang/pi-agent-team
```

Record:

- passing/failing test count;
- wall-clock test duration;
- current snapshot shape for start → settle → continue;
- current cancel → prompt behavior;
- current bounded event/result limits.

A known pre-change failure that demonstrates the lifecycle ambiguity is allowed only in the new focused evaluation test. All pre-existing tests must remain green before implementation begins.

## Acceptance command

After implementation:

```sh
npm test --workspace @geminixiang/pi-agent-team
npm run check --workspace @geminixiang/pi-agent-team
```

The change is accepted only when:

1. Gates A–E pass in automated tests.
2. No pre-existing test is deleted or weakened to pass.
3. Public API changes are documented with a migration note.
4. A reviewer verifies that production code remains scenario-neutral.
5. The final diff contains no werewolf-specific production symbols.

## Before / after evaluation table

| Question | Before | Required after |
|---|---|---|
| What does the retained ID identify? | Team and latest run are conflated | Stable team handle only |
| Can a cancelled team continue? | Cancellation leaves an unusable retained record | A new round starts on the same team |
| Can two rounds be distinguished? | No first-class round identity/history | Unique `roundId`, index, bounded summaries |
| What does status describe? | Manager/run status mixes control plane and outcome | Team lifecycle and round lifecycle are separate |
| Are member histories retained? | Yes | Yes, with the same session identities |
| Does this add domain logic? | No | No; lifecycle remains rule-agnostic |

## Non-goals for this change

Do not combine these into the lifecycle patch:

- werewolf/game state;
- durable recovery after parent-process restart;
- typed claim namespaces;
- block-reason confidentiality redesign;
- wait-abort UI wording;
- semantic/idempotent group aliases.

Those require separate evaluations and diffs. Keeping them out makes lifecycle regression attribution possible.
72 changes: 61 additions & 11 deletions packages/pi-agent-team/src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -128,15 +128,16 @@ export default function agentTeam(pi: ExtensionAPI): void {
params.members.map((member) => [member.id, new PiTeamAgent(member, ctx.cwd, ctx)]),
);
let runtime!: TeamRuntime;
const roundId = crypto.randomUUID();
runtime = new TeamRuntime(params.objective, agents, {
reactionDelayMs: { min: 50, max: 500 },
waitForIntervention: true,
reporterId,
reportPrompt: params.reportPrompt,
onActivity: (activity) => runs.observeActivity(runtime.teamId, activity),
onProgress: (progress) => runs.observeProgress(runtime.teamId, progress),
onActivity: (activity) => runs.observeActivity(runtime.teamId, activity, roundId),
onProgress: (progress) => runs.observeProgress(runtime.teamId, progress, roundId),
});
const snapshot = runs.start(runtime, initial);
const snapshot = runs.start(runtime, initial, { roundId });
return {
content: [{ type: "text", text: renderRunSnapshot(snapshot) }],
details: snapshot,
Expand Down Expand Up @@ -198,10 +199,17 @@ export default function agentTeam(pi: ExtensionAPI): void {
let retained = false;
try {
const result = await runtime.run(initial, signal);
runs.retain(runtime, result);
const retainedSnapshot = runs.retain(runtime, result);
retained = true;
return {
content: [{ type: "text", text: renderFinalContent(result, params.members) }],
content: [{
type: "text",
text: renderFinalContent(result, params.members, {
roundId: retainedSnapshot.roundId,
roundIndex: retainedSnapshot.roundIndex,
objective: retainedSnapshot.objective,
}),
}],
details: finalDetails(details(), result),
};
} finally {
Expand Down Expand Up @@ -315,21 +323,34 @@ function registerTeamCancel(pi: ExtensionAPI, runs: TeamRunManager): void {
{
runId: Type.String({ minLength: 1 }),
reason: Type.Optional(Type.String({ minLength: 1, maxLength: 1_000 })),
roundId: Type.Optional(
Type.String({
minLength: 1,
description: "Optional observed round identity; a stale retry becomes a no-op instead of cancelling a newer round.",
}),
),
},
{ additionalProperties: false },
),
async execute(_id, params) {
return snapshotResult(runs.cancel(params.runId, params.reason));
return snapshotResult(runs.cancel(params.runId, params.reason, params.roundId));
},
});
}

function renderRunSnapshot(snapshot: TeamRunSnapshot): string {
const lines = [
`team run: ${snapshot.runId}`,
`status: ${snapshot.status}`,
`team: ${snapshot.teamId}`,
`team lifecycle: ${snapshot.lifecycle}`,
`round: ${snapshot.roundId} (#${snapshot.roundIndex})`,
`objective: ${snapshot.objective}`,
`round status: ${snapshot.status}`,
`stateChangeSeq: ${snapshot.stateChangeSeq}`,
];
if (snapshot.cancellation)
lines.push(
`cancellation: requested at ${new Date(snapshot.cancellation.requestedAt).toISOString()} · ${snapshot.cancellation.reason}`,
);
if (snapshot.progress)
lines.push(
`progress: ${snapshot.progress.turns} turns · ${snapshot.progress.finished.length} finished · ${snapshot.progress.blocked.length} blocked`,
Expand Down Expand Up @@ -364,6 +385,10 @@ function renderRunSnapshot(snapshot: TeamRunSnapshot): string {
);
}
if (snapshot.error) lines.push(`error: ${snapshot.error}`);
if (snapshot.rounds.length)
lines.push(
`round history: ${snapshot.rounds.map((round) => `#${round.roundIndex} ${round.roundId} ${round.status}`).join(" · ")}`,
);
const latest = snapshot.events.at(-1);
if (latest) lines.push(`latest event: #${latest.sequence} ${latest.type} · ${latest.summary}`);
return lines.join("\n");
Expand All @@ -384,7 +409,21 @@ function summarizeLive(details: TeamDisplayDetails): string {
export function renderFinalContent(
result: TeamResult,
members: readonly { id: string; name: string }[],
round?: { roundId: string; roundIndex: number; objective?: string },
): string {
const resultRound = result as TeamResult & {
roundId?: string;
roundIndex?: number;
objective?: string;
};
const manifestRound = round ??
(resultRound.roundId !== undefined && resultRound.roundIndex !== undefined
? {
roundId: resultRound.roundId,
roundIndex: resultRound.roundIndex,
objective: resultRound.objective,
}
: undefined);
const nameOf = (id: string) => members.find((member) => member.id === id)?.name ?? id;
const lines: string[] = [];
if (result.report) {
Expand All @@ -404,8 +443,19 @@ export function renderFinalContent(
}
lines.push(
"TEAM MANIFEST",
`team: ${result.teamId}`,
`settlement: ${result.settlement.kind} (${result.settlement.meaning}; objective correctness unverified)`,
`team handle: ${result.teamId}`,
...(manifestRound
? [
`round id: ${manifestRound.roundId}`,
`round index: ${manifestRound.roundIndex}`,
...(manifestRound.objective !== undefined
? [`objective: ${manifestRound.objective}`]
: []),
`round outcome: ${result.settlement.kind} (${result.settlement.meaning}; objective correctness unverified)`,
]
: [
`settlement: ${result.settlement.kind} (${result.settlement.meaning}; objective correctness unverified)`,
]),
"members:",
);
for (const member of result.members) {
Expand All @@ -419,7 +469,7 @@ export function renderFinalContent(
`messages: ${result.publicTranscript.length} public · ${result.restrictedMessages.length} restricted (bodies not included here)`,
`audit: ${result.events.length} events · head ${result.auditHead}`,
"Each member's full first-person history is in its session file listed above.",
`The team remains available in this parent session: use team_prompt with runId ${result.teamId} and a member id to start a continuation round.`,
`The team remains available in this parent session: use team_prompt with runId ${result.teamId} (the stable team handle) and a member id to start a continuation round.`,
);
return lines.join("\n");
}
Expand Down
Loading
Loading