diff --git a/.github/dco.yml b/.github/dco.yml new file mode 100644 index 0000000000..5078cb8789 --- /dev/null +++ b/.github/dco.yml @@ -0,0 +1,11 @@ +# DCO app (probot/dco) configuration. +# +# Allow an author to vouch for their own earlier commits with a DCO +# remediation commit, the mechanism the DCO app documents for commits whose +# author email has no matching Signed-off-by. This is needed when a v5 -> v6 +# top-up carries commits whose author and sign-off identities differ (see +# hivecommons/hive#18697); it does not allow anyone to sign off on behalf of +# another author. +allowRemediationCommits: + individual: true + thirdParty: false diff --git a/.github/scripts/integration-governance.cjs b/.github/scripts/integration-governance.cjs index c1e2309e67..e3c4ee5769 100644 --- a/.github/scripts/integration-governance.cjs +++ b/.github/scripts/integration-governance.cjs @@ -9,7 +9,7 @@ const MAX_COMMITS = 1000; const MAX_AUTHOR_BATCH = 50; // This literal is inside the enrolled .github closure. A caller-supplied // preparedBy.sourceSHA must never choose a new executable verifier tree. -const TRUSTED_SCRIPTS_TREE = '46005db3bdfd434e5a143bde8590e3b05e0892a2'; +const TRUSTED_SCRIPTS_TREE = 'bf12ae550a3a4d306821d0ae56cc22850f035e3c'; const SOURCE = 'https://github.com/kubernetes-sigs/kubebuilder-release-tools/tree/012269a88fa4c034a0acf1ba84c26b195c0dbab4'; const sha = value => crypto.createHash('sha256').update(value).digest('hex'); const git = (args, cwd = process.cwd()) => diff --git a/.github/workflows/dco-squash-attribution.yml b/.github/workflows/dco-squash-attribution.yml index 0594c12bd6..ac1ce7820a 100644 --- a/.github/workflows/dco-squash-attribution.yml +++ b/.github/workflows/dco-squash-attribution.yml @@ -54,7 +54,12 @@ jobs: steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - fetch-depth: 0 + # Shallow on purpose: `fetch-depth: 0` fetches EVERY branch and tag + # (`+refs/heads/*` `+refs/tags/*`). On this repo's ref count that + # took 17-35s on a good day and, on #18482 (run 43960382195), hung + # for the whole 5-minute job budget before the check ever ran. The + # step below fetches only the histories the checker reads. + fetch-depth: 1 # The merge commit GitHub synthesises for a PR is not what gets # squashed; the PR's own commits are. Check those. ref: ${{ github.event.pull_request.head.sha }} @@ -62,17 +67,48 @@ jobs: - name: Self-test the attribution checker run: bash src/scripts/test-check-squash-signoff-attribution.sh - - name: Make the default branch resolvable + - name: Fetch the PR, base and default-branch histories env: DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + BASE_SHA: ${{ github.event.pull_request.base.sha }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + # A healthy targeted fetch finishes well inside this; a stalled + # connection is killed and retried instead of eating the job budget. + FETCH_ATTEMPT_TIMEOUT_SECS: '60' + FETCH_ATTEMPTS: '3' run: | set -euo pipefail - # Commits already on the default branch are excluded from the check - # (see the script). Without this ref a sync PR re-flags every commit - # it carries forward, so fetch it explicitly rather than relying on - # checkout having populated remote-tracking refs. - git fetch --no-tags origin \ - "+refs/heads/${DEFAULT_BRANCH}:refs/remotes/origin/${DEFAULT_BRANCH}" + # The checker walks BASE_SHA..HEAD_SHA, and commits already on the + # default branch are excluded (see the script): without that ref a + # sync PR re-flags every commit it carries forward. Fetch exactly + # those three histories, in full, and nothing else. + # + # Blobless (--filter=blob:none): the checker reads only commits and + # trees (rev-list, log formats, diff-tree --name-only), never file + # contents, and git marks origin as a promisor so anything that did + # need a blob would lazy-fetch it. Unshallowing WITH blobs measured + # 38-42s on the self-hosted pool (#18609), slower than the old + # all-refs checkout; blobless cuts the transfer ~10x (277 MB -> 24 MB + # measured for v5's history). + for attempt in $(seq 1 "$FETCH_ATTEMPTS"); do + unshallow=() + if [ "$(git rev-parse --is-shallow-repository)" = "true" ]; then + unshallow=(--unshallow) + fi + if timeout "$FETCH_ATTEMPT_TIMEOUT_SECS" git fetch --no-tags --filter=blob:none ${unshallow[@]+"${unshallow[@]}"} origin \ + "$HEAD_SHA" "$BASE_SHA" \ + "+refs/heads/${DEFAULT_BRANCH}:refs/remotes/origin/${DEFAULT_BRANCH}"; then + break + fi + if [ "$attempt" = "$FETCH_ATTEMPTS" ]; then + echo "::error::git fetch failed or stalled ${FETCH_ATTEMPTS} times (${FETCH_ATTEMPT_TIMEOUT_SECS}s each)" >&2 + exit 1 + fi + echo "::warning::git fetch attempt ${attempt} failed or stalled; retrying" >&2 + done + test "$(git rev-parse --is-shallow-repository)" = "false" + git rev-parse --verify "${BASE_SHA}^{commit}" >/dev/null + git rev-parse --verify "${HEAD_SHA}^{commit}" >/dev/null git rev-parse --verify "refs/remotes/origin/${DEFAULT_BRANCH}^{commit}" >/dev/null - name: Check sign-off attribution across the PR's commits diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index 3aa9f1dcca..1d87faa8e6 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -311,8 +311,15 @@ jobs: with: driver-opts: network=host + # Also logs in on same-repo PR builds so the whisper-builder warm step + # can READ the GHCR registry cache below. Fork PRs never reach this job + # (see `if:` above), and nothing in a PR build pushes: the image output + # has push=false and the registry cache-to is gated on push == 'true'. - name: Log in to GHCR - if: needs.gate.outputs.push == 'true' + if: >- + needs.gate.outputs.push == 'true' || + (github.event_name == 'pull_request' && + github.event.pull_request.head.repo.full_name == github.repository) uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0 with: registry: ghcr.io @@ -362,6 +369,10 @@ jobs: BASE_REGISTRY=${{ env.BASE_REGISTRY }} provenance: false sbom: false + # Materialize the stage into this builder (see the note on the + # whisper warm step below); without an output a cache hit stays a + # lazy remote reference and the main build recompiles tmux. + outputs: type=tar,dest=/dev/null cache-from: type=gha,scope=tmux-${{ steps.slug.outputs.slug }} cache-to: type=gha,scope=tmux-${{ steps.slug.outputs.slug }},mode=max @@ -369,7 +380,15 @@ jobs: # whisper-builder's inputs are the pinned base image and two pinned, # checksummed tarballs (whisper.cpp, FFmpeg), so its layers are safe to # serve from cache and it is not in no-cache-filters. Compiling it cold - # costs several minutes per arch. + # costs several minutes per arch (far more for arm64 under QEMU). + # + # Its cache lives in a GHCR registry cache, NOT the Actions cache. The + # Actions cache sits over its 10 GB cap, so LRU eviction kept dropping + # the whisper blobs and every build recompiled whisper.cpp + FFmpeg. + # The registry cache is not subject to that cap. It is a dedicated + # package (hive-buildcache), never a release image tag, with one tag + # per arch so amd64 and arm64 never overwrite each other. Only + # push=true builds export it; same-repo PR builds read it only. - name: Warm whisper-builder layer cache continue-on-error: true uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 @@ -382,8 +401,21 @@ jobs: BASE_REGISTRY=${{ env.BASE_REGISTRY }} provenance: false sbom: false - cache-from: type=gha,scope=whisper-${{ steps.slug.outputs.slug }} - cache-to: type=gha,scope=whisper-${{ steps.slug.outputs.slug }},mode=max + # MATERIALIZE the result. Measured 2026-11-17 on v5: with no output + # this step reported every whisper-builder layer CACHED in ~1s, and + # the main build below then recompiled FFmpeg + whisper.cpp anyway + # (and tmux, same pattern, on runs before that). A warm step that + # hits its remote cache without an output never loads the layers + # into the builder's local cache, and the main build cannot be + # served from a remote cache for these stages: they share their + # FROM image with `runtime`, which no-cache-filters marks + # ignore-cache. The only builds where the main step showed these + # stages CACHED were the ones where the warm step MISSED and + # compiled locally. Exporting the stage (to /dev/null, nothing is + # kept) forces the layers to be pulled and recorded locally. + outputs: type=tar,dest=/dev/null + cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/hive-buildcache:whisper-${{ steps.slug.outputs.slug }} + cache-to: ${{ needs.gate.outputs.push == 'true' && format('type=registry,ref=ghcr.io/{0}/hive-buildcache:whisper-{1},mode=max,image-manifest=true,oci-mediatypes=true,ignore-error=true', github.repository_owner, steps.slug.outputs.slug) || '' }} - name: Build and push by digest id: build @@ -764,8 +796,11 @@ jobs: sbom: false # Owner-derived image name (dual-publish) — see the main build step. outputs: type=image,name=ghcr.io/${{ github.repository_owner }}/hive-contributor,push-by-digest=true,name-canonical=true,push=${{ needs.gate.outputs.push }},oci-mediatypes=false + # Only push=true builds export: a PR-ref Actions cache entry is only + # visible to that same PR, yet it counts against the repo's 10 GB + # cap and evicts the long-lived branch entries every PR reads. cache-from: type=gha,scope=contributor-${{ steps.slug.outputs.slug }} - cache-to: type=gha,scope=contributor-${{ steps.slug.outputs.slug }},mode=max + cache-to: ${{ needs.gate.outputs.push == 'true' && format('type=gha,scope=contributor-{0},mode=max', steps.slug.outputs.slug) || '' }} - name: Export digest if: needs.gate.outputs.push == 'true' @@ -987,8 +1022,11 @@ jobs: sbom: false # Owner-derived image name (dual-publish) — see the main build step. outputs: type=image,name=ghcr.io/${{ github.repository_owner }}/hive-hub,push-by-digest=true,name-canonical=true,push=${{ needs.gate.outputs.push }},oci-mediatypes=false - cache-from: type=gha,scope=hub-${{ steps.slug.outputs.slug }} - cache-to: type=gha,scope=hub-${{ steps.slug.outputs.slug }},mode=max + # No layer cache. src/Dockerfile.hub has exactly two stages, builder + # and runtime, and no-cache-filters above forces both to re-execute, + # so a cache could never serve a layer here. The old gha mode=max + # export only filled the Actions cache (and evicted entries that are + # reused, like whisper's) for zero hits. - name: Export digest if: needs.gate.outputs.push == 'true' @@ -1215,8 +1253,8 @@ jobs: sbom: false # Owner-derived image name (dual-publish) — see the main build step. outputs: type=image,name=ghcr.io/${{ github.repository_owner }}/hive-mx,push-by-digest=true,name-canonical=true,push=${{ needs.gate.outputs.push }},oci-mediatypes=false - cache-from: type=gha,scope=mx-${{ steps.slug.outputs.slug }} - cache-to: type=gha,scope=mx-${{ steps.slug.outputs.slug }},mode=max + # No layer cache, for the same reason as the hub build: both stages + # of src/Dockerfile.mx (builder, runtime) are in no-cache-filters. - name: Export digest if: needs.gate.outputs.push == 'true' diff --git a/.github/workflows/integration-candidate.yml b/.github/workflows/integration-candidate.yml index 25c3e6250f..baa6fcef75 100644 --- a/.github/workflows/integration-candidate.yml +++ b/.github/workflows/integration-candidate.yml @@ -757,13 +757,15 @@ jobs: - ci-required-context-scope if: ${{ always() && needs.subject.result == 'success' && (always() && (needs.ci-required-context-scope.result != 'success' || needs.ci-required-context-scope.outputs.run-ci == 'true')) }} runs-on: ubuntu-latest - timeout-minutes: 10 + timeout-minutes: 20 steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 with: persist-credentials: false - name: Set up Docker Buildx uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 + with: + cleanup: false - name: Resolve runner ref name id: ref_name run: | @@ -783,7 +785,7 @@ jobs: outputs: type=oci,oci-mediatypes=true,dest=/tmp/hive-oci.tar,tar=true cache-from: | type=gha,scope=tmux-linux-amd64 - type=gha,scope=whisper-linux-amd64 + type=registry,ref=ghcr.io/${{ github.repository_owner }}/hive-buildcache:whisper-linux-amd64 build-args: | BASE_REGISTRY=${{ env.BASE_REGISTRY }} GIT_HASH=${{ github.sha }} diff --git a/.github/workflows/integration-technical-v2-ci.yml b/.github/workflows/integration-technical-v2-ci.yml index 1127eb20f1..ae549e44b5 100644 --- a/.github/workflows/integration-technical-v2-ci.yml +++ b/.github/workflows/integration-technical-v2-ci.yml @@ -595,13 +595,15 @@ jobs: needs: required-context-scope if: ${{ always() && (needs.required-context-scope.result != 'success' || needs.required-context-scope.outputs.run-ci == 'true') }} runs-on: ubuntu-latest - timeout-minutes: 10 + timeout-minutes: 20 steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 with: persist-credentials: false - name: Set up Docker Buildx uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 + with: + cleanup: false - name: Resolve runner ref name id: ref_name run: | @@ -621,7 +623,7 @@ jobs: outputs: type=oci,oci-mediatypes=true,dest=/tmp/hive-oci.tar,tar=true cache-from: | type=gha,scope=tmux-linux-amd64 - type=gha,scope=whisper-linux-amd64 + type=registry,ref=ghcr.io/${{ github.repository_owner }}/hive-buildcache:whisper-linux-amd64 build-args: | BASE_REGISTRY=${{ env.BASE_REGISTRY }} GIT_HASH=${{ github.sha }} diff --git a/.github/workflows/issue-closed-reopen-hint.yml b/.github/workflows/issue-closed-reopen-hint.yml index d0eea0c7c6..4e5142acf1 100644 --- a/.github/workflows/issue-closed-reopen-hint.yml +++ b/.github/workflows/issue-closed-reopen-hint.yml @@ -11,7 +11,11 @@ name: Issue Closed Hint # instead of depending on whoever closed the issue remembering to say so. # # Skipped when the reporter closed the issue themselves (nothing to tell them) -# and for pull requests (the `issues` event never fires for those). +# and for pull requests (the `issues` event never fires for those). Also +# skipped when the reporter is a bot (the hive's own issues: nobody reads the +# hint) or a maintainer (OWNER / MEMBER / COLLABORATOR, who can reopen the +# issue themselves): those were most of the hint's comments and none of them +# told anyone something they could not already do (#18607). on: issues: @@ -26,7 +30,11 @@ jobs: name: Point the reporter at /reopen runs-on: ubuntu-latest timeout-minutes: 5 - if: github.event.issue.user.login != github.event.sender.login + if: >- + github.event.issue.user.login != github.event.sender.login && + github.event.issue.user.type != 'Bot' && + !endsWith(github.event.issue.user.login, '[bot]') && + !contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.issue.author_association) steps: - name: Comment env: diff --git a/.github/workflows/v2-ci.yml b/.github/workflows/v2-ci.yml index 083d3b2d75..aa2c6511b4 100644 --- a/.github/workflows/v2-ci.yml +++ b/.github/workflows/v2-ci.yml @@ -1526,7 +1526,21 @@ jobs: needs: required-context-scope if: ${{ always() && (needs.required-context-scope.result != 'success' || needs.required-context-scope.outputs.run-ci == 'true') }} runs-on: ubuntu-latest - timeout-minutes: 10 + # Sized for the COLD-cache path, not the warm one. Measured 2026-11-16/17 + # on ubuntu-latest: with the whisper-builder layers served from the gha + # cache the job takes 6.5-8 min (buildx step 4.3-5.8 min). But the repo's + # Actions cache sits at 12-23 GB against a 10 GB cap, so GitHub evicts LRU + # entries continuously (oldest surviving entry ~2 days) and the + # whisper-linux-amd64 blobs are regularly gone — docker.yml's own v5 warm + # step recompiles whisper several times a day. On a miss this 4-vCPU + # runner compiles FFmpeg + whisper.cpp from source: a serial ~8.5 min + # chain (apt ~35s, FFmpeg 95-150s, whisper.cpp 210-335s), so the build + # step alone is ~10.5 min and the job ~13 min. Every overlayfs-exec-guard + # cancellation sampled (#18432 x3, #18446, #18500, #18527) was that path: + # the build step cancelled at 9.5 min with whisper still compiling, or + # finishing at 7.7 min and the containerd run cut off. 20 min covers the + # cold path with ~50% headroom; the warm path is unchanged. + timeout-minutes: 20 # No `needs: build-and-test` — see the note above the `docker` job. steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 @@ -1535,6 +1549,10 @@ jobs: - name: Set up Docker Buildx uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 + with: + # Ephemeral hosted runner: tearing the builder container down in the + # post step bought nothing and measured 16-36s on every run. + cleanup: false # Build a single-arch image the SAME way docker.yml builds the shipped # image — buildx (BuildKit), attestations OFF (provenance:false/sbom:false @@ -1555,6 +1573,12 @@ jobs: # tmux-builder (~51s) at its third layer while its own apt layer takes # ~20s, so a hit removes ~30s from a ~3.5 min build; a whole-image # cache-to would instead ADD the 130-225s export docker.yml measured. + # + # whisper-builder layers come from docker.yml's GHCR registry cache + # (hive-buildcache:whisper-linux-amd64), which v5 pushes export and + # which is not subject to the Actions cache cap. Its inputs are pinned, + # checksummed tarballs, so the same stale-COPY argument holds. The read + # is anonymous; a miss or an unreadable ref just recompiles whisper. - name: Resolve runner ref name id: ref_name run: | @@ -1576,7 +1600,7 @@ jobs: outputs: type=oci,oci-mediatypes=true,dest=/tmp/hive-oci.tar,tar=true cache-from: | type=gha,scope=tmux-linux-amd64 - type=gha,scope=whisper-linux-amd64 + type=registry,ref=ghcr.io/${{ github.repository_owner }}/hive-buildcache:whisper-linux-amd64 build-args: | BASE_REGISTRY=${{ env.BASE_REGISTRY }} GIT_HASH=${{ github.sha }} diff --git a/CHANGELOG.md b/CHANGELOG.md index 965b65c28b..8fe73e9a26 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -344,6 +344,117 @@ Hive did not historically maintain a complete changelog. This file starts a prag - The signed-commit reconciler's intentional refusal to rewrite a foreign or merge commit no longer files repeated unclassified-error fleet reports; its warning and PR remediation remain visible ([#17924](https://github.com/hivecommons/hive/issues/17924)). - Dashboard logs one summary warning per five minutes for dropped SSE events instead of one warning per event, so a stalled browser tab no longer triggers repeated-error fleet reports (#17995). +## 2026-11-17 (v5.298.2) + +### Fixed + +- **A dashboard Copilot re-login now reaches running Copilot agents without a restart** ([#18671](https://github.com/hivecommons/hive/issues/18671)). The Copilot CLI's own save, seconds after launch, replaced the freshly relinked `~/.copilot/config.json` symlink with a private copy again, so a later login written to the shared file only reached each agent after its next restart. The pane poller now checks the link on every tick and relinks it within one poll interval of a CLI save, first merging the fields the CLI wrote into the shared file so nothing it saved is lost; the shared login always wins over the token in the private copy, and a private copy holding the only login is still left in place. + +## 2026-11-17 (v5.298.1) + +### Fixed + +- Stop retrying the closing-issue link when GitHub rejects it with the permanent "exceeds manual reference limit" error, so the PR-claim sweep no longer repeats WARN logs and selfreports (#18666). + +## 2026-11-17 (v5.298.0) + +### Added + +- **Workflow models: simulating a merged PR reports its observed `pipeline.status`** (part of [#18356](https://github.com/hivecommons/hive/issues/18356)). When the workflow simulate fetches a merged PR by repo and number, it now asks the GitHub Actions post-merge observer about the PR's merge commit on the default branch and fills `pr.pipeline` (`green`, `red`, `running` or `unknown` with the reason), so a model's Main CI, Release and Deploy transitions can be checked against real runs. If the read fails, `pr.pipeline` stays `unknown`, a note says why, and the simulate still answers. +- **Workflow models: GitHub Actions post-merge run observer feeds `pipeline.status`** ([#18645](https://github.com/hivecommons/hive/issues/18645), part of [#18356](https://github.com/hivecommons/hive/issues/18356)). Default-branch push runs (Main CI, deciding by the exact merge SHA when the range can't), release runs and deploy runs (per environment) of the workflows an ingested model binds are now attributed to merged PRs through the range-attribution core (`workflowmodel.AttributeRuns`), so the `pipeline.status` fact can report `green`, `red` or `running`. Nothing calls the observer yet, so the fact stays `unknown` in production until the follow-up wiring lands. Pull-request and merge-queue runs never count, a deploy run that does not report which environment it targeted credits none of them, and a run whose revision or range can't be determined leaves the status `unknown` with the reason recorded. + +## 2026-11-17 (v5.297.0) + +### Added + +- Workflow models gain a Tekton PipelineRun observer and `tektonRuns` config for post-merge runs; the stage engine does not call it yet, so no column changes until the follow-up wiring lands ([#18646](https://github.com/hivecommons/hive/issues/18646)). A Tekton pipeline's new `tektonRuns` block names the cluster (`context` or `inCluster`), `namespace` and `labelSelector` explicitly (nothing is auto-discovered); each run's revision is read from the `pipelinesascode.tekton.dev/sha` annotation, then the `revision` param (both renameable), and fed through range attribution. A run whose revision is not a full commit SHA leaves the column `unknown` with the reason recorded instead of guessing. Existing models are unchanged. + +## 2026-11-17 (v5.296.0) + +### Added + +- feat: range attribution core for post-merge runs (#18653) + +## 2026-11-17 (v5.295.0) + +### Added + +- Each Sessions pane now opens in its own git worktree of the hive's one shared clone (`/.sessions///`, branch `hive-session/`) instead of the shared checkout, so two people running different specs on one hive no longer see, block or discard each other's Spektacular workflow, and a branch switched in one session no longer moves the others. The shared clone's git config and hooks are now locked: root rewrites `.git/config` at every boot as read-only to sessions, empties `.git/hooks`, turns on `extensions.worktreeConfig` for personal settings (`git config --worktree`), and the hive's own git always runs with hooks and `core.fsmonitor` off. One session can no longer plant a config entry, hook, filter or alias that runs as another session's uid. Deleting a session auto-saves its uncommitted work as a commit on its branch before the worktree goes, and keeps the branch when the clone's `HEAD` lacks it. The periodic sweep removes worktrees of deleted sessions, but never while the sessions store has failed to load. Existing sessions move into their own worktree when resumed. Writes to the shared config (`git push -u`), deleting refs still in `packed-refs`, gc and submodules no longer work from a session ([#18588](https://github.com/hivecommons/hive/issues/18588)). +- **Betty's answers are timed stage by stage.** The meeting's live state (`/api/meetings/{id}/live`) now lists each recent answer's retrieval, model, answer-ready and delivery times, its source and its budget (4 s for questions about the hive's live state, 8 s otherwise), with a median and slowest summary; slow answers are logged. Questions about the hive no longer wait for slower sources such as web search once the live state is in. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- **Betty says who and what each answer is for when several requests overlap.** With more than one request in flight, after other people spoke, or when an answer is late, every answer opens with the asker's name and a short topic ("Andy (re: open PRs): ..." in chat, "Andy, about open PRs: ..." spoken), without @mentions that ping. With two or more requests in hand she posts one short "I have N questions; answering each as it's ready." note (at most once a minute), and a spoken acknowledgement names its asker when others are waiting. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- **Betty says where her answers come from and when she is not sure.** Chat answers end with the sources they used (a web link, a work item id, the knowledge fact or "hive live status"), the model is told to give issue and PR numbers and never guess, and a low-confidence answer opens with "I'm not sure, but ..." instead of sounding certain. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- **Betty works on several requests at once.** One meeting may now run up to `meetings.llm.per_meeting_in_flight` (default 3, never above `meetings.llm.max_in_flight`) model calls in parallel, so a second question no longer waits for the first answer. On the Claude CLI backend the meeting keeps a small pool of warm CLIs that grows only when requests overlap, and every call in flight reserves budget so parallel answers cannot overspend the meeting's token budget. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- **Betty speaks several ready answers back to back, shortest first, and moves the rest to chat.** Spoken answers ready together no longer wait behind a long one; an answer that would wait behind more than 20 s of queued speech goes to chat with a spoken "I've put the rest in chat.", and a stale acknowledgement is skipped. Answers to addressed requests may take `meetings.chat_answer_burst` (default 3) extra chat posts a minute beyond `meetings.chat_posts_per_minute`, so parallel answers are not held for a minute. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- Meetings: per-user desktop companion tokens (`/companion/token`), admitted only on the companion meeting routes at the user's live role; hosted hubs pass `/api/companion/info` and `/api/companion/meetings/*` through to the hive (#18596). +- Meetings: desktop companion meetings (`/api/companion/*`): the companion app streams a person's own meeting client audio and microphone, after a versioned consent confirmation, as a live meeting Betty takes part in; her replies go to the companion window (#18596). +- Meetings: finished meetings (ended, failed, blocked, declined, left) and stale join requests get a 🗑 Delete action with an in-page confirm, plus **Clear finished** to delete all finished meetings at once. Delete removes the meeting record, its transcript and live state on the hive, erases it in Vexa with Quorum `meeting.delete` (best effort; a failure is reported and the record is still removed), and drops the Calendar entry that existed only for that meeting. Past Calendar entries get the same 🗑 Delete; scheduled ones keep Skip and Cancel. A meeting hive is still in answers 409 (leave it first). Knowledge facts and drafts made from a meeting are not touched. Owner only, audited as `meeting_delete` (`DELETE /api/meetings/{id}`, `POST /api/meetings/clear-finished`, `DELETE /api/meetings/schedule/{eid}`). + +### Changed + +- Meetings: the hive image ships quorum v0.3.0, which adds the desktop companion platform (#18596). + +### Fixed + +- Meetings no longer restart-loop the Spektacular live child when the installed spektacular has no `meeting` command (before hivecommons/spektacular#126): hive probes `spektacular meeting --help` once per installed binary, keeps live mode off without starting the child, stops forwarding transcript lines to it, and the Meetings panel still says notes and clarifying questions are off. A supported child that keeps exiting still restarts with capped exponential backoff, warns at most once per backoff window, and is self-reported as `spek_stream_unavailable` instead of `unclassified_error_repeated` ([#18630](https://github.com/hivecommons/hive/issues/18630)). +- Hosted spokes on push-reported clusters (such as vllm-d and a-ks-wec2) now get the cluster's `host_aliases` from clusters.json: the hub sends them on every heartbeat and the spoke converges its own Deployment, idempotently and keeping unrelated aliases, and both sides log a WARN when the aliases cannot be applied (#18637). + +## 2026-11-17 (v5.294.0) + +### Added + +- Reference repos: the own-item follow-up now also routes comments and reviews on issues and PRs the hive opened as its GitHub App bot (`project.remote_write_identity`), listing each repo under the login its writes appear as and reading it with that repo's own credential (#18516). +- The Activity Monitor offers owners **Send to *profile*** on a row of a model-bound repository whose state the model hands to an agent profile (Send to techdocs-drafter on an accepted TechDocs request, Send to techdocs-verifier on a draft PR), backed by the owner-only `POST /api/workflows/dispatch`, which claims the issue, kicks the agent and records the hand-off (audited `workflow_dispatch`). +- A repository bound to a workflow model now hands each item to the agent its model names: the curated TechDocs model sends an accepted request (`triage/accepted`, no covering PR) to the `techdocs-drafter` agent once, claiming the issue for it first, and runs `techdocs-verifier` on each `/verify` on the draft PR; model-owned items are withheld from the scanner and other generic lanes, and hand-offs are recorded in `workflow-dispatch.json` so a restart never repeats one (cncf/techdocs#395). + +### Changed + +- Notifications: in `github.notifications.mode: quiet` (the default) three more notes stop posting comments of their own and become `hive:status` card sections, edited in place: the review Follow-ups checklist (`review-followups`, collapsible, ticked entries kept), the 090 ready-for-human-merge entry when the author's trust tier is the only objection (`stage-ready`), and the issue-side stage note naming the PR that covers the issue (`issue-pr-link`). Every 085 needs-decision entry and a held or manual-merge 090 still post a notifying comment; `mode: normal` is unchanged (hivecommons/hive#18606). +- CI: the advisory mutation report no longer opens a PR comment for an `error` verdict (gremlins wrote no report, so there is nothing to act on); the report stays in the job summary and an earlier finding comment is edited instead. The closed-issue `/reopen` hint no longer comments when the reporter is a bot or a maintainer who can reopen it themselves (hivecommons/hive#18607). + +### Fixed + +- ioscan: a kick withheld under fail-closed because untrusted issue/PR text tripped the injection policy is no longer fleet-reported as `ioscan_classifier_unavailable`. That block is the defense working as designed, so it now logs `ioscan fail-closed withheld kick on an injection finding` (or the held-PR coordination variant) and stays in local logs and the audit trail; only a kick withheld while the classifier is failing keeps the `ioscan fail-closed blocked kick` wording, now with the classifier error and fix on the held-PR coordination path too ([#18459](https://github.com/hivecommons/hive/issues/18459)). +- The open-PR guard now writes the `has-open-pr` label on its own time budget, remembers a failed write per issue instead of retrying it on every claim pass, logs refusals (no write access, issue gone or locked) at INFO, and reports repeated transient failures under a new `open_pr_guard_label_failed` fleet-report class instead of `unclassified_error_repeated` (#18589). +- **Betty always closes the loop on a deep dive, and meeting research finds its agents again.** The research agents (`research-1..N`) live only in the runtime, so every config reload or reviewer autoscale step removed them and each later research request failed silently with "no research agent available"; they are now kept across reconciles, and a dispatch that finds none registers them again. When a deep dive cannot start, the agent reports an error, or it runs past 5 minutes, Betty says "I couldn't look that up: ." in the channel she was asked in (spoken for a spoken question); one running past 30 seconds gets one "still looking" note, and a spoken question's answer is spoken. Settings, Meeting attendee shows the research agents, how many are available, and a "No research agent available" warning. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- Meetings keep nothing from before the consent notice is delivered: until Quorum confirms the notice in chat (or hive speaks it), every transcript segment, draft and chat line is discarded rather than stored or handed to Betty, Spektacular, research, knowledge facts or the Inbox, and lines that arrive later but belong before the notice are discarded too; only a `pre_consent_dropped` count is kept, a leave phrase still removes the bot, and a reconnect or restart while the notice is pending no longer replays Quorum's buffer ([#18590](https://github.com/hivecommons/hive/issues/18590)). +- **A failed meetings call now names its error in the hive log.** Every meetings API failure (join, approve, decline, leave, schedule, actions) is logged with its method, path, status and error, and a Discord `/hive join` request whose approval fails keeps the reason on the request and stays approvable. Before, a Quorum refusal such as `blocked: the bot has no permission to connect to that voice channel` reached a hosted owner only as the ingress's generic "hive unavailable" page and left no line in the spoke log The Meetings panel also stops calling a blocked Discord join "The meeting refused a guest bot" (Discord has no lobby): it says the bot lacks View Channel, Connect or Speak on that voice channel, and a pending join request shows why its approval failed ([#18598](https://github.com/hivecommons/hive/issues/18598)). + +## 2026-11-17 (v5.293.0) + +### Added + +- New owner setting **Advisory report in every repo** (Settings, Governor Config, Advisory; `governor.advisory.per_repo`, default off, `HIVE_ADVISORY_PER_REPO` sets the default when the key is absent). Off keeps one Hive Advisory Report in the primary repo covering all repos. On, each monitored repo also gets exactly one report, found by title plus the `hive/advisory` label or a hive marker so a restart never files a second; a report a person closed is left closed and no longer posted to; repos with Issues disabled or no write access are skipped with one warning. Turning it off stops updating those reports without closing them ([#18448](https://github.com/hivecommons/hive/issues/18448)). +- **Release-feed watcher: kick an agent when a tracked upstream dependency releases.** `governor.release_feeds.agents..sources` lists GitHub releases or tags (`owner/repo`), PyPI packages, or a generic index URL. The governor polls them every `interval_s` (default `HIVE_RELEASE_FEED_INTERVAL`, else 15 minutes) with `If-None-Match` / `If-Modified-Since`, through the spoke's upstream-proxy rules, and keeps last-seen versions in `/data/release-feeds.json`. A new version queues a kick for the subscribed agent through the kick queue with a `${NEW_RELEASES}` block (package, old, new, URL) and records the reason on the governor kick-history entry; the first answer is a baseline and an unchanged feed kicks nothing. Kicks pass the same gates as the webhook CI-failure kick and are bounded by a per-agent cooldown (default `HIVE_RELEASE_FEED_COOLDOWN`, else 1 hour). Private indexes authenticate with `auth_var`, an existing per-agent or hive-level variable whose value is never logged. See `src/docs/release-feeds.md` ([#18494](https://github.com/hivecommons/hive/issues/18494)). +- Settings → Repos → Reference repos: owner-only "Post on other projects as" control (Hive bot, falling back to me / Hive bot only / Me) for `project.remote_write_identity`, a per-repo line saying which identity posts there and why, `PUT /api/config/reference-repos/write-identity`, and the docs (#18516). +- Mail-in to `hive@` now accepts a plain-text or HTML body as a Markdown note (a body-only mail becomes a `message.md` Inbox item), accepts `.mp4`, `.mov` and `.mkv` video (MP4 and QuickTime audio is transcribed), and raises the SMTP SIZE limit to 35 MiB with 25 MB per file; hive-mx spools mail to disk, the hub relays large deliveries to the spoke in signed pieces under the 1 MiB ingress limit, and each hive gets a daily byte budget (#18564). +- **Sandboxed agents can open issues and keep topic claims.** The sandbox request spool now carries `issue` and `topic` requests as well as reviews, signed with the same per-kick key and attributed to the kick's own agent. `hive-open-issue` and the new `hive-topic` relay ship on every sandbox's `PATH` and write to the spool when they run in a sandbox. Issue requests go through the issue relay with all its gates (ACMM mode, lane write allowlist, repo scope, dedupe by title, canary scan, audit); topic requests land in the agent's own bead store, and `hive-topic claim ` without `--pr` claims the PR hive opens from the kick. `hive-review` is unchanged. ([#18572](https://github.com/hivecommons/hive/issues/18572), [#18573](https://github.com/hivecommons/hive/issues/18573)) +- **Sandboxed fix kicks push to the agent's own existing PR, and sandbox-opened PRs are credited to their agent.** A sandboxed kick that names one of the agent's open PRs (an explicit `[hive:target-pr #]` line, the red-PR or red-CI kick header, the first FIX-BEFORE-NEW row, or a single topic claim) checks out that PR's head branch and adds its commits there. Hive pushes only if the PR is open, same-repo, opened by this hive for this agent, still at the head the kick checked out (`--force-with-lease`), and the usual broker scans pass; otherwise the kick fails with the reason and nothing is pushed. A PR a sandboxed kick opens now gets the attribution trailer and `agent_pr_created` audit entry a relay-opened PR gets, so red CI on it kicks the agent that opened it instead of `scanner`. ([#18574](https://github.com/hivecommons/hive/issues/18574), [#18575](https://github.com/hivecommons/hive/issues/18575)) +- **A calendar email whose proof mail was refused now says why** under **Settings, Meeting attendee, My calendar emails**, instead of only "pending": the reason in plain words with what to do (not authenticated by the sender's mail server, sent from a different address, naming the From address hive-mx saw, or the code expired), when it arrived, and the SPF, DKIM and alignment results hive-mx saw (results and domains only, never the message), with a **Check again** button. The calendar-emails API carries the same summary as `last_failure`. A proof mail whose code matches no address shows, sender masked, under the owner's **Recent proof attempts** (`recent_proof_attempts`, owner only). ([#18579](https://github.com/hivecommons/hive/issues/18579)) +- **Betty answers questions about the hive from its live state.** "Betty, how many open issues are there?" or "what PRs are open in the hive repo?" is answered in seconds from the same in-memory status the Overview and the dashboard chat read (open, actionable, held and blocked counts, agents and what they are doing, recent completions, open PRs), and is no longer sent to the research agents. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- Workflow models resolve where each agent profile takes over an item (`spec.agents[].dispatch`, or by convention the grant's first state, on entry or on the gate reply for a reviewing profile), and `workflow.dispatch.profiles..agent` maps a profile to a hive agent (default: an agent named after the profile, else a writer, docs or feature agent); the curated TechDocs model hands Accepted to `techdocs-drafter` and `/verify` to `techdocs-verifier`. + +### Changed + +- Notifications: in `github.notifications.mode: quiet` (the default) the auto-merge note ("merged by Hive trusted-author auto-merge …" / "merged by Hive on behalf of …") is now the `auto-merge` section of the PR's `hive:status` card, edited in place with its @-mentions neutralised, instead of a fresh comment on every merged PR (about 32 notifying comments a day on hivecommons/hive). `mode: normal` keeps the standalone comment, and a merge that waits on a person still gets its own 085 / 090 entry comment (hivecommons/hive#18452). +- Betty speaks her acknowledgement as soon as a partial transcript names her with a request, instead of after the final transcript, so the room hears she is on it within a couple of seconds (hivecommons/hive#18590). + +### Fixed + +- The kick freshness guard now logs a ref it withheld because its repo's snapshot is stale (last-good data after a failed reconcile or a refused enumeration) at INFO, and a kick withheld only by stale, closed or deferred refs summarises at INFO, so a stale repo no longer files a `kick_freshness_withheld` self-report on every cycle; the stale repo itself is still logged at WARN where it is produced ([#18470](https://github.com/hivecommons/hive/issues/18470)). +- **A Copilot agent's config being re-linked to the shared login is no longer reported as a fleet error.** The routine, successful repair logged at WARN, so a hive where the CLI kept saving over the link filed an `unclassified_error_repeated` self-report (16 in 10 minutes); it now logs at INFO, while failed repairs still warn ([#18497](https://github.com/hivecommons/hive/issues/18497)). +- The dashboard's per-agent **Run in sandbox** toggle no longer wedges an agent ([#18525](https://github.com/hivecommons/hive/issues/18525)). Turning it on used to write `sandbox.enabled: true` alone, so with no `agent_sandbox` image every kick failed with "sandbox image is not configured". It now writes a complete block, prefilled from the `agent_sandbox` defaults or copied (`copy_from`) from another agent that has one, minus that agent's `publish.verify`, and refuses with a 400 that names what is missing when neither exists. The General tab shows a picker for the source when the toggle is turned on, with an inline error when there is none. Turning it off keeps the block and sets `enabled: false`. Switching an agent into sandbox mode now stops its tmux session, and a refused kick names the real state ("agent has a live interactive session; sandbox kicks are blocked until it stops") instead of "sandbox execution already running". Every sandbox Kubernetes Job now carries `activeDeadlineSeconds` (the kick timeout plus a 15-minute margin), and at boot hive reaps unfinished `app.kubernetes.io/managed-by=hive-kubejob` sandbox Jobs that no live kick owns, logging and auditing each one (`sandbox_job_reaped`). Before this, one Job sat Pending for 11 days. +- Repeated inference backend 401 warnings now produce an operator-action self-report identifying the rejected credential instead of an unclassified error. +- The "failed to send kick" log for a busy shared CLI session pool is now INFO instead of WARN, so routine pool contention is no longer fleet-reported as a repeated runtime error (#18544). +- The hold guard no longer logs a WARN (which self-reported as a defect) when GitHub has not yet shown the new head right after keeping a held PR mergeable; it retries next tick. +- CI: label flips no longer cancel the required `verify PR contents`, `Sign-off survives the squash` and `changelog-fragment-guard` checks. Their concurrency groups now carry the head SHA, and label, edit and reopen re-checks each run in their own group, so a burst of `lgtm`/`approved`/size labels can neither cancel the run in flight nor drop a queued run. Those cancelled runs used to leave PRs BLOCKED and jam Tide. A new guard, `.github/scripts/test-required-check-concurrency.py`, evaluates every required-check workflow's concurrency for a label burst and fails on any shared group (#18578). +- **Betty answers a spoken "introduce yourself" by voice.** The intro reply did not carry the reply kind, so it went to the meeting chat even when the request was spoken and voice was on; typed intro requests still get a chat reply. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- Meetings: the consent notice now counts as posted only when Quorum confirms the line reached the meeting. In a live Teams test the bot could not open the meeting chat (`chat_send_failed reason=no_composer`, chat closed to guests) yet hive recorded `consent_posted_at` and kept listening (#18590, finding 11). Hive now posts the notice with `chat.post {confirm:true}` (quorum 0.2.2) and treats anything but `delivery: confirmed`, including an older Quorum that cannot say, as not posted. When chat cannot deliver it and the meeting has voice on, Betty speaks the notice instead (`consent_via: voice`, shown in the Meetings panel); when neither works she leaves at once with state reason `consent_undelivered`, and the panel says "Left: the consent notice could not be delivered (chat unavailable (...), voice off)". Transient failures (bot not admitted yet, rate limit, a dropped call) are retried up to three times before that. The image now ships quorum v0.2.2. +- **The Meetings panel says plainly when notes and clarifying questions are off.** When Spektacular live mode is not running for a meeting the panel now leads with "Notes and clarifying questions are off" (or "paused" while hive cannot reach a restarting child, cleared when it writes again) instead of showing nothing until the child gives up. ([#18590](https://github.com/hivecommons/hive/issues/18590)) +- **A hosted hive's own error is no longer hidden behind "hive unavailable".** The hosted ingress replaces any 502 with its generic "this hive is restarting" page, so an error the spoke actually answered (a Discord join refused for a missing permission, a failed upstream call) read as an outage. On a request that came through the hosted ingress, a JSON 502 from a dashboard API handler is now sent as 424 with the original status in `X-Hive-Status`, which the ingress passes through with the spoke's message; the generic page still appears when the spoke cannot be reached ([#18598](https://github.com/hivecommons/hive/issues/18598)). +- Dashboard: the Collaborate sidebar group (Inbox, Calendar, Meetings, Inception (spek)) no longer disappears into the Dashboard group when the section layout is applied, and now sits right after Dashboard, before Agents. +- Stale Dependabot PRs no longer get stuck behind refused `@dependabot rebase` comments: the hive never posts Dependabot commands (Dependabot ignores them from the App), updates a merely-behind Dependabot PR with the update-branch API, leaves a conflicting one to Dependabot's own rebase, and closes a conflicting one whose branch someone else pushed to so Dependabot opens a fresh PR (#18441). +- Avoid spurious runtime self-reports when automerge safely refuses a squash of unsigned commits; the refusal remains visible in logs. + ## 2026-11-17 (v5.292.0) ### Added diff --git a/changelog.d/added-18692-kick-refusals-docs.md b/changelog.d/added-18692-kick-refusals-docs.md new file mode 100644 index 0000000000..737ad8d29b --- /dev/null +++ b/changelog.d/added-18692-kick-refusals-docs.md @@ -0,0 +1 @@ +- Added a [Kick refusals](src/docs/kick-refusals.md) docs page that explains `kick_refusal` and every reason a kick is refused (the agent's model declining, startup deferral, paused, failed start, stopped, provider backoff, restart hold, cancelled by restart, delivery already in progress, no session, sandbox misconfiguration, unknown agent), what to do about each, and when it is normal. The dashboard's "Kick failed" toast now has a "What does this mean?" button, the navbar health hover names the page for `kick_refusal`, and `POST /api/kick/{agent}`, `GET /api/kick/{agent}/status` and the deep-health `kick_refusal` check return a `docs_url` field when the failure matches a documented reason. Fixes #18692. diff --git a/changelog.d/changed-dco-allow-individual-remediation.md b/changelog.d/changed-dco-allow-individual-remediation.md new file mode 100644 index 0000000000..f9fad193eb --- /dev/null +++ b/changelog.d/changed-dco-allow-individual-remediation.md @@ -0,0 +1 @@ +- CI: the DCO app now honours an author's own DCO remediation commit (`.github/dco.yml`, `allowRemediationCommits.individual`), so a v5 -> v6 top-up that carries commits whose author email has no matching sign-off can be unblocked by the author without rewriting history (#18697). diff --git a/changelog.d/fixed-18690-workflow-dispatch-standard-fleet.md b/changelog.d/fixed-18690-workflow-dispatch-standard-fleet.md new file mode 100644 index 0000000000..86bdd702a1 --- /dev/null +++ b/changelog.d/fixed-18690-workflow-dispatch-standard-fleet.md @@ -0,0 +1 @@ +- **A bound workflow now starts on a standard hive, and a profile nothing can fill is no longer silent** ([#18690](https://github.com/hivecommons/hive/issues/18690)). A workflow model hands some states to an agent profile (TechDocs: an Accepted request to `techdocs-drafter`, a draft to `techdocs-verifier`), but the hive only accepted a pinned agent, an agent named after the profile, or a `writer`, `docs` or `feature` agent. A default roster has none of those, so every pass logged `no agent fills the profile`, the item stayed withheld from every other lane, and nothing in the dashboard said why. The resolver now ends with a last resort: the hive's issue worker (the `scanner` role) for a profile that opens a pull request and the `reviewer` for one that reviews, in both cases only an agent allowed to work on the bound repository; a supervisor, guide or brainstorm agent is never picked on its own. When still nothing fills a profile, the repository's workflow row on Settings → Repos warns and offers an agent picker that saves `workflow.dispatch.profiles..agent` (owner only, `PUT /api/workflows/dispatch/profiles`, no restart), the item's Activity Monitor row wears a `no agent for ` badge with the reason, and **Send to *profile*** answers with the reason and where to fix it. Operators who relied on a profile staying unfilled should set `workflow.dispatch.profiles..disabled: true`. diff --git a/dashboard/openapi.json b/dashboard/openapi.json index 794c9c7c29..a944c64238 100644 --- a/dashboard/openapi.json +++ b/dashboard/openapi.json @@ -98,6 +98,564 @@ } ], "paths": { + "/api/companion/info": { + "get": { + "tags": [ + "Meetings" + ], + "summary": "Desktop companion settings", + "description": "Owner or read-write. What the desktop companion shows and sends before a capture: the consent notice and its version, the PCM sample rate, the channels and the chunk limit.", + "responses": { + "200": { + "description": "Companion settings", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "consent_version": { + "type": "string" + }, + "consent_text": { + "type": "string" + }, + "sample_rate": { + "type": "integer" + }, + "channels": { + "type": "array", + "items": { + "type": "string" + } + }, + "max_chunk_bytes": { + "type": "integer" + } + } + } + } + } + }, + "403": { + "description": "Not owner or read-write, or another person's companion meeting", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "503": { + "description": "Meetings or companion meetings are not enabled on this hive", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + } + } + } + }, + "/api/companion/meetings": { + "post": { + "tags": [ + "Meetings" + ], + "summary": "Start a desktop companion meeting", + "description": "Owner or read-write. Starts a live meeting fed by the desktop companion (the person's own meeting client audio and microphone). The request must name the current consent_version and confirm the participants were told (consent_attested). Who attested, when and the notice version are stored with the meeting.", + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "title": { + "type": "string" + }, + "app": { + "type": "string" + }, + "host_name": { + "type": "string" + }, + "consent_version": { + "type": "string" + }, + "consent_attested": { + "type": "boolean" + } + }, + "required": [ + "consent_version", + "consent_attested" + ] + } + } + } + }, + "responses": { + "201": { + "description": "The meeting record", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "platform": { + "type": "string" + }, + "state": { + "type": "string" + } + } + } + } + } + }, + "400": { + "description": "Consent not acknowledged or a field too long", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "403": { + "description": "Not owner or read-write, or another person's companion meeting", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "503": { + "description": "Meetings or companion meetings are not enabled on this hive", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + } + } + } + }, + "/api/companion/meetings/{id}/audio": { + "post": { + "tags": [ + "Meetings" + ], + "summary": "Upload one companion audio chunk", + "description": "The companion meeting's starter (or an owner). The body is raw 16 kHz mono s16le PCM, at most max_chunk_bytes. Chunks are accepted in order per channel from seq 0; a gap answers 409 with expected_seq and a resend of a stored seq is acknowledged with duplicate: true.", + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "schema": { + "type": "string" + }, + "description": "Meeting id from the start response" + }, + { + "name": "channel", + "in": "query", + "required": true, + "schema": { + "type": "string", + "enum": [ + "call", + "mic" + ] + }, + "description": "call = the meeting app's audio, mic = the person's microphone" + }, + { + "name": "seq", + "in": "query", + "required": true, + "schema": { + "type": "integer", + "minimum": 0 + }, + "description": "Chunk number per channel, from 0" + } + ], + "requestBody": { + "required": true, + "content": { + "application/octet-stream": { + "schema": { + "type": "string", + "format": "binary" + } + } + } + }, + "responses": { + "200": { + "description": "Stored or recognised", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "seq": { + "type": "integer" + }, + "duplicate": { + "type": "boolean" + } + } + } + } + } + }, + "400": { + "description": "Bad channel, seq or sample count", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "404": { + "description": "No such meeting", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "409": { + "description": "Out of order (expected_seq) or the meeting is not live", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + }, + "expected_seq": { + "type": "integer" + } + } + } + } + } + }, + "413": { + "description": "Chunk too large", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "403": { + "description": "Not owner or read-write, or another person's companion meeting", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "503": { + "description": "Meetings or companion meetings are not enabled on this hive", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + } + } + } + }, + "/api/companion/meetings/{id}/replies": { + "get": { + "tags": [ + "Meetings" + ], + "summary": "Betty's lines for the companion window", + "description": "The companion meeting's starter (or an owner). Lines hive posted for the meeting (the consent notice first) after the cursor, and the meeting state so the window notices the meeting ended.", + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "schema": { + "type": "string" + }, + "description": "Meeting id from the start response" + }, + { + "name": "after", + "in": "query", + "required": false, + "schema": { + "type": "integer", + "minimum": 0 + }, + "description": "Cursor: the next_seq of the previous call" + } + ], + "responses": { + "200": { + "description": "Lines and state", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "lines": { + "type": "array", + "items": { + "type": "object", + "properties": { + "seq": { + "type": "integer" + }, + "text": { + "type": "string" + }, + "at": { + "type": "string" + } + } + } + }, + "next_seq": { + "type": "integer" + }, + "state": { + "type": "string" + } + } + } + } + } + }, + "400": { + "description": "Bad cursor", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "404": { + "description": "No such meeting", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "403": { + "description": "Not owner or read-write, or another person's companion meeting", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "503": { + "description": "Meetings or companion meetings are not enabled on this hive", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + } + } + } + }, + "/api/companion/meetings/{id}/end": { + "post": { + "tags": [ + "Meetings" + ], + "summary": "End a companion meeting", + "description": "The companion meeting's starter (or an owner): the companion stopped capturing. Speech still buffered is transcribed first.", + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "schema": { + "type": "string" + }, + "description": "Meeting id from the start response" + } + ], + "responses": { + "200": { + "description": "The meeting record", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "state": { + "type": "string" + } + } + } + } + } + }, + "404": { + "description": "No such meeting", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "403": { + "description": "Not owner or read-write, or another person's companion meeting", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + }, + "503": { + "description": "Meetings or companion meetings are not enabled on this hive", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "error": { + "type": "string" + } + } + } + } + } + } + } + } + }, "/api/meetings/{id}/approve": { "post": { "tags": [ @@ -157,6 +715,152 @@ } } }, + "/api/meetings/clear-finished": { + "post": { + "tags": [ + "Meetings" + ], + "summary": "Clear finished meetings", + "description": "Owner only. Deletes every meeting in a final state that finished before `before` (RFC 3339; empty or later than now means now), each exactly like DELETE /api/meetings/{id}. Live meetings and pending join requests stay. Each deleted meeting is audited as meeting_delete.", + "requestBody": { + "required": false, + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "before": { + "type": "string", + "format": "date-time" + } + } + } + } + } + }, + "responses": { + "200": { + "description": "Deleted meetings", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "deleted": { + "type": "array", + "items": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "state": { + "type": "string", + "description": "The state the meeting was in" + }, + "upstream_error": { + "type": "string", + "description": "Set when Vexa (Quorum meeting.delete) could not confirm the erase; the record on this hive is removed anyway" + }, + "calendar_removed": { + "type": "integer", + "description": "Calendar entries removed because they existed only for this meeting" + } + } + } + }, + "count": { + "type": "integer" + } + } + } + } + } + }, + "400": { + "description": "before is not an RFC 3339 time" + }, + "403": { + "description": "Owner access required" + }, + "503": { + "description": "Meetings are not enabled on this hive" + } + } + } + }, + "/api/meetings/schedule/{eid}": { + "delete": { + "tags": [ + "Meetings" + ], + "summary": "Delete a past Calendar entry", + "description": "Owner only. Deletes a finished Calendar entry (ended, cancelled, declined, failed, blocked, skipped) and the meetings it created, each like DELETE /api/meetings/{id}. Entries still scheduled, awaiting approval or live keep Skip and Cancel and answer 409, as does an entry whose meeting hive is still in. Audited as meeting_schedule_delete, plus meeting_delete for each meeting.", + "parameters": [ + { + "name": "eid", + "in": "path", + "required": true, + "schema": { + "type": "string" + }, + "description": "Schedule event id" + } + ], + "responses": { + "200": { + "description": "Deleted", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "meetings": { + "type": "array", + "items": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "state": { + "type": "string", + "description": "The state the meeting was in" + }, + "upstream_error": { + "type": "string", + "description": "Set when Vexa (Quorum meeting.delete) could not confirm the erase; the record on this hive is removed anyway" + }, + "calendar_removed": { + "type": "integer", + "description": "Calendar entries removed because they existed only for this meeting" + } + } + } + } + } + } + } + } + }, + "403": { + "description": "Owner access required" + }, + "404": { + "description": "Unknown event" + }, + "409": { + "description": "The entry is still to come or live, or hive is still in its meeting" + }, + "503": { + "description": "Meetings are not enabled on this hive" + } + } + } + }, "/api/meetings/{id}/decline": { "post": { "tags": [ @@ -446,6 +1150,64 @@ } }, "/api/meetings/{id}": { + "delete": { + "tags": [ + "Meetings" + ], + "summary": "Delete a finished meeting or a pending join request", + "description": "Owner only. Allowed for a meeting in a final state (ended, failed, blocked, declined, left, handed_off, purged) and for a pending platform join request (declined with meeting.leave, then removed). Removes the meeting record, its transcript and live state on this hive, erases it in Vexa with Quorum meeting.delete (best effort: not_found is fine, any other error is returned as upstream_error while the record is still removed) and removes the Calendar entry that existed only for this meeting. Knowledge facts and drafts already made from the meeting are not touched. Audited as meeting_delete.", + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "Deleted", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "state": { + "type": "string", + "description": "The state the meeting was in" + }, + "upstream_error": { + "type": "string", + "description": "Set when Vexa (Quorum meeting.delete) could not confirm the erase; the record on this hive is removed anyway" + }, + "calendar_removed": { + "type": "integer", + "description": "Calendar entries removed because they existed only for this meeting" + } + } + } + } + } + }, + "403": { + "description": "Owner access required" + }, + "404": { + "description": "Unknown meeting" + }, + "409": { + "description": "Hive is still joining, in the lobby of or live in the meeting: leave the meeting first" + }, + "503": { + "description": "Meetings are not enabled on this hive" + } + } + }, "get": { "tags": [ "Meetings" @@ -42416,6 +43178,176 @@ } } }, + "/api/workflows/dispatch": { + "post": { + "tags": [ + "Workflows" + ], + "summary": "Send a model-owned item to its agent profile", + "description": "Owner only. The stage tracker's \"Send to \" on a row of a repository bound to a workflow model whose current state the model hands to an agent profile (cncf/techdocs#395: Accepted -> techdocs-drafter, Drafting -> techdocs-verifier). Reads the item fresh, resolves its model state and the hive agent that fills the profile (workflow.dispatch.profiles..agent; a profile disabled for automatic dispatch still resolves), takes the agent claim on an issue first, and kicks the agent with the dispatch prompt. Once the kick lands the hand-off is recorded in the dispatch log, so the automatic pass does not repeat it. Audited as workflow_dispatch.", + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "type": "object", + "required": [ + "repo", + "number" + ], + "properties": { + "repo": { + "type": "string", + "description": "owner/repo" + }, + "number": { + "type": "integer" + } + } + } + } + } + }, + "responses": { + "200": { + "description": "Kick delivered", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "status": { + "type": "string", + "enum": [ + "sent" + ] + }, + "ref": { + "type": "string" + }, + "model": { + "type": "string" + }, + "state": { + "type": "string" + }, + "profile": { + "type": "string" + }, + "agent": { + "type": "string" + }, + "result": { + "type": "object", + "description": "Tracker toast: action dispatched, outcome, next" + } + } + } + } + } + }, + "400": { + "description": "Body is not {repo, number}" + }, + "403": { + "description": "Not an owner" + }, + "404": { + "description": "The bound model does not hand the item's current state to a profile (or workflow.enabled is off)" + }, + "409": { + "description": "No enabled agent fills the profile, someone else holds the issue's claim, or the agent is busy" + }, + "502": { + "description": "The item could not be read or the kick failed" + }, + "503": { + "description": "Dispatch is not wired on this hive" + } + } + } + }, + "/api/workflows/dispatch/profiles": { + "put": { + "tags": [ + "Workflows" + ], + "summary": "Pick the hive agent a workflow profile is handed to", + "description": "Owner only. The agent picker on a repository's workflow row in Settings -> Repos (hivecommons/hive#18690). Pins workflow.dispatch.profiles..agent, or clears the pin when agent is empty so the profile resolves by default again (an agent named after the profile, then a writer, docs or feature agent, then the hive's issue worker or, for a reviewing profile, the reviewer). The profile's prompt_template and disabled settings are kept. The next scheduler pass reads the live config, so no restart is needed. Audited as workflow_dispatch_agent.", + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "type": "object", + "required": [ + "profile" + ], + "properties": { + "profile": { + "type": "string", + "description": "Model profile name" + }, + "agent": { + "type": "string", + "description": "An enabled agent that is not a replica; empty clears the pin" + } + } + } + } + } + }, + "responses": { + "200": { + "description": "Pin saved (or already as requested)", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "changed": { + "type": "boolean" + }, + "profile": { + "type": "string" + }, + "agent": { + "type": "string" + }, + "dispatch": { + "type": "array", + "description": "Every bound repository's profiles: repo, profile, agent, source, pinned, disabled, problem (why nothing fills it), states", + "items": { + "type": "object" + } + }, + "agents": { + "type": "array", + "description": "Agents that can be picked", + "items": { + "type": "string" + } + } + } + } + } + } + }, + "400": { + "description": "Body is not {profile, agent}, or agent is not an enabled agent of this hive" + }, + "403": { + "description": "Not an owner" + }, + "500": { + "description": "The config could not be saved" + }, + "503": { + "description": "Config unavailable" + } + } + } + }, "/api/workflows/enabled": { "put": { "tags": [ diff --git a/dashboard/test/meetings-calendar-role-race.test.js b/dashboard/test/meetings-calendar-role-race.test.js index b2caab7f22..eb7c174fc4 100644 --- a/dashboard/test/meetings-calendar-role-race.test.js +++ b/dashboard/test/meetings-calendar-role-race.test.js @@ -36,11 +36,12 @@ function extractConst(name) { const GH_AUTH = ['hiveRoleSettled']; const MEETINGS = ['meetingsIsOwner', 'meetingsRoleKnown', 'meetingsOnRoleReady', 'fetchMeetings', 'renderMeetings', 'meetingsJoinFormHtml', - 'meetingsRowHtml', 'meetingsFormatTime', 'meetingsFriendlyReason', 'meetingsReasonHtml', 'meetingsLabel']; + 'meetingsRowHtml', 'meetingsFormatTime', 'meetingsFriendlyReason', 'meetingsReasonHtml', 'meetingsLabel', + 'meetingsFinished', 'meetingsCanDelete', 'meetingsDeleteButtonHtml']; const CALENDAR = ['calendarRoleKnown', 'calendarOnRoleReady', 'calendarCanView', 'calendarIsOwner', 'fetchCalendar', 'calendarRefreshMeetings', 'fetchMeetingDrafts', 'renderCalendar', 'renderMeetingDrafts', 'meetingDraftsSplit', 'calendarFormHtml', 'calendarViewerTimeZone', 'calendarRowHtml', 'calendarFormatTime', 'calendarPlatformLabel', 'calendarMeetingLink', 'calendarActionButtons', 'renderCalendarIndicator', - 'calendarIndicatorState', 'calendarBadgeCount', 'calendarMeetingList']; + 'calendarIndicatorState', 'calendarBadgeCount', 'calendarMeetingList', 'calendarCanDelete']; function makeSandbox(opts) { const els = {}; @@ -73,7 +74,7 @@ function makeSandbox(opts) { vm.createContext(sb); const consts = ['MEETINGS_ERROR_INLINE_CHARS', 'MEETINGS_PLATFORM_LABELS', 'MEETINGS_PILL_STATUS', 'MEETINGS_LIVE_STATES', 'CALENDAR_ACTIVE_STATES', 'CALENDAR_DEFAULT_NAME', 'CALENDAR_PLATFORM_LABELS', 'CALENDAR_PLATFORM_ICONS', 'CALENDAR_SOURCE_LABELS', - 'CALENDAR_STATUS_LABELS', 'CALENDAR_PILL_STATUS'].map(extractConst); + 'CALENDAR_STATUS_LABELS', 'CALENDAR_PILL_STATUS', 'MEETINGS_FINAL_STATES', 'CALENDAR_FINISHED_STATUSES'].map(extractConst); // const declarations stay script-scoped in a vm context; re-declare the // mutable module state with var so the extracted functions share it. vm.runInContext(consts.join('\n') + '\n' + GH_AUTH.concat(MEETINGS, CALENDAR).map(extractFunction).join('\n'), sb); diff --git a/src/Dockerfile b/src/Dockerfile index 051594f3b6..d0ef4f72ae 100644 --- a/src/Dockerfile +++ b/src/Dockerfile @@ -447,7 +447,7 @@ RUN npm install -g --ignore-scripts --no-fund --no-audit @openai/codex@${CODEX_V echo "WARN: OpenAI Codex CLI install failed — skipping" # --- Layer 7: Claude Code (most frequent updates) --- -ARG CLAUDE_CODE_VERSION=2.1.325 +ARG CLAUDE_CODE_VERSION=2.1.326 # --ignore-scripts (kept, deliberately: no arbitrary postinstall runs during the # build) skips claude-code's install.cjs, which is what LINKS the platform # native binary into the package's bin/. The binary itself ships in the @@ -530,8 +530,8 @@ RUN ARCH="${TARGETARCH:-$(dpkg --print-architecture)}" && \ # Bump note: see src/docs/cli-pins.md (quorum row); update QUORUM_VERSION and # QUORUM_SHA256 together from the release's quorum.mjs.sha256, and # QUORUM_DAVEY_VERSION from quorum's package-lock.json. -ARG QUORUM_VERSION=v0.2.2 -ARG QUORUM_SHA256=f36ab9b3618f9a0b9a017f67a8404001e449255f74ce8b34f2c70f287c08e1fc +ARG QUORUM_VERSION=v0.3.0 +ARG QUORUM_SHA256=bda595d8fa3ce98771b5be74c90dcbbbca0ef0cf919b4ab50fdb37b63103fc5c ARG QUORUM_DAVEY_VERSION=0.1.12 RUN if [ -n "${QUORUM_VERSION}" ]; then \ test -n "${QUORUM_SHA256}" || { echo "QUORUM_SHA256 is required with QUORUM_VERSION" >&2; exit 1; } && \ @@ -577,9 +577,9 @@ RUN ARCH="${TARGETARCH:-$(dpkg --print-architecture)}" && \ # --- Layer 8.27: OMP CLI (Oh My Pi — backend: omp) --- # Mirrored from Dockerfile.contributor so hub agents can launch backend: omp. # Downloads are optional like Goose/agy, but checksum mismatches are fatal. -ARG OMP_VERSION=18.20.0 -ARG OMP_SHA256_AMD64=c8694be8c6e6a1f45e3636dd8a8e9eaa78878ad629a89000b403ec5e456d3f89 -ARG OMP_SHA256_ARM64=8b5e197db7c58b7ebabf83c6f03597f89bae3a766a84c6481b3a942a125f5e22 +ARG OMP_VERSION=18.22.0 +ARG OMP_SHA256_AMD64=f706b27ba845834d8f8360705a2e9ba1534f3593fa24411d41358272de33642b +ARG OMP_SHA256_ARM64=7238dd213d5c61bc9bf51735d552bc71901fdd44329f862b0a971146b1cdac3e RUN ARCH="${TARGETARCH:-$(dpkg --print-architecture)}" \ && case "$ARCH" in \ amd64|x86_64) OMP_ASSET=omp-linux-x64; OMP_SHA256="$OMP_SHA256_AMD64" ;; \ diff --git a/src/Dockerfile.contributor b/src/Dockerfile.contributor index 140dc3f028..5904721081 100644 --- a/src/Dockerfile.contributor +++ b/src/Dockerfile.contributor @@ -263,7 +263,7 @@ RUN mkdir -p /etc/apt/keyrings \ # npm cache is cleaned in the same layer it's populated (issue #2613, # Observation 2 "also minor") so the cache doesn't linger in this layer's # diff, mirroring the cleanup already done for the bobshell install below. -ARG CLAUDE_CODE_VERSION=2.1.325 +ARG CLAUDE_CODE_VERSION=2.1.326 ARG COPILOT_VERSION=1.0.88 # --ignore-scripts (kept, deliberately: no arbitrary postinstall runs during # the build) skips claude-code's install.cjs, which is what LINKS the platform @@ -521,9 +521,9 @@ RUN ARCH="${TARGETARCH:-$(dpkg --print-architecture)}" \ # No credential is baked in: omp reads the selected model's provider # credential from its documented environment variables (ANTHROPIC_API_KEY, # OPENAI_API_KEY, …), which the Justfile's container arm already forwards. -ARG OMP_VERSION=18.20.0 -ARG OMP_SHA256_AMD64=c8694be8c6e6a1f45e3636dd8a8e9eaa78878ad629a89000b403ec5e456d3f89 -ARG OMP_SHA256_ARM64=8b5e197db7c58b7ebabf83c6f03597f89bae3a766a84c6481b3a942a125f5e22 +ARG OMP_VERSION=18.22.0 +ARG OMP_SHA256_AMD64=f706b27ba845834d8f8360705a2e9ba1534f3593fa24411d41358272de33642b +ARG OMP_SHA256_ARM64=7238dd213d5c61bc9bf51735d552bc71901fdd44329f862b0a971146b1cdac3e RUN ARCH="${TARGETARCH:-$(dpkg --print-architecture)}" \ && case "$ARCH" in \ amd64) OMP_ASSET=omp-linux-x64; OMP_SHA256="$OMP_SHA256_AMD64" ;; \ diff --git a/src/cmd/hive/main.go b/src/cmd/hive/main.go index 40bb3794b5..c72f80a328 100644 --- a/src/cmd/hive/main.go +++ b/src/cmd/hive/main.go @@ -96,6 +96,7 @@ import ( "github.com/hivecommons/hive/pkg/telegram" "github.com/hivecommons/hive/pkg/tokens" "github.com/hivecommons/hive/pkg/tracing" + "github.com/hivecommons/hive/pkg/workflowmodel" "github.com/hivecommons/hive/pkg/worksource" "github.com/hivecommons/hive/pkg/worksource/wavefront" // registers the additive Wavefront source (#8362) "gopkg.in/natefinch/lumberjack.v2" @@ -1128,8 +1129,12 @@ func (b *boot) wireBootClosures() { WorkflowItemSnapshot: func(ctx context.Context, repo string, number int) (github.WorkflowItemSnapshot, error) { return b.ghClient.WorkflowItemSnapshot(ctx, repo, number) }, + WorkflowPostMerge: func(ctx context.Context, repo string, m *workflowmodel.Model, defaultBranch string, prs []workflowmodel.MergedPR) (map[int]*stageengine.PostMergeFacts, error) { + return b.ghClient.ObservePostMerge(ctx, repo, m, defaultBranch, prs) + }, WorkflowRepoFile: workflowRepoFile(func() *github.Client { return b.ghClient }), WorkflowProvisionLabels: workflowProvisionLabels(b.cfg, func() *github.Client { return b.ghClient }, b.logger), + WorkflowDispatchPrepare: workflowDispatchPrepare(b.cfg, func() *github.Client { return b.ghClient }, b.logger), WorkflowEmitChecks: workflowEmitChecks(b.cfg, func() *github.Client { return b.ghClient }), WorkflowClusterRunner: workflowClusterRunner(), WorkflowSpecLLM: newWorkflowSpecRunner(func() *config.Config { return b.cfg }), @@ -3821,6 +3826,9 @@ func (b *boot) bootDashboardAPIWith(deps bootDashboardAPIDeps) { rt = b.agentMgr } researchPool := wireMeetingsResearch(b.cfg, rt, b.logger) + if fn := meetingsResearchStatus(b.cfg, researchPool); fn != nil { + b.dashSrv.SetMeetingsResearchStatus(fn) + } if lr := wireMeetingsLive(b.cfg, mgr, store, b.knowledgeAPI, func() map[string]*beads.Store { return b.beadStores }, researchPool, b.logger, b.dashSrv.HiveStatusBrief); lr != nil { b.meetingsLive = lr b.cleanup.push(lr.Close) @@ -3829,12 +3837,15 @@ func (b *boot) bootDashboardAPIWith(deps bootDashboardAPIDeps) { // Drafts people ask Betty for, approved in the dashboard (#18406). b.wireMeetingsActions(store) // Owner API, live panel and /api/events frames (#18292). - b.dashSrv.SetMeetingsService(&meetingsDashboard{mgr: mgr, live: b.meetingsLive}) - // Scheduled meetings and the Calendar section (#18394). + md := &meetingsDashboard{mgr: mgr, live: b.meetingsLive} + // Scheduled meetings and the Calendar section (#18394). Wired + // first so a deleted meeting's Calendar entry goes with it. if svc, stop := wireMeetingsSchedule(b.cfg, mgr, b.logger); svc != nil { b.cleanup.push(stop) b.dashSrv.SetMeetingsScheduleService(svc) + md.sched = svc.sched } + b.dashSrv.SetMeetingsService(md) // Ended meetings become inbox items; transcripts are purged // after meetings.retention_days (#18293). b.wireMeetingsInbox(mgr, store) @@ -4243,7 +4254,7 @@ func (b *boot) bootWatchersWith(deps bootWatchersDeps) { if err := b.cfg.ExpandAgentReplicas(); err != nil { b.logger.Warn("failed to expand agent replicas after config reload", "error", err) } - addedAgents := b.agentMgr.ReconcileAgents(b.cfg.EnabledAgents()) + addedAgents := b.agentMgr.ReconcileAgents(withResearchAgents(b.cfg.EnabledAgents())) for _, added := range addedAgents { if ac, ok := b.cfg.Agents[added]; ok && !ac.OnDemand { deps.startAgent(b.ctx, b.agentMgr, added, b.logger) diff --git a/src/cmd/hive/main_eval_cycle.go b/src/cmd/hive/main_eval_cycle.go index c0207ecefb..10628a4864 100644 --- a/src/cmd/hive/main_eval_cycle.go +++ b/src/cmd/hive/main_eval_cycle.go @@ -1110,6 +1110,10 @@ func runEvalCycle( }() } kickActionable = dashSrv.IntegrationKickAdmission(kickActionable) + // A workflow-model state handed to an agent profile is that profile's + // work, not the generic lanes' (cncf/techdocs#395): withhold it here, + // before the cache every kick (manual ones included) is built from. + kickActionable = withholdWorkflowOwnedForKicks(cfg, ghClient, kickActionable, logger) sched.SetLastActionable(kickActionable) // Shadow advisory engine (hivecommons/hive#18464 phase 1): default off, @@ -1167,6 +1171,20 @@ func runEvalCycle( messages = append(messages, scheduler.KickMessage{Agent: k.Agent, Message: k.Message, IssueRefs: []string{k.PRRef}}) reviewKickByMessage[k.Agent+"\x00"+k.Message] = k } + // Workflow-model hand-offs (cncf/techdocs#395): each owned item is + // claimed for the agent that fills its profile, then kicked once; the + // durable record is written on delivery. They go ahead of the + // governor kicks: a lane takes one kick per pass, and an agent kicked + // every cycle would otherwise defer its hand-off forever. + workflowKickByMessage := map[string]workflowDispatchKick{} + var workflowMessages []scheduler.KickMessage + for _, k := range planWorkflowDispatchKicks(ctx, cfg, ghClient, actionable, dashSrv.IssueClaims(), func(agent string) bool { + return gov.AgentEligibleForCELKick(agent) && !agentMgr.IsPaused(agent) + }, logger) { + workflowMessages = append(workflowMessages, k.Message) + workflowKickByMessage[workflowDispatchKickKey(k.Message)] = k + } + messages = append(workflowMessages, messages...) // A fix kick (a maintainer's changes_requested / `/hive decision`, or // the swarm's) goes ahead of the governor kick for the same agent, or // the governor kick owns the agent and the fix kick times out every @@ -1709,8 +1727,16 @@ func runEvalCycle( persistReviewDispatchStateInProgress(reviewPlan, deliveredReviewKicks, logger) } }, - onDeferred: fixQueue.Deferred, + onDeferred: func(msg scheduler.KickMessage) { + if k, ok := workflowKickByMessage[workflowDispatchKickKey(msg)]; ok { + releaseWorkflowDispatch(k) + } + fixQueue.Deferred(msg) + }, onFailed: func(msg scheduler.KickMessage, err error) { + if k, ok := workflowKickByMessage[workflowDispatchKickKey(msg)]; ok { + releaseWorkflowDispatch(k) + } gov.RecordKickFailure(msg.Agent, err, time.Now()) if k, ok := reviewKickByMessage[reviewKickKey(msg)]; ok { logUndeliveredReviewKick(logger, dashSrv, k, err) @@ -1726,6 +1752,10 @@ func runEvalCycle( } gov.RecordKickForRepoAt(msg.Agent, msg.Repo, deliveredAt) dashSrv.AuditLog("governor", "kick", "trigger=governor-eval", msg.Agent) + if k, ok := workflowKickByMessage[workflowDispatchKickKey(msg)]; ok { + recordWorkflowDispatchDelivered(cfg, k, deliveredAt, logger) + dashSrv.AuditLog("governor", "workflow-dispatch", "item="+k.Entry.Ref+" profile="+k.Entry.Profile+" state="+k.Entry.State, msg.Agent) + } // Record issue-scoped kicks into the lifecycle timeline. Cheap, // guarded, and nil-safe (Record no-ops on a nil dashboard/store). diff --git a/src/cmd/hive/meetings_companion.go b/src/cmd/hive/meetings_companion.go new file mode 100644 index 0000000000..952e1ceea1 --- /dev/null +++ b/src/cmd/hive/meetings_companion.go @@ -0,0 +1,89 @@ +package main + +import ( + "context" + "errors" + "fmt" + + "github.com/hivecommons/hive/pkg/dashboard" + "github.com/hivecommons/hive/pkg/meetings" +) + +// Companion side of meetingsDashboard (hivecommons/hive#18596): the desktop +// companion streams a person's own meeting client audio as a live meeting. + +// companionInfo is GET /api/companion/info. +type companionInfo struct { + ConsentVersion string `json:"consent_version"` + ConsentText string `json:"consent_text"` + SampleRate int `json:"sample_rate"` + Channels []string `json:"channels"` + MaxChunkBytes int `json:"max_chunk_bytes"` +} + +// companionDashboardError adds the companion errors to +// meetingsDashboardError. +func companionDashboardError(err error) error { + var gap *meetings.ChunkGapError + switch { + case err == nil: + return nil + case errors.As(err, &gap): + return &dashboard.CompanionGapError{Expected: gap.Expected} + case errors.Is(err, meetings.ErrCompanionNotYours): + return fmt.Errorf("%w: %v", dashboard.ErrMeetingForbidden, err) + case errors.Is(err, meetings.ErrCompanionUnavailable): + return fmt.Errorf("%w: %v", dashboard.ErrMeetingUnavailable, err) + case errors.Is(err, meetings.ErrNotCompanion): + return fmt.Errorf("%w: %v", dashboard.ErrMeetingInvalid, err) + default: + return meetingsDashboardError(err) + } +} + +func (d *meetingsDashboard) CompanionInfo() any { + return companionInfo{ + ConsentVersion: meetings.CompanionConsentVersion, + ConsentText: meetings.CompanionConsentText, + SampleRate: meetings.CompanionSampleRate, + Channels: []string{meetings.CompanionChannelCall, meetings.CompanionChannelMic}, + MaxChunkBytes: meetings.CompanionMaxChunkBytes, + } +} + +func (d *meetingsDashboard) StartCompanion(ctx context.Context, req dashboard.CompanionStartRequest, user string) (any, error) { + m, err := d.mgr.StartCompanion(ctx, meetings.CompanionStart{ + Title: req.Title, App: req.App, HostName: req.HostName, + ConsentVersion: req.ConsentVersion, Attested: req.ConsentAttested, User: user, + }) + if err != nil { + return nil, companionDashboardError(err) + } + return m, nil +} + +func (d *meetingsDashboard) CompanionAudio(ctx context.Context, id, user string, asOwner bool, channel string, seq int, pcm []byte) (any, error) { + res, err := d.mgr.CompanionAudio(ctx, id, user, asOwner, channel, seq, pcm) + if err != nil { + return nil, companionDashboardError(err) + } + return res, nil +} + +func (d *meetingsDashboard) CompanionReplies(ctx context.Context, id, user string, asOwner bool, after int64) (any, error) { + res, err := d.mgr.CompanionReplies(ctx, id, user, asOwner, after) + if err != nil { + return nil, companionDashboardError(err) + } + return res, nil +} + +func (d *meetingsDashboard) EndCompanion(ctx context.Context, id, user string, asOwner bool) (any, error) { + m, err := d.mgr.EndCompanion(ctx, id, user, asOwner) + if err != nil { + return nil, companionDashboardError(err) + } + return m, nil +} + +var _ dashboard.CompanionMeetingsService = (*meetingsDashboard)(nil) diff --git a/src/cmd/hive/meetings_companion_test.go b/src/cmd/hive/meetings_companion_test.go new file mode 100644 index 0000000000..3c2724001c --- /dev/null +++ b/src/cmd/hive/meetings_companion_test.go @@ -0,0 +1,113 @@ +package main + +import ( + "context" + "errors" + "testing" + + "github.com/hivecommons/hive/pkg/dashboard" + "github.com/hivecommons/hive/pkg/meetings" +) + +// companionStubQuorum is stubQuorum plus the companion routes; joins start +// live companion meetings. +type companionStubQuorum struct { + stubQuorum + gap bool +} + +func (q *companionStubQuorum) Join(context.Context, meetings.JoinArgs) (meetings.JoinResult, error) { + return meetings.JoinResult{MeetingID: "qc", Platform: meetings.PlatformCompanion, State: meetings.StateLive}, nil +} + +func (q *companionStubQuorum) CompanionAudio(_ context.Context, _, _ string, seq int, _ []byte) (meetings.CompanionChunk, error) { + if q.gap { + return meetings.CompanionChunk{}, &meetings.ChunkGapError{Expected: 2} + } + return meetings.CompanionChunk{Seq: seq}, nil +} + +func (q *companionStubQuorum) CompanionOutbox(context.Context, string, int64) ([]meetings.CompanionLine, int64, error) { + return []meetings.CompanionLine{{Seq: 1, Text: "consent"}}, 1, nil +} + +func TestMeetingsDashboardCompanion(t *testing.T) { + store, err := meetings.NewStore(t.TempDir()) + if err != nil { + t.Fatal(err) + } + q := &companionStubQuorum{stubQuorum: stubQuorum{events: make(chan meetings.Event, 8)}} + mgr, err := meetings.NewManager(meetings.Options{Store: store, Quorum: q, ConsentText: "consent"}) + if err != nil { + t.Fatal(err) + } + defer mgr.Close() + d := &meetingsDashboard{mgr: mgr} + ctx := context.Background() + + info, ok := d.CompanionInfo().(companionInfo) + if !ok || info.ConsentVersion != meetings.CompanionConsentVersion || info.SampleRate != meetings.CompanionSampleRate || len(info.Channels) != 2 { + t.Fatalf("info = %+v", d.CompanionInfo()) + } + if _, err := d.StartCompanion(ctx, dashboard.CompanionStartRequest{ConsentVersion: "old", ConsentAttested: true}, "ana"); !errors.Is(err, dashboard.ErrMeetingInvalid) { + t.Fatalf("old consent = %v", err) + } + v, err := d.StartCompanion(ctx, dashboard.CompanionStartRequest{ + Title: "Standup", App: "zoom", ConsentVersion: meetings.CompanionConsentVersion, ConsentAttested: true, + }, "ana") + if err != nil { + t.Fatal(err) + } + mt, ok := v.(meetings.Meeting) + if !ok || mt.Platform != meetings.PlatformCompanion { + t.Fatalf("start = %+v", v) + } + if _, err := d.CompanionAudio(ctx, mt.ID, "ana", false, "mic", 0, []byte{0, 0}); err != nil { + t.Fatal(err) + } + if _, err := d.CompanionAudio(ctx, mt.ID, "bob", false, "mic", 0, nil); !errors.Is(err, dashboard.ErrMeetingForbidden) { + t.Fatalf("other user = %v", err) + } + q.gap = true + var gap *dashboard.CompanionGapError + if _, err := d.CompanionAudio(ctx, mt.ID, "ana", false, "call", 5, nil); !errors.As(err, &gap) || gap.Expected != 2 { + t.Fatalf("gap = %v", err) + } + if r, err := d.CompanionReplies(ctx, mt.ID, "ana", false, 0); err != nil || r.(meetings.CompanionReplies).NextSeq != 1 { + t.Fatalf("replies = %+v, %v", r, err) + } + if _, err := d.CompanionReplies(ctx, "mtg-missing", "ana", false, 0); !errors.Is(err, dashboard.ErrMeetingNotFound) { + t.Fatalf("missing = %v", err) + } + if v, err := d.EndCompanion(ctx, mt.ID, "ana", false); err != nil || v.(meetings.Meeting).State != meetings.StateEnded { + t.Fatalf("end = %+v, %v", v, err) + } + if _, err := d.EndCompanion(ctx, "mtg-missing", "ana", false); !errors.Is(err, dashboard.ErrMeetingNotFound) { + t.Fatalf("end missing = %v", err) + } + + plain := &meetingsDashboard{mgr: companionTestManager(t, &stubQuorum{events: make(chan meetings.Event, 1)})} + if _, err := plain.StartCompanion(ctx, dashboard.CompanionStartRequest{ConsentVersion: meetings.CompanionConsentVersion, ConsentAttested: true}, "ana"); !errors.Is(err, dashboard.ErrMeetingUnavailable) { + t.Fatalf("no companion quorum = %v", err) + } + if err := companionDashboardError(meetings.ErrNotCompanion); !errors.Is(err, dashboard.ErrMeetingInvalid) { + t.Fatalf("not companion = %v", err) + } + if companionDashboardError(nil) != nil { + t.Fatal("nil error") + } +} + +func companionTestManager(t *testing.T, q meetings.Quorum) *meetings.Manager { + t.Helper() + store, err := meetings.NewStore(t.TempDir()) + if err != nil { + t.Fatal(err) + } + mgr, err := meetings.NewManager(meetings.Options{Store: store, Quorum: q, ConsentText: "consent"}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(mgr.Close) + return mgr +} diff --git a/src/cmd/hive/meetings_dashboard.go b/src/cmd/hive/meetings_dashboard.go index 8bc8aeb0d6..299ffb0cf2 100644 --- a/src/cmd/hive/meetings_dashboard.go +++ b/src/cmd/hive/meetings_dashboard.go @@ -4,10 +4,12 @@ import ( "context" "errors" "fmt" + "time" "github.com/hivecommons/hive/pkg/dashboard" "github.com/hivecommons/hive/pkg/meetings" "github.com/hivecommons/hive/pkg/meetings/live" + "github.com/hivecommons/hive/pkg/meetings/schedule" ) // meetingsDashboard implements dashboard.MeetingsService over the meeting @@ -16,6 +18,8 @@ import ( type meetingsDashboard struct { mgr *meetings.Manager live *live.Runner + // sched, when the Calendar runs, forgets a deleted meeting's entry. + sched *schedule.Scheduler } // meetingsDashboardError maps meetings errors onto the dashboard's @@ -28,7 +32,8 @@ func meetingsDashboardError(err error) error { return fmt.Errorf("%w: %v", dashboard.ErrMeetingNotFound, err) case errors.Is(err, meetings.ErrInvalidRequest), meetings.IsCode(err, meetings.CodeInvalidArgument): return fmt.Errorf("%w: %v", dashboard.ErrMeetingInvalid, err) - case errors.Is(err, live.ErrNotPending), errors.Is(err, live.ErrSessionEnded), errors.Is(err, meetings.ErrNotLive), errors.Is(err, meetings.ErrNotRequested): + case errors.Is(err, live.ErrNotPending), errors.Is(err, live.ErrSessionEnded), errors.Is(err, meetings.ErrNotLive), errors.Is(err, meetings.ErrNotRequested), + errors.Is(err, meetings.ErrStillRunning): return fmt.Errorf("%w: %v", dashboard.ErrMeetingConflict, err) default: return err @@ -101,6 +106,33 @@ func (d *meetingsDashboard) DeclineRequest(ctx context.Context, id string) (any, return m, nil } +// meetingDeleted is the dashboard's view of a deleted meeting; with the +// Calendar running, the entry that existed only for it goes too. +func meetingDeleted(res meetings.DeleteResult, sched *schedule.Scheduler) dashboard.MeetingDeleted { + out := dashboard.MeetingDeleted{ID: res.Meeting.ID, State: string(res.Meeting.State), UpstreamError: res.UpstreamError} + if sched != nil { + out.CalendarRemoved = sched.ForgetMeeting(res.Meeting.ID) + } + return out +} + +func (d *meetingsDashboard) Delete(ctx context.Context, id string) (dashboard.MeetingDeleted, error) { + res, err := d.mgr.Delete(ctx, id) + if err != nil { + return dashboard.MeetingDeleted{}, meetingsDashboardError(err) + } + return meetingDeleted(res, d.sched), nil +} + +func (d *meetingsDashboard) ClearFinished(ctx context.Context, before time.Time) ([]dashboard.MeetingDeleted, error) { + list, err := d.mgr.ClearFinished(ctx, before) + out := make([]dashboard.MeetingDeleted, 0, len(list)) + for _, res := range list { + out = append(out, meetingDeleted(res, d.sched)) + } + return out, meetingsDashboardError(err) +} + // meetingLive is the panel snapshot. type meetingLive struct { Meeting meetings.Meeting `json:"meeting"` diff --git a/src/cmd/hive/meetings_delete_test.go b/src/cmd/hive/meetings_delete_test.go new file mode 100644 index 0000000000..76d7ef7edb --- /dev/null +++ b/src/cmd/hive/meetings_delete_test.go @@ -0,0 +1,102 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "testing" + "time" + + "github.com/hivecommons/hive/pkg/config" + "github.com/hivecommons/hive/pkg/dashboard" + "github.com/hivecommons/hive/pkg/meetings" + "github.com/hivecommons/hive/pkg/meetings/schedule" +) + +// TestMeetingsDeleteAdapters: deleting a meeting drops the Calendar entry +// that existed only for it; deleting a past Calendar entry deletes its +// meeting; live meetings and entries still to come answer 409. +func TestMeetingsDeleteAdapters(t *testing.T) { + dir := t.TempDir() + store, err := meetings.NewStore(dir) + if err != nil { + t.Fatal(err) + } + now := time.Now().UTC().Truncate(time.Second) + past := now.Add(-2 * time.Hour) + for _, m := range []meetings.Meeting{ + {ID: "mtg-ended", NativeID: "q-e", State: meetings.StateEnded, EndedAt: &past, CreatedAt: past}, + {ID: "mtg-cal", NativeID: "q-c", State: meetings.StateFailed, EndedAt: &past, CreatedAt: past}, + {ID: "mtg-live", NativeID: "q-l", State: meetings.StateLive, CreatedAt: past}, + } { + if err := store.Save(m); err != nil { + t.Fatal(err) + } + } + ev := func(uid string, st schedule.Status, start time.Time, mid string) schedule.Event { + e := schedule.Event{UID: uid, Start: start, Status: st, Platform: schedule.PlatformGoogleMeet, + JoinURL: "https://meet.google.com/abc", Source: schedule.SourceManual, MeetingID: mid, CreatedAt: past, UpdatedAt: now} + if mid != "" { + e.History = []schedule.Occurrence{{Start: start, MeetingID: mid, Status: st}} + } + e.ID = schedule.EventID(uid, nil) + return e + } + ended := ev("u-ended", schedule.StatusEnded, past, "mtg-ended") + failed := ev("u-cal", schedule.StatusFailed, past, "mtg-cal") + future := ev("u-future", schedule.StatusScheduled, now.Add(24*time.Hour), "") + data, err := json.Marshal(map[string]any{"version": 1, "events": []schedule.Event{ended, failed, future}}) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, schedule.FileName), data, 0o600); err != nil { + t.Fatal(err) + } + mgr, err := meetings.NewManager(meetings.Options{Store: store, Quorum: &stubQuorum{events: make(chan meetings.Event, 1)}}) + if err != nil { + t.Fatal(err) + } + defer mgr.Close() + cfg := &config.Config{} + cfg.Meetings.DataDir = dir + svc, stop := wireMeetingsSchedule(cfg, mgr, nil) + if svc == nil { + t.Fatal("schedule not wired") + } + defer stop() + md := &meetingsDashboard{mgr: mgr, sched: svc.sched} + ctx := context.Background() + + if _, err := md.Delete(ctx, "mtg-live"); !errors.Is(err, dashboard.ErrMeetingConflict) { + t.Fatalf("delete live = %v, want conflict", err) + } + if _, err := md.Delete(ctx, "mtg-none"); !errors.Is(err, dashboard.ErrMeetingNotFound) { + t.Fatalf("delete unknown = %v, want not found", err) + } + d, err := md.Delete(ctx, "mtg-ended") + if err != nil || d.ID != "mtg-ended" || d.State != "ended" || d.CalendarRemoved != 1 { + t.Fatalf("delete ended = %+v, %v", d, err) + } + if _, err := svc.sched.Get(ended.ID); !errors.Is(err, schedule.ErrNotFound) { + t.Fatalf("calendar entry kept: %v", err) + } + + if _, err := svc.Delete(ctx, future.ID); !errors.Is(err, dashboard.ErrMeetingConflict) { + t.Fatalf("delete future entry = %v, want conflict", err) + } + cd, err := svc.Delete(ctx, failed.ID) + if err != nil || len(cd.Meetings) != 1 || cd.Meetings[0].ID != "mtg-cal" { + t.Fatalf("delete past entry = %+v, %v", cd, err) + } + if _, err := mgr.Get("mtg-cal"); !errors.Is(err, meetings.ErrNotFound) { + t.Fatalf("entry's meeting kept: %v", err) + } + if _, err := svc.sched.Get(future.ID); err != nil { + t.Fatalf("future entry touched: %v", err) + } + if _, err := mgr.Get("mtg-live"); err != nil { + t.Fatalf("live meeting touched: %v", err) + } +} diff --git a/src/cmd/hive/meetings_research.go b/src/cmd/hive/meetings_research.go index 762b32a383..192534d3ed 100644 --- a/src/cmd/hive/meetings_research.go +++ b/src/cmd/hive/meetings_research.go @@ -3,9 +3,12 @@ package main import ( "errors" "log/slog" + "maps" "sort" + "sync/atomic" "github.com/hivecommons/hive/pkg/config" + "github.com/hivecommons/hive/pkg/dashboard" "github.com/hivecommons/hive/pkg/meetings/live" "github.com/hivecommons/hive/pkg/research" ) @@ -14,22 +17,53 @@ import ( // the research agents reuse (hivecommons/hive#18294). const meetingsResearchBaseAgent = "scanner" +// researchBaseName is the configured agent researchBase copies from, "" +// when there is none. +func researchBaseName(cfg *config.Config) string { + if cfg == nil || len(cfg.Agents) == 0 { + return "" + } + if _, ok := cfg.Agents[meetingsResearchBaseAgent]; ok { + return meetingsResearchBaseAgent + } + names := make([]string, 0, len(cfg.Agents)) + for n := range cfg.Agents { + names = append(names, n) + } + sort.Strings(names) + return names[0] +} + +// researchStatusSource is the slice of *research.Pool the settings page +// reads. +type researchStatusSource interface { + Status() research.Status +} + +// meetingsResearchStatus reports the pool for Settings, Meeting attendee, +// or nil when d is not a pool. +func meetingsResearchStatus(cfg *config.Config, d live.ResearchDispatcher) dashboard.MeetingsResearchStatusFunc { + src, ok := d.(researchStatusSource) + if !ok { + return nil + } + base := researchBase(cfg) + name := researchBaseName(cfg) + return func() dashboard.MeetingsResearchStatus { + st := src.Status() + return dashboard.MeetingsResearchStatus{Agents: st.Agents, Available: st.Available, BaseAgent: name, Backend: base.Backend, Model: base.Model} + } +} + // researchBase picks the backend and model for research-1..N: the scanner's // when it exists, else the first configured agent's by name. Only those two // fields are copied; research.AgentConfig forces the read-only rest. func researchBase(cfg *config.Config) config.AgentConfig { - if cfg == nil || len(cfg.Agents) == 0 { + name := researchBaseName(cfg) + if name == "" { return config.AgentConfig{} } - a, ok := cfg.Agents[meetingsResearchBaseAgent] - if !ok { - names := make([]string, 0, len(cfg.Agents)) - for n := range cfg.Agents { - names = append(names, n) - } - sort.Strings(names) - a = cfg.Agents[names[0]] - } + a := cfg.Agents[name] return config.AgentConfig{Backend: a.Backend, Model: a.Model} } @@ -53,13 +87,51 @@ func wireMeetingsResearch(cfg *config.Config, rt any, logger *slog.Logger) live. } return nil } - pool.Ensure(researchBase(cfg)) + base := researchBase(cfg) + pool.Ensure(base) + keepResearchAgents(pool.Agents(), research.AgentConfig(base)) if logger != nil { logger.Info("[meetings] research pool ready", "agents", pool.Agents()) } return pool } +// researchAgents is the research pool's agent set once it is wired. The +// agents are runtime-only (not in hive.yaml), so every ReconcileAgents call +// (config reload, reviewer autoscale) must carry them, or it removes them as +// "no longer configured" and every meeting research request then fails with +// "no research agent available" (hivecommons/hive#18590 finding 6). +var researchAgents atomic.Pointer[map[string]config.AgentConfig] + +// keepResearchAgents records the pool's agents for withResearchAgents. +func keepResearchAgents(names []string, cfg config.AgentConfig) { + set := make(map[string]config.AgentConfig, len(names)) + for _, n := range names { + set[n] = cfg + } + researchAgents.Store(&set) +} + +// withResearchAgents returns configs plus the research pool's agents (when +// the pool is wired and a name is not already configured), for +// ReconcileAgents. configs is not modified. +func withResearchAgents(configs map[string]config.AgentConfig) map[string]config.AgentConfig { + set := researchAgents.Load() + if set == nil || len(*set) == 0 { + return configs + } + out := maps.Clone(configs) + if out == nil { + out = make(map[string]config.AgentConfig, len(*set)) + } + for n, c := range *set { + if _, ok := out[n]; !ok { + out[n] = c + } + } + return out +} + // researchObserver is the slice of *research.Pool that reports progress. type researchObserver interface { Observe(fn func(research.Event)) (remove func()) diff --git a/src/cmd/hive/meetings_research_test.go b/src/cmd/hive/meetings_research_test.go index ec9dfaca30..e48a45a3ec 100644 --- a/src/cmd/hive/meetings_research_test.go +++ b/src/cmd/hive/meetings_research_test.go @@ -85,6 +85,7 @@ func TestWireMeetingsResearch(t *testing.T) { if d := wireMeetingsResearch(on, struct{}{}, discardLogger()); d != nil { t.Fatal("an unsupported runtime must leave research unwired") } + t.Cleanup(func() { researchAgents.Store(nil) }) rt := &fakeResearchRuntime{added: map[string]config.AgentConfig{}} d := wireMeetingsResearch(on, rt, discardLogger()) if d == nil || len(rt.added) != 3 { @@ -126,6 +127,7 @@ func (s *fakeSpekForResearch) Lines() <-chan []byte { return s.lines } func (s *fakeSpekForResearch) Close(any) {} func TestRouteResearch(t *testing.T) { + t.Cleanup(func() { researchAgents.Store(nil) }) if remove := routeResearch(nil, nil); remove == nil { t.Fatal("always returns a func") } @@ -174,3 +176,58 @@ func newResearchTestRunner(t *testing.T, d live.ResearchDispatcher) *live.Runner t.Cleanup(lr.Close) return lr } + +// TestWithResearchAgentsSurvivesReconcile pins hivecommons/hive#18590 +// finding 6: the research agents are runtime-only, so the reconcile set +// must carry them or a config reload removes them. +func TestWithResearchAgentsSurvivesReconcile(t *testing.T) { + t.Cleanup(func() { researchAgents.Store(nil) }) + researchAgents.Store(nil) + cfgs := map[string]config.AgentConfig{"scanner": {Backend: "claude"}} + if got := withResearchAgents(cfgs); len(got) != 1 { + t.Fatalf("unwired pool adds nothing: %v", got) + } + on := &config.Config{Meetings: config.MeetingsConfig{Enabled: true, ResearchPoolSize: 2}, AgentRuntimeConfig: config.AgentRuntimeConfig{Agents: cfgs}} + if wireMeetingsResearch(on, &fakeResearchRuntime{added: map[string]config.AgentConfig{}}, nil) == nil { + t.Fatal("pool not wired") + } + got := withResearchAgents(cfgs) + if len(got) != 3 || got["research-1"].Mode != research.Mode || got["research-2"].Role != research.Role || got["scanner"].Backend != "claude" { + t.Fatalf("reconcile set = %+v", got) + } + if len(cfgs) != 1 { + t.Fatal("the caller's map is not modified") + } + // A configured agent of the same name wins. + cfgs["research-1"] = config.AgentConfig{Backend: "copilot"} + if withResearchAgents(cfgs)["research-1"].Backend != "copilot" { + t.Fatal("hive.yaml wins over the pool's default") + } + if got := withResearchAgents(nil); len(got) != 2 { + t.Fatalf("nil configs = %v", got) + } +} + +func TestMeetingsResearchStatus(t *testing.T) { + if meetingsResearchStatus(nil, nil) != nil { + t.Fatal("no pool, no report") + } + t.Cleanup(func() { researchAgents.Store(nil) }) + cfg := &config.Config{Meetings: config.MeetingsConfig{Enabled: true, ResearchPoolSize: 2}, + AgentRuntimeConfig: config.AgentRuntimeConfig{Agents: map[string]config.AgentConfig{"scanner": {Backend: "claude", Model: "m1"}, "a": {Backend: "codex"}}}} + d := wireMeetingsResearch(cfg, &fakeResearchRuntime{added: map[string]config.AgentConfig{}}, nil) + fn := meetingsResearchStatus(cfg, d) + if fn == nil { + t.Fatal("a pool reports") + } + st := fn() + if len(st.Agents) != 2 || st.Available != 2 || st.BaseAgent != "scanner" || st.Backend != "claude" || st.Model != "m1" { + t.Fatalf("status = %+v", st) + } + if got := researchBaseName(&config.Config{AgentRuntimeConfig: config.AgentRuntimeConfig{Agents: map[string]config.AgentConfig{"b": {}, "a": {}}}}); got != "a" { + t.Fatalf("first by name, got %q", got) + } + if researchBaseName(nil) != "" { + t.Fatal("nil config") + } +} diff --git a/src/cmd/hive/meetings_schedule.go b/src/cmd/hive/meetings_schedule.go index 06d2143aef..a7da1ae4c8 100644 --- a/src/cmd/hive/meetings_schedule.go +++ b/src/cmd/hive/meetings_schedule.go @@ -31,6 +31,8 @@ const ( // meeting scheduler (hivecommons/hive#18394). type meetingsSchedule struct { sched *schedule.Scheduler + // mgr deletes the meetings a deleted Calendar entry created. + mgr *meetings.Manager // attendee is the name the dashboard's "in a meeting" indicator uses. attendee string now func() time.Time @@ -129,6 +131,53 @@ func (d *meetingsSchedule) Act(_ context.Context, eid, action, by string) (any, return ev.Redacted(), nil } +// Delete removes a finished Calendar entry and the meetings it created. +// An entry still to come or live answers 409 (skip or cancel it); so does +// one whose meeting hive is still in. Meetings already gone are skipped. +func (d *meetingsSchedule) Delete(ctx context.Context, eid string) (dashboard.MeetingScheduleDeleted, error) { + out := dashboard.MeetingScheduleDeleted{ID: eid, Meetings: []dashboard.MeetingDeleted{}} + e, err := d.sched.Get(eid) + if err != nil { + return out, meetingsScheduleError(err) + } + if !e.Status.Terminal() { + return out, fmt.Errorf("%w: this meeting is %s; skip or cancel it instead", dashboard.ErrMeetingConflict, e.Status) + } + ids := schedule.LinkedMeetings(e) + if d.mgr != nil { + for _, id := range ids { + m, err := d.mgr.Get(id) + if errors.Is(err, meetings.ErrNotFound) { + continue + } + if err != nil { + return out, err + } + if !meetings.Deletable(m) { + return out, fmt.Errorf("%w: hive is still in this meeting; leave the meeting first", dashboard.ErrMeetingConflict) + } + } + } + if _, err := d.sched.Delete(eid); err != nil { + return out, meetingsScheduleError(err) + } + if d.mgr == nil { + return out, nil + } + for _, id := range ids { + res, err := d.mgr.Delete(ctx, id) + if errors.Is(err, meetings.ErrNotFound) { + continue + } + if err != nil { + out.Meetings = append(out.Meetings, dashboard.MeetingDeleted{ID: id, UpstreamError: err.Error()}) + continue + } + out.Meetings = append(out.Meetings, meetingDeleted(res, nil)) + } + return out, nil +} + // wireMeetingsSchedule starts the meeting scheduler over the manager // (hivecommons/hive#18394): schedule.json in the meetings data directory, // joins HIVE_MEETINGS_SCHEDULE_LEAD_S before start, scans every @@ -186,5 +235,5 @@ func wireMeetingsSchedule(cfg *config.Config, mgr *meetings.Manager, logger *slo <-done sched.Close() } - return &meetingsSchedule{sched: sched, attendee: attendeeName(cfg.Meetings.BotNameOrDefault()), now: time.Now}, stop + return &meetingsSchedule{sched: sched, mgr: mgr, attendee: attendeeName(cfg.Meetings.BotNameOrDefault()), now: time.Now}, stop } diff --git a/src/cmd/hive/meetingswire.go b/src/cmd/hive/meetingswire.go index b14aaef683..571c42905b 100644 --- a/src/cmd/hive/meetingswire.go +++ b/src/cmd/hive/meetingswire.go @@ -160,6 +160,12 @@ func wireMeetingsManager(cfg *config.Config, sup quorumSupervisor, logger *slog. return mgr, store } +// meetingSpekProbeArgs checks once per installed binary that spektacular +// has the live `meeting` command (hivecommons/spektacular#126); a release +// without it answers "unknown command" and live mode stays off without a +// child restart loop (hivecommons/hive#18630). +var meetingSpekProbeArgs = []string{"meeting", "--help"} + // meetingSpekArgs is the contract's live-mode command line // (quorum.contract/v1 section 2). func meetingSpekArgs(m meetings.Meeting, contextDir string) []string { @@ -205,7 +211,9 @@ func wireMeetingsLive(cfg *config.Config, mgr *meetings.Manager, store *meetings Store: store, NewSpek: func(mt meetings.Meeting) live.Spek { env := meetingSpekEnv(os.Getenv, llm, hiveBin, meetingUsageFile(store, mt.ID), slotsDir, maxTokens) - return spektacular.StreamExec(binary, meetingSpekArgs(mt, filepath.Join(store.Dir(), mt.ID)), spektacular.StreamOptions{Logger: logger, Env: env}) + return spektacular.StreamExec(binary, meetingSpekArgs(mt, filepath.Join(store.Dir(), mt.ID)), spektacular.StreamOptions{ + Logger: logger, Env: env, Probe: meetingSpekProbeArgs, + }) }, SpekMode: llm.Label(), UsageFile: func(id string) string { @@ -221,6 +229,7 @@ func wireMeetingsLive(cfg *config.Config, mgr *meetings.Manager, store *meetings Notes: notes, Policy: live.Policy{ ChatPostsPerMinute: m.ChatPostsPerMinuteOrDefault(), + ChatAnswerBurst: m.ChatAnswerBurstOrDefault(), ResearchPerMeeting: m.ResearchPerMeetingOrDefault(), VoiceOnAddress: m.VoiceOnAddressEnabled(), MaxTokensPerMeeting: m.Budget.MaxTokensPerMeetingOrDefault(), @@ -296,6 +305,9 @@ func meetingPersonaFactory(cfg *config.Config, persona config.ResolvedMeetingsPe Timeout: betty.DefaultAddressedTimeout, UsageFile: meetingUsageFile(store, mt.ID), MaxTokens: maxTokens, SlotsDir: slotsDir, MaxInFlight: llm.MaxInFlight, + // Several requests are answered at once, each call reserving + // budget against the others in flight (#18590). + Gate: meetllm.NewGate(llm.PerMeetingInFlight), } switch llm.Backend { case config.MeetingsLLMModeOpenAI: @@ -320,9 +332,10 @@ func meetingPersonaFactory(cfg *config.Config, persona config.ResolvedMeetingsPe // meetingCLIModel is the persona's model on an agent backend // (hivecommons/hive#18517, #18549), in the backend's no-tools completion -// profile with the agents' sign-in: one warm CLI for the meeting, started -// now so the first question finds it ready, on a backend with a warm mode; -// one CLI per call otherwise. It reports false when the backend has no +// profile with the agents' sign-in: a pool of warm CLIs for the meeting +// (one per concurrent answer, #18590), the first started now so the first +// question finds it ready, on a backend with a warm mode; one CLI per call +// otherwise. It reports false when the backend has no // profile. func meetingCLIModel(llm config.ResolvedMeetingsLLM, mc meetllm.Config, meetingID string, logger *slog.Logger) (betty.MeetLLM, bool) { mc.Provider, mc.Backend, mc.Model = meetllm.ProviderAgentBackend, llm.AgentBackend, llm.Model @@ -338,21 +351,30 @@ func meetingCLIModel(llm config.ResolvedMeetingsLLM, mc meetllm.Config, meetingI } return betty.MeetLLM{Config: mc}, true } - sess, err := meetllm.NewCLISession(mc) + pool, err := meetllm.NewCLIPool(mc, meetingCLIPoolSize(llm, mc)) if err != nil { if logger != nil { logger.Error("[meetings] persona model not started", "meeting", meetingID, "backend", llm.AgentBackend, "error", err) } return betty.MeetLLM{}, false } - if err := sess.Warm(); err != nil && logger != nil { + if err := pool.Warm(); err != nil && logger != nil { // The first call retries the start. logger.Warn("[meetings] persona CLI not warmed", "meeting", meetingID, "backend", llm.AgentBackend, "error", err) } - mc.Session = sess + mc.Pool = pool return betty.MeetLLM{Config: mc}, true } +// meetingCLIPoolSize is how many warm CLIs one meeting may run: its +// concurrent answers (the gate's size), else the resolved per-meeting cap. +func meetingCLIPoolSize(llm config.ResolvedMeetingsLLM, mc meetllm.Config) int { + if mc.Gate != nil { + return mc.Gate.Size() + } + return llm.PerMeetingInFlight +} + // applyMeetingCLITimeouts widens the persona's per-call timeouts to the CLI // mode's: a warm turn is slower than an API call. func applyMeetingCLITimeouts(s *betty.Settings, cli config.ResolvedMeetingsCLI) { diff --git a/src/cmd/hive/meetingswire_cli_test.go b/src/cmd/hive/meetingswire_cli_test.go index e909163ac6..5e17328c10 100644 --- a/src/cmd/hive/meetingswire_cli_test.go +++ b/src/cmd/hive/meetingswire_cli_test.go @@ -97,8 +97,8 @@ func TestMeetingPersonaFactoryAgentBackend(t *testing.T) { e.Close() } missing := cliLLM(filepath.Join(dir, "absent"), dir) - if m, ok := meetingCLIModel(missing, meetllm.Config{MaxInFlight: 1}, "m3", logger); !ok || m.Config.Session == nil { - t.Fatal("a missing binary still yields a session that retries the start") + if m, ok := meetingCLIModel(missing, meetllm.Config{MaxInFlight: 1}, "m3", logger); !ok || m.Config.Pool == nil { + t.Fatal("a missing binary still yields a pool that retries the start") } else { _ = m.Close() } diff --git a/src/cmd/hive/reference_repos_identity_test.go b/src/cmd/hive/reference_repos_identity_test.go index 3e96699607..4447a0abd2 100644 --- a/src/cmd/hive/reference_repos_identity_test.go +++ b/src/cmd/hive/reference_repos_identity_test.go @@ -208,15 +208,22 @@ func TestReferenceRepoIdentity_PerRepoCredential(t *testing.T) { } } } - // The own-item follow-up reads under the owner token, so it watches only - // the owner-identity repos. + // The own-item follow-up watches every write-tier repo under the login + // its writes appear as: the bot's items on the bot repo, the owner's on + // the owner repo. res, err := buildReferenceRepoPolicies(context.Background(), refs, withIdentity(testReferenceRepoDeps(), config.RemoteWriteIdentityBot, app), testLogger()) if err != nil { t.Fatal(err) } - repos, login := remoteItemRepos(res.repos) - if len(repos) != 1 || repos[0] != "acme/site" || login != testOwnerLogin { - t.Fatalf("follow-up must watch only owner-identity repos, got %v as %q", repos, login) + targets := remoteItemTargets(res.repos) + want := map[string]string{"hivecommons/docs": testBotLogin, "acme/site": testOwnerLogin} + if len(targets) != len(want) { + t.Fatalf("follow-up targets = %+v", targets) + } + for _, tg := range targets { + if want[tg.Repo] != tg.Login { + t.Fatalf("follow-up for %s must list items by %q, got %q", tg.Repo, want[tg.Repo], tg.Login) + } } } diff --git a/src/cmd/hive/remote_items.go b/src/cmd/hive/remote_items.go index f501d0bdc9..bbafe18ba2 100644 --- a/src/cmd/hive/remote_items.go +++ b/src/cmd/hive/remote_items.go @@ -14,15 +14,17 @@ import ( // // Every reference-repo pass (boot, config reload, the derive poller, an // owner-identity change) ends by polling the write-tier reference repos for -// the issues and PRs the hive opened there as the owner and routing new -// feedback to the agents that opened them. The poller is bounded by design: -// it lists creator= only and reads comments on tracked items only. +// the issues and PRs the hive opened there (as the owner, or as the App bot +// on a repo that writes as the bot, #18516) and routing new feedback to the +// agents that opened them. The poller is bounded by design: it lists +// creator= only and reads comments on tracked items only. // Nothing here scans a remote backlog, kicks from a remote queue or runs a // stage on a remote item. -// newRemoteItemsPoller builds the poller over the owner token source. +// newRemoteItemsPoller builds the poller over the reference-repo credentials: +// each repo's proxy policy token when a proxy is wired, else the owner token. func (b *boot) newRemoteItemsPoller(rt *referenceRepoRuntime) *remoteitems.Poller { - if rt == nil || rt.deps.ownerToken == nil { + if rt == nil || (rt.deps.ownerToken == nil && b.githubProxy == nil) { return nil } apiURL := "" @@ -31,9 +33,24 @@ func (b *boot) newRemoteItemsPoller(rt *referenceRepoRuntime) *remoteitems.Polle apiURL = auth.APIURL() } } + fetch := &github.RemoteItemsFetcher{Token: rt.deps.ownerToken, APIURL: apiURL} + if b.githubProxy != nil { + // Each watched repo is read under its own reference-repo credential, + // the one the proxy injects on its writes: the scoped bot token on a + // repo that writes as the App bot, the owner's token (or the + // write_token_file) elsewhere (hivecommons/hive#18516). + proxy := b.githubProxy + fetch.TokenFor = func(repo string) (string, bool) { + pol, ok := proxy.ReferenceRepoPolicyFor(repo) + if !ok || pol.Token == nil { + return "", false + } + return pol.Token() + } + } return &remoteitems.Poller{ Store: remoteitems.NewStore(""), - Fetch: &github.RemoteItemsFetcher{Token: rt.deps.ownerToken, APIURL: apiURL}, + Fetch: fetch, Marker: github.HiveMarkerAgent, ResolveAgent: func(name string) string { return resolveMarkerAgent(b.cfg, name) }, Logger: b.logger, @@ -65,43 +82,37 @@ func resolveMarkerAgent(cfg *config.Config, name string) string { return "" } -// remoteItemRepos returns the reference repos the poller watches: write tier -// with a resolved credential login, which is the owner's login. Entries that -// write as the App bot (project.remote_write_identity) are left out: the -// poller reads under the owner token and lists creator=, so it would -// never find the bot's items there. -func remoteItemRepos(refs []config.ReferenceRepo) (repos []string, login string) { +// remoteItemTargets returns the reference repos the poller watches: write +// tier with a resolved credential login, each under the login its writes +// appear as. That is the App bot's on a repo that writes as the bot +// (project.remote_write_identity, hivecommons/hive#18516), the owner's or the +// write_token_file user's elsewhere. +func remoteItemTargets(refs []config.ReferenceRepo) []remoteitems.Target { + var out []remoteitems.Target for _, rr := range refs { if rr.ReadOnly() || rr.CredentialState != config.ReferenceCredentialWriteTier || rr.CredentialLogin == "" { continue } - if rr.CredentialIdentity == config.ReferenceIdentityBot { - continue - } - repos = append(repos, rr.Repo) - if login == "" { - login = rr.CredentialLogin - } + out = append(out, remoteitems.Target{Repo: rr.Repo, Login: rr.CredentialLogin}) } - return repos, login + return out } -// pollRemoteItems runs one follow-up pass over refs. Entries whose -// credential is a write_token_file are listed under that file's login too -// only when it matches the owner's; a mixed set follows the first login. +// pollRemoteItems runs one follow-up pass over refs, each repo listed under +// the login its writes appear as and read under its own credential. func (b *boot) pollRemoteItems(ctx context.Context, rt *referenceRepoRuntime, refs []config.ReferenceRepo) { if rt == nil || rt.remoteItems == nil || ctx == nil || ctx.Err() != nil { return } - repos, login := remoteItemRepos(refs) - if len(repos) == 0 || login == "" { + targets := remoteItemTargets(refs) + if len(targets) == 0 { return } var resumer remoteitems.Resumer if b.agentMgr != nil { resumer = b.agentMgr } - out := rt.remoteItems.Run(ctx, repos, login, resumer, time.Now()) + out := rt.remoteItems.RunTargets(ctx, targets, resumer, time.Now()) if out.Added+out.Pruned+out.Routed+out.Queued+out.Unrouted+out.ListErrs+out.FetchErrs > 0 { b.logger.Info("remote items: follow-up pass", "repos", out.Repos, "tracked", out.Tracked, "added", out.Added, "pruned", out.Pruned, "routed", out.Routed, "queued", out.Queued, "unrouted", out.Unrouted, "list_errors", out.ListErrs, "fetch_errors", out.FetchErrs) diff --git a/src/cmd/hive/remote_items_test.go b/src/cmd/hive/remote_items_test.go index 1b14dffe67..a345680c22 100644 --- a/src/cmd/hive/remote_items_test.go +++ b/src/cmd/hive/remote_items_test.go @@ -4,6 +4,7 @@ import ( "testing" "github.com/hivecommons/hive/pkg/config" + "github.com/hivecommons/hive/pkg/remoteitems" ) // hivecommons/hive#18199: which repos the remote-item pass watches and how a @@ -15,12 +16,17 @@ func TestRemoteItemRepos(t *testing.T) { {Repo: "hivecommons/hive", Write: []string{"issues", "prs"}, CredentialState: config.ReferenceCredentialWriteTier, CredentialLogin: "nate-double-u"}, {Repo: "hivecommons/site", Write: []string{"issues"}, DerivedFrom: "cncf/techdocs#395", CredentialState: config.ReferenceCredentialWriteTier, CredentialLogin: "nate-double-u"}, } - repos, login := remoteItemRepos(refs) - if len(repos) != 2 || repos[0] != "hivecommons/hive" || repos[1] != "hivecommons/site" || login != "nate-double-u" { - t.Fatalf("repos=%v login=%q", repos, login) + targets := remoteItemTargets(refs) + if len(targets) != 2 || targets[0] != (remoteitems.Target{Repo: "hivecommons/hive", Login: "nate-double-u"}) || targets[1] != (remoteitems.Target{Repo: "hivecommons/site", Login: "nate-double-u"}) { + t.Fatalf("targets=%+v", targets) } - if repos, login := remoteItemRepos(refs[:2]); len(repos) != 0 || login != "" { - t.Fatalf("read-only and downgraded entries must not be watched: %v %q", repos, login) + if targets := remoteItemTargets(refs[:2]); len(targets) != 0 { + t.Fatalf("read-only and downgraded entries must not be watched: %+v", targets) + } + // hivecommons/hive#18516: a bot-identity repo is watched under the bot. + bot := config.ReferenceRepo{Repo: "hivecommons/docs", Write: []string{"issues", "prs"}, DerivedFrom: "cncf/techdocs#395", CredentialState: config.ReferenceCredentialWriteTier, CredentialLogin: "hivecommons-hive[bot]", CredentialIdentity: config.ReferenceIdentityBot} + if targets := remoteItemTargets([]config.ReferenceRepo{bot}); len(targets) != 1 || targets[0].Login != "hivecommons-hive[bot]" { + t.Fatalf("bot repo must be watched under the bot login: %+v", targets) } } diff --git a/src/cmd/hive/reviewer_autoscale.go b/src/cmd/hive/reviewer_autoscale.go index a29e9c7d68..cd86d8f70c 100644 --- a/src/cmd/hive/reviewer_autoscale.go +++ b/src/cmd/hive/reviewer_autoscale.go @@ -249,7 +249,7 @@ func (r *reviewerAutoscaleRuntime) reconcile(ctx context.Context, cfg *config.Co return } enabled := cfg.EnabledAgents() - for _, added := range agentMgr.ReconcileAgents(enabled) { + for _, added := range agentMgr.ReconcileAgents(withResearchAgents(enabled)) { if ac, ok := cfg.Agents[added]; ok && !ac.OnDemand { if err := agentMgr.Start(ctx, added); err != nil { logger.Warn("reviewer autoscale: failed to start reviewer", "agent", added, "error", err) diff --git a/src/cmd/hive/workflow_dispatch.go b/src/cmd/hive/workflow_dispatch.go new file mode 100644 index 0000000000..768664162b --- /dev/null +++ b/src/cmd/hive/workflow_dispatch.go @@ -0,0 +1,397 @@ +package main + +import ( + "context" + "errors" + "fmt" + "log/slog" + "path/filepath" + "sync" + "time" + + "github.com/hivecommons/hive/pkg/claims" + "github.com/hivecommons/hive/pkg/config" + "github.com/hivecommons/hive/pkg/dashboard" + "github.com/hivecommons/hive/pkg/github" + "github.com/hivecommons/hive/pkg/scheduler" + "github.com/hivecommons/hive/pkg/workflowmodel" + wfstore "github.com/hivecommons/hive/pkg/workflowmodel/store" +) + +// Workflow-model agent dispatch, eval-cycle wiring (cncf/techdocs#395). +// +// Per cycle, after the generic kicks are planned from an actionable set +// that no longer offers model-owned items (scheduler.WithholdWorkflowOwned): +// +// 1. collect every enumerated item whose bound model hands its state to a +// profile, plus the open drafts of repositories whose model dispatches +// a pull-request state (the enumeration leaves drafts out); +// 2. plan the hand-offs (scheduler.PlanWorkflowDispatch): the agent that +// fills the profile, the triggering reply for a reply-started profile, +// and the idempotency key; +// 3. for an issue, take the agent claim on the issue-claims ledger FIRST; +// a claim someone else holds stops the hand-off; +// 4. append one kick per hand-off to the cycle's messages. On delivery the +// key is written to the durable dispatch log, so neither a later cycle +// nor a restart repeats it; a failed or deferred kick leaves the key +// free for the next cycle. + +// workflowDispatchLogFile is the durable log under the workflow store root. +const workflowDispatchLogFile = "workflow-dispatch.json" + +// workflowDispatchPendingTTL bounds how long a planned, not yet delivered +// hand-off blocks re-planning in this process. A kick a later gate +// withholds (provider budget, quota hold, upgrade drain) reports nothing, +// so the in-flight mark lapses on its own and the next cycle plans again. +const workflowDispatchPendingTTL = 30 * time.Minute + +// workflowDraftListTTL bounds how often one bound repository's open drafts +// are listed. Webhook-driven cycles can run minutes apart or less; a draft +// that appeared since is picked up within this window. +const workflowDraftListTTL = 5 * time.Minute + +// workflowDispatchClaimTTL is the agent claim taken before the drafter is +// kicked: long enough for a draft to reach its PR (the covering PR then +// holds the issue), short enough that an abandoned hand-off frees it. +const workflowDispatchClaimTTL = 8 * time.Hour + +type workflowDispatchRuntime struct { + once sync.Once + log *scheduler.WorkflowDispatchLog + mu sync.Mutex + pending map[string]time.Time + // replies caches LatestDispatchReply per item ref, keyed on the item's + // updated_at: a new comment moves it, so an unchanged item costs no call. + replies map[string]workflowReplyCacheEntry + // drafts caches ListWorkflowDraftPRs per repository for + // workflowDraftListTTL. + drafts map[string]workflowDraftCacheEntry +} + +type workflowDraftCacheEntry struct { + at time.Time + prs []github.PullRequest +} + +type workflowReplyCacheEntry struct { + updatedAt time.Time + reply github.DispatchReply + found bool +} + +var workflowDispatchRT = &workflowDispatchRuntime{pending: map[string]time.Time{}, replies: map[string]workflowReplyCacheEntry{}, drafts: map[string]workflowDraftCacheEntry{}} + +// draftsFor returns repo's open drafts, listing them at most once per +// workflowDraftListTTL. +func (rt *workflowDispatchRuntime) draftsFor(ctx context.Context, ghClient *github.Client, repo string, now time.Time) ([]github.PullRequest, error) { + rt.mu.Lock() + if rt.drafts == nil { + rt.drafts = map[string]workflowDraftCacheEntry{} + } + cached, ok := rt.drafts[repo] + rt.mu.Unlock() + if ok && now.Sub(cached.at) < workflowDraftListTTL { + return cached.prs, nil + } + prs, err := ghClient.ListWorkflowDraftPRs(ctx, repo) + if err != nil { + return nil, err + } + rt.mu.Lock() + rt.drafts[repo] = workflowDraftCacheEntry{at: now, prs: prs} + rt.mu.Unlock() + return prs, nil +} + +// dispatchLog opens the durable log once per process. +func (rt *workflowDispatchRuntime) dispatchLog(cfg *config.Config, logger *slog.Logger) *scheduler.WorkflowDispatchLog { + rt.once.Do(func() { + path := "" + if cfg != nil { + path = filepath.Join(wfstore.Root(cfg.Data.WorkflowsDir), workflowDispatchLogFile) + } + l, err := scheduler.OpenWorkflowDispatchLog(path) + if err != nil && logger != nil { + logger.Warn("workflow dispatch log unreadable; starting empty", "path", path, "error", err) + } + rt.log = l + }) + return rt.log +} + +func (rt *workflowDispatchRuntime) inFlight(key string, now time.Time) bool { + rt.mu.Lock() + defer rt.mu.Unlock() + at, ok := rt.pending[key] + if ok && now.Sub(at) > workflowDispatchPendingTTL { + delete(rt.pending, key) + return false + } + return ok +} + +func (rt *workflowDispatchRuntime) markPending(key string, now time.Time) { + rt.mu.Lock() + defer rt.mu.Unlock() + rt.pending[key] = now +} + +func (rt *workflowDispatchRuntime) clearPending(key string) { + rt.mu.Lock() + defer rt.mu.Unlock() + delete(rt.pending, key) +} + +// workflowDispatchKick is one planned hand-off and the record its delivery +// writes. +type workflowDispatchKick struct { + Message scheduler.KickMessage + Entry scheduler.WorkflowDispatchEntry +} + +// workflowDispatchActive reports whether the cycle dispatches at all. +func workflowDispatchActive(cfg *config.Config) bool { + return cfg != nil && cfg.Workflow.Enabled && cfg.Workflow.Dispatch.DispatchEnabled() +} + +// withholdWorkflowOwnedForKicks removes model-owned items from the set the +// generic kicks are built from. Model ownership is the model's, so it holds +// whatever workflow.dispatch says; only workflow.enabled (no model derives +// anything) turns it off. +func withholdWorkflowOwnedForKicks(cfg *config.Config, ghClient *github.Client, actionable *github.ActionableResult, logger *slog.Logger) *github.ActionableResult { + if cfg == nil || !cfg.Workflow.Enabled || ghClient == nil { + return actionable + } + out, withheld := scheduler.WithholdWorkflowOwned(actionable, ghClient) + if withheld > 0 && logger != nil { + logger.Info("workflow dispatch: model-owned items withheld from generic kicks", "withheld", withheld) + } + return out +} + +// workflowDispatchItems collects the cycle's model-owned items. +func workflowDispatchItems(ctx context.Context, cfg *config.Config, ghClient *github.Client, actionable *github.ActionableResult, logger *slog.Logger) []github.WorkflowDispatch { + var items []github.WorkflowDispatch + if actionable != nil { + for _, issue := range actionable.Issues.Items { + if d, ok := ghClient.WorkflowDispatchForIssue(issue); ok { + items = append(items, d) + } + } + for _, pr := range actionable.PRs.Items { + if d, ok := ghClient.WorkflowDispatchForPR(pr); ok { + items = append(items, d) + } + } + } + for _, b := range cfg.Project.WorkflowBindings { + p := ghClient.WorkflowModelFor(b.Repo) + if !dispatchesPRState(p) { + continue + } + drafts, err := workflowDispatchRT.draftsFor(ctx, ghClient, qualifyClaimRepo(cfg.Project.Org, b.Repo), time.Now()) + if err != nil { + if logger != nil { + logger.Warn("workflow dispatch: listing drafts failed", "repo", b.Repo, "error", err) + } + continue + } + for _, pr := range drafts { + if d, ok := ghClient.WorkflowDispatchForPR(pr); ok { + items = append(items, d) + } + } + } + return items +} + +func dispatchesPRState(p *workflowmodel.Profile) bool { + if p == nil { + return false + } + for _, t := range p.Dispatch { + if t.AppliesTo(workflowmodel.KindPR) { + return true + } + } + return false +} + +// workflowDispatchReplyFor is the cached reply lookup the planner uses. +func workflowDispatchReplyFor(ctx context.Context, ghClient *github.Client) func(d github.WorkflowDispatch) (github.DispatchReply, bool, error) { + rt := workflowDispatchRT + return func(d github.WorkflowDispatch) (github.DispatchReply, bool, error) { + ref := d.Ref() + "|" + d.Target.Reply + at := d.UpdatedAt + rt.mu.Lock() + cached, ok := rt.replies[ref] + rt.mu.Unlock() + if ok && !at.IsZero() && cached.updatedAt.Equal(at) { + return cached.reply, cached.found, nil + } + r, found, err := ghClient.LatestDispatchReply(ctx, d.Repo, d.Number, d.Target.Reply) + if err != nil { + return github.DispatchReply{}, false, err + } + rt.mu.Lock() + rt.replies[ref] = workflowReplyCacheEntry{updatedAt: at, reply: r, found: found} + rt.mu.Unlock() + return r, found, nil + } +} + +// resolveWorkflowDispatchAgent resolves the hive agent that fills the +// item's profile (config.ResolveWorkflowDispatch), telling the resolver the +// item's repository and whether the profile opens a pull request or reviews +// one, which the last-resort pick depends on (hivecommons/hive#18690). +func resolveWorkflowDispatchAgent(cfg *config.Config, d github.WorkflowDispatch, manual bool) config.WorkflowDispatchResolution { + return cfg.ResolveWorkflowDispatch(d.Target.Profile, config.WorkflowDispatchNeed{Repo: d.Repo, Reviews: !d.Target.ProducesChange()}, manual) +} + +// planWorkflowDispatchKicks plans the cycle's hand-offs and claims each +// issue for its agent before returning the kick. +func planWorkflowDispatchKicks(ctx context.Context, cfg *config.Config, ghClient *github.Client, actionable *github.ActionableResult, ledger *claims.Ledger, eligible func(agent string) bool, logger *slog.Logger) []workflowDispatchKick { + if !workflowDispatchActive(cfg) || ghClient == nil { + return nil + } + rt := workflowDispatchRT + dlog := rt.dispatchLog(cfg, logger) + now := time.Now() + items := workflowDispatchItems(ctx, cfg, ghClient, actionable, logger) + if len(items) == 0 { + return nil + } + cands, skips := scheduler.PlanWorkflowDispatch(items, scheduler.WorkflowDispatchPlanDeps{ + Agent: func(d github.WorkflowDispatch) (string, bool) { + r := resolveWorkflowDispatchAgent(cfg, d, false) + return r.Agent, r.OK() + }, + Eligible: eligible, + Reply: workflowDispatchReplyFor(ctx, ghClient), + Done: func(key string) bool { return dlog.Done(key) || rt.inFlight(key, now) }, + }) + for _, s := range skips { + if logger != nil && s.Reason != scheduler.WorkflowSkipDone && s.Reason != scheduler.WorkflowSkipNoReply { + logger.Info("workflow dispatch: item not handed off", "item", s.Ref, "profile", s.Profile, "reason", s.Reason, + "fix", "pin an agent on the repository's workflow row in Settings → Repos (workflow.dispatch.profiles."+s.Profile+".agent)") + } + } + var out []workflowDispatchKick + for _, c := range cands { + if !claimForWorkflowDispatch(cfg, ledger, c, logger) { + continue + } + set := cfg.Workflow.Dispatch.Profile(c.Dispatch.Target.Profile) + msg := scheduler.BuildWorkflowDispatchPrompt(c, set.PromptTemplate) + var refs []string + if c.Dispatch.Kind == workflowmodel.KindIssue { + refs = []string{c.Dispatch.Ref()} + } + rt.markPending(c.Key, now) + out = append(out, workflowDispatchKick{ + Message: scheduler.KickMessage{Agent: c.Agent, Message: msg, IssueRefs: refs}, + Entry: scheduler.WorkflowDispatchEntry{ + Key: c.Key, Ref: c.Dispatch.Ref(), Profile: c.Dispatch.Target.Profile, + State: c.Dispatch.State, Agent: c.Agent, By: "governor", + }, + }) + if logger != nil { + logger.Info("workflow dispatch: handing item to agent", "item", c.Dispatch.Ref(), "model", c.Dispatch.Model, + "state", c.Dispatch.State, "profile", c.Dispatch.Target.Profile, "agent", c.Agent) + } + } + return out +} + +// claimForWorkflowDispatch takes the agent claim on an issue before the +// kick. A pull request is not claimed (an issue claim on a PR stages it as +// an issue); its idempotency is the dispatch key alone. With no ledger the +// hand-off proceeds: the item is already withheld from every generic lane. +func claimForWorkflowDispatch(cfg *config.Config, ledger *claims.Ledger, c scheduler.WorkflowDispatchCandidate, logger *slog.Logger) bool { + if c.Dispatch.Kind != workflowmodel.KindIssue || ledger == nil { + return true + } + res, err := ledger.Claim(claims.Request{ + Repo: qualifyClaimRepo(cfg.Project.Org, c.Dispatch.Repo), + Issue: c.Dispatch.Number, + Holder: c.Agent, + HolderID: c.Agent, + Kind: claims.KindAgent, + TTL: workflowDispatchClaimTTL, + }) + if err != nil { + if logger != nil { + logger.Warn("workflow dispatch: claim refused; not handing off", "item", c.Dispatch.Ref(), "agent", c.Agent, "error", err) + } + return false + } + if !res.Outcome.Changed() { + if logger != nil { + logger.Info("workflow dispatch: item held by another claimant; not handing off", "item", c.Dispatch.Ref(), "agent", c.Agent, "outcome", string(res.Outcome)) + } + return false + } + return true +} + +// recordWorkflowDispatchDelivered writes the durable record of a delivered +// hand-off. +func recordWorkflowDispatchDelivered(cfg *config.Config, k workflowDispatchKick, at time.Time, logger *slog.Logger) { + rt := workflowDispatchRT + e := k.Entry + e.At = at + if err := rt.dispatchLog(cfg, logger).Record(e); err != nil && logger != nil { + logger.Warn("workflow dispatch: recording hand-off failed", "item", e.Ref, "error", err) + } + rt.clearPending(e.Key) +} + +// releaseWorkflowDispatch frees a hand-off whose kick did not land. +func releaseWorkflowDispatch(k workflowDispatchKick) { workflowDispatchRT.clearPending(k.Entry.Key) } + +// workflowDispatchKickKey identifies a workflow kick among the cycle's +// messages. +func workflowDispatchKickKey(msg scheduler.KickMessage) string { + return msg.Agent + "\x00" + msg.Message +} + +// workflowDispatchPrepare is the dashboard's WorkflowDispatchPrepare: the +// owner's "Send to " through the same steps as the automatic +// pass (fresh item read, model state, agent, claim first, prompt). The +// handler kicks the agent and calls Record once the kick landed, so the +// automatic pass does not hand the same state off again. +func workflowDispatchPrepare(cfg *config.Config, client func() *github.Client, logger *slog.Logger) func(ctx context.Context, repo string, number int, by string, ledger *claims.Ledger) (dashboard.WorkflowDispatchPlan, error) { + return func(ctx context.Context, repo string, number int, by string, ledger *claims.Ledger) (dashboard.WorkflowDispatchPlan, error) { + gh := client() + if cfg == nil || gh == nil { + return dashboard.WorkflowDispatchPlan{}, errors.New("GitHub client unavailable") + } + if !cfg.Workflow.Enabled { + return dashboard.WorkflowDispatchPlan{}, fmt.Errorf("%w (workflow.enabled is off)", dashboard.ErrWorkflowDispatchNotOwned) + } + d, ok, err := gh.WorkflowDispatchForRef(ctx, qualifyClaimRepo(cfg.Project.Org, repo), number) + if err != nil { + return dashboard.WorkflowDispatchPlan{}, err + } + if !ok { + return dashboard.WorkflowDispatchPlan{}, dashboard.ErrWorkflowDispatchNotOwned + } + res := resolveWorkflowDispatchAgent(cfg, d, true) + if !res.OK() { + return dashboard.WorkflowDispatchPlan{}, dashboard.WorkflowDispatchNoAgentError(res) + } + agent := res.Agent + c := scheduler.WorkflowDispatchCandidate{Dispatch: d, Agent: agent, Key: scheduler.WorkflowDispatchKey(d, nil), ManualBy: by} + if !claimForWorkflowDispatch(cfg, ledger, c, logger) { + return dashboard.WorkflowDispatchPlan{}, dashboard.ErrWorkflowDispatchHeld + } + entry := scheduler.WorkflowDispatchEntry{Key: c.Key, Ref: d.Ref(), Profile: d.Target.Profile, State: d.State, Agent: agent, By: by} + return dashboard.WorkflowDispatchPlan{ + Ref: d.Ref(), Model: d.Model, State: d.State, Profile: d.Target.Profile, Agent: agent, + Message: scheduler.BuildWorkflowDispatchPrompt(c, cfg.Workflow.Dispatch.Profile(d.Target.Profile).PromptTemplate), + Record: func(at time.Time) { + recordWorkflowDispatchDelivered(cfg, workflowDispatchKick{Entry: entry}, at, logger) + }, + }, nil + } +} diff --git a/src/cmd/hive/workflow_dispatch_test.go b/src/cmd/hive/workflow_dispatch_test.go new file mode 100644 index 0000000000..804351af99 --- /dev/null +++ b/src/cmd/hive/workflow_dispatch_test.go @@ -0,0 +1,144 @@ +package main + +import ( + "path/filepath" + "testing" + "time" + + "github.com/hivecommons/hive/pkg/claims" + "github.com/hivecommons/hive/pkg/config" + "github.com/hivecommons/hive/pkg/github" + "github.com/hivecommons/hive/pkg/scheduler" + "github.com/hivecommons/hive/pkg/workflowmodel" +) + +func workflowCandidate(kind workflowmodel.ItemKind, number int) scheduler.WorkflowDispatchCandidate { + return scheduler.WorkflowDispatchCandidate{ + Dispatch: github.WorkflowDispatch{Repo: "cncf/techdocs", Number: number, Kind: kind, State: "accepted", + Target: workflowmodel.DispatchTarget{Profile: "techdocs-drafter"}}, + Agent: "writer", + Key: "k", + } +} + +// The claim comes first: a free issue is claimed for the drafter's agent, +// an issue a person holds is not handed off, and a pull request is never +// claimed as an issue. +func TestClaimForWorkflowDispatch(t *testing.T) { + cfg := &config.Config{} + cfg.Project.Org = "cncf" + l, err := claims.New(filepath.Join(t.TempDir(), "claims.json"), claims.DefaultPolicy(), claims.Hooks{}) + if err != nil { + t.Fatal(err) + } + if !claimForWorkflowDispatch(cfg, l, workflowCandidate(workflowmodel.KindIssue, 395), nil) { + t.Fatal("free issue not handed off") + } + c, ok := l.Lookup("cncf/techdocs", 395) + if !ok || c.Holder != "writer" || c.Kind != claims.KindAgent { + t.Fatalf("claim %+v %v", c, ok) + } + // Same agent again (a retry after a deferred kick) renews. + if !claimForWorkflowDispatch(cfg, l, workflowCandidate(workflowmodel.KindIssue, 395), nil) { + t.Fatal("renewal refused") + } + + if _, err := l.Claim(claims.Request{Repo: "cncf/techdocs", Issue: 396, Holder: "nate", Kind: claims.KindHuman}); err != nil { + t.Fatal(err) + } + if claimForWorkflowDispatch(cfg, l, workflowCandidate(workflowmodel.KindIssue, 396), nil) { + t.Fatal("an issue a person holds was handed off") + } + + if !claimForWorkflowDispatch(cfg, l, workflowCandidate(workflowmodel.KindPR, 400), nil) { + t.Fatal("pull request hand-off refused") + } + if _, ok := l.Lookup("cncf/techdocs", 400); ok { + t.Fatal("a pull request was claimed as an issue") + } +} + +func TestWorkflowDispatchPendingLapses(t *testing.T) { + rt := &workflowDispatchRuntime{pending: map[string]time.Time{}, replies: map[string]workflowReplyCacheEntry{}} + now := time.Now() + rt.markPending("k", now) + if !rt.inFlight("k", now.Add(time.Minute)) { + t.Fatal("fresh hand-off not in flight") + } + if rt.inFlight("k", now.Add(workflowDispatchPendingTTL+time.Minute)) { + t.Fatal("in-flight mark never lapses") + } + rt.markPending("k", now) + rt.clearPending("k") + if rt.inFlight("k", now) { + t.Fatal("cleared hand-off still in flight") + } +} + +func TestWorkflowDispatchActive(t *testing.T) { + off := false + cfg := &config.Config{} + if workflowDispatchActive(cfg) { + t.Fatal("active with workflow.enabled off") + } + cfg.Workflow.Enabled = true + if !workflowDispatchActive(cfg) { + t.Fatal("dispatch must default on") + } + cfg.Workflow.Dispatch.Enabled = &off + if workflowDispatchActive(cfg) { + t.Fatal("workflow.dispatch.enabled: false ignored") + } + if workflowDispatchActive(nil) { + t.Fatal("nil config active") + } +} + +// On a standard fleet (no writer, docs or feature agent) the drafter's +// item goes to the scanner and the verifier's to the reviewer, so the +// TechDocs workflow starts out of the box; an agent scoped away from the +// item's repository is not picked (hivecommons/hive#18690). +func TestResolveWorkflowDispatchAgentStandardFleet(t *testing.T) { + cfg := &config.Config{} + cfg.Project.Org = "cncf" + cfg.Agents = map[string]config.AgentConfig{} + for _, name := range []string{"brainstorm", "ci-maintainer", "guide", "quality", "reviewer", "scanner", "supervisor"} { + cfg.Agents[name] = config.AgentConfig{Enabled: true} + } + drafter := github.WorkflowDispatch{Repo: "techdocs", Number: 395, Kind: workflowmodel.KindIssue, State: "accepted", + Target: workflowmodel.DispatchTarget{Profile: "techdocs-drafter", Produces: []string{"pull-request", "push"}}} + verifier := github.WorkflowDispatch{Repo: "techdocs", Number: 400, Kind: workflowmodel.KindPR, State: "drafting", + Target: workflowmodel.DispatchTarget{Profile: "techdocs-verifier", Reply: "/verify", Produces: []string{"comment"}}} + if got := resolveWorkflowDispatchAgent(cfg, drafter, false); got.Agent != "scanner" { + t.Fatalf("drafter = %+v; want scanner", got) + } + if got := resolveWorkflowDispatchAgent(cfg, verifier, false); got.Agent != "reviewer" { + t.Fatalf("verifier = %+v; want reviewer", got) + } + cands, skips := scheduler.PlanWorkflowDispatch([]github.WorkflowDispatch{drafter, verifier}, scheduler.WorkflowDispatchPlanDeps{ + Agent: func(d github.WorkflowDispatch) (string, bool) { + r := resolveWorkflowDispatchAgent(cfg, d, false) + return r.Agent, r.OK() + }, + }) + if len(cands) != 1 || cands[0].Agent != "scanner" || len(skips) != 1 || skips[0].Reason != scheduler.WorkflowSkipNoReply { + t.Fatalf("plan = %+v skips %+v", cands, skips) + } + + scoped := cfg.Agents["scanner"] + scoped.Repos = []string{"cncf/website"} + cfg.Agents["scanner"] = scoped + got := resolveWorkflowDispatchAgent(cfg, drafter, true) + if got.OK() || got.Problem == "" { + t.Fatalf("a scanner scoped to another repository was picked: %+v", got) + } + _, skips = scheduler.PlanWorkflowDispatch([]github.WorkflowDispatch{drafter}, scheduler.WorkflowDispatchPlanDeps{ + Agent: func(d github.WorkflowDispatch) (string, bool) { + r := resolveWorkflowDispatchAgent(cfg, d, false) + return r.Agent, r.OK() + }, + }) + if len(skips) != 1 || skips[0].Reason != scheduler.WorkflowSkipNoAgent || skips[0].Profile != "techdocs-drafter" { + t.Fatalf("skips = %+v", skips) + } +} diff --git a/src/crux/agent/copilot_config_bridge.go b/src/crux/agent/copilot_config_bridge.go index eab63c8ecc..e1a4d1405d 100644 --- a/src/crux/agent/copilot_config_bridge.go +++ b/src/crux/agent/copilot_config_bridge.go @@ -1,15 +1,19 @@ package agent import ( + "context" "errors" "fmt" "io/fs" + "log/slog" "os" "path/filepath" + "reflect" "strings" "sync" "time" + "github.com/hivecommons/hive/crux/agent/copilottoken" "github.com/hivecommons/hive/crux/sharedfile" ) @@ -37,13 +41,37 @@ import ( // none, the private copy may be the only login the fleet has, so it is left in // place (logged once). A later dashboard login or token sync fills the shared // file and the next launch relinks. +// +// Keeping the link alive while the agent runs (#18671): the CLI does its own +// atomic save a few seconds after it starts, so a launch-time relink alone is +// undone almost at once and a later dashboard login never reaches the running +// agent. The pane poller therefore checks the link on every tick +// (relinkCopilotConfigAfterCLISave, one Lstat while the link is intact) and +// repairs it within one poll interval of the save. Before the private copy is +// moved aside, the fields the CLI wrote to it are merged into the shared file +// (mergeCopilotConfigIntoShared), so relinking loses nothing the CLI saved. +// The credential fields are the one exception: the shared file's login always +// wins, because the private copy holds whatever token the CLI had in memory, +// which is older than a dashboard login made since. // Repair triggers, recorded in the log. const ( copilotBridgeTriggerLaunch = "launch" copilotBridgeTriggerLoginPrompt = "login-prompt" + copilotBridgeTriggerCLISave = "cli-save" ) +// copilotConfigCredentialKeys are the config.json fields that together make up +// the login. The merge never copies them from a private copy into the shared +// file: the shared login is authoritative (dashboard device flow, token sync), +// and mixing one file's token map with another's identity yields a config the +// CLI reads as logged out. +var copilotConfigCredentialKeys = map[string]bool{ + "copilotTokens": true, + "lastLoggedInUser": true, + "loggedInUsers": true, +} + // copilotConfigBackupInfix names a private config moved aside by the repair. const copilotConfigBackupInfix = ".bak-" @@ -142,34 +170,105 @@ func copilotConfigDiverged(link, target string) bool { // with a nil error when there is nothing to repair (a symlink, an absent path, // a directory). func repairCopilotConfigBridge(link, target string) (string, error) { + backup, _, err := repairCopilotConfigBridgeMerged(link, target) + return backup, err +} + +// repairCopilotConfigBridgeMerged is repairCopilotConfigBridge that also +// reports whether CLI-written fields were merged into the shared file first. +// When another hive writer holds the shared config it does nothing this time +// ("" and a nil error): the poller retries on its next tick. +func repairCopilotConfigBridgeMerged(link, target string) (string, bool, error) { copilotBridgeMu.Lock() defer copilotBridgeMu.Unlock() info, err := os.Lstat(link) if err != nil || !info.Mode().IsRegular() { - return "", nil + return "", false, nil } same, err := claudeBridgeLinkIsTarget(link, target) if err != nil { - return "", nil + return "", false, nil } if same { - return "", fmt.Errorf("%w: %s", errCopilotBridgeLinkIsTarget, link) + return "", false, fmt.Errorf("%w: %s", errCopilotBridgeLinkIsTarget, link) } if copilotCredentialFileHasTokens(link) && !copilotCredentialFileHasTokens(target) { - return "", errCopilotBridgeSharedEmpty + return "", false, errCopilotBridgeSharedEmpty + } + // Best effort: a private copy or shared file that cannot be read or parsed + // is relinked without a merge, exactly as before #18671; the backup still + // holds every field the CLI wrote. + merged, busy, _ := mergeCopilotConfigIntoShared(link, target) + if busy { + return "", false, nil } backup, err := moveAsideCopilotConfig(link, copilotBridgeNow()) if err != nil { - return "", fmt.Errorf("move private copilot config aside: %w", err) + return "", merged, fmt.Errorf("move private copilot config aside: %w", err) } pruneCopilotConfigBackups(link, backup) if err := os.Symlink(target, link); err != nil { - // EEXIST: the CLI saved again between the rename and now. The next - // launch repairs that one. - return backup, fmt.Errorf("restore copilot config symlink: %w", err) + // EEXIST: the CLI saved again between the rename and now. The poller + // repairs that one on its next tick. + return backup, merged, fmt.Errorf("restore copilot config symlink: %w", err) + } + return backup, merged, nil +} + +// mergeCopilotConfigIntoShared folds the fields the Copilot CLI wrote to the +// private copy at link into the shared config at target, so relinking does not +// throw them away. Credential fields (copilotConfigCredentialKeys) are never +// copied: the shared login wins. The shared file is rewritten only when the +// merge changes it, through copilottoken.WriteConfig (temp file + rename, the +// shared mode, owner preserved), and only while holding sharedCopilotConfigMu +// so it cannot race a dashboard login. busy reports that another hive writer +// holds that lock; nothing was read or written. Contents are never logged. +func mergeCopilotConfigIntoShared(link, target string) (merged, busy bool, err error) { + privateData, err := sharedfile.ReadNoFollow(link, copilotConfigMaxBytes) + if err != nil { + return false, false, err } - return backup, nil + private, err := copilottoken.ParseConfig(privateData) + if err != nil { + return false, false, err + } + if !sharedCopilotConfigMu.TryLock() { + return false, true, nil + } + defer sharedCopilotConfigMu.Unlock() + sharedData, err := sharedfile.ReadNoFollow(target, copilotConfigMaxBytes) + if err != nil { + return false, false, err + } + shared, err := copilottoken.ParseConfig(sharedData) + if err != nil { + return false, false, err + } + out := mergeCopilotConfigFields(shared, private) + if reflect.DeepEqual(out, shared) { + return false, false, nil + } + if err := copilottoken.WriteConfig(target, out, sharedConfigDesiredMode); err != nil { + return false, false, err + } + return true, false, nil +} + +// mergeCopilotConfigFields returns shared overlaid with every non-credential +// field of private. Neither input is modified. +func mergeCopilotConfigFields(shared, private map[string]interface{}) map[string]interface{} { + out := make(map[string]interface{}, len(shared)+len(private)) + for k, v := range shared { + out[k] = v + } + for k, v := range private { + if copilotConfigCredentialKeys[k] { + continue + } + out[k] = v + } + return out } // moveAsideCopilotConfig renames path to a free .bak-[-N] @@ -215,7 +314,7 @@ func pruneCopilotConfigBackups(link, keep string) { // rebridgeCopilotConfig runs the repair for one agent and logs the outcome. It // reports whether the link was restored. func (m *Manager) rebridgeCopilotConfig(agentName, link, target, trigger string) bool { - backup, err := repairCopilotConfigBridge(link, target) + backup, merged, err := repairCopilotConfigBridgeMerged(link, target) if err != nil { if prev, loaded := copilotBridgeWarned.Swap(link, err.Error()); loaded && prev == err.Error() { return false @@ -229,16 +328,46 @@ func (m *Manager) rebridgeCopilotConfig(agentName, link, target, trigger string) return false } // A completed repair is routine self-healing, not a fault: WARN made the - // fleet self-report it as an unclassified repeated error (#18497). - m.logger.Info("copilot config bridge: a private config.json had replaced the shared-login symlink; re-linked", + // fleet self-report it as an unclassified repeated error (#18497). The + // running-agent repair follows every CLI save, so it logs at DEBUG; the + // launch and login-prompt repairs stay at INFO. + level := slog.LevelInfo + if trigger == copilotBridgeTriggerCLISave { + level = slog.LevelDebug + } + m.logger.Log(context.Background(), level, "copilot config bridge: a private config.json had replaced the shared-login symlink; re-linked", "agent", agentName, "trigger", trigger, "backup", backup, + "merged_cli_fields", merged, "cause", "Copilot CLI saves its config with a temp file renamed over the link", "see", "hivecommons/hive#18443") return true } +// relinkCopilotConfigAfterCLISave is the pane poller's per-tick hook (#18671): +// when the Copilot CLI's own atomic save has replaced a running agent's +// config.json symlink with a private regular file, merge the CLI's fields into +// the shared file and relink, so a later dashboard login reaches the agent +// without a restart. An intact link costs one Lstat. The #18445 guard holds: a +// private copy holding the only login is left in place. +func (m *Manager) relinkCopilotConfigAfterCLISave(agent *AgentProcess) bool { + m.mu.RLock() + name, uid, backend := agent.Name, agent.UID, effectiveBackend(agent) + m.mu.RUnlock() + if backend != "copilot" { + return false + } + link, target, ok := copilotConfigBridgePaths(name, uid, backend) + if !ok { + return false + } + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + return false + } + return m.rebridgeCopilotConfig(name, link, target, copilotBridgeTriggerCLISave) +} + // repairCopilotConfigOnLoginPrompt is the poller's hook: a Copilot agent stuck // at a login prompt whose config.json diverged from the shared login is // relinked, so the credential probe then evaluates the shared login and the diff --git a/src/crux/agent/copilot_config_bridge_sync_test.go b/src/crux/agent/copilot_config_bridge_sync_test.go new file mode 100644 index 0000000000..98cfa28314 --- /dev/null +++ b/src/crux/agent/copilot_config_bridge_sync_test.go @@ -0,0 +1,215 @@ +package agent + +import ( + "os" + "strings" + "testing" + + "github.com/hivecommons/hive/crux/agent/copilottoken" + "github.com/hivecommons/hive/crux/config" +) + +// Running-agent relink after the Copilot CLI's own save (#18671). Every token +// value carries copilotBridgeTestSentinel so assertNoSentinelLogged proves no +// log line leaks token material. + +const ( + // copilotBridgeTestCLISaved is what the CLI's save leaves behind: the token + // it had in memory plus fields it wrote itself. + copilotBridgeTestCLISaved = `{"copilotTokens":{"https://github.com:fake-user":"gho_older_` + copilotBridgeTestSentinel + `"},"banner":"never","trustedFolders":["/work"]}` + // copilotBridgeTestNewToken is the token a later dashboard login writes. + copilotBridgeTestNewToken = "gho_dashboard_" + copilotBridgeTestSentinel +) + +func readCopilotBridgeConfig(t *testing.T, path string) map[string]interface{} { + t.Helper() + cfg, err := copilottoken.ReadConfig(path) + if err != nil { + t.Fatalf("read %s: %v", path, err) + } + return cfg +} + +func copilotBridgeTokenAt(t *testing.T, path string) string { + t.Helper() + tokens, ok := readCopilotBridgeConfig(t, path)["copilotTokens"].(map[string]interface{}) + if !ok { + t.Fatalf("%s has no copilotTokens map", path) + } + tok, _ := tokens["https://github.com:fake-user"].(string) + return tok +} + +// simulateDashboardLogin writes a new login to the shared file the way the +// dashboard device flow does: a whole-file atomic rewrite of the shared path. +func simulateDashboardLogin(t *testing.T, target, token string) { + t.Helper() + cfg := readCopilotBridgeConfig(t, target) + cfg["copilotTokens"] = map[string]interface{}{"https://github.com:fake-user": token} + if err := copilottoken.WriteConfig(target, cfg, sharedConfigDesiredMode); err != nil { + t.Fatal(err) + } +} + +func TestRelinkCopilotConfigAfterCLISave_RelinksAndMergesCLIFields(t *testing.T) { + m, buf, ap, link, target := copilotBridgeFixture(t) + simulateCopilotSave(t, link, copilotBridgeTestCLISaved) + + // One poll tick. + if !m.relinkCopilotConfigAfterCLISave(ap) { + t.Fatal("a private copy left by the CLI's save must be relinked on the next poll") + } + assertCopilotLinked(t, link, target) + + cfg := readCopilotBridgeConfig(t, target) + if cfg["banner"] != "never" { + t.Fatalf("CLI-written field lost by the relink: banner=%v", cfg["banner"]) + } + if folders, ok := cfg["trustedFolders"].([]interface{}); !ok || len(folders) != 1 || folders[0] != "/work" { + t.Fatalf("CLI-written field lost by the relink: trustedFolders=%v", cfg["trustedFolders"]) + } + // The shared login wins over the token the CLI had in memory. + if got := copilotBridgeTokenAt(t, target); got != "gho_"+copilotBridgeTestSentinel { + t.Fatal("the merge must keep the shared login, not the private copy's older token") + } + if b := copilotBackups(t, link); len(b) != 1 { + t.Fatalf("want one backup of the private copy, got %v", b) + } + + // The next tick, with the link intact, does nothing. + if m.relinkCopilotConfigAfterCLISave(ap) { + t.Fatal("an intact link must not be repaired again") + } + assertNoSentinelLogged(t, buf) +} + +func TestRelinkCopilotConfigAfterCLISave_DashboardLoginReachesRunningAgent(t *testing.T) { + m, buf, ap, link, target := copilotBridgeFixture(t) + + // The CLI's save seconds after launch, then the poller's relink. + simulateCopilotSave(t, link, copilotBridgeTestLoggedIn) + if !m.relinkCopilotConfigAfterCLISave(ap) { + t.Fatal("private copy must be relinked") + } + + // A dashboard re-login updates only the shared file. The running agent's + // effective config (the file at its own path) sees it with no restart. + simulateDashboardLogin(t, target, copilotBridgeTestNewToken) + if got := copilotBridgeTokenAt(t, link); got != copilotBridgeTestNewToken { + t.Fatal("running agent's config.json must see the dashboard login without a restart") + } + + // The CLI saves again with the token it still holds in memory. The next + // tick relinks without rolling the fleet back to that older token. + simulateCopilotSave(t, link, copilotBridgeTestCLISaved) + if !m.relinkCopilotConfigAfterCLISave(ap) { + t.Fatal("second CLI save must be relinked too") + } + assertCopilotLinked(t, link, target) + if got := copilotBridgeTokenAt(t, link); got != copilotBridgeTestNewToken { + t.Fatal("a later CLI save must not overwrite the dashboard login") + } + if got := copilotBridgeTokenAt(t, target); got != copilotBridgeTestNewToken { + t.Fatal("shared login must still be the dashboard login") + } + if b := copilotBackups(t, link); len(b) != 1 { + t.Fatalf("only the newest backup is kept, got %v", b) + } + assertNoSentinelLogged(t, buf) +} + +func TestRelinkCopilotConfigAfterCLISave_KeepsOnlyLoginWhenSharedEmpty(t *testing.T) { + m, buf, ap, link, target := copilotBridgeFixture(t) + simulateCopilotSave(t, link, copilotBridgeTestLoggedIn) + writeCopilotBridgeFile(t, target, copilotBridgeTestNoLogin) + + for i := 0; i < 3; i++ { + if m.relinkCopilotConfigAfterCLISave(ap) { + t.Fatal("a private copy holding the only login must not be relinked") + } + } + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Fatalf("private login must stay in place: info=%v err=%v", info, err) + } + if data, err := os.ReadFile(target); err != nil || string(data) != copilotBridgeTestNoLogin { + t.Fatalf("shared file must not be touched by a refused repair: err=%v", err) + } + if got := strings.Count(buf.String(), "repair not applied"); got != 1 { + t.Fatalf("refusal must be logged once across polls, got %d:\n%s", got, buf.String()) + } + assertNoSentinelLogged(t, buf) +} + +func TestRelinkCopilotConfigAfterCLISave_WaitsForSharedWriter(t *testing.T) { + m, _, ap, link, target := copilotBridgeFixture(t) + simulateCopilotSave(t, link, copilotBridgeTestCLISaved) + + sharedCopilotConfigMu.Lock() + relinked := m.relinkCopilotConfigAfterCLISave(ap) + sharedCopilotConfigMu.Unlock() + if relinked { + t.Fatal("must not relink while another writer holds the shared config") + } + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Fatalf("private copy must stay until the next tick: info=%v err=%v", info, err) + } + if b := copilotBackups(t, link); len(b) != 0 { + t.Fatalf("nothing may be moved aside while busy, got %v", b) + } + + if !m.relinkCopilotConfigAfterCLISave(ap) { + t.Fatal("next tick must relink once the writer is done") + } + assertCopilotLinked(t, link, target) + if readCopilotBridgeConfig(t, target)["banner"] != "never" { + t.Fatal("CLI-written field must be merged on the retry") + } +} + +func TestRelinkCopilotConfigAfterCLISave_IgnoresOtherAgents(t *testing.T) { + m, _, _, link, _ := copilotBridgeFixture(t) + simulateCopilotSave(t, link, copilotBridgeTestNoLogin) + other := &AgentProcess{Name: copilotBridgeTestAgentName, UID: copilotBridgeTestUID, Config: config.AgentConfig{Backend: "claude"}} + if m.relinkCopilotConfigAfterCLISave(other) { + t.Fatal("non-copilot backends are not relinked by the copilot hook") + } + noHome := &AgentProcess{Name: "planner", Config: config.AgentConfig{Backend: "copilot"}} + if m.relinkCopilotConfigAfterCLISave(noHome) { + t.Fatal("agent without a per-agent home has nothing to relink") + } + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Fatalf("file must stay in place: info=%v err=%v", info, err) + } +} + +func TestMergeCopilotConfigFields(t *testing.T) { + shared := map[string]interface{}{ + "copilotTokens": map[string]interface{}{"k": "shared"}, + "lastLoggedInUser": "https://github.com:shared", + "model": "shared-model", + "sharedOnly": true, + } + private := map[string]interface{}{ + "copilotTokens": map[string]interface{}{"k": "private"}, + "loggedInUsers": []interface{}{"https://github.com:private"}, + "model": "cli-model", + "privateOnly": "x", + } + out := mergeCopilotConfigFields(shared, private) + + if tok := out["copilotTokens"].(map[string]interface{})["k"]; tok != "shared" { + t.Fatalf("copilotTokens must come from shared, got %v", tok) + } + if out["lastLoggedInUser"] != "https://github.com:shared" { + t.Fatalf("identity must come from shared, got %v", out["lastLoggedInUser"]) + } + if _, ok := out["loggedInUsers"]; ok { + t.Fatal("a credential field absent from shared must not be imported from private") + } + if out["model"] != "cli-model" || out["privateOnly"] != "x" || out["sharedOnly"] != true { + t.Fatalf("non-credential fields: want private over shared, got %v", out) + } + if shared["model"] != "shared-model" || len(shared) != 4 { + t.Fatal("inputs must not be modified") + } +} diff --git a/src/crux/agent/copilot_token_store.go b/src/crux/agent/copilot_token_store.go index e5384909e7..7893bb811a 100644 --- a/src/crux/agent/copilot_token_store.go +++ b/src/crux/agent/copilot_token_store.go @@ -1,6 +1,8 @@ package agent import ( + "sync" + "github.com/hivecommons/hive/crux" "github.com/hivecommons/hive/crux/agent/copilottoken" ) @@ -28,15 +30,30 @@ func readCopilotConfig(path string) (map[string]interface{}, error) { return copilottoken.ReadConfig(path) } +// sharedCopilotConfigMu serializes the hive's own writes to the shared +// Copilot config.json: the token writers below (dashboard login, token sync, +// expired-token clear) and the config bridge's merge of CLI-written fields +// (mergeCopilotConfigIntoShared, #18671). Without it a merge that read the +// shared file just before a dashboard login could rename its stale copy over +// the new login. The bridge only ever TryLocks it, so a token writer blocked +// on a GitHub login lookup never stalls an agent's pane poller. +var sharedCopilotConfigMu sync.Mutex + func clearExpiredTokens() error { + sharedCopilotConfigMu.Lock() + defer sharedCopilotConfigMu.Unlock() return copilottoken.ClearExpiredTokens(sharedCopilotConfigPath, sharedConfigDesiredMode) } func restoreCopilotTokens(path, token string) error { + sharedCopilotConfigMu.Lock() + defer sharedCopilotConfigMu.Unlock() return copilottoken.RestoreTokens(path, token, sharedConfigDesiredMode, githubTokenLogin) } func replaceCopilotTokens(path, token string) error { + sharedCopilotConfigMu.Lock() + defer sharedCopilotConfigMu.Unlock() return copilottoken.ReplaceTokens(path, token, sharedConfigDesiredMode, githubTokenLogin) } diff --git a/src/crux/agent/copilottoken/copilottoken.go b/src/crux/agent/copilottoken/copilottoken.go index e9d97f78a6..8a696b922c 100644 --- a/src/crux/agent/copilottoken/copilottoken.go +++ b/src/crux/agent/copilottoken/copilottoken.go @@ -119,6 +119,13 @@ func ReadConfig(path string) (map[string]interface{}, error) { if err != nil { return nil, err } + return ParseConfig(data) +} + +// ParseConfig is ReadConfig over bytes the caller already read (for example +// with sharedfile.ReadNoFollow): it strips the CLI's // comment lines and +// unmarshals the remainder, returning an empty map for a JSON null. +func ParseConfig(data []byte) (map[string]interface{}, error) { var cleaned []byte for _, line := range strings.Split(string(data), "\n") { if strings.HasPrefix(strings.TrimSpace(line), "//") { diff --git a/src/crux/agent/kick_restart_recovery_test.go b/src/crux/agent/kick_restart_recovery_test.go index c5542f7856..9460ce5405 100644 --- a/src/crux/agent/kick_restart_recovery_test.go +++ b/src/crux/agent/kick_restart_recovery_test.go @@ -99,8 +99,13 @@ func quiesceThenMarkReady(t *testing.T, m *Manager, name string) (wait func()) { // One poll interval (3s) with margin, so the poller has observed // ctx.Done before anything below makes the pane interesting. time.Sleep(4 * time.Second) - _ = testTmuxCommand("send-keys", "-t", session, "-l", ": goose is ready").Run() - _ = testTmuxCommand("send-keys", "-t", session, "Enter").Run() + if out, err := testTmuxCommand("send-keys", "-t", session, "-l", ": goose is ready").CombinedOutput(); err != nil { + t.Errorf("inject readiness marker into %s: %v: %s", session, err, out) + return + } + if out, err := testTmuxCommand("send-keys", "-t", session, "Enter").CombinedOutput(); err != nil { + t.Errorf("submit readiness marker into %s: %v: %s", session, err, out) + } return } }() @@ -120,11 +125,12 @@ func TestSendKick_RestartsCrashedCLIThenDelivers(t *testing.T) { t.Setenv("HIVE_WORK_DIR", t.TempDir()) forceFastPaneShell(t) + const name = "kick-crash-recovery" m := NewManager(map[string]config.AgentConfig{ - "worker": makeAgentConfig("claude", "sonnet"), + name: makeAgentConfig("claude", "sonnet"), }, discardLogger(), ProjectContext{}) m.mu.RLock() - agent := m.agents["worker"] + agent := m.agents[name] m.mu.RUnlock() // A real session whose pane is a bare shell: session exists, CLI does not. @@ -133,11 +139,11 @@ func TestSendKick_RestartsCrashedCLIThenDelivers(t *testing.T) { agent.State = StateRunning m.mu.Unlock() - wait := quiesceThenMarkReady(t, m, "worker") + wait := quiesceThenMarkReady(t, m, name) + defer cleanupAgent(t, m, name) defer wait() - defer cleanupAgent(t, m, "worker") - if err := m.SendKick("worker", "recover and continue the sweep"); err != nil { + if err := m.SendKick(name, "recover and continue the sweep"); err != nil { t.Fatalf("SendKick on crashed CLI: %v", err) } @@ -168,20 +174,23 @@ func TestRestartThenSendKick_CleanSlateThenDelivers(t *testing.T) { t.Setenv("HIVE_WORK_DIR", t.TempDir()) forceFastPaneShell(t) + // Other shuffled tests use "worker" and can leave delayed tmux operations + // against hive-worker. Give this test a session they cannot tear down. + const name = "kick-clean-slate" m := NewManager(map[string]config.AgentConfig{ - "worker": makeAgentConfig("claude", "sonnet"), + name: makeAgentConfig("claude", "sonnet"), }, discardLogger(), ProjectContext{}) - wait := quiesceThenMarkReady(t, m, "worker") + wait := quiesceThenMarkReady(t, m, name) + defer cleanupAgent(t, m, name) defer wait() - defer cleanupAgent(t, m, "worker") - if err := m.RestartThenSendKick(context.Background(), "worker", "fresh context, take the next issue"); err != nil { - t.Fatalf("RestartThenSendKick: %v", err) + if err := m.RestartThenSendKick(context.Background(), name, "fresh context, take the next issue"); err != nil { + t.Fatalf("RestartThenSendKick: %v; pane: %q", err, m.captureVisiblePaneForAgent(m.agents[name])) } m.mu.RLock() - agent := m.agents["worker"] + agent := m.agents[name] restarts := agent.RestartCount lastMsg := agent.LastKickMessage m.mu.RUnlock() diff --git a/src/crux/agent/manager_poll.go b/src/crux/agent/manager_poll.go index 2690144f5c..c60f77b15d 100644 --- a/src/crux/agent/manager_poll.go +++ b/src/crux/agent/manager_poll.go @@ -27,6 +27,12 @@ func (m *Manager) pollTmuxOutputForAgent(agent *AgentProcess, ctx context.Contex case <-ctx.Done(): return case <-ticker.C: + // The Copilot CLI's own atomic save replaces the config.json + // symlink seconds after launch; relink it within one poll interval + // so a later dashboard login reaches this running agent (#18671). + // One Lstat while the link is intact; a no-op for other backends. + m.relinkCopilotConfigAfterCLISave(agent) + // Scrollback capture for dashboard display + ring buffer diff output := m.captureTmuxPaneForAgent(agent) if output == "" { diff --git a/src/crux/agent/usersession.go b/src/crux/agent/usersession.go index 706e96405b..49f3480f38 100644 --- a/src/crux/agent/usersession.go +++ b/src/crux/agent/usersession.go @@ -129,8 +129,12 @@ type UserSessionRuntime struct { // orphaned judges whether an attached tmux client has lost its terminal // attacher (usersession_clients.go); nil disables the reaper. orphaned clientOrphanFunc - // workspaceMu serialises workspace clone and spek init across sessions. + // workspaceMu serialises workspace clone, the lock of its git config, + // worktree add/remove and spek init across sessions. workspaceMu sync.Mutex + // securedClones records the shared clones whose git config this process + // has locked (userSessionWorkspaceSecureScript); guarded by workspaceMu. + securedClones map[string]bool // bufSeq makes every paste buffer name unique, so concurrent inputs to one // session cannot paste each other's text. bufSeq atomic.Uint64 @@ -248,8 +252,9 @@ func (r *UserSessionRuntime) Start(spec crux.UserSessionSpec) error { if err := r.prepareSandbox(spec.ID, uid); err != nil { return err } - // The pane opens in the hive's shared workspace checkout when the spec - // names one, otherwise in the session's own empty work dir. + // The pane opens in the session's own worktree of the hive's shared + // workspace clone when the spec names one, otherwise in the session's own + // empty work dir. cwd, workspace := userSessionWorkDir(spec.ID), "" if spec.WorkspaceRepo != "" { ws, err := r.ensureWorkspace(spec, uid) diff --git a/src/crux/agent/usersession_async_test.go b/src/crux/agent/usersession_async_test.go index d4b0714d30..7c22bb3ca0 100644 --- a/src/crux/agent/usersession_async_test.go +++ b/src/crux/agent/usersession_async_test.go @@ -166,18 +166,26 @@ func TestUserSessionWorkspaceConcurrentStartsCloneOnce(t *testing.T) { if n := len(privCallsWithScript(p, userSessionWorkspaceOwnScript)); n != 1 { t.Errorf("hand-over ran %d times, want once", n) } - ws := r.WorkspaceDir(testWorkspaceRepo) - respawns := 0 + // Each opened in its own worktree of that one clone (#18588), added once + // per session. + clone := r.WorkspaceDir(testWorkspaceRepo) + cwds := map[string]bool{} for _, c := range f.calls { if c.args[0] == "respawn-pane" { - respawns++ - if c.args[indexOf(c.args, "-c")+1] != ws { - t.Errorf("session %s did not open in the shared workspace: %v", c.socket, c.args) + id := strings.TrimPrefix(c.socket, UserSessionTmuxPrefix) + cwd := c.args[indexOf(c.args, "-c")+1] + if cwd != r.SessionWorkspaceDir(id, testWorkspaceRepo) { + t.Errorf("session %s opened in %s, want its own worktree", id, cwd) } + cwds[cwd] = true } } - if respawns != 2 { - t.Errorf("%d sessions launched, want both", respawns) + if len(cwds) != 2 { + t.Errorf("sessions opened in %v, want two distinct worktrees", cwds) + } + adds := privCallsWithScript(p, userSessionWorktreeAddScript) + if len(adds) != 2 || adds[0].argv[4] != clone || adds[1].argv[4] != clone || adds[0].argv[5] == adds[1].argv[5] { + t.Errorf("worktree adds = %+v, want one per session against the one clone %s", adds, clone) } } diff --git a/src/crux/agent/usersession_inbox.go b/src/crux/agent/usersession_inbox.go index 7a9a44abe4..4c1911105a 100644 --- a/src/crux/agent/usersession_inbox.go +++ b/src/crux/agent/usersession_inbox.go @@ -14,11 +14,12 @@ import ( // Inbox seed files (POST /api/sessions/{id}/seed). The dashboard sends Inbox // items into a session as markdown files under /.hive/inbox, where cwd is -// the session's shared workspace checkout (or its own work dir when it has -// none). Like an upload, the file is written AS THE SESSION'S UID through the -// su-exec helper, never as the hive's uid into a directory the session -// controls: a symlink planted by any session uid only reaches what that uid -// can already reach, and the script refuses one at .hive or .hive/inbox. +// the session's worktree of the shared workspace clone (or its own work dir +// when it has none). Like an upload, the file is written AS THE SESSION'S UID +// through the su-exec helper, never as the hive's uid into a directory the +// session controls: a symlink planted by any session uid only reaches what +// that uid can already reach, and the script refuses one at .hive or +// .hive/inbox. const ( // userSessionInboxDir is the seed directory relative to the session cwd. userSessionInboxDir = ".hive/inbox" @@ -59,19 +60,18 @@ if [ -d "$d/$name" ] || [ -L "$d/$name" ]; then rm -f -- "$t"; exit ` + userSess mv -f -- "$t" "$d/$name"` // userSessionInboxCwd validates the working directory a seed is written -// under: "" is the session's own work dir; anything else must be exactly a -// workspace checkout path (//). +// under: "" is the session's own work dir; anything else must be exactly this +// session's worktree (/.sessions///) or, for a +// session recorded before per-session worktrees, the shared clone +// (//) it still runs in. func userSessionInboxCwd(id, workspace string) (string, error) { if strings.TrimSpace(workspace) == "" { return userSessionWorkDir(id), nil } clean := filepath.Clean(workspace) - rel, err := filepath.Rel(userSessionWorkspaceRoot(), clean) - if err != nil { - return "", fmt.Errorf("workspace %q is not a session workspace", workspace) - } - org, name, ok := strings.Cut(filepath.ToSlash(rel), "/") - if !ok || !userSessionRepoPartPattern.MatchString(org) || !userSessionRepoPartPattern.MatchString(name) || clean != userSessionWorkspaceDir(org, name) { + org, name := filepath.Base(filepath.Dir(clean)), filepath.Base(clean) + if !userSessionRepoPartPattern.MatchString(org) || !userSessionRepoPartPattern.MatchString(name) || + (clean != userSessionWorktreeDir(id, org, name) && clean != userSessionWorkspaceDir(org, name)) { return "", fmt.Errorf("workspace %q is not a session workspace", workspace) } return clean, nil @@ -79,9 +79,9 @@ func userSessionInboxCwd(id, workspace string) (string, error) { // StoreInboxFile writes data as /.hive/inbox/, as the session // uid, owned by that uid with mode 0640 and the directory's group (the -// sessions group in the shared workspace), and returns -// its absolute path. workspace is the session's workspace checkout ("" for a -// session without one). fileName must be a plain name. +// sessions group in the setgid worktree), and returns its absolute path. +// workspace is the session's worktree ("" for a session without one). +// fileName must be a plain name. func (r *UserSessionRuntime) StoreInboxFile(id, workspace, fileName string, data []byte) (string, error) { if err := checkUserSessionID(id); err != nil { return "", err diff --git a/src/crux/agent/usersession_inbox_test.go b/src/crux/agent/usersession_inbox_test.go index cdd46f2c31..775b35bca0 100644 --- a/src/crux/agent/usersession_inbox_test.go +++ b/src/crux/agent/usersession_inbox_test.go @@ -53,12 +53,20 @@ func TestUserSessionStoreInboxFileRunsAsSessionUIDInWorkDir(t *testing.T) { func TestUserSessionStoreInboxFileWorkspaceAndValidation(t *testing.T) { r, _, p := startedRuntime(t) p.handler = inboxScriptHandler - ws := userSessionWorkspaceDir("hivecommons", "hive") - path, err := r.StoreInboxFile(testSessionID, ws, testInboxFileName, []byte("x")) - if err != nil || path != filepath.Join(ws, ".hive", "inbox", testInboxFileName) { - t.Fatalf("workspace seed = %q, %v", path, err) + // The session's own worktree is where its pane runs; the shared clone is + // still accepted for a record from before per-session worktrees. + for _, ws := range []string{userSessionWorktreeDir(testSessionID, "hivecommons", "hive"), userSessionWorkspaceDir("hivecommons", "hive")} { + path, err := r.StoreInboxFile(testSessionID, ws, testInboxFileName, []byte("x")) + if err != nil || path != filepath.Join(ws, ".hive", "inbox", testInboxFileName) { + t.Fatalf("workspace seed = %q, %v", path, err) + } } - for _, bad := range []string{"/etc", filepath.Join(userSessionWorkspaceRoot(), "only-org"), filepath.Join(ws, "sub"), filepath.Join(userSessionWorkspaceRoot(), "..", "x", "y")} { + ws := userSessionWorkspaceDir("hivecommons", "hive") + for _, bad := range []string{ + "/etc", filepath.Join(userSessionWorkspaceRoot(), "only-org"), filepath.Join(ws, "sub"), filepath.Join(userSessionWorkspaceRoot(), "..", "x", "y"), + userSessionWorktreeDir("fedcba987654", "hivecommons", "hive"), // another session's worktree + filepath.Join(userSessionWorktreesDir(testSessionID), "hivecommons"), + } { if _, err := r.StoreInboxFile(testSessionID, bad, testInboxFileName, []byte("x")); err == nil { t.Errorf("workspace %q accepted", bad) } diff --git a/src/crux/agent/usersession_priv.go b/src/crux/agent/usersession_priv.go index 24b3b7bbcb..749ac64e73 100644 --- a/src/crux/agent/usersession_priv.go +++ b/src/crux/agent/usersession_priv.go @@ -354,11 +354,13 @@ func (r *UserSessionRuntime) openSocketDirToAttacher(uid int) error { return nil } -// Remove ends the session and deletes everything it owned: the sandbox (home, -// work dir, uploads, tmp) and the uid's tmux socket dir, so a later session -// that is allocated the same uid starts clean. Directories earlier releases -// created for the id are removed too. A failure is returned and the caller -// must keep the session record (and its uid slot) so the removal is retried. +// Remove ends the session and deletes everything it owned: its worktree of the +// shared workspace clone (uncommitted work is auto-saved onto its branch first, +// see userSessionWorktreeRemoveScript), the sandbox (home, work dir, uploads, +// tmp) and the uid's tmux socket dir, so a later session that is allocated the +// same uid starts clean. Directories earlier releases created for the id are +// removed too. A failure is returned and the caller must keep the session +// record (and its uid slot) so the removal is retried. func (r *UserSessionRuntime) Remove(id string) error { if err := checkUserSessionID(id); err != nil { return err @@ -369,6 +371,12 @@ func (r *UserSessionRuntime) Remove(id string) error { r.logger.Warn("user session kill before removal failed", "id", id, "error", err) } } + r.workspaceMu.Lock() + err := r.removeWorktrees(id) + r.workspaceMu.Unlock() + if err != nil { + return fmt.Errorf("removing session worktree: %w", err) + } dir := userSessionSandboxDir(id) if _, err := os.Lstat(dir); err == nil { if out, err := r.privRun(userSessionRootSpec, nil, "rm", "-rf", "--one-file-system", "--", dir); err != nil { diff --git a/src/crux/agent/usersession_workspace.go b/src/crux/agent/usersession_workspace.go index adca2a4025..932b4c98bb 100644 --- a/src/crux/agent/usersession_workspace.go +++ b/src/crux/agent/usersession_workspace.go @@ -18,24 +18,43 @@ import ( // The per-hive session workspace. A session is a person using a CLI the way // they would on a laptop: inside a checkout of the project that has been // `spektacular init `-ed, so the CLI finds .spektacular/ and the spek -// skills. The hive keeps ONE such checkout per repository: +// skills. The hive keeps ONE clone per repository and gives every session its +// own git worktree of it: // -// // root: 2770, files g+rw +// // the clone, root: 2770, files g+rw +// /.sessions/// the session's worktree, : 2770 // // where the workspace root defaults to /workspace (hive-owned, -// 0711, like the sandbox root itself). Every session of the hive shares that -// checkout: two sessions editing the same files are in exactly the position -// of two people on one checkout, and that sharing is the point (a "jam"). The -// checkout is cloned once, by the hive, with a short-lived token in a -// one-shot credential helper that never reaches the clone's config or -// remotes, then handed to the sessions group; the remote stays the plain -// https URL and no session ever holds a hive credential, so pushing is the -// person's own business with their own login. `git fetch` is never automatic. +// 0711, like the sandbox root itself). The clone is cloned once, by the hive, +// with a short-lived token in a one-shot credential helper that never reaches +// the clone's config or remotes, then handed to the sessions group; the remote +// stays the plain https URL and no session ever holds a hive credential, so +// pushing is the person's own business with their own login. `git fetch` is +// never automatic. // -// `spektacular init ` runs once per vendor per workspace, as the -// starting session's uid, only when the vendor is one spek supports and the -// binary is installed; a failure or an absent binary is logged and the -// session still starts. Clone and init are serialised by workspaceMu. +// Sessions share the clone's objects and refs but not a working tree: each +// one has its own checked-out branch (hive-session/) and its own +// .spektacular/ (spek keeps ONE in-progress workflow per directory, so a +// shared tree let one person's unfinished spec block or be discarded by +// another's, #18588). People sharing one session (a "jam") still share +// everything; two sessions no longer see each other's uncommitted edits. The +// worktree is added by the session uid (hooks off), never by root, and is +// removed with the session; work left uncommitted is auto-saved onto the +// session's branch first and a branch holding commits HEAD lacks is kept. +// +// The clone's git config and hooks are locked (userSessionWorkspaceSecureScript) +// before any session gets a worktree of it and again at every boot: root owns +// them and no session can write them, so no session can plant a config entry, +// hook, filter or alias that another session's git, or the hive's git for +// that session, would run as the other uid. Objects and refs stay +// group-writable. The hive itself runs as a uid outside the sessions group, so +// it cannot look inside the clone or a worktree: every "is it there" check +// runs in the scripts, as the session uid. +// +// `spektacular init ` runs once per vendor per session worktree, as +// the session's uid, only when the vendor is one spek supports and the binary +// is installed; a failure or an absent binary is logged and the session still +// starts. Clone, worktree add/remove and init are serialised by workspaceMu. const ( // userSessionWorkspaceDirEnv relocates the workspace root (default // /workspace). @@ -57,11 +76,39 @@ const ( // userSessionSpekInitTimeout bounds one `spektacular init`. userSessionSpekInitTimeout = 2 * time.Minute // userSessionWorkspaceStateDirName, under the sessions data dir, holds the - // hive-owned "init ran" markers (//) outside the - // checkout so they never show up in git status. - userSessionWorkspaceStateDirName = "workspace-init" + // hive-owned "init ran" markers (/) outside the + // worktree so they never show up in git status. + userSessionWorkspaceStateDirName = "worktree-init" userSessionWorkspaceStateDirMode = 0o700 userSessionWorkspaceStateMode = 0o600 + // userSessionWorktreeDirName, under the workspace root, holds the + // per-session worktrees (//). Its leading dot keeps it + // apart from every : userSessionRepoPartPattern never starts with one. + userSessionWorktreeDirName = ".sessions" + // userSessionWorktreeBranchPrefix names the branch a session's worktree + // starts on (hive-session/). + userSessionWorktreeBranchPrefix = userSessionWorktreeBranchDir + "/" + // userSessionWorktreeTimeout bounds one worktree add (a full checkout of + // the project) or removal. + userSessionWorktreeTimeout = 10 * time.Minute + // userSessionWorktreeStagingMode is the mode the hive creates an empty + // worktree directory with, before root hands it to the session as 2770. + userSessionWorktreeStagingMode = 0o700 + // userSessionWorktreeAddedLine is what userSessionWorktreeAddScript prints + // when it added a fresh worktree (and not when the session's worktree was + // already there), so the hive knows to clear stale spek init markers. + userSessionWorktreeAddedLine = "hive: worktree added" + // userSessionWorktreeCloneGoneLine is what userSessionWorktreeRemoveScript + // prints when the shared clone behind the worktree no longer exists, so + // there is no git to save the work through. + userSessionWorktreeCloneGoneLine = "hive: shared clone gone" + // userSessionWorktreeAutoSaveMessage is the commit message of the + // auto-save userSessionWorktreeRemoveScript makes of uncommitted work. + userSessionWorktreeAutoSaveMessage = "hive: auto-save of work left uncommitted when the session was deleted" + // userSessionWorktreeBranchDir is the directory under refs/heads (and + // logs/refs/heads) that holds every session branch; the lock script makes + // it group-writable so each session can create its branch there. + userSessionWorktreeBranchDir = "hive-session" // userSessionWorkspaceCloningSuffix names the staging directory a clone // lands in before root hands it to the sessions group and moves it into // place, so a half-finished clone is never mistaken for a workspace. @@ -77,6 +124,11 @@ const ( userSessionWorkspaceEnv = "HIVE_SESSION_WORKSPACE" ) +// userSessionHiveGitArgs precede every git subcommand the hive runs for the +// session workspace: no hook and no fsmonitor command ever runs on its behalf. +// The worktree scripts' g() passes the same two. +var userSessionHiveGitArgs = []string{"-c", "core.hooksPath=/dev/null", "-c", "core.fsmonitor=false"} + // userSessionRepoPartPattern admits one GitHub owner or repository name. var userSessionRepoPartPattern = regexp.MustCompile(`^[A-Za-z0-9_][A-Za-z0-9._-]{0,99}$`) @@ -100,8 +152,8 @@ find "$src" -type f -exec chmod ug+rw,o-rwx {} + mv -- "$src" "$dst"` // userSessionSpekInitScript runs `spektacular init ` inside the -// workspace as the session uid. $1 is the workspace, $2 the session HOME, $3 -// the spektacular binary, $4 the vendor. +// session's worktree as the session uid. $1 is the worktree, $2 the session +// HOME, $3 the spektacular binary, $4 the vendor. const userSessionSpekInitScript = `set -e; cd -- "$1"; HOME=$2; export HOME; exec "$3" init "$4"` // gitRunFunc runs one git invocation as the hive itself (the clone). It is @@ -145,9 +197,10 @@ func userSessionWorkspaceDir(org, name string) string { return filepath.Join(userSessionWorkspaceRoot(), org, name) } -// WorkspaceDir is the path sessions of the given "org/repo" open in, or "" +// WorkspaceDir is the path of the hive's shared clone of "org/repo", or "" // when repo is empty or malformed. It is the same whether or not the clone -// exists yet; Start creates it. +// exists yet; the first Start creates it. Sessions open in their own worktree +// of it (SessionWorkspaceDir). func (r *UserSessionRuntime) WorkspaceDir(repo string) string { if strings.TrimSpace(repo) == "" { return "" @@ -159,28 +212,40 @@ func (r *UserSessionRuntime) WorkspaceDir(repo string) string { return userSessionWorkspaceDir(org, name) } -// ensureWorkspace returns the workspace for spec.WorkspaceRepo, cloning it on -// first use and running spek init for the vendor once. The sandbox (HOME) of -// the session must already exist. A clone failure is returned: a session that -// silently opened somewhere else would not be the workspace the record names. +// ensureWorkspace returns the session's worktree for spec.WorkspaceRepo, +// cloning the shared clone on first use, locking its git config once per hive +// process, adding the worktree on the session's first start and running spek +// init in it for the vendor once. The sandbox (HOME) of the session must +// already exist. A clone, lock or worktree failure is returned: a session that +// silently opened somewhere else, or in a clone whose config any session can +// still write, would not be the workspace the record names. func (r *UserSessionRuntime) ensureWorkspace(spec crux.UserSessionSpec, uid int) (string, error) { org, name, err := splitUserSessionRepo(spec.WorkspaceRepo) if err != nil { return "", err } - ws := userSessionWorkspaceDir(org, name) + clone := userSessionWorkspaceDir(org, name) r.workspaceMu.Lock() defer r.workspaceMu.Unlock() - if _, err := os.Lstat(ws); err != nil { + if _, err := os.Lstat(clone); err != nil { if !errors.Is(err, os.ErrNotExist) { - return "", fmt.Errorf("checking session workspace %s: %w", ws, err) + return "", fmt.Errorf("checking session workspace %s: %w", clone, err) } - if err := r.cloneWorkspace(org, name, ws); err != nil { + if err := r.cloneWorkspace(org, name, clone); err != nil { return "", fmt.Errorf("preparing session workspace: %w", err) } - r.logger.Info("session workspace cloned", "repo", org+"/"+name, "path", ws) + r.logger.Info("session workspace cloned", "repo", org+"/"+name, "path", clone) + } + if !r.securedClones[clone] { + if err := r.secureClone(org, name, clone); err != nil { + return "", fmt.Errorf("locking the session workspace's git config: %w", err) + } } - r.spekInit(spec, uid, ws, org, name) + ws, err := r.ensureWorktree(spec.ID, uid, org, name, clone) + if err != nil { + return "", fmt.Errorf("preparing session worktree: %w", err) + } + r.spekInit(spec, uid, ws) return ws, nil } @@ -189,7 +254,9 @@ func (r *UserSessionRuntime) ensureWorkspace(spec crux.UserSessionSpec, uid int) // it to ws. The token travels only in a one-shot `git -c credential.helper` // (before the subcommand, so git does not persist it) and its file is removed // before this returns; `core.sharedRepository=group` is the ONLY config -// written into the clone. +// written into the clone, and the lock that follows +// (userSessionWorkspaceSecureScript, run by the caller) rewrites that config +// from scratch anyway. func (r *UserSessionRuntime) cloneWorkspace(org, name, ws string) error { parent := filepath.Dir(ws) for _, dir := range []string{userSessionWorkspaceRoot(), parent} { @@ -215,7 +282,7 @@ func (r *UserSessionRuntime) cloneWorkspace(org, name, ws string) error { authArgs, cleanupAuth := r.workspaceCloneAuth(ctx, repo, parent) defer cleanupAuth() url := "https://github.com/" + repo + ".git" - args := append(append([]string{}, authArgs...), "clone", "-c", "core.sharedRepository=group", "--", url, staging) + args := append(append(append([]string{}, userSessionHiveGitArgs...), authArgs...), "clone", "-c", "core.sharedRepository=group", "--", url, staging) if out, err := r.git(ctx, "", pushbroker.PushEnv(os.Environ()), args...); err != nil { _ = os.RemoveAll(staging) return fmt.Errorf("git clone %s: %w: %s", repo, err, strings.TrimSpace(string(out))) @@ -271,17 +338,21 @@ func (r *UserSessionRuntime) workspaceCloneAuth(ctx context.Context, repo, dir s return []string{"-c", "credential.helper=store --file=" + path}, func() { _ = os.Remove(path) } } -// userSessionSpekInitMarker is the hive-owned file recording that -// `spektacular init ` succeeded in the workspace. -func userSessionSpekInitMarker(org, name, vendor string) string { - return filepath.Join(userSessionsDir(), userSessionWorkspaceStateDirName, org, name, vendor) +// userSessionSpekInitMarkerDir holds the hive-owned files recording that +// `spektacular init ` succeeded in session id's worktree. +func userSessionSpekInitMarkerDir(id string) string { + return filepath.Join(userSessionsDir(), userSessionWorkspaceStateDirName, id) +} + +func userSessionSpekInitMarker(id, vendor string) string { + return filepath.Join(userSessionSpekInitMarkerDir(id), vendor) } // spekInit runs `spektacular init ` in ws as the session uid, once per -// vendor per workspace. Nothing here fails the session: an unsupported vendor -// is skipped silently, a missing binary is logged at info, a failed init at -// warn (and retried on the next start, since no marker is written). -func (r *UserSessionRuntime) spekInit(spec crux.UserSessionSpec, uid int, ws, org, name string) { +// vendor per session worktree. Nothing here fails the session: an unsupported +// vendor is skipped silently, a missing binary is logged at info, a failed +// init at warn (and retried on the next start, since no marker is written). +func (r *UserSessionRuntime) spekInit(spec crux.UserSessionSpec, uid int, ws string) { if !userSessionSpekAgents[spec.Vendor] { return } @@ -291,10 +362,10 @@ func (r *UserSessionRuntime) spekInit(spec crux.UserSessionSpec, uid int, ws, or } path, err := r.lookPath(bin) if err != nil { - r.logger.Info("session workspace: spektacular is not installed; the workspace gets no spek skills", "binary", bin, "repo", org+"/"+name) + r.logger.Info("session workspace: spektacular is not installed; the workspace gets no spek skills", "binary", bin, "repo", spec.WorkspaceRepo) return } - marker := userSessionSpekInitMarker(org, name, spec.Vendor) + marker := userSessionSpekInitMarker(spec.ID, spec.Vendor) if _, err := os.Stat(marker); err == nil { return } @@ -302,7 +373,7 @@ func (r *UserSessionRuntime) spekInit(spec crux.UserSessionSpec, uid int, ws, or defer cancel() out, err := r.priv(ctx, userSessionSpec(uid), nil, "sh", "-c", userSessionSpekInitScript, "sh", ws, userSessionHome(spec.ID), path, spec.Vendor) if err != nil { - r.logger.Warn("session workspace: spektacular init failed; the session starts without spek skills", "vendor", spec.Vendor, "repo", org+"/"+name, "error", outputErr("spektacular init", err, out)) + r.logger.Warn("session workspace: spektacular init failed; the session starts without spek skills", "vendor", spec.Vendor, "repo", spec.WorkspaceRepo, "error", outputErr("spektacular init", err, out)) return } err = os.MkdirAll(filepath.Dir(marker), userSessionWorkspaceStateDirMode) @@ -312,22 +383,17 @@ func (r *UserSessionRuntime) spekInit(spec crux.UserSessionSpec, uid int, ws, or if err != nil { r.logger.Warn("session workspace: recording spektacular init failed; it will run again next start", "vendor", spec.Vendor, "error", err) } - r.logger.Info("session workspace: spektacular init ran", "vendor", spec.Vendor, "repo", org+"/"+name, "path", ws) + r.logger.Info("session workspace: spektacular init ran", "vendor", spec.Vendor, "repo", spec.WorkspaceRepo, "path", ws) } // userSessionWorkspaceEnvPairs are the extra session variables for a session -// opened in a workspace. The checkout is root-owned (so a session cannot open -// it to other uids) and git refuses to work in a directory owned by another -// user unless it is marked safe; the environment form of that config applies -// to every git the CLI spawns without writing any file. +// opened in a workspace. No safe.directory is set: the session's worktree, +// its .git file and its entry under the clone's worktrees/ are all the +// session's own, so git's ownership check passes on its own, and it keeps +// refusing a .git another uid swapped in (the worktree is group-writable). func userSessionWorkspaceEnvPairs(workspace string) [][2]string { if workspace == "" { return nil } - return [][2]string{ - {userSessionWorkspaceEnv, workspace}, - {"GIT_CONFIG_COUNT", "1"}, - {"GIT_CONFIG_KEY_0", "safe.directory"}, - {"GIT_CONFIG_VALUE_0", workspace}, - } + return [][2]string{{userSessionWorkspaceEnv, workspace}} } diff --git a/src/crux/agent/usersession_workspace_test.go b/src/crux/agent/usersession_workspace_test.go index 226a897d25..67c92f9612 100644 --- a/src/crux/agent/usersession_workspace_test.go +++ b/src/crux/agent/usersession_workspace_test.go @@ -73,7 +73,9 @@ func workspaceSpec(id string, slot int, vendor string) crux.UserSessionSpec { } // workspaceRuntime wires the fakes. The priv fake simulates the root hand-over -// script by moving the staging clone into place, as root's mv would. +// script by moving the staging clone into place, as root's mv would, and the +// worktree scripts by creating (a .git file, as `git worktree add` writes) or +// deleting the session's worktree. func workspaceRuntime(t *testing.T) (*UserSessionRuntime, *fakeTmux, *fakePriv, *fakeGit) { t.Helper() f := sessionLifecycleTmux("") @@ -82,8 +84,19 @@ func workspaceRuntime(t *testing.T) (*UserSessionRuntime, *fakeTmux, *fakePriv, r.git = g.run r.cloneMinter = &fakeWorkspaceMinter{token: testWorkspaceToken} p.handler = func(c fakePrivCall) (bool, []byte, error) { - if len(c.argv) >= 7 && c.argv[0] == "sh" && c.argv[2] == userSessionWorkspaceOwnScript { + if len(c.argv) < 7 || c.argv[0] != "sh" { + return false, nil, nil + } + switch c.argv[2] { + case userSessionWorkspaceOwnScript: return true, nil, os.Rename(c.argv[4], c.argv[5]) + case userSessionWorktreeAddScript: + if _, err := os.Lstat(filepath.Join(c.argv[5], ".git")); err == nil { + return true, nil, nil + } + return true, []byte(userSessionWorktreeAddedLine + "\n"), os.WriteFile(filepath.Join(c.argv[5], ".git"), []byte("gitdir: "+c.argv[4]+"/.git/worktrees/x\n"), 0o644) + case userSessionWorktreeRemoveScript: + return true, nil, os.RemoveAll(c.argv[5]) } return false, nil, nil } @@ -131,6 +144,13 @@ func TestUserSessionWorkspaceCloneLeavesNoTokenBehind(t *testing.T) { if clone < 0 || helper < 0 || helper > clone || args[helper-1] != "-c" { t.Fatalf("credential helper must be a one-shot `git -c` BEFORE the clone subcommand (clone -c would persist it): %v", args) } + // The hive's own git runs no hook and no fsmonitor command, whatever the + // repository or its config would name. + for _, off := range []string{"core.hooksPath=/dev/null", "core.fsmonitor=false"} { + if i := indexOf(args, off); i < 1 || i > clone || args[i-1] != "-c" { + t.Errorf("the clone lacks a one-shot -c %s before the subcommand: %v", off, args) + } + } shared := indexOf(args, "core.sharedRepository=group") if shared < clone || args[shared-1] != "-c" { t.Errorf("core.sharedRepository=group must be the clone's own (persisted) config: %v", args) @@ -211,20 +231,39 @@ func TestUserSessionWorkspaceOwnedBySessionsGroup(t *testing.T) { t.Errorf("workspace %q is not under the sandbox root's workspace dir", ws) } for _, c := range p.calls { - if c.user == userSessionRootSpec && c.argv[0] == "sh" && c.argv[2] != userSessionPrepareScript && c.argv[2] != userSessionWorkspaceOwnScript && c.argv[2] != userSessionSocketDirScript { + if c.user == userSessionRootSpec && c.argv[0] == "sh" && c.argv[2] != userSessionPrepareScript && c.argv[2] != userSessionWorkspaceOwnScript && c.argv[2] != userSessionSocketDirScript && c.argv[2] != userSessionWorktreeOwnScript && c.argv[2] != userSessionWorkspaceSecureScript { t.Errorf("unexpected root script: %v", c.argv) } + // git in the shared clone never runs as root: a hook or config any + // session planted there would run with root's rights. + if c.user == userSessionRootSpec && len(c.argv) > 2 && (c.argv[2] == userSessionWorktreeAddScript || c.argv[2] == userSessionWorktreeRemoveScript) { + t.Errorf("worktree git ran as root: %v", c.argv) + } + } + wt := privCallsWithScript(p, userSessionWorktreeOwnScript) + if len(wt) != 1 || wt[0].user != userSessionRootSpec || wt[0].argv[4] != r.SessionWorkspaceDir(testSessionID, testWorkspaceRepo) || + wt[0].argv[5] != strconv.Itoa(testSessionUID) || wt[0].argv[6] != strconv.Itoa(defaultUserSessionGID) || wt[0].argv[7] != strconv.Itoa(os.Getuid()) { + t.Errorf("worktree hand-over = %+v, want root handing the session's own dir to its uid and the sessions group", wt) + } + for _, bad := range []string{"777", "o+", "a+", "-R"} { + if strings.Contains(userSessionWorktreeOwnScript, bad) { + t.Errorf("worktree hand-over script widens access (%q)", bad) + } } } -// The invariant: the pane opens IN the workspace and git in the session is -// told that root-owned checkout is safe; nothing else about the sandbox moves. +// The invariant: the pane opens IN the session's own worktree, git in the +// session is not told any directory is safe, and nothing else about the +// sandbox moves. func TestUserSessionWorkspaceIsTheSessionCwd(t *testing.T) { r, f, _, _ := workspaceRuntime(t) if err := r.Start(workspaceSpec(testSessionID, testSessionSlot, "claude")); err != nil { t.Fatal(err) } - ws := r.WorkspaceDir(testWorkspaceRepo) + ws := r.SessionWorkspaceDir(testSessionID, testWorkspaceRepo) + if want := filepath.Join(userSessionWorkspaceRoot(), userSessionWorktreeDirName, testSessionID, "acme", "widgets"); ws != want { + t.Fatalf("session worktree = %q, want %q", ws, want) + } cwds := 0 var env string for _, c := range f.calls { @@ -244,7 +283,6 @@ func TestUserSessionWorkspaceIsTheSessionCwd(t *testing.T) { } for _, want := range []string{ userSessionWorkspaceEnv + " " + ws, - "GIT_CONFIG_KEY_0 safe.directory", "GIT_CONFIG_VALUE_0 " + ws, "HOME " + userSessionHome(testSessionID), "TMPDIR " + userSessionTmpDir(testSessionID), } { if !strings.Contains(env, want) { @@ -257,36 +295,74 @@ func TestUserSessionWorkspaceIsTheSessionCwd(t *testing.T) { if r.WorkDir(testSessionID) != userSessionWorkDir(testSessionID) { t.Error("uploads must stay under the per-session dir, not the shared workspace") } + // No safe.directory: the worktree is the session's own, so git's + // ownership check stays on and refuses a .git another uid swapped in. + if strings.Contains(env, "GIT_CONFIG_COUNT") || strings.Contains(env, "safe.directory") { + t.Errorf("session env marks a directory safe for git:\n%s", env) + } if pairs := userSessionWorkspaceEnvPairs(""); pairs != nil { t.Errorf("no workspace must add no env: %v", pairs) } } -func TestUserSessionWorkspaceIsClonedOnceAndShared(t *testing.T) { - r, f, _, g := workspaceRuntime(t) +// The invariant (#18588): one clone per repository, but every session opens in +// its own worktree of it, added by `git worktree add` against that one clone +// as the session's own uid; a resume reuses the worktree. +func TestUserSessionWorkspaceIsClonedOnceWithAWorktreePerSession(t *testing.T) { + r, f, p, g := workspaceRuntime(t) if err := r.Start(workspaceSpec(testSessionID, testSessionSlot, "claude")); err != nil { t.Fatal(err) } if err := r.Start(workspaceSpec(testSecondSessionID, testSecondSlot, "codex")); err != nil { t.Fatal(err) } - // Resume of the first session: still no second clone, no fetch. + // Resume of the first session: still no second clone, no fetch, and no + // second worktree. if err := r.Start(workspaceSpec(testSessionID, testSessionSlot, "claude")); err != nil { t.Fatal(err) } - if len(g.calls) != 1 { - t.Fatalf("git ran %d times, want one clone and never a fetch: %v", len(g.calls), g.calls) + if len(g.calls) != 1 || indexOf(g.calls[0], "clone") < 0 { + t.Fatalf("git ran %v, want one clone and never a fetch", g.calls) + } + clone := r.WorkspaceDir(testWorkspaceRepo) + ws1, ws2 := r.SessionWorkspaceDir(testSessionID, testWorkspaceRepo), r.SessionWorkspaceDir(testSecondSessionID, testWorkspaceRepo) + if ws1 == ws2 || ws1 == clone || ws2 == clone { + t.Fatalf("worktrees %q and %q must be distinct and not the clone %q", ws1, ws2, clone) + } + adds := privCallsWithScript(p, userSessionWorktreeAddScript) + secondUser := strconv.Itoa(defaultUserSessionUIDBase+testSecondSlot-1) + ":" + strconv.Itoa(defaultUserSessionGID) + // The add script runs on every start, as the session uid, because only + // that uid can see whether the worktree is already there: the resume of + // the first session runs it again (a no-op inside the script). + want := [][3]string{{testSessionSpecUser, ws1, userSessionWorktreeBranch(testSessionID)}, {secondUser, ws2, userSessionWorktreeBranch(testSecondSessionID)}, {testSessionSpecUser, ws1, userSessionWorktreeBranch(testSessionID)}} + if len(adds) != len(want) { + t.Fatalf("worktree add ran %d times, want once per start: %+v", len(adds), adds) + } + for i, c := range adds { + if c.user != want[i][0] || c.argv[4] != clone || c.argv[5] != want[i][1] || c.argv[6] != want[i][2] { + t.Errorf("worktree add %d = %s %v, want %v against the one clone %s", i, c.user, c.argv[4:], want[i], clone) + } } - ws := r.WorkspaceDir(testWorkspaceRepo) + for _, s := range []string{"worktree add", "core.hooksPath=/dev/null", "core.fsmonitor=false", `safe.directory="$clone"`} { + if !strings.Contains(userSessionWorktreeAddScript, s) { + t.Errorf("worktree add script lacks %q", s) + } + } + wantCwd := []string{ws1, ws2, ws1} + var cwds []string for _, c := range f.calls { - if c.args[0] == "respawn-pane" && c.args[indexOf(c.args, "-c")+1] != ws { - t.Errorf("session %s did not open in the shared workspace: %v", c.socket, c.args) + if c.args[0] == "respawn-pane" { + cwds = append(cwds, c.args[indexOf(c.args, "-c")+1]) } } + if strings.Join(cwds, " ") != strings.Join(wantCwd, " ") { + t.Errorf("panes opened in %v, want %v", cwds, wantCwd) + } } -// The invariant: `spektacular init ` runs as the session uid, in the -// workspace, once per vendor, and only for vendors spek supports. +// The invariant: `spektacular init ` runs as the session uid, in that +// session's worktree, once per vendor per session, and only for vendors spek +// supports. func TestUserSessionWorkspaceSpekInitOncePerVendor(t *testing.T) { r, _, p, _ := workspaceRuntime(t) for _, spec := range []crux.UserSessionSpec{ @@ -301,32 +377,30 @@ func TestUserSessionWorkspaceSpekInitOncePerVendor(t *testing.T) { } } inits := privCallsWithScript(p, userSessionSpekInitScript) - if len(inits) != 2 { - t.Fatalf("spektacular init ran %d times, want once for claude and once for codex: %+v", len(inits), inits) - } - ws := r.WorkspaceDir(testWorkspaceRepo) - wantVendors := []string{"claude", "codex"} - // claude was initialised by the first session (slot 3), codex by the - // second (slot 4): each init runs as the STARTING session's uid with the - // shared sessions group. - wantUsers := []string{testSessionSpecUser, strconv.Itoa(defaultUserSessionUIDBase+testSecondSlot-1) + ":" + strconv.Itoa(defaultUserSessionGID)} + if len(inits) != 3 { + t.Fatalf("spektacular init ran %d times, want once per supported vendor per session: %+v", len(inits), inits) + } + wantWS := []string{r.SessionWorkspaceDir(testSessionID, testWorkspaceRepo), r.SessionWorkspaceDir(testSecondSessionID, testWorkspaceRepo), r.SessionWorkspaceDir(testSecondSessionID, testWorkspaceRepo)} + wantVendors := []string{"claude", "claude", "codex"} + secondUser := strconv.Itoa(defaultUserSessionUIDBase+testSecondSlot-1) + ":" + strconv.Itoa(defaultUserSessionGID) + // Each init runs as the session's own uid with the shared sessions + // group, in that session's worktree. + wantUsers := []string{testSessionSpecUser, secondUser, secondUser} + wantHomes := []string{userSessionHome(testSessionID), userSessionHome(testSecondSessionID), userSessionHome(testSecondSessionID)} for i, c := range inits { if c.user != wantUsers[i] { t.Errorf("init %d ran as %q, want the session uid %q", i, c.user, wantUsers[i]) } - if c.argv[4] != ws || c.argv[6] != "/usr/bin/"+userSessionSpekDefaultBinary || c.argv[7] != wantVendors[i] { - t.Errorf("init args = %v", c.argv) - } - if c.argv[5] != userSessionHome(testSessionID) && c.argv[5] != userSessionHome(testSecondSessionID) { - t.Errorf("init HOME = %q, want a session home", c.argv[5]) + if c.argv[4] != wantWS[i] || c.argv[5] != wantHomes[i] || c.argv[6] != "/usr/bin/"+userSessionSpekDefaultBinary || c.argv[7] != wantVendors[i] { + t.Errorf("init %d args = %v", i, c.argv) } } - for _, v := range wantVendors { - if _, err := os.Stat(userSessionSpekInitMarker("acme", "widgets", v)); err != nil { - t.Errorf("no init marker for %s: %v", v, err) + for _, m := range [][2]string{{testSessionID, "claude"}, {testSecondSessionID, "claude"}, {testSecondSessionID, "codex"}} { + if _, err := os.Stat(userSessionSpekInitMarker(m[0], m[1])); err != nil { + t.Errorf("no init marker for %v: %v", m, err) } } - if _, err := os.Stat(userSessionSpekInitMarker("acme", "widgets", "goose")); err == nil { + if _, err := os.Stat(userSessionSpekInitMarker(testSecondSessionID, "goose")); err == nil { t.Error("init marker written for a vendor spek does not support") } for v := range userSessionSpekAgents { @@ -356,7 +430,7 @@ func TestUserSessionWorkspaceMissingSpekDoesNotFailStart(t *testing.T) { if indexOf(f.verbs(), "respawn-pane") < 0 { t.Error("CLI not launched") } - if _, err := os.Stat(userSessionSpekInitMarker("acme", "widgets", "claude")); err == nil { + if _, err := os.Stat(userSessionSpekInitMarker(testSessionID, "claude")); err == nil { t.Error("marker written although init never ran") } } @@ -378,7 +452,7 @@ func TestUserSessionWorkspaceSpekInitFailureIsRetriedNotFatal(t *testing.T) { if n := len(privCallsWithScript(p, userSessionSpekInitScript)); n != 2 { t.Errorf("failed init should be retried on the next start, ran %d times", n) } - if _, err := os.Stat(userSessionSpekInitMarker("acme", "widgets", "claude")); err == nil { + if _, err := os.Stat(userSessionSpekInitMarker(testSessionID, "claude")); err == nil { t.Error("marker written for a failed init") } if indexOf(f.verbs(), "respawn-pane") < 0 { @@ -436,6 +510,9 @@ func TestUserSessionWorkspaceRejectsBadRepo(t *testing.T) { if r.WorkspaceDir(repo) != "" { t.Errorf("WorkspaceDir(%q) = %q, want empty", repo, r.WorkspaceDir(repo)) } + if got := r.SessionWorkspaceDir(testSessionID, repo); got != "" { + t.Errorf("SessionWorkspaceDir(%q) = %q, want empty", repo, got) + } } if len(g.calls) != 0 || indexOf(f.verbs(), "respawn-pane") >= 0 { t.Error("a malformed repo reached git or launched a CLI") @@ -447,6 +524,11 @@ func TestUserSessionWorkspaceRejectsBadRepo(t *testing.T) { if got := r.WorkspaceDir("Acme-Inc/my.repo_1"); got != filepath.Join("/x/ws", "Acme-Inc", "my.repo_1") { t.Errorf("WorkspaceDir = %q", got) } + for _, id := range []string{"", "../x", "ABCDEF012345", testSessionID + "/.."} { + if got := r.SessionWorkspaceDir(id, testWorkspaceRepo); got != "" { + t.Errorf("SessionWorkspaceDir(%q) = %q, want empty", id, got) + } + } } func TestUserSessionWithoutWorkspaceKeepsEmptyWorkDir(t *testing.T) { @@ -495,3 +577,130 @@ func TestUserSessionGIDGuards(t *testing.T) { t.Error("default gid changed") } } + +// The invariant (the owner's lock design for #18588): root locks the shared +// clone's git config and hooks before any session gets a worktree of it, once +// per clone per hive process, and with the clone's plain https origin. +func TestUserSessionWorkspaceLockedBeforeAnyWorktree(t *testing.T) { + r, _, p, _ := workspaceRuntime(t) + if err := r.Start(workspaceSpec(testSessionID, testSessionSlot, "claude")); err != nil { + t.Fatal(err) + } + if err := r.Start(workspaceSpec(testSecondSessionID, testSecondSlot, "claude")); err != nil { + t.Fatal(err) + } + locks := privCallsWithScript(p, userSessionWorkspaceSecureScript) + if len(locks) != 1 { + t.Fatalf("lock ran %d times, want once per clone per process: %+v", len(locks), locks) + } + c := locks[0] + if c.user != userSessionRootSpec || c.argv[4] != r.WorkspaceDir(testWorkspaceRepo) || c.argv[5] != strconv.Itoa(defaultUserSessionGID) || + c.argv[6] != "https://github.com/"+testWorkspaceRepo+".git" || c.argv[7] != userSessionWorktreeBranchDir { + t.Errorf("lock = %s %v", c.user, c.argv[4:]) + } + lockAt, addAt := -1, -1 + for i, call := range p.calls { + if len(call.argv) > 2 && call.argv[2] == userSessionWorkspaceSecureScript && lockAt < 0 { + lockAt = i + } + if len(call.argv) > 2 && call.argv[2] == userSessionWorktreeAddScript && addAt < 0 { + addAt = i + } + } + if lockAt < 0 || addAt < 0 || lockAt > addAt { + t.Errorf("lock at call %d, first worktree add at %d: the clone must be locked first", lockAt, addAt) + } + for _, want := range []string{ + "worktreeConfig = true", "sharedRepository = group", "auto = 0", + "chmod 0750 -- \"$gd\"", "chmod 3770 -- \"$gd\"", "chmod 3770 -- \"$gd/worktrees\"", + "for d in hooks info modules", "mkdir -m 0755", "-type f -exec chmod 0644", + "mktemp", "echo . > \"$t\"", + } { + if !strings.Contains(userSessionWorkspaceSecureScript, want) { + t.Errorf("lock script lacks %q", want) + } + } + // The lock never runs git as root, and the hive's own git (the clone) + // runs no hook and no fsmonitor. + if strings.Contains(userSessionWorkspaceSecureScript, "git ") { + t.Error("the lock script runs git as root") + } + for _, want := range []string{"core.hooksPath=/dev/null", "core.fsmonitor=false"} { + if !strings.Contains(strings.Join(userSessionHiveGitArgs, " "), want) || !strings.Contains(userSessionWorktreeGit, want) { + t.Errorf("hive git lacks -c %s", want) + } + } +} + +// A clone whose lock fails is not handed out: the start fails closed, before +// any worktree is added or any CLI is launched, and the next start retries. +func TestUserSessionWorkspaceLockFailureFailsClosed(t *testing.T) { + r, f, p, _ := workspaceRuntime(t) + inner := p.handler + fail := true + p.handler = func(c fakePrivCall) (bool, []byte, error) { + if fail && len(c.argv) > 2 && c.argv[2] == userSessionWorkspaceSecureScript { + return true, []byte("chmod: Operation not permitted"), errors.New("exit 1") + } + return inner(c) + } + err := r.Start(workspaceSpec(testSessionID, testSessionSlot, "claude")) + if err == nil || !strings.Contains(err.Error(), "Operation not permitted") { + t.Fatalf("Start = %v, want the lock failure", err) + } + if n := len(privCallsWithScript(p, userSessionWorktreeAddScript)); n != 0 { + t.Errorf("worktree added %d times into an unlocked clone", n) + } + if indexOf(f.verbs(), "respawn-pane") >= 0 { + t.Error("CLI launched in an unlocked clone") + } + fail = false + if err := r.Start(workspaceSpec(testSessionID, testSessionSlot, "claude")); err != nil { + t.Fatalf("retry after the lock recovers = %v", err) + } + if n := len(privCallsWithScript(p, userSessionWorkspaceSecureScript)); n != 2 { + t.Errorf("lock ran %d times, want the failed attempt and its retry", n) + } +} + +// At boot every clone already on disk is locked (a clone from an earlier +// release had a group-writable config and hooks), and nothing else is: not +// the per-session worktrees dir, not a half-finished staging clone, not a +// stray file. +func TestUserSessionSecureWorkspacesLocksEveryCloneOnDisk(t *testing.T) { + r, _, p, _ := workspaceRuntime(t) + root := userSessionWorkspaceRoot() + for _, d := range []string{"acme/widgets", "acme/gadgets", "other/thing", "acme/half" + userSessionWorkspaceCloningSuffix, userSessionWorktreeDirName + "/" + testSessionID} { + if err := os.MkdirAll(filepath.Join(root, d), 0o700); err != nil { + t.Fatal(err) + } + } + if err := os.WriteFile(filepath.Join(root, "acme", "notes.txt"), nil, 0o600); err != nil { + t.Fatal(err) + } + r.SecureWorkspaces() + var got []string + for _, c := range privCallsWithScript(p, userSessionWorkspaceSecureScript) { + if c.user != userSessionRootSpec { + t.Errorf("lock ran as %q", c.user) + } + got = append(got, c.argv[4]+" "+c.argv[6]) + } + want := []string{ + filepath.Join(root, "acme", "gadgets") + " https://github.com/acme/gadgets.git", + filepath.Join(root, "acme", "widgets") + " https://github.com/acme/widgets.git", + filepath.Join(root, "other", "thing") + " https://github.com/other/thing.git", + } + if strings.Join(got, "\n") != strings.Join(want, "\n") { + t.Errorf("locked %v, want %v", got, want) + } + // Those clones are now locked for this process: a start does not lock + // again. + p.calls = nil + if err := r.Start(workspaceSpec(testSessionID, testSessionSlot, "claude")); err != nil { + t.Fatal(err) + } + if n := len(privCallsWithScript(p, userSessionWorkspaceSecureScript)); n != 0 { + t.Errorf("a clone locked at boot was locked again on start (%d)", n) + } +} diff --git a/src/crux/agent/usersession_worktree.go b/src/crux/agent/usersession_worktree.go new file mode 100644 index 0000000000..5d15181050 --- /dev/null +++ b/src/crux/agent/usersession_worktree.go @@ -0,0 +1,423 @@ +package agent + +import ( + "context" + "errors" + "fmt" + "os" + "path/filepath" + "strconv" + "strings" +) + +// Per-session git worktrees of the shared workspace clone, and the lock of +// that clone's git config and hooks (#18588). See usersession_workspace.go for +// the layout; everything here that looks inside the clone or a worktree runs +// as root or as the session uid, never as the hive's own uid, which is +// outside the sessions group. + +// userSessionWorkspaceSecureScript (run as root) locks the shared clone's git +// configuration so no session can plant anything that another session's git, +// or the hive's own worktree git, would run as that other uid: $1 the clone, +// $2 the sessions gid, $3 the origin URL, $4 the session branch directory +// under refs/heads. It runs on every clone and once per clone per hive +// process (a boot fixes a clone from an earlier release), never runs git, and +// writes nothing through a path a session could have planted: +// +// - .git itself is first made group read-only, so nothing in it can be swapped +// while it is rebuilt, and symlinks at its top level are removed; +// - .git/config is rewritten from scratch (whatever a session ever put there, +// a core.fsmonitor, a filter or alias, an include, is dropped) as +// root: 0644 with extensions.worktreeConfig, so personal settings go in +// `git config --worktree` (the session's own config.worktree); +// - the main worktree's config.worktree and a commondir of "." are root-owned +// placeholders, so neither can be planted to redirect git elsewhere; +// - hooks/, info/ and modules/ are emptied and become root 0755; +// - objects/, refs/, logs/ (and the session branch directories) stay setgid +// group-writable so commits, branches and fetches work, and worktrees/ is +// group-writable with the sticky bit, so a session adds its own worktree +// entry but cannot rename or delete another's; +// - every other top-level file becomes root: 0644, and .git (and the +// clone directory above it) end as 3770: the sticky bit stops a session +// from renaming or replacing anything root owns there. +// +// The main worktree's branch keeps its upstream when HEAD names one. gc and +// maintenance are off: they would rewrite root-owned files and fail. +const userSessionWorkspaceSecureScript = `set -e +c=$1; g=$2; url=$3; bp=$4 +gd=$c/.git +[ -d "$c" ] && [ ! -L "$c" ] && [ -d "$gd" ] && [ ! -L "$gd" ] +chown root:"$g" -- "$c" "$gd" +chmod 0750 -- "$gd" +chmod 3770 -- "$c" +find "$gd" -mindepth 1 -maxdepth 1 -type l -exec rm -f -- {} + +[ -f "$gd/HEAD" ] +head=$(sed -n '1s#^ref: refs/heads/\([A-Za-z0-9][A-Za-z0-9._/-]*\)$#\1#p' "$gd/HEAD") +t=$(mktemp "$gd/config.hive.XXXXXX") +cat > "$t" <> "$t"; fi +mv -f -- "$t" "$gd/config" +t=$(mktemp "$gd/config.hive.XXXXXX") +mv -f -- "$t" "$gd/config.worktree" +t=$(mktemp "$gd/config.hive.XXXXXX") +echo . > "$t" +mv -f -- "$t" "$gd/commondir" +for d in hooks info modules; do + rm -rf --one-file-system -- "$gd/$d" + mkdir -m 0755 -- "$gd/$d" +done +for d in objects refs logs worktrees refs/heads logs/refs logs/refs/heads "refs/heads/$bp" "logs/refs/heads/$bp"; do + [ ! -L "$gd/$d" ] + [ -d "$gd/$d" ] || mkdir -- "$gd/$d" + chmod 2770 -- "$gd/$d" +done +chmod 3770 -- "$gd/worktrees" +find "$gd" -mindepth 1 -maxdepth 1 -exec chown -h root:"$g" -- {} + +find "$gd" -mindepth 1 -maxdepth 1 -type f -exec chmod 0644 -- {} + +chown -R root:"$g" -- "$gd/hooks" "$gd/info" "$gd/modules" +chmod 3770 -- "$gd"` + +// userSessionWorktreeOwnScript (run as root) hands the hive-made, empty +// worktree directory to the session: $1 the directory, $2 the session uid, $3 +// the sessions gid, $4 the hive's own uid. Its parents are hive-owned and +// writable by nobody else; a symlink at the name fails closed. A directory the +// session already owns (a resume) is left alone, and one owned by anybody else +// but the hive is refused, so root never hands a directory from one session to +// another. +const userSessionWorktreeOwnScript = `set -e +d=$1; u=$2; g=$3; h=$4 +[ ! -L "$d" ] +[ -d "$d" ] +o=$(stat -c %u -- "$d") +if [ "$o" = "$u" ]; then exit 0; fi +[ "$o" = "$h" ] +chown -- "$u:$g" "$d" +chmod 2770 -- "$d"` + +// userSessionWorktreeGit defines what the worktree scripts share. g is the git +// every one of them runs: the root-owned clone is marked safe for this one +// call (the session's own worktree needs no such mark, so git's ownership +// check still guards it), hooks and core.fsmonitor are off. owned checks, as +// the session uid, that $ws/.git is the session's own gitfile pointing at its +// own entry under the clone's worktrees/ directory, whose commondir leads back +// to the clone, and sets $a to that entry: with the clone's config locked +// (userSessionWorkspaceSecureScript) that is everything git reads config +// from, so nothing another session wrote runs here. The scripts set $clone +// and $ws first. +const userSessionWorktreeGit = `g() { git -c safe.directory="$clone" -c core.hooksPath=/dev/null -c core.fsmonitor=false "$@"; } +owned() { + [ -f "$ws/.git" ] && [ ! -L "$ws/.git" ] && [ -O "$ws/.git" ] || return 1 + a=$(sed -n '1s/^gitdir: //p' "$ws/.git") + cg=$(cd -P -- "$clone/.git" && pwd -P) || return 1 + case $a in "$cg/worktrees/"*/*|*/../*|*/./*) return 1 ;; "$cg/worktrees/"?*) ;; *) return 1 ;; esac + [ -d "$a" ] && [ ! -L "$a" ] && [ -O "$a" ] || return 1 + [ -f "$a/commondir" ] && [ ! -L "$a/commondir" ] && [ -O "$a/commondir" ] || return 1 + [ "$(cat "$a/commondir")" = "../.." ] +}` + +// userSessionWorktreeAddScript (run as the session uid:sessions gid) adds the +// session's worktree of the clone: $1 the clone, $2 the worktree dir, $3 the +// session branch. Every existence check runs here, as the session uid, +// because the hive's own uid can see inside neither the root: 2770 clone +// nor the session's 2770 worktree. A clone without .git is reported, not +// guessed at. A worktree that is already there (a resume, or a start after a +// restart) is left as it is when it is the session's own, and refused +// otherwise. A fresh one is added under umask 022 with +// core.sharedRepository=0644, so its entry under worktrees/ is never +// group-writable (not even between its creation and its checkout), and gets +// an empty config.worktree of its own (git keeps that file's mode when it +// rewrites it). The branch is created at the clone's HEAD, or reused when a +// removal kept it. Stale entries are pruned first so a kept branch whose +// worktree vanished can be checked out again (best effort: another session's +// entry is not ours to prune). +const userSessionWorktreeAddScript = `set -e +clone=$1; ws=$2; b=$3 +` + userSessionWorktreeGit + ` +if [ ! -d "$clone/.git" ]; then echo "` + userSessionWorktreeCloneGoneLine + `"; exit 0; fi +if [ -e "$ws/.git" ] || [ -L "$ws/.git" ]; then + owned || { echo "hive: $ws/.git is not this session's worktree of $clone" >&2; exit 1; } + exit 0 +fi +umask 022 +g -C "$clone" worktree prune 2>/dev/null || true +if g -C "$clone" show-ref --verify --quiet "refs/heads/$b"; then + g -c core.sharedRepository=0644 -C "$clone" worktree add "$ws" "$b" +else + g -c core.sharedRepository=0644 -C "$clone" worktree add -b "$b" "$ws" HEAD +fi +owned +: > "$a/config.worktree" +echo "` + userSessionWorktreeAddedLine + `"` + +// userSessionWorktreeRemoveScript (run as the worktree's owner uid:sessions +// gid) removes a session's worktree without losing work: $1 the clone, $2 the +// worktree, $3 the session branch. Uncommitted changes (untracked files +// included, ignored ones not) are first committed onto whatever the worktree +// has checked out as an auto-save, a detached HEAD is saved as +// -detached, then the worktree's entry under the clone's worktrees/ +// is deleted and the clone pruned; the hive deletes the worktree directory +// itself afterwards (its parent is the hive's, so the session uid cannot). +// The session branches are deleted only when the clone's HEAD already +// contains them; otherwise they are kept and named on stdout. A failing git +// aborts before anything is removed, and so does a worktree whose .git is not +// the session's own. When the clone itself is gone there is no git to save +// through: the script says so and does nothing. +const userSessionWorktreeRemoveScript = `set -e +clone=$1; ws=$2; b=$3 +` + userSessionWorktreeGit + ` +if [ ! -d "$clone/.git" ]; then echo "` + userSessionWorktreeCloneGoneLine + `"; exit 0; fi +if [ -e "$ws/.git" ] || [ -L "$ws/.git" ]; then + owned || { echo "hive: $ws/.git is not this session's worktree of $clone; its work is not saved and nothing is removed" >&2; exit 1; } + if [ -n "$(g -C "$ws" status --porcelain)" ]; then + g -C "$ws" add -A + g -C "$ws" -c user.name=Hive -c user.email=hive@localhost -c commit.gpgsign=false commit -q --no-verify -m "` + userSessionWorktreeAutoSaveMessage + `" + echo "auto-saved uncommitted work" + fi + if ! g -C "$ws" symbolic-ref -q HEAD >/dev/null; then + g -C "$ws" branch -f "$b-detached" HEAD + fi + rm -rf -- "$a" +fi +g -C "$clone" worktree prune 2>/dev/null || true +for x in "$b" "$b-detached"; do + if g -C "$clone" show-ref --verify --quiet "refs/heads/$x"; then + if g -C "$clone" merge-base --is-ancestor "$x" HEAD; then + g -C "$clone" branch -D "$x" >/dev/null 2>&1 + else + echo "kept branch $x" + fi + fi +done` + +// userSessionWorktreesDir holds every worktree of one session. +func userSessionWorktreesDir(id string) string { + return filepath.Join(userSessionWorkspaceRoot(), userSessionWorktreeDirName, id) +} + +// userSessionWorktreeDir is the session's own worktree of /. +func userSessionWorktreeDir(id, org, name string) string { + return filepath.Join(userSessionWorktreesDir(id), org, name) +} + +// userSessionWorktreeBranch is the branch a session's worktree starts on. +func userSessionWorktreeBranch(id string) string { return userSessionWorktreeBranchPrefix + id } + +// SessionWorkspaceDir is the path session id opens in for "org/repo": its own +// worktree of the shared clone. "" when the id or repo is empty or malformed. +// Start creates it. +func (r *UserSessionRuntime) SessionWorkspaceDir(id, repo string) string { + if checkUserSessionID(id) != nil || strings.TrimSpace(repo) == "" { + return "" + } + org, name, err := splitUserSessionRepo(repo) + if err != nil { + return "" + } + return userSessionWorktreeDir(id, org, name) +} + +// secureClone has root lock the shared clone's git config and hooks +// (userSessionWorkspaceSecureScript) and records it for this process. The +// caller holds workspaceMu. +func (r *UserSessionRuntime) secureClone(org, name, clone string) error { + url := "https://github.com/" + org + "/" + name + ".git" + if out, err := r.privRunFor(userSessionWorkspaceHandoffTimeout, userSessionRootSpec, nil, "sh", "-c", userSessionWorkspaceSecureScript, "sh", clone, strconv.Itoa(userSessionGID()), url, userSessionWorktreeBranchDir); err != nil { + return outputErr("locking "+clone, err, out) + } + if r.securedClones == nil { + r.securedClones = map[string]bool{} + } + r.securedClones[clone] = true + return nil +} + +// SecureWorkspaces locks the git config and hooks of every shared clone +// already on disk (userSessionWorkspaceSecureScript), so a clone made by an +// earlier release, whose config and hooks every session could write, is fixed +// at boot rather than on its next session start. Failures are logged; the next +// start of a session in that clone retries and fails closed. +func (r *UserSessionRuntime) SecureWorkspaces() { + r.workspaceMu.Lock() + defer r.workspaceMu.Unlock() + root := userSessionWorkspaceRoot() + orgs, err := os.ReadDir(root) + if err != nil { + if !errors.Is(err, os.ErrNotExist) { + r.logger.Warn("session workspace: listing shared clones to lock failed", "path", root, "error", err) + } + return + } + for _, o := range orgs { + org := o.Name() + // The pattern never admits a leading dot, so the worktrees dir + // (userSessionWorktreeDirName) is skipped here. + if !o.IsDir() || !userSessionRepoPartPattern.MatchString(org) { + continue + } + repos, err := os.ReadDir(filepath.Join(root, org)) + if err != nil { + r.logger.Warn("session workspace: listing shared clones to lock failed", "path", filepath.Join(root, org), "error", err) + continue + } + for _, e := range repos { + name := e.Name() + if !e.IsDir() || strings.HasSuffix(name, userSessionWorkspaceCloningSuffix) || !userSessionRepoPartPattern.MatchString(name) { + continue + } + clone := userSessionWorkspaceDir(org, name) + if err := r.secureClone(org, name, clone); err != nil { + r.logger.Warn("session workspace: locking a shared clone's git config failed; sessions in it fail to start until it succeeds", "path", clone, "error", err) + continue + } + r.logger.Info("session workspace: shared clone's git config and hooks locked", "path", clone) + } + } +} + +// ensureWorktree returns the session's worktree of clone, adding it when it +// does not exist yet: the hive makes the empty directory under hive-owned, +// traverse-only parents, root hands it to the session uid, and the session +// uid runs `git worktree add` (userSessionWorktreeAddScript). Whether the +// worktree is already there is decided by that script, as the session uid: +// the hive's own uid cannot look inside the session's 2770 directory. No +// second clone is ever made. The caller holds workspaceMu. +func (r *UserSessionRuntime) ensureWorktree(id string, uid int, org, name, clone string) (string, error) { + ws := userSessionWorktreeDir(id, org, name) + for _, dir := range []string{filepath.Dir(userSessionWorktreesDir(id)), userSessionWorktreesDir(id), filepath.Dir(ws)} { + if err := os.MkdirAll(dir, userSessionWorkspaceParentMode); err != nil { + return "", fmt.Errorf("creating %s: %w", dir, err) + } + if err := os.Chmod(dir, userSessionWorkspaceParentMode); err != nil { + return "", fmt.Errorf("securing %s: %w", dir, err) + } + } + if err := os.Mkdir(ws, userSessionWorktreeStagingMode); err != nil && !errors.Is(err, os.ErrExist) { + return "", fmt.Errorf("creating %s: %w", ws, err) + } + if out, err := r.privRun(userSessionRootSpec, nil, "sh", "-c", userSessionWorktreeOwnScript, "sh", ws, strconv.Itoa(uid), strconv.Itoa(userSessionGID()), strconv.Itoa(os.Getuid())); err != nil { + return "", outputErr("handing the worktree to the session", err, out) + } + ctx, cancel := context.WithTimeout(context.Background(), userSessionWorktreeTimeout) + defer cancel() + out, err := r.priv(ctx, userSessionSpec(uid), nil, "sh", "-c", userSessionWorktreeAddScript, "sh", clone, ws, userSessionWorktreeBranch(id)) + if err != nil { + return "", outputErr("git worktree add", err, out) + } + if strings.Contains(string(out), userSessionWorktreeCloneGoneLine) { + return "", fmt.Errorf("the shared clone %s has no .git; remove that directory so the next start clones it again", clone) + } + if !strings.Contains(string(out), userSessionWorktreeAddedLine) { + // The session's own worktree was already there (a resume). + return ws, nil + } + r.logger.Info("session worktree added", "id", id, "repo", org+"/"+name, "path", ws, "branch", userSessionWorktreeBranch(id)) + // A fresh worktree has none of spek's files, whatever an earlier one had. + if err := os.RemoveAll(userSessionSpekInitMarkerDir(id)); err != nil { + r.logger.Warn("session worktree: clearing stale spektacular init markers failed", "id", id, "error", err) + } + return ws, nil +} + +// removeWorktrees removes every worktree session id has (see +// userSessionWorktreeRemoveScript), then its whole worktrees directory and +// its spek init markers. A worktree that was never handed to a session uid +// holds no work and goes with the directory. A failure is returned before +// anything of that worktree is deleted, so the caller keeps the session and +// retries. The caller holds workspaceMu. +func (r *UserSessionRuntime) removeWorktrees(id string) error { + dir := userSessionWorktreesDir(id) + if _, err := os.Lstat(dir); err != nil { + if errors.Is(err, os.ErrNotExist) { + return os.RemoveAll(userSessionSpekInitMarkerDir(id)) + } + return err + } + matches, err := filepath.Glob(filepath.Join(dir, "*", "*")) + if err != nil { + return err + } + for _, ws := range matches { + org, name := filepath.Base(filepath.Dir(ws)), filepath.Base(ws) + if !userSessionRepoPartPattern.MatchString(org) || !userSessionRepoPartPattern.MatchString(name) { + continue + } + if err := r.removeWorktree(id, org, name, ws); err != nil { + return err + } + } + if out, err := r.privRunFor(userSessionWorktreeTimeout, userSessionRootSpec, nil, "rm", "-rf", "--one-file-system", "--", dir); err != nil { + return outputErr("removing session worktrees dir", err, out) + } + return os.RemoveAll(userSessionSpekInitMarkerDir(id)) +} + +// removeWorktree runs userSessionWorktreeRemoveScript for one worktree as the +// uid that owns it, and logs what was kept. +func (r *UserSessionRuntime) removeWorktree(id, org, name, ws string) error { + uid, err := r.statUID(ws) + if err != nil || !userSessionUIDInRange(uid) { + return nil + } + repo := org + "/" + name + clone := userSessionWorkspaceDir(org, name) + ctx, cancel := context.WithTimeout(context.Background(), userSessionWorktreeTimeout) + defer cancel() + out, err := r.priv(ctx, userSessionSpec(uid), nil, "sh", "-c", userSessionWorktreeRemoveScript, "sh", clone, ws, userSessionWorktreeBranch(id)) + if err != nil { + return outputErr("removing session worktree", err, out) + } + if strings.Contains(string(out), userSessionWorktreeCloneGoneLine) { + r.logger.Warn("session worktree: the shared clone is gone, so its worktree is deleted without git; anything uncommitted in it is lost", "id", id, "repo", repo, "path", ws) + return nil + } + if kept := strings.TrimSpace(string(out)); kept != "" { + r.logger.Info("session worktree removed; its work stays on a branch of the shared clone", "id", id, "repo", repo, "clone", clone, "detail", strings.ReplaceAll(kept, "\n", "; ")) + } else { + r.logger.Info("session worktree removed", "id", id, "repo", repo) + } + return nil +} + +// SweepWorktrees removes the worktrees of every session live reports gone +// (its record no longer exists), as Remove would. It runs under workspaceMu, +// so a session whose record is saved before its start can never lose a +// worktree it is still being given. Failures are logged and retried by the +// next sweep. +func (r *UserSessionRuntime) SweepWorktrees(live func(id string) bool) { + r.workspaceMu.Lock() + defer r.workspaceMu.Unlock() + entries, err := os.ReadDir(filepath.Join(userSessionWorkspaceRoot(), userSessionWorktreeDirName)) + if err != nil { + if !errors.Is(err, os.ErrNotExist) { + r.logger.Warn("session worktree sweep: listing worktrees failed", "error", err) + } + return + } + for _, e := range entries { + id := e.Name() + if checkUserSessionID(id) != nil || live(id) { + continue + } + if err := r.removeWorktrees(id); err != nil { + r.logger.Warn("session worktree sweep: removing an orphaned worktree failed; retried next sweep", "id", id, "error", err) + continue + } + r.logger.Info("session worktree sweep: removed the worktree of a deleted session", "id", id) + } +} diff --git a/src/crux/agent/usersession_worktree_test.go b/src/crux/agent/usersession_worktree_test.go new file mode 100644 index 0000000000..e0548680eb --- /dev/null +++ b/src/crux/agent/usersession_worktree_test.go @@ -0,0 +1,467 @@ +package agent + +import ( + "errors" + "os" + "os/exec" + "path/filepath" + "runtime" + "strconv" + "strings" + "testing" + + "github.com/hivecommons/hive/crux" +) + +// startTwoWorkspaceSessions starts two sessions on the same workspace repo and +// returns their worktrees. +func startTwoWorkspaceSessions(t *testing.T, r *UserSessionRuntime) (string, string) { + t.Helper() + for _, s := range []crux.UserSessionSpec{workspaceSpec(testSessionID, testSessionSlot, "claude"), workspaceSpec(testSecondSessionID, testSecondSlot, "claude")} { + if err := r.Start(s); err != nil { + t.Fatal(err) + } + } + return r.SessionWorkspaceDir(testSessionID, testWorkspaceRepo), r.SessionWorkspaceDir(testSecondSessionID, testWorkspaceRepo) +} + +// Deleting a session removes its own worktree (as its uid, through git, with +// the session's branch) and nothing of another session's. +func TestUserSessionRemoveRemovesOnlyItsOwnWorktree(t *testing.T) { + r, _, p, _ := workspaceRuntime(t) + ws1, ws2 := startTwoWorkspaceSessions(t, r) + p.calls = nil + if err := r.Remove(testSessionID); err != nil { + t.Fatal(err) + } + removes := privCallsWithScript(p, userSessionWorktreeRemoveScript) + if len(removes) != 1 { + t.Fatalf("worktree remove ran %d times, want once: %+v", len(removes), removes) + } + c := removes[0] + if c.user != testSessionSpecUser || c.argv[4] != r.WorkspaceDir(testWorkspaceRepo) || c.argv[5] != ws1 || c.argv[6] != userSessionWorktreeBranch(testSessionID) { + t.Errorf("worktree remove = %s %v, want the session uid removing %s from the clone", c.user, c.argv[4:], ws1) + } + for _, gone := range []string{ws1, userSessionWorktreesDir(testSessionID), userSessionSpekInitMarkerDir(testSessionID), userSessionSandboxDir(testSessionID)} { + if _, err := os.Lstat(gone); err == nil { + t.Errorf("%s survived Remove", gone) + } + } + if _, err := os.Stat(filepath.Join(ws2, ".git")); err != nil { + t.Errorf("the other session's worktree was touched: %v", err) + } + if _, err := os.Stat(userSessionSpekInitMarker(testSecondSessionID, "claude")); err != nil { + t.Errorf("the other session's init marker was touched: %v", err) + } +} + +// A worktree that cannot be removed safely (git failed, so its work was not +// saved) keeps the session: nothing of it is deleted and Remove fails so the +// caller keeps the record and retries. +func TestUserSessionRemoveKeepsEverythingWhenTheWorktreeCannotBeSaved(t *testing.T) { + r, _, p, _ := workspaceRuntime(t) + ws1, _ := startTwoWorkspaceSessions(t, r) + inner := p.handler + p.handler = func(c fakePrivCall) (bool, []byte, error) { + if len(c.argv) > 2 && c.argv[2] == userSessionWorktreeRemoveScript { + return true, []byte("fatal: unable to write new index file"), errors.New("exit 128") + } + return inner(c) + } + err := r.Remove(testSessionID) + if err == nil || !strings.Contains(err.Error(), "unable to write new index file") { + t.Fatalf("Remove = %v, want the worktree failure with git's output", err) + } + for _, kept := range []string{filepath.Join(ws1, ".git"), userSessionSandboxDir(testSessionID)} { + if _, err := os.Lstat(kept); err != nil { + t.Errorf("%s was deleted although the worktree could not be saved: %v", kept, err) + } + } +} + +// The sweep removes only worktrees whose session is gone, and leaves anything +// under the worktrees dir that is not a session id alone. +func TestUserSessionSweepWorktreesKeepsLiveSessions(t *testing.T) { + r, _, p, _ := workspaceRuntime(t) + ws1, ws2 := startTwoWorkspaceSessions(t, r) + stray := filepath.Join(userSessionWorkspaceRoot(), userSessionWorktreeDirName, "not-a-session") + if err := os.MkdirAll(stray, 0o700); err != nil { + t.Fatal(err) + } + p.calls = nil + var asked []string + r.SweepWorktrees(func(id string) bool { + asked = append(asked, id) + return id == testSessionID + }) + if len(asked) != 2 { + t.Errorf("liveness asked for %v, want both session ids and nothing else", asked) + } + removes := privCallsWithScript(p, userSessionWorktreeRemoveScript) + if len(removes) != 1 || removes[0].argv[5] != ws2 { + t.Fatalf("sweep removed %+v, want only %s", removes, ws2) + } + if _, err := os.Stat(filepath.Join(ws1, ".git")); err != nil { + t.Errorf("a live session's worktree was swept: %v", err) + } + for _, gone := range []string{ws2, userSessionWorktreesDir(testSecondSessionID)} { + if _, err := os.Lstat(gone); err == nil { + t.Errorf("%s survived the sweep", gone) + } + } + if _, err := os.Stat(stray); err != nil { + t.Errorf("a non-session entry was swept: %v", err) + } + // Nothing left to sweep: no git runs at all. + p.calls = nil + r.SweepWorktrees(func(id string) bool { return id == testSessionID }) + if n := len(privCallsWithScript(p, userSessionWorktreeRemoveScript)); n != 0 { + t.Errorf("second sweep ran %d worktree removals", n) + } +} + +// realGitWorkspace is a shared clone, made the way the hive makes one (a +// clone with core.sharedRepository=group) and then locked by +// userSessionWorkspaceSecureScript, all as the test's own uid. Root's chown is +// replaced by a no-op on PATH, since the test cannot change owners: everything +// else the lock script does (rewriting the config, emptying hooks, the mode +// bits) runs for real. It returns the clone, a git helper, a script runner and +// the environment both use. +type realGitWorkspace struct { + dir, clone string + env []string + t *testing.T +} + +func newRealGitWorkspace(t *testing.T) *realGitWorkspace { + t.Helper() + if runtime.GOOS != "linux" { + t.Skip("the workspace scripts target the Linux image (GNU coreutils)") + } + if _, err := exec.LookPath("git"); err != nil { + t.Skip("no git") + } + dir := t.TempDir() + shim := filepath.Join(dir, "bin") + if err := os.MkdirAll(shim, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(shim, "chown"), []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil { + t.Fatal(err) + } + w := &realGitWorkspace{dir: dir, clone: filepath.Join(dir, "clone"), t: t, + env: append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null", "GIT_CONFIG_NOSYSTEM=1", "HOME="+dir, "PATH="+shim+":"+os.Getenv("PATH"))} + origin := filepath.Join(dir, "origin") + w.git(dir, "init", "-q", "-b", "main", origin) + if err := os.WriteFile(filepath.Join(origin, "README"), []byte("hi\n"), 0o644); err != nil { + t.Fatal(err) + } + w.git(origin, "add", "README") + w.git(origin, "commit", "-qm", "init") + w.git(dir, append(append([]string{}, userSessionHiveGitArgs...), "clone", "-q", "-c", "core.sharedRepository=group", "--", origin, w.clone)...) + // What a session could plant while the config and hooks were + // group-writable (before this release): all of it must go. + gd := filepath.Join(w.clone, ".git") + w.git(w.clone, "config", "core.fsmonitor", "touch "+filepath.Join(dir, "pwned-fsmonitor")) + w.git(w.clone, "config", "alias.st", "!touch "+filepath.Join(dir, "pwned-alias")) + if err := os.WriteFile(filepath.Join(gd, "hooks", "post-checkout"), []byte("#!/bin/sh\ntouch "+filepath.Join(dir, "pwned-hook")+"\n"), 0o775); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(gd, "info", "attributes"), []byte("* filter=evil\n"), 0o664); err != nil { + t.Fatal(err) + } + w.script(userSessionWorkspaceSecureScript, w.clone, strconv.Itoa(os.Getgid()), "https://github.com/acme/widgets.git", userSessionWorktreeBranchDir) + return w +} + +func (w *realGitWorkspace) git(in string, args ...string) string { + w.t.Helper() + out, err := w.gitErr(in, args...) + if err != nil { + w.t.Fatalf("git %v: %v: %s", args, err, out) + } + return out +} + +func (w *realGitWorkspace) gitErr(in string, args ...string) (string, error) { + cmd := exec.Command("git", append([]string{"-c", "user.name=t", "-c", "user.email=t@example.com"}, args...)...) + cmd.Dir, cmd.Env = in, w.env + out, err := cmd.CombinedOutput() + return strings.TrimSpace(string(out)), err +} + +func (w *realGitWorkspace) scriptErr(s string, args ...string) (string, error) { + cmd := exec.Command("sh", append([]string{"-c", s, "sh"}, args...)...) + cmd.Env = w.env + out, err := cmd.CombinedOutput() + return string(out), err +} + +func (w *realGitWorkspace) script(s string, args ...string) string { + w.t.Helper() + out, err := w.scriptErr(s, args...) + if err != nil { + w.t.Fatalf("script %v: %v: %s", args, err, out) + } + return out +} + +// add makes the session's worktree dir and runs the add script, as the hive +// would (minus root's hand-over, which only changes the owner). +func (w *realGitWorkspace) add(name, branch string) (string, string) { + w.t.Helper() + ws := filepath.Join(w.dir, "wt", name) + if err := os.MkdirAll(ws, 0o700); err != nil { + w.t.Fatal(err) + } + return ws, w.script(userSessionWorktreeAddScript, w.clone, ws, branch) +} + +// remove runs the remove script and then deletes the worktree dir, as root's +// rm -rf in removeWorktrees does. +func (w *realGitWorkspace) remove(ws, branch string) string { + w.t.Helper() + out := w.script(userSessionWorktreeRemoveScript, w.clone, ws, branch) + if err := os.RemoveAll(ws); err != nil { + w.t.Fatal(err) + } + return out +} + +func modeOf(t *testing.T, path string) os.FileMode { + t.Helper() + fi, err := os.Lstat(path) + if err != nil { + t.Fatal(err) + } + return fi.Mode() +} + +// The lock (the owner's chosen design for #18588): the shared clone's config, +// hooks, info and modules can be written by root alone, while objects, refs +// and logs stay group-writable for commits and fetches. Whatever a session +// planted before is gone, and a session's own settings live in its worktree's +// config.worktree. +func TestUserSessionWorkspaceLockOnRealGit(t *testing.T) { + w := newRealGitWorkspace(t) + gd := filepath.Join(w.clone, ".git") + const groupWrite = 0o020 + for _, f := range []string{"config", "config.worktree", "commondir", "HEAD", "packed-refs"} { + p := filepath.Join(gd, f) + if _, err := os.Lstat(p); errors.Is(err, os.ErrNotExist) && f == "packed-refs" { + continue + } + if m := modeOf(t, p); !m.IsRegular() || m.Perm() != 0o644 { + t.Errorf(".git/%s is %v, want a regular file 0644 (not group-writable)", f, m) + } + } + for _, d := range []string{"hooks", "info", "modules"} { + m := modeOf(t, filepath.Join(gd, d)) + if !m.IsDir() || m.Perm()&groupWrite != 0 || m.Perm() != 0o755 { + t.Errorf(".git/%s is %v, want a 0755 directory", d, m) + } + if entries, err := os.ReadDir(filepath.Join(gd, d)); err != nil || len(entries) != 0 { + t.Errorf(".git/%s still holds %v (%v)", d, entries, err) + } + } + for _, d := range []string{"objects", "refs", "logs", "refs/heads/" + userSessionWorktreeBranchDir, "logs/refs/heads/" + userSessionWorktreeBranchDir} { + if m := modeOf(t, filepath.Join(gd, d)); !m.IsDir() || m.Perm() != 0o770 || m&os.ModeSetgid == 0 { + t.Errorf(".git/%s is %v, want a setgid group-writable directory", d, m) + } + } + for _, d := range []string{gd, filepath.Join(gd, "worktrees"), w.clone} { + if m := modeOf(t, d); m.Perm() != 0o770 || m&os.ModeSticky == 0 || m&os.ModeSetgid == 0 { + t.Errorf("%s is %v, want 3770: group-writable, but only an entry's owner may rename or delete it", d, m) + } + } + cfg, err := os.ReadFile(filepath.Join(gd, "config")) + if err != nil { + t.Fatal(err) + } + for _, bad := range []string{"fsmonitor", "alias", "pwned"} { + if strings.Contains(string(cfg), bad) { + t.Errorf("the rewritten config kept a planted %q:\n%s", bad, cfg) + } + } + if got := w.git(w.clone, "-c", "safe.directory=*", "config", "extensions.worktreeConfig"); got != "true" { + t.Errorf("extensions.worktreeConfig = %q", got) + } + if got := w.git(w.clone, "-c", "safe.directory=*", "config", "remote.origin.url"); got != "https://github.com/acme/widgets.git" { + t.Errorf("origin = %q, want the plain https URL", got) + } + if got := w.git(w.clone, "-c", "safe.directory=*", "config", "branch.main.merge"); got != "refs/heads/main" { + t.Errorf("branch.main.merge = %q", got) + } + wantCommon, err := filepath.EvalSymlinks(gd) + if err != nil { + t.Fatal(err) + } + if got, err := filepath.EvalSymlinks(w.git(w.clone, "-c", "safe.directory=*", "rev-parse", "--path-format=absolute", "--git-common-dir")); err != nil || got != wantCommon { + t.Errorf("the placeholder commondir moved the common dir to %q (%v), want %q", got, err, wantCommon) + } + // Locking twice (every boot) is a no-op. + w.script(userSessionWorkspaceSecureScript, w.clone, strconv.Itoa(os.Getgid()), "https://github.com/acme/widgets.git", userSessionWorktreeBranchDir) + + // git works under the lock: add, commit, branch, worktree config. + ws, out := w.add("work", "hive-session/aaaa") + if !strings.Contains(out, userSessionWorktreeAddedLine) { + t.Fatalf("add reported %q", out) + } + if err := os.WriteFile(filepath.Join(ws, "f"), []byte("x\n"), 0o644); err != nil { + t.Fatal(err) + } + w.git(ws, "add", "f") + w.git(ws, "commit", "-qm", "work") + w.git(ws, "branch", "scratch") + w.git(ws, "branch", "-D", "scratch") + w.git(ws, "config", "--worktree", "user.name", "Session Person") + if got := w.git(ws, "config", "--worktree", "--get", "user.name"); got != "Session Person" { + t.Errorf("worktree config user.name = %q", got) + } + entry := w.git(ws, "rev-parse", "--absolute-git-dir") + for _, f := range []string{"config.worktree", "commondir", "gitdir"} { + if m := modeOf(t, filepath.Join(entry, f)); m.Perm()&groupWrite != 0 { + t.Errorf("worktree entry %s is %v: another session could redirect or configure this one", f, m) + } + } + if m := modeOf(t, entry); m.Perm()&groupWrite != 0 { + t.Errorf("worktree entry %s is %v: another session could plant files in it", entry, m) + } + if strings.TrimSpace(w.git(ws, "log", "-1", "--format=%s")) != "work" { + t.Error("commit in the worktree did not land") + } + for _, f := range []string{"pwned-fsmonitor", "pwned-alias", "pwned-hook"} { + if _, err := os.Stat(filepath.Join(w.dir, f)); err == nil { + t.Errorf("a planted command ran (%s)", f) + } + } +} + +// The worktree scripts against a real git under the lock (as the test's own +// uid): a resume is a no-op that keeps the work, a clean session leaves +// nothing behind, uncommitted and detached work is auto-saved on a branch of +// the clone, a kept branch is checked out again by a later add, a missing +// clone is reported, and a worktree whose .git points anywhere else is +// refused without touching anything. +func TestUserSessionWorktreeScriptsOnRealGit(t *testing.T) { + w := newRealGitWorkspace(t) + worktrees := func() int { + return len(strings.Split(w.git(w.clone, "-c", "safe.directory=*", "worktree", "list", "--porcelain"), "\n\n")) + } + inClone := func(args ...string) string { + return w.git(w.clone, append([]string{"-c", "safe.directory=*"}, args...)...) + } + + // Resume: re-adding an existing worktree (which the hive's own uid + // cannot see inside) is a no-op that keeps the work and reports no add. + ws, _ := w.add("resume", "hive-session/rrrr") + if err := os.WriteFile(filepath.Join(ws, "wip.md"), []byte("wip\n"), 0o644); err != nil { + t.Fatal(err) + } + if _, out := w.add("resume", "hive-session/rrrr"); strings.Contains(out, userSessionWorktreeAddedLine) { + t.Errorf("re-adding an existing worktree reported a fresh add: %q", out) + } + if b, err := os.ReadFile(filepath.Join(ws, "wip.md")); err != nil || string(b) != "wip\n" { + t.Errorf("re-adding an existing worktree touched its work: %q, %v", b, err) + } + w.remove(ws, "hive-session/rrrr") + inClone("branch", "-D", "hive-session/rrrr") + + // A missing clone: both scripts say so and touch nothing. + orphan := filepath.Join(w.dir, "orphan") + if err := os.MkdirAll(orphan, 0o700); err != nil { + t.Fatal(err) + } + for _, s := range []string{userSessionWorktreeRemoveScript, userSessionWorktreeAddScript} { + if out := w.script(s, filepath.Join(w.dir, "no-clone"), orphan, "hive-session/oooo"); !strings.Contains(out, userSessionWorktreeCloneGoneLine) { + t.Errorf("script against a missing clone reported %q", out) + } + } + if entries, err := os.ReadDir(orphan); err != nil || len(entries) != 0 { + t.Errorf("a script against a missing clone changed the worktree dir: %v, %v", entries, err) + } + + // Clean session: worktree and branch go, nothing is reported kept. + ws, _ = w.add("clean", "hive-session/aaaa") + if got := w.git(ws, "rev-parse", "--abbrev-ref", "HEAD"); got != "hive-session/aaaa" { + t.Fatalf("worktree is on %q, want its session branch", got) + } + if out := w.remove(ws, "hive-session/aaaa"); strings.TrimSpace(out) != "" { + t.Errorf("clean removal reported %q", out) + } + if worktrees() != 1 || inClone("branch", "--list", "hive-session/*") != "" { + t.Errorf("clean removal left a worktree or branch behind: %s", inClone("worktree", "list")) + } + + // Dirty session: tracked and untracked changes are auto-saved onto its + // branch, which is kept. + ws, _ = w.add("dirty", "hive-session/bbbb") + if err := os.WriteFile(filepath.Join(ws, "spec.md"), []byte("draft\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(ws, "README"), []byte("edited\n"), 0o644); err != nil { + t.Fatal(err) + } + out := w.remove(ws, "hive-session/bbbb") + if !strings.Contains(out, "auto-saved uncommitted work") || !strings.Contains(out, "kept branch hive-session/bbbb") { + t.Errorf("dirty removal reported %q", out) + } + if worktrees() != 1 { + t.Error("dirty worktree's entry not removed after its work was saved") + } + if got := inClone("show", "hive-session/bbbb:spec.md"); got != "draft" { + t.Errorf("untracked file on the kept branch = %q", got) + } + if got := inClone("show", "hive-session/bbbb:README"); got != "edited" { + t.Errorf("tracked edit on the kept branch = %q", got) + } + if got := inClone("log", "-1", "--format=%s", "hive-session/bbbb"); got != userSessionWorktreeAutoSaveMessage { + t.Errorf("auto-save commit message = %q", got) + } + + // A later worktree for the same branch checks the kept work out again. + ws, _ = w.add("again", "hive-session/bbbb") + if b, err := os.ReadFile(filepath.Join(ws, "spec.md")); err != nil || string(b) != "draft\n" { + t.Errorf("re-added worktree lacks the kept work: %q, %v", b, err) + } + w.remove(ws, "hive-session/bbbb") + + // Detached HEAD with a commit: saved as -detached; the session + // branch itself held nothing and goes. + ws, _ = w.add("detached", "hive-session/cccc") + w.git(ws, "checkout", "-q", "--detach") + if err := os.WriteFile(filepath.Join(ws, "detached.md"), []byte("x\n"), 0o644); err != nil { + t.Fatal(err) + } + w.git(ws, "add", "detached.md") + w.git(ws, "commit", "-qm", "detached work") + out = w.remove(ws, "hive-session/cccc") + if !strings.Contains(out, "kept branch hive-session/cccc-detached") || strings.Contains(out, "kept branch hive-session/cccc\n") { + t.Errorf("detached removal reported %q", out) + } + if got := inClone("show", "hive-session/cccc-detached:detached.md"); got != "x" { + t.Errorf("detached commit not kept: %q", got) + } + if inClone("branch", "--list", "hive-session/cccc") != "" { + t.Error("an empty session branch was kept") + } + + // A worktree whose .git was swapped for one pointing elsewhere (the + // worktree dir is group-writable): neither script runs git there, and + // the removal fails without deleting anything. + ws, _ = w.add("swapped", "hive-session/dddd") + evil := filepath.Join(w.dir, "evil") + w.git(w.dir, "init", "-q", "--bare", evil) + if err := os.WriteFile(filepath.Join(ws, ".git"), []byte("gitdir: "+evil+"\n"), 0o644); err != nil { + t.Fatal(err) + } + if out, err := w.scriptErr(userSessionWorktreeAddScript, w.clone, ws, "hive-session/dddd"); err == nil { + t.Errorf("resume into a swapped .git succeeded: %q", out) + } + if out, err := w.scriptErr(userSessionWorktreeRemoveScript, w.clone, ws, "hive-session/dddd"); err == nil || !strings.Contains(out, "not this session's worktree") { + t.Errorf("removal of a swapped .git = %q, %v; want a refusal", out, err) + } + if _, err := os.Stat(filepath.Join(ws, "README")); err != nil { + t.Errorf("a refused removal deleted the worktree's files: %v", err) + } +} diff --git a/src/docs/agent-configuration.md b/src/docs/agent-configuration.md index 0966cf2aea..9f0d23ce85 100644 --- a/src/docs/agent-configuration.md +++ b/src/docs/agent-configuration.md @@ -489,6 +489,8 @@ Rounding out the schema — fields you will rarely touch: | `stats_display` | Custom sidebar metrics (key, label, source, field, style, optional icon/target). The Diagnostics **Quality stats** card mirrors the `quality` agent's configured entries. A `pct`/`pct-bar` stat with no measurement renders `—`, not `0%`. | none (an agent starts with no stats unless it is a built-in with defaults) | | `hidden` (packs only) | Keep a pack agent out of the default roster view | false | +If a kick to an agent fails or an agent is flagged `kick_refusal`, see [Kick refusals](kick-refusals.md). + ## Explain mode (debugging agent behaviour) Agents are told to act, not narrate. Every policy carries an "Output Rules — Terse Mode" block, and on inference backends the agent manager appends an explicit `EXECUTE, DO NOT NARRATE` instruction to each kick. That rule earns its keep — weak models otherwise answer a kick with a plan for someone else to run instead of running it — but it also means that when an agent does the wrong thing, there is nothing in the log saying *why*. @@ -1389,11 +1391,11 @@ How it works and what it never does: - **Per-repo status.** Settings shows, on each write entry, which identity will actually post and why: `posts as hivecommons-hive[bot]: App installed on `, `posts as : App not installed on `, `read-only: connect your GitHub account`, or `read-only: bot only, App not installed on `. A change that would leave a hand-written write entry without a credential is refused before it is saved, with the error boot would stop on. - **Upgrade.** Hives that never set the key read as `bot`. Where the App is not installed, nothing changes. A granted repo whose owner installed the App now gets posts as the bot instead of the owner. Choose *Me* to keep the previous behaviour everywhere. -The own-item follow-up below reads under the owner token, so for now it follows only the items opened as the owner. +The own-item follow-up below follows the items opened under either identity, each repo read with its own credential. **Fork-and-PR.** The owner's account cannot push to a repository it does not own, so a PR against a remote repo goes through the owner's fork: `gh repo fork / --clone=false`, push the topic branch to `https:////`, then `gh pr create --repo / --head :`. The proxy allows exactly that (fork creation, pushes to the fork with its default branch protected, the PR against the remote) and refuses everything else on the fork. The kick spells the commands out, so `git push` and `gh pr create` need no agent-side special casing. -**Own-item follow-up.** Remote repos send no webhooks and are not enumerated, so the hive polls them: it lists the open issues and PRs created by the owner login, keeps the ones whose body carries a hive marker (the `— hive:` attribution trailer or the `🐝 **Hive Agent**` footer), fetches new comments, review comments and reviews, and delivers them to the agent the marker names, into its live session or as a `🌐 REMOTE ITEM FOLLOW-UP` section on its next kick. State is in `/data/turn/remote-items.json` (`HIVE_REMOTE_ITEMS_FILE` overrides). Nothing else happens on the remote: no backlog scan, no queue, no stage engine ([#18199](https://github.com/hivecommons/hive/issues/18199)). +**Own-item follow-up.** Remote repos send no webhooks and are not enumerated, so the hive polls them: it lists the open issues and PRs created by the login each repo writes as (the owner's, or the App bot's on a repo that writes as the bot, [#18516](https://github.com/hivecommons/hive/issues/18516)), reading each repo under its own reference-repo credential, keeps the ones whose body carries a hive marker (the `— hive:` attribution trailer or the `🐝 **Hive Agent**` footer), fetches new comments, review comments and reviews, and delivers them to the agent the marker names, into its live session or as a `🌐 REMOTE ITEM FOLLOW-UP` section on its next kick. State is in `/data/turn/remote-items.json` (`HIVE_REMOTE_ITEMS_FILE` overrides). Nothing else happens on the remote: no backlog scan, no queue, no stage engine ([#18199](https://github.com/hivecommons/hive/issues/18199)). What agents see: the `📚 REFERENCE REPOS` block appends `(from owner/repo#N)` to each derived line, `(from owner/repo#N; read-only: )` when the credential fell back, and `(read-only this pass: )` while an App lookup is unresolved. The proxy rules are the ones in the table above: a derived entry unlocks opening issues and PRs on that repo and nothing else, and the read-only fallback unlocks nothing. On the dashboard, Settings shows derived entries read-only with a link to the request issue and the credential state, and the API answers an edit of one with `409`. diff --git a/src/docs/api-reference.md b/src/docs/api-reference.md index 84aa1474ab..2a49d49ae6 100644 --- a/src/docs/api-reference.md +++ b/src/docs/api-reference.md @@ -106,10 +106,10 @@ summary and the `health_deep_read` diagnostic tool. | `POST` | `/api/github-app/recheck` | Dashboard auth/session | GitHub App Recheck | `pkg/dashboard/server.go:1970` | | `POST` | `/api/github-app/install-clicked` | Dashboard auth/session | GitHub App Install Clicked | `pkg/dashboard/server.go:1971` | | `GET` | `/gh-setup` | Public | GitHub App Setup Callback | `pkg/dashboard/server.go:1972` | -| `GET` | `/api/github-owner-identity` | Owner only | The hive owner's GitHub identity for remote repos ([#18197](https://github.com/hivecommons/hive/issues/18197)): `connected`, `login`, `scope`, `connected_at`, `usable`, `reason`, `verified_at`, `scope_required` (`public_repo`), `private_repos_supported` (false) and `file` (the token file path on the spoke). Never the token | `pkg/dashboard/api.go:599` | -| `POST` | `/api/github-owner-identity/start` | Owner only | Start the consent: a GitHub device flow with the spoke's `oauth_client_id` asking for `public_repo`, separate from the scope-free login flow. Returns `user_code`, `verification_uri`, `expires_in`, `interval`, `scope` and the `flow_id` the poll must present. 502 when GitHub refused the device-code request | `pkg/dashboard/api.go:600` | -| `POST` | `/api/github-owner-identity/poll` | Owner only | Poll the consent with `{"flow_id"}`: `status` `pending` / `slow_down` (poll again), `complete` (with `login` and the identity) or `error`. On completion the granted scope must cover `public_repo`, the token must validate and the account must be the logged-in owner's; the identity is then stored on the spoke (`/data/secrets/github-owner-token`, `HIVE_GITHUB_OWNER_TOKEN_FILE` overrides) and the reference-repo policies re-resolve. 400 without a live flow or with the wrong `flow_id`; audited as `github_owner_identity_connected` / `_rejected` | `pkg/dashboard/api.go:601` | -| `DELETE` | `/api/github-owner-identity` | Owner only | Disconnect: removes the token file, drops every write-tier reference repo to read-only on the next pass, returns the previous `login` and a `note` that GitHub-side revocation is done under Settings → Applications on github.com; audited as `github_owner_identity_disconnected` | `pkg/dashboard/api.go:602` | +| `GET` | `/api/github-owner-identity` | Owner only | The hive owner's GitHub identity for remote repos ([#18197](https://github.com/hivecommons/hive/issues/18197)): `connected`, `login`, `scope`, `connected_at`, `usable`, `reason`, `verified_at`, `scope_required` (`public_repo`), `private_repos_supported` (false) and `file` (the token file path on the spoke). Never the token | `pkg/dashboard/api.go:601` | +| `POST` | `/api/github-owner-identity/start` | Owner only | Start the consent: a GitHub device flow with the spoke's `oauth_client_id` asking for `public_repo`, separate from the scope-free login flow. Returns `user_code`, `verification_uri`, `expires_in`, `interval`, `scope` and the `flow_id` the poll must present. 502 when GitHub refused the device-code request | `pkg/dashboard/api.go:602` | +| `POST` | `/api/github-owner-identity/poll` | Owner only | Poll the consent with `{"flow_id"}`: `status` `pending` / `slow_down` (poll again), `complete` (with `login` and the identity) or `error`. On completion the granted scope must cover `public_repo`, the token must validate and the account must be the logged-in owner's; the identity is then stored on the spoke (`/data/secrets/github-owner-token`, `HIVE_GITHUB_OWNER_TOKEN_FILE` overrides) and the reference-repo policies re-resolve. 400 without a live flow or with the wrong `flow_id`; audited as `github_owner_identity_connected` / `_rejected` | `pkg/dashboard/api.go:603` | +| `DELETE` | `/api/github-owner-identity` | Owner only | Disconnect: removes the token file, drops every write-tier reference repo to read-only on the next pass, returns the previous `login` and a `note` that GitHub-side revocation is done under Settings → Applications on github.com; audited as `github_owner_identity_disconnected` | `pkg/dashboard/api.go:604` | ## Configuration @@ -315,7 +315,7 @@ Read the result from `GET /api/kick/{agent}/status`, which returns `status` of ` | `POST` | `/api/repos/{owner}/{repo}/items/{number}/hold` | Owner or repo write/maintain/admin | Add or remove a dashboard hold chip for an issue or PR. Body `{"held": true}` adds the hive's exact-match canonical `hive-pause/` label; `{"held": false}` removes exactly the labels causing the hold (`hive-pause/` and/or generic hold labels) and never removes `hive/` provenance. Every toggle writes an audit-log entry; the #8851 upgrade migration writes `/data/hive-hold-migration-.json` with audit-backed, provenance-only, and ambiguous legacy `hive/` classifications. | `pkg/dashboard/api.go:352`, `pkg/dashboard/api_repo_hold.go` | | `POST` | `/api/repos/{owner}/{repo}/items/{number}/decision` | Owner or repo write/maintain/admin, signed-in GitHub user | Record an operator decision from the activity monitor's row menu / selection bar. Body `{"decision": "approve"|"direction"|"close-superseded"|"close-wontfix", "comment": "…", "type": "issue"|"pr", "superseded_by": "#458"}`. On an issue parked at 045, `approve`/`direction` act exactly as a maintainer's `/hive approve` / `/hive decision ` comment would (labels, `approved-direction`, stage-engine Unparked fact, reply comment naming the operator). On a PR parked at 085 (`type: "pr"`), `approve` posts the `/lgtm` and `/approve` commands as two comments attributed to the operator, records the approval against the head as the human answer to a `requires_human` / decision-requested verdict (the lanes and the stage engine honour it like a trusted merger's Approve review), lifts `needs-human`, clears the sentinel label and acknowledges the bundle's sentinel findings for that head, re-derives the stage and drops the tracker cache — the response adds `released` (`approval-recorded`, `needs-human`, `sentinel-label`, `sentinel-acknowledged`) and `stage`; the merge gate decides from there (no review as the operator, no merge) — and `direction` records the text as a human change request against the head — the `/hive decision` fix lane — on a hive-authored PR only (409 otherwise). The close variants post a comment naming the operator (and the superseding PR when `superseded_by` is given) and close the item as not planned. Never merges. Every decision writes an audit-log entry (`repo_item_decision_*`). | `pkg/dashboard/api.go:353`, `pkg/dashboard/api_repo_item_decision.go` | | `POST` / `GET` | `/api/repos/{owner}/{repo}/items/{number}/decide-delegated` | Same as `…/hold` (owner or repo write), signed-in GitHub user; `decisions.delegate.enabled` (403 when off) | *Let hive decide*: enqueue a delegated-decision job in which the hive decides the parked item itself as the owner's delegate under `pkg/delegatedecision.PolicyText` and records it as a human decision (`/hive decision` comment with the `` marker, park labels off, `triage/accepted`, queue approval on a PR head). Body `{"type": "issue"|"pr", "reason", "caption", "sentinel_reason"}`. `POST` answers 202 with the job (`{id, status: "running"|"done"|"failed", decision: {verdict, rule, text, rationale, actions, keeps_park}, notes, error}`); 409 with the running job while one is in flight; `GET` reads it, from memory while this process holds the job and otherwise from the operator-action ledger (`from_ledger: true`; status, requester, start, verdict and rule, with the outcome's words in `notes` or `error`), so the last result outlives the 30-minute in-memory eviction and a restart. A job still running when the hive restarts is never resumed: the next process marks its ledger record `failed` (*interrupted by a hive restart before it finished, and not resumed …*) with a `repo_item_decision_delegated_failed` audit entry, and the operator asks again. Rails: never merges, refuses items wearing `hold`, never grants secrets/permissions (`needs-owner` + `hold` instead), recommends only on hive policy/governance questions (keeps 085). Audited as `repo_item_decision_delegated[_failed]`. See [dashboard-sections.md](dashboard-sections.md#let-hive-decide). | `pkg/dashboard/api.go:329`, `pkg/dashboard/api_repo_item_decide_delegated.go` | -| `POST` | `/api/repos/{owner}/{repo}/items/{number}/accept-triage` | Owner or repo write/maintain/admin | Accept triage on an issue from the activity monitor's row menu — what a maintainer accepting triage on GitHub does by hand: adds the reporter-trust triage label (`triage/accepted`, or the first of `issue_filter.reporter_trust.untrusted_require_labels`; the label is created in the repo if missing) and removes the awaiting label (`needs-triage` / `reporter_trust.awaiting_label`) and a Hive-written `stage/015-triage-needed`. Empty body. Returns `{ok, repo, number, acceptLabel, added, removed, alreadyAccepted, minStatusSeq}`; an issue already carrying the accept label with nothing left to remove is a no-op (`alreadyAccepted: true`, no write, no audit). 404 for a pull request or an unknown item. The status snapshot's labels are patched so the next reconcile reads 010 Actionable before the engine's own 015 → 010 derivation. Every write is audited as `repo_item_accept_triage` under the dashboard user. | `pkg/dashboard/api.go:587`, `pkg/dashboard/api_repo_item_accept_triage.go` | +| `POST` | `/api/repos/{owner}/{repo}/items/{number}/accept-triage` | Owner or repo write/maintain/admin | Accept triage on an issue from the activity monitor's row menu — what a maintainer accepting triage on GitHub does by hand: adds the reporter-trust triage label (`triage/accepted`, or the first of `issue_filter.reporter_trust.untrusted_require_labels`; the label is created in the repo if missing) and removes the awaiting label (`needs-triage` / `reporter_trust.awaiting_label`) and a Hive-written `stage/015-triage-needed`. Empty body. Returns `{ok, repo, number, acceptLabel, added, removed, alreadyAccepted, minStatusSeq}`; an issue already carrying the accept label with nothing left to remove is a no-op (`alreadyAccepted: true`, no write, no audit). 404 for a pull request or an unknown item. The status snapshot's labels are patched so the next reconcile reads 010 Actionable before the engine's own 015 → 010 derivation. Every write is audited as `repo_item_accept_triage` under the dashboard user. | `pkg/dashboard/api.go:589`, `pkg/dashboard/api_repo_item_accept_triage.go` | | `GET` | `/api/acmm/evaluation` | Dashboard auth/session | ACMMEvaluation — combined codebase + operational result, cached server-side for 1 hour (`acmmEvalTTL`). `?refresh=1` (the dashboard's "🔄 Re-evaluate" button, #5877) bypasses the hourly TTL but is debounced server-side: requests within 1 minute of the last evaluation (`acmmRefreshDebounce`) still serve the cache, since a full refresh costs up to ~29 GitHub GetContents calls per repo. The response's `last_evaluated_at` timestamp reports when the cached evaluation was computed. When the cache has expired (or `?refresh=1` asks) while the GitHub API budget is conserve or critical, the pass is skipped with no GitHub calls: the last result is served with `stale: true` and `stale_reason`, or, before any pass has run, an empty result with `error` set to the same reason | `pkg/dashboard/api.go:368`, `pkg/dashboard/api_acmm_eval.go` | | `POST` | `/api/acmm/issue` | Owner only | ACMMCreate Issue — files on GitHub or, with `governor.acmm.issue_tracker: work_source` / body `tracker: "work_source"` on a Linear-sourced hive, on Linear; response `tracker` says which. See [ACMM policy matrix](acmm-policy-matrix.md#where-acmm-gap-issues-are-filed) | `pkg/dashboard/api.go:369` | | `GET` | `/api/acmm-recommendation` | Dashboard auth/session | Advisory level-up recommendation (`acmmadvisor.Recommendation`, JSON): never changes the applied level — see [ACMM advisor](acmm-advisor.md) | `pkg/dashboard/api.go:370` | diff --git a/src/docs/api-reference/auth.md b/src/docs/api-reference/auth.md index 9caa3c0cfb..a3273d7e65 100644 --- a/src/docs/api-reference/auth.md +++ b/src/docs/api-reference/auth.md @@ -27,7 +27,7 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f | `POST` | `/api/github-app/recheck` | Dashboard auth/session | GitHub App Recheck | `pkg/dashboard/server.go:1970` | | `POST` | `/api/github-app/install-clicked` | Dashboard auth/session | GitHub App Install Clicked | `pkg/dashboard/server.go:1971` | | `GET` | `/gh-setup` | Public | GitHub App Setup Callback | `pkg/dashboard/server.go:1972` | -| `GET` | `/api/github-owner-identity` | Owner only | The hive owner's GitHub identity for remote repos ([#18197](https://github.com/hivecommons/hive/issues/18197)): `connected`, `login`, `scope`, `connected_at`, `usable`, `reason`, `verified_at`, `scope_required` (`public_repo`), `private_repos_supported` (false) and `file` (the token file path on the spoke). Never the token | `pkg/dashboard/api.go:599` | -| `POST` | `/api/github-owner-identity/start` | Owner only | Start the consent: a GitHub device flow with the spoke's `oauth_client_id` asking for `public_repo`, separate from the scope-free login flow. Returns `user_code`, `verification_uri`, `expires_in`, `interval`, `scope` and the `flow_id` the poll must present. 502 when GitHub refused the device-code request | `pkg/dashboard/api.go:600` | -| `POST` | `/api/github-owner-identity/poll` | Owner only | Poll the consent with `{"flow_id"}`: `status` `pending` / `slow_down` (poll again), `complete` (with `login` and the identity) or `error`. On completion the granted scope must cover `public_repo`, the token must validate and the account must be the logged-in owner's; the identity is then stored on the spoke (`/data/secrets/github-owner-token`, `HIVE_GITHUB_OWNER_TOKEN_FILE` overrides) and the reference-repo policies re-resolve. 400 without a live flow or with the wrong `flow_id`; audited as `github_owner_identity_connected` / `_rejected` | `pkg/dashboard/api.go:601` | -| `DELETE` | `/api/github-owner-identity` | Owner only | Disconnect: removes the token file, drops every write-tier reference repo to read-only on the next pass, returns the previous `login` and a `note` that GitHub-side revocation is done under Settings → Applications on github.com; audited as `github_owner_identity_disconnected` | `pkg/dashboard/api.go:602` | +| `GET` | `/api/github-owner-identity` | Owner only | The hive owner's GitHub identity for remote repos ([#18197](https://github.com/hivecommons/hive/issues/18197)): `connected`, `login`, `scope`, `connected_at`, `usable`, `reason`, `verified_at`, `scope_required` (`public_repo`), `private_repos_supported` (false) and `file` (the token file path on the spoke). Never the token | `pkg/dashboard/api.go:601` | +| `POST` | `/api/github-owner-identity/start` | Owner only | Start the consent: a GitHub device flow with the spoke's `oauth_client_id` asking for `public_repo`, separate from the scope-free login flow. Returns `user_code`, `verification_uri`, `expires_in`, `interval`, `scope` and the `flow_id` the poll must present. 502 when GitHub refused the device-code request | `pkg/dashboard/api.go:602` | +| `POST` | `/api/github-owner-identity/poll` | Owner only | Poll the consent with `{"flow_id"}`: `status` `pending` / `slow_down` (poll again), `complete` (with `login` and the identity) or `error`. On completion the granted scope must cover `public_repo`, the token must validate and the account must be the logged-in owner's; the identity is then stored on the spoke (`/data/secrets/github-owner-token`, `HIVE_GITHUB_OWNER_TOKEN_FILE` overrides) and the reference-repo policies re-resolve. 400 without a live flow or with the wrong `flow_id`; audited as `github_owner_identity_connected` / `_rejected` | `pkg/dashboard/api.go:603` | +| `DELETE` | `/api/github-owner-identity` | Owner only | Disconnect: removes the token file, drops every write-tier reference repo to read-only on the next pass, returns the previous `login` and a `note` that GitHub-side revocation is done under Settings → Applications on github.com; audited as `github_owner_identity_disconnected` | `pkg/dashboard/api.go:604` | diff --git a/src/docs/api-reference/configuration.md b/src/docs/api-reference/configuration.md index c90588f491..6e1f83381a 100644 --- a/src/docs/api-reference/configuration.md +++ b/src/docs/api-reference/configuration.md @@ -98,7 +98,7 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f | `PUT` | `/api/config/escalation` | Owner only | Escalation Config Set | `pkg/dashboard/api.go:228` | | `GET` | `/api/config/write-surface` | Owner only | Lane write allowlist (`write_surface.allowlist`, [#9587](https://github.com/hivecommons/hive/issues/9587)): `allowlist`, `ops`, `warnings` | `pkg/dashboard/api.go:556` | | `PUT` | `/api/config/write-surface` | Owner only | Replace the lane write allowlist; `{"allowlist": {}}` clears it; unknown operation or bad lane name is a 400 with nothing changed | `pkg/dashboard/api.go:557` | -| `PUT` | `/api/config/reference-repos` | Owner only | Replace `project.reference_repos`; `{"reference_repos": []}` clears it. Refused with a 400 and nothing changed when the config loader would reject the list (bare name, repo also in `project.repos`, duplicate, unknown write kind, token file without write) or when a write entry has no credential boot would accept — the same message boot exits 22 with — or when an entry carries a `write_token_file` the config file does not already hold for that repo (the path is operator-only; a dashboard write opt-in writes as the hive owner's connected GitHub account). Entries with `derived_from` ([#18139](https://github.com/hivecommons/hive/issues/18139)) are the request issue's grant: echo them back as served; changing one's write set, leaving one out, sending one the hive did not derive or adding a hand-written entry for the same repo is a 409 naming the issue to change instead ([#18161](https://github.com/hivecommons/hive/issues/18161)). They are never stored: the saved list holds the hand-written entries and the derived set is merged back in memory after the save. 500 when the file could not be written (proxy restored, nothing changed). On success the GitHub proxy's reference policies are replaced without a restart; audited as `config_reference_repos` | `pkg/dashboard/api.go:594` | +| `PUT` | `/api/config/reference-repos` | Owner only | Replace `project.reference_repos`; `{"reference_repos": []}` clears it. Refused with a 400 and nothing changed when the config loader would reject the list (bare name, repo also in `project.repos`, duplicate, unknown write kind, token file without write) or when a write entry has no credential boot would accept — the same message boot exits 22 with — or when an entry carries a `write_token_file` the config file does not already hold for that repo (the path is operator-only; a dashboard write opt-in writes as the hive owner's connected GitHub account). Entries with `derived_from` ([#18139](https://github.com/hivecommons/hive/issues/18139)) are the request issue's grant: echo them back as served; changing one's write set, leaving one out, sending one the hive did not derive or adding a hand-written entry for the same repo is a 409 naming the issue to change instead ([#18161](https://github.com/hivecommons/hive/issues/18161)). They are never stored: the saved list holds the hand-written entries and the derived set is merged back in memory after the save. 500 when the file could not be written (proxy restored, nothing changed). On success the GitHub proxy's reference policies are replaced without a restart; audited as `config_reference_repos` | `pkg/dashboard/api.go:596` | | `GET` | `/api/config/review` | Dashboard auth/session | Review Config Get | `pkg/dashboard/api.go:235` | | `PUT` | `/api/config/review` | Owner only | Review Config Set | `pkg/dashboard/api.go:238` | | `PUT` | `/api/config/governor/features` | Owner only | Governor Features | `pkg/dashboard/api.go:244` | @@ -109,10 +109,10 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f | `GET` | `/api/config/github/webhooks` | Owner only | Config GitHub Webhooks (payload URL, events, secret source, receiver health, App hook permissions) | `pkg/dashboard/api_github_webhook_setup.go:88` | | `POST` | `/api/config/github/webhooks/secret` | Owner only | Config GitHub Webhooks Generate Secret (returned once) | `pkg/dashboard/api_github_webhook_setup.go:89` | | `POST` | `/api/config/github/webhooks/create` | Owner only | Config GitHub Webhooks Create Hook (through the App installation) | `pkg/dashboard/api_github_webhook_setup.go:90` | -| `GET` | `/api/config/prompt-sources` | Owner only | Repo-sourced prompt and definition gate ([#18490](https://github.com/hivecommons/hive/issues/18490)): `repos` (one row per managed repo, then any owner approval whose repo is no longer managed, each `{repo, managed, owner_approved, seed_allowed, allowed, approved_by, approved_at}`), `seed_enabled`, `seed_allowlist`, `trust_notice`, `seed_only_note` | `pkg/dashboard/api.go:605` | -| `PUT` | `/api/config/prompt-sources/owner-repos` | Owner only | `{"repo": "org/name", "allowed": true, "acknowledge_trust": true}` approves one of the hive's managed repos as a `prompt_source` / `definition_source` repo; `"allowed": false` revokes. 400 with nothing changed for a repo outside `project.repos` (still needs the seed edit), a malformed slug, or an approval without `acknowledge_trust: true`. Persisted to `variables.security.owner_github_prompt_repos` in the running config; audited as `config_prompt_source_repo`. Responds with the GET body | `pkg/dashboard/api.go:606` | -| `GET` | `/api/config/reference-repos` | Owner only | `project.reference_repos` ([#18136](https://github.com/hivecommons/hive/issues/18136)): `reference_repos` (`{repo, write, write_token_file}` plus `derived_from` on a derived entry and, on every write entry, `credential_state`, `credential_reason` and `credential_login`, the GitHub account the writes appear under ([#18197](https://github.com/hivecommons/hive/issues/18197)), plus `credential_identity` (`bot`, `owner` or `token_file`) and `credential_note`, the one-line why ([#18516](https://github.com/hivecommons/hive/issues/18516)); always an array), `write_kinds`, `write_identity` (`project.remote_write_identity` with the default applied), `write_identities` and `bot_login` (the App bot login, when known) | `pkg/dashboard/api.go:593` | -| `PUT` | `/api/config/reference-repos/write-identity` | Owner only | Set `project.remote_write_identity` ([#18516](https://github.com/hivecommons/hive/issues/18516)): body `{"write_identity": "bot"}`, one of `bot`, `bot_only`, `owner`. 400 on an unknown value, or when a hand-written write entry would have no credential under the new value (the message boot exits 22 with); nothing is stored then. 500 when the file could not be written. On success the reference repos are re-resolved before the response, which is the same body as `GET /api/config/reference-repos`; audited as `config_remote_write_identity` with `previous` and `write_identity` | `pkg/dashboard/api.go:596` | +| `GET` | `/api/config/prompt-sources` | Owner only | Repo-sourced prompt and definition gate ([#18490](https://github.com/hivecommons/hive/issues/18490)): `repos` (one row per managed repo, then any owner approval whose repo is no longer managed, each `{repo, managed, owner_approved, seed_allowed, allowed, approved_by, approved_at}`), `seed_enabled`, `seed_allowlist`, `trust_notice`, `seed_only_note` | `pkg/dashboard/api.go:607` | +| `PUT` | `/api/config/prompt-sources/owner-repos` | Owner only | `{"repo": "org/name", "allowed": true, "acknowledge_trust": true}` approves one of the hive's managed repos as a `prompt_source` / `definition_source` repo; `"allowed": false` revokes. 400 with nothing changed for a repo outside `project.repos` (still needs the seed edit), a malformed slug, or an approval without `acknowledge_trust: true`. Persisted to `variables.security.owner_github_prompt_repos` in the running config; audited as `config_prompt_source_repo`. Responds with the GET body | `pkg/dashboard/api.go:608` | +| `GET` | `/api/config/reference-repos` | Owner only | `project.reference_repos` ([#18136](https://github.com/hivecommons/hive/issues/18136)): `reference_repos` (`{repo, write, write_token_file}` plus `derived_from` on a derived entry and, on every write entry, `credential_state`, `credential_reason` and `credential_login`, the GitHub account the writes appear under ([#18197](https://github.com/hivecommons/hive/issues/18197)), plus `credential_identity` (`bot`, `owner` or `token_file`) and `credential_note`, the one-line why ([#18516](https://github.com/hivecommons/hive/issues/18516)); always an array), `write_kinds`, `write_identity` (`project.remote_write_identity` with the default applied), `write_identities` and `bot_login` (the App bot login, when known) | `pkg/dashboard/api.go:595` | +| `PUT` | `/api/config/reference-repos/write-identity` | Owner only | Set `project.remote_write_identity` ([#18516](https://github.com/hivecommons/hive/issues/18516)): body `{"write_identity": "bot"}`, one of `bot`, `bot_only`, `owner`. 400 on an unknown value, or when a hand-written write entry would have no credential under the new value (the message boot exits 22 with); nothing is stored then. 500 when the file could not be written. On success the reference repos are re-resolved before the response, which is the same body as `GET /api/config/reference-repos`; audited as `config_remote_write_identity` with `previous` and `write_identity` | `pkg/dashboard/api.go:598` | | `PATCH` | `/api/config/auto-merge` | Owner only | Partially update auto-merge settings, including nested `merge_train`; omitted fields stay unchanged; returns effective settings and bot-author metadata | `pkg/dashboard/api_config_automerge.go` | | `GET` | `/api/config/convergence` | Owner only | Convergence Config Get | `pkg/dashboard/api.go:257` | | `PUT` | `/api/config/convergence` | Owner only | Convergence Config Set | `pkg/dashboard/api.go:259` | @@ -145,3 +145,5 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f | `GET` | `/api/config/governor/inference-auth` | Owner only | Governor Inference Auth Get | `pkg/dashboard/api.go:296` | | `PUT` | `/api/config/governor/inference-auth` | Owner only | Governor Inference Auth Set | `pkg/dashboard/api.go:297` | | `PUT` | `/api/workflows/enabled` | Owner only | Turn model-based stage derivation (`workflow.enabled`, [#17620](https://github.com/hivecommons/hive/issues/17620)) on or off from the dashboard, for hosted hives with no `hive.yaml` access. Body `{enabled: bool}`; persists `hive.yaml`; the stage engine and tracker read the live flag so it applies on the next derivation pass without a restart. Turning on also provisions every bound repository's model labels now (`warnings[]` lists repositories that could not be provisioned; the next sweep retries). Returns `{enabled, changed, bindings[], warnings?[]}`. Audited `workflow_enabled` / `workflow_disabled`; `400` on a non-boolean body | `pkg/dashboard/api_workflows.go:29` | +| `POST` | `/api/workflows/dispatch` | Owner only | Send a model-owned item to its agent profile: the stage tracker's **Send to *profile*** on a row whose bound model hands its current state to a profile ([cncf/techdocs#395](https://github.com/cncf/techdocs/issues/395): Accepted → `techdocs-drafter`, Drafting → `techdocs-verifier`). Body `{repo, number}`. Reads the item fresh, resolves the agent (`workflow.dispatch.profiles..agent`, then the default order ending at the hive's issue worker or reviewer; a profile disabled for automatic dispatch still resolves), takes the agent claim on an issue first, kicks the agent, and records the hand-off once the kick lands so the automatic pass does not repeat it. Returns `{status: "sent", ref, model, state, profile, agent, result}`. Audited `workflow_dispatch`; `404` when the model does not hand the state to a profile, `409` when no agent fills it (the message gives the reason and names the Settings → Repos picker), someone else holds the claim, or the agent is busy | `pkg/dashboard/api_workflows.go:30` | +| `PUT` | `/api/workflows/dispatch/profiles` | Owner only | Pick the hive agent a bound model's profile is handed to ([#18690](https://github.com/hivecommons/hive/issues/18690)): the agent picker on a repository's workflow row in Settings → Repos. Body `{profile, agent}`; an empty `agent` clears the pin so the profile resolves by default again. Persists `workflow.dispatch.profiles..agent` in `hive.yaml` (the profile's `prompt_template` and `disabled` are kept); the next scheduler pass reads the live config, so no restart is needed. `agent` must be an enabled agent that is not a replica (`400` otherwise). Returns `{changed, profile, agent, dispatch, agents}`, where `dispatch` lists every bound repository's profiles as `{repo, profile, agent, source, pinned, disabled, problem, states}` (`problem` is why nothing fills the profile) and `agents` the agents that can be picked; `GET /api/workflows` and `PUT /api/workflows/bindings` carry the same two fields. Audited `workflow_dispatch_agent` | `pkg/dashboard/api_workflows.go:31` | diff --git a/src/docs/api-reference/knowledge.md b/src/docs/api-reference/knowledge.md index 9a8c8534ab..104eeb14cc 100644 --- a/src/docs/api-reference/knowledge.md +++ b/src/docs/api-reference/knowledge.md @@ -90,23 +90,26 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f | `DELETE` | `/api/inbox/recordings/{id}` | The recorder, or an owner | Abort an in-progress recording and delete its chunks; audited as `inbox_recording_abort` (403, 404, 409 already ended) | `pkg/dashboard/api_inbox_recordings.go:23` | | `GET` | `/api/inbox/recordings/{id}/audio` | Owner or read-write | Play a finished recording from this hive's disk on the dashboard origin: `http.ServeContent` (Range, 206), `Content-Type` audio/webm or audio/mp4, `Cache-Control: no-store`, `Cross-Origin-Resource-Policy: same-origin`. Opening a playback is audited as `inbox_recording_audio` (seeks are not). `GET /api/inbox/hive-recording/{id}/audio/stream` serves the same file (404 unknown or not finished, 410 audio expired after `inbox.recording.retention_days`) | `pkg/dashboard/api_inbox_recordings.go:24` | | `DELETE` | `/api/inbox/hive-recording/{id}` | An owner, or the read-write member who recorded it | Delete a finished recording for good: its stored audio and chunks, the Inbox item and its state, and its Knowledge fact with the machine transcript. What was already shared from it (RFC, issues, DIBS ideas, spek seeds) is not touched. 200 `{id, deleted, fact_removed, shares_kept}`; audited as `inbox_recording_delete` (403 read role or someone else's recording, 404 unknown, 409 still in progress or being transcribed) | `pkg/dashboard/api_inbox_recordings.go:25` | -| `GET` | `/api/meetings/settings` | Owner only | Meetings settings: `enabled`, `enabled_env_override`, `running`, `vexa_url`, `vexa_key_set`, `discord_token_set`, `transcribe_token_set`, `tts_token_set` (secrets are never returned) and `effective` values of the other `meetings.*` keys ([env vars](../env-vars.md#meetings-quorum)) | `pkg/dashboard/api_meetings.go:123` | -| `POST` | `/api/meetings/settings` | Owner only | Update `meetings.enabled` and `meetings.vexa.url`, and write the Vexa tenant key, Discord bot token, transcription service token and TTS service token to keystore-sealed secret files (`""` clears one): `{enabled?, vexa_url?, vexa_api_key?, discord_token?, transcribe_token?, tts_token?}`. Returns the settings with `restart_required`; 400 on an invalid value. Audited as `meetings_settings` | `pkg/dashboard/api_meetings.go:124` | -| `POST` | `/api/meetings` | Owner only | Send hive to a meeting: `{url}` for Google Meet, Teams or Zoom, or `{platform: "discord", guild_id, channel_id}`, plus `title?` and `voice?`. Returns the meeting record; 400 invalid request, 502 Quorum refused, 503 meetings off. Audited as `meeting_join` | `pkg/dashboard/api_meetings.go:125` | -| `GET` | `/api/meetings` | Owner only | `{meetings: [...]}`, newest first | `pkg/dashboard/api_meetings.go:126` | -| `GET` | `/api/meetings/{id}` | Owner only | `{meeting, transcript, live}`: the record, stored final segments and chat, and the live-loop state (notes, research, asks, budget); 404 unknown | `pkg/dashboard/api_meetings.go:127` | -| `POST` | `/api/meetings/{id}/leave` | Owner only | Make hive leave; returns the record. Audited as `meeting_leave` | `pkg/dashboard/api_meetings.go:128` | -| `POST` | `/api/meetings/{id}/voice` | Owner only | `{enabled}`: allow hive to answer aloud when addressed (with `meetings.voice_on_address`). Audited as `meeting_voice` | `pkg/dashboard/api_meetings.go:129` | -| `POST` | `/api/meetings/{id}/research/{rid}/approve` | Owner only | Dispatch a research item waiting past the per-meeting cap; 409 when it is not waiting or the live session ended. Audited as `meeting_research_approve` | `pkg/dashboard/api_meetings.go:130` | -| `POST` | `/api/meetings/{id}/approve` | Owner only | Approve a platform join request (Discord `/hive join`): joins that channel; optional `{voice}`. 409 when the meeting is not a pending request. Audited as `meeting_request_approve` | `pkg/dashboard/api_meetings.go:131` | -| `POST` | `/api/meetings/{id}/decline` | Owner only | Decline a platform join request with `meeting.leave`. Audited as `meeting_request_decline` | `pkg/dashboard/api_meetings.go:132` | -| `GET` | `/api/meetings/{id}/ws` | Owner only | WebSocket live panel: a `snapshot` frame, then `meeting`, `line`, `intent`, `research`, `notes` and `dropped` frames for the meeting | `pkg/dashboard/api_meetings.go:133` | -| `GET` | `/api/meetings/schedule` | Owner or read-write | The Calendar: `{upcoming, recent, attendee_name}`; upcoming events (scheduled, awaiting approval, live) by start and events finished in the last 14 days, newest first. Each event carries its platform, source, status, next occurrences and the meeting it created; a passcode is shown only as `set`. See [Calendar](../meetings.md#calendar) | `pkg/dashboard/api_meetings_schedule.go:83` | -| `POST` | `/api/meetings/schedule` | Owner or read-write | Schedule a meeting: `{url, start, end?, title?, rrule?, timezone?}` (RFC 3339 times, RFC 5545 `RRULE` expanded in an IANA `timezone`). The platform comes from the link (Google Meet, Teams, Zoom). The owner's add is scheduled; a read-write user's add awaits approval. 400 invalid or already ended. Audited as `meeting_schedule_add` | `pkg/dashboard/api_meetings_schedule.go:84` | -| `POST` | `/api/meetings/schedule/{eid}/approve` | Owner only | Schedule an event awaiting approval. 409 when it is not awaiting approval. Audited as `meeting_schedule_approve` | `pkg/dashboard/api_meetings_schedule.go:85` | -| `POST` | `/api/meetings/schedule/{eid}/decline` | Owner only | Decline an event awaiting approval; later updates of the same invite stay declined. Audited as `meeting_schedule_decline` | `pkg/dashboard/api_meetings_schedule.go:86` | -| `POST` | `/api/meetings/schedule/{eid}/cancel` | Owner only | Stop hive joining the event (every occurrence of a series); a meeting already live is not left. Audited as `meeting_schedule_cancel` | `pkg/dashboard/api_meetings_schedule.go:87` | -| `POST` | `/api/meetings/schedule/{eid}/skip` | Owner only | Skip the next occurrence hive has not joined: a single meeting becomes `skipped`, a series moves on. Audited as `meeting_schedule_skip` | `pkg/dashboard/api_meetings_schedule.go:88` | +| `GET` | `/api/meetings/settings` | Owner only | Meetings settings: `enabled`, `enabled_env_override`, `running`, `vexa_url`, `vexa_key_set`, `discord_token_set`, `transcribe_token_set`, `tts_token_set` (secrets are never returned) and `effective` values of the other `meetings.*` keys ([env vars](../env-vars.md#meetings-quorum)) | `pkg/dashboard/api_meetings.go:129` | +| `POST` | `/api/meetings/settings` | Owner only | Update `meetings.enabled` and `meetings.vexa.url`, and write the Vexa tenant key, Discord bot token, transcription service token and TTS service token to keystore-sealed secret files (`""` clears one): `{enabled?, vexa_url?, vexa_api_key?, discord_token?, transcribe_token?, tts_token?}`. Returns the settings with `restart_required`; 400 on an invalid value. Audited as `meetings_settings` | `pkg/dashboard/api_meetings.go:130` | +| `POST` | `/api/meetings` | Owner only | Send hive to a meeting: `{url}` for Google Meet, Teams or Zoom, or `{platform: "discord", guild_id, channel_id}`, plus `title?` and `voice?`. Returns the meeting record; 400 invalid request, 502 Quorum refused, 503 meetings off. Audited as `meeting_join` | `pkg/dashboard/api_meetings.go:131` | +| `GET` | `/api/meetings` | Owner only | `{meetings: [...]}`, newest first | `pkg/dashboard/api_meetings.go:132` | +| `GET` | `/api/meetings/{id}` | Owner only | `{meeting, transcript, live}`: the record, stored final segments and chat, and the live-loop state (notes, research, asks, budget); 404 unknown | `pkg/dashboard/api_meetings.go:133` | +| `POST` | `/api/meetings/{id}/leave` | Owner only | Make hive leave; returns the record. Audited as `meeting_leave` | `pkg/dashboard/api_meetings.go:134` | +| `POST` | `/api/meetings/{id}/voice` | Owner only | `{enabled}`: allow hive to answer aloud when addressed (with `meetings.voice_on_address`). Audited as `meeting_voice` | `pkg/dashboard/api_meetings.go:135` | +| `POST` | `/api/meetings/{id}/research/{rid}/approve` | Owner only | Dispatch a research item waiting past the per-meeting cap; 409 when it is not waiting or the live session ended. Audited as `meeting_research_approve` | `pkg/dashboard/api_meetings.go:136` | +| `POST` | `/api/meetings/{id}/approve` | Owner only | Approve a platform join request (Discord `/hive join`): joins that channel; optional `{voice}`. 409 when the meeting is not a pending request. Audited as `meeting_request_approve` | `pkg/dashboard/api_meetings.go:137` | +| `POST` | `/api/meetings/{id}/decline` | Owner only | Decline a platform join request with `meeting.leave`. Audited as `meeting_request_decline` | `pkg/dashboard/api_meetings.go:138` | +| `DELETE` | `/api/meetings/{id}` | Owner only | Delete a meeting in a final state (ended, failed, blocked, declined, left, handed off, purged) or a pending join request (declined with `meeting.leave`, then removed): the record, transcript and live state on this hive, the meeting in Vexa (Quorum `meeting.delete`, best effort: `not_found` is fine, any other error comes back as `upstream_error` while the record is still removed), and the Calendar entry that existed only for it. Knowledge facts and drafts made from it are not touched. 200 `{id, state, upstream_error?, calendar_removed?}`; 404 unknown; 409 while hive is joining, in the lobby or live (leave the meeting first). Audited as `meeting_delete` | `pkg/dashboard/api_meetings_delete.go:44` | +| `POST` | `/api/meetings/clear-finished` | Owner only | Delete every meeting in a final state that finished before `{before?}` (RFC 3339; empty or later than now means now), each as `DELETE /api/meetings/{id}`; live meetings and pending join requests stay. 200 `{deleted, count}`; 400 bad `before`. Each meeting audited as `meeting_delete` | `pkg/dashboard/api_meetings_delete.go:45` | +| `GET` | `/api/meetings/{id}/ws` | Owner only | WebSocket live panel: a `snapshot` frame, then `meeting`, `line`, `intent`, `research`, `notes` and `dropped` frames for the meeting | `pkg/dashboard/api_meetings.go:139` | +| `GET` | `/api/meetings/schedule` | Owner or read-write | The Calendar: `{upcoming, recent, attendee_name}`; upcoming events (scheduled, awaiting approval, live) by start and events finished in the last 14 days, newest first. Each event carries its platform, source, status, next occurrences and the meeting it created; a passcode is shown only as `set`. See [Calendar](../meetings.md#calendar) | `pkg/dashboard/api_meetings_schedule.go:86` | +| `POST` | `/api/meetings/schedule` | Owner or read-write | Schedule a meeting: `{url, start, end?, title?, rrule?, timezone?}` (RFC 3339 times, RFC 5545 `RRULE` expanded in an IANA `timezone`). The platform comes from the link (Google Meet, Teams, Zoom). The owner's add is scheduled; a read-write user's add awaits approval. 400 invalid or already ended. Audited as `meeting_schedule_add` | `pkg/dashboard/api_meetings_schedule.go:87` | +| `POST` | `/api/meetings/schedule/{eid}/approve` | Owner only | Schedule an event awaiting approval. 409 when it is not awaiting approval. Audited as `meeting_schedule_approve` | `pkg/dashboard/api_meetings_schedule.go:88` | +| `POST` | `/api/meetings/schedule/{eid}/decline` | Owner only | Decline an event awaiting approval; later updates of the same invite stay declined. Audited as `meeting_schedule_decline` | `pkg/dashboard/api_meetings_schedule.go:89` | +| `POST` | `/api/meetings/schedule/{eid}/cancel` | Owner only | Stop hive joining the event (every occurrence of a series); a meeting already live is not left. Audited as `meeting_schedule_cancel` | `pkg/dashboard/api_meetings_schedule.go:90` | +| `POST` | `/api/meetings/schedule/{eid}/skip` | Owner only | Skip the next occurrence hive has not joined: a single meeting becomes `skipped`, a series moves on. Audited as `meeting_schedule_skip` | `pkg/dashboard/api_meetings_schedule.go:91` | +| `DELETE` | `/api/meetings/schedule/{eid}` | Owner only | Delete a past Calendar entry (ended, cancelled, declined, failed, blocked, skipped) and the meetings it created, each as `DELETE /api/meetings/{id}`. 200 `{id, meetings}`; 404 unknown; 409 for an entry still scheduled, awaiting approval or live (skip or cancel it) or whose meeting hive is still in. Audited as `meeting_schedule_delete`, plus `meeting_delete` per meeting | `pkg/dashboard/api_meetings_delete.go:46` | | `GET` | `/api/meetings/actions` | Owner or read-write | Drafts from meetings: `{drafts, pending}`, newest first. Each draft carries `id`, `meeting_id`, `meeting_title`, `meeting_link`, `type` (`issue`, `rfc`, `spek`, `knowledge`), `title`, `body`, `repo`, `excerpt`, `requested_by` (a meeting display name), `request`, `spoken`, `status` (`pending`, `approving`, `done`, `failed`, `rejected`), `auto_approved`, `result_url`, `result_note`, `error`, `announced` (`chat` or `notes`), `edited_by`, `decided_by`, `decided_at`, `created_at`, `updated_at`. See [Asking Betty to do things](../meetings.md#asking-betty-to-do-things) | `pkg/dashboard/api_meetings_actions.go:69` | | `GET` | `/api/meetings/actions/settings` | Owner or read-write | `meetings.persona.actions` as set and in effect: `enabled`, `types`, `all_types`, `default_repo`, `effective_default_repo`, `repos` (the monitored repos a draft may target), `auto_approve` (per type), `auto_approvable` (`knowledge`) and `env_overrides` | `pkg/dashboard/api_meetings_actions.go:70` | | `PUT` | `/api/meetings/actions/settings` | Owner only | Save `{enabled?, types?, default_repo?, auto_approve?}` into `meetings.persona.actions`; applies at once. 400 for an unknown type, a repo that is not `owner/repo`, or auto-approve on anything but `knowledge`. Audited as `meetings_actions_settings` | `pkg/dashboard/api_meetings_actions.go:71` | @@ -115,10 +118,10 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f | `POST` | `/api/meetings/actions/{aid}/edit` | Owner or read-write | Change a pending or failed draft: `{type?, title?, body?, repo?}`; the repo must be a monitored one. 404 unknown, 409 already decided. Audited as `meeting_action_edit` | `pkg/dashboard/api_meetings_actions.go:72` | | `POST` | `/api/meetings/actions/{aid}/approve` | Owner or read-write | Create the draft (issue or RFC on GitHub, Spek session through the Inbox `spek_rfc` destination, knowledge note in the `meetings` vault) and report the result to the meeting. Returns the draft, `done` with `result_url` or `failed` with `error`. 409 when it is not pending or failed, or action requests are off. Audited as `meeting_action_approve` | `pkg/dashboard/api_meetings_actions.go:73` | | `POST` | `/api/meetings/actions/{aid}/reject` | Owner or read-write | Reject a pending or failed draft; nothing is created. Audited as `meeting_action_reject` | `pkg/dashboard/api_meetings_actions.go:74` | -| `GET` | `/api/meetings/persona` | Owner or read-write | Settings, Meeting attendee: `{effective, presets, overridden, voices, voice_types, restart_required, running, tts_configured, email}`. `effective` is the resolved persona (fields resolved as if on, `enabled` separately) with the rendered `intro` and `voice_on_address`; `presets` lists the built-in personas, the default first; `overridden` maps each `meetings.persona` key not taken from the preset or defaults to `config` or `env`; `voices` are the Kokoro voices with language, accent and voice type; `restart_required` is true when the saved persona differs from the one the hive booted with; `email` is the attendee address (`hive@`), the access list with registered calendar emails and proof status, the approval queue and the email-in switches (`available: false` until email-in is enabled). See [Meeting attendee settings](../meetings.md#meeting-attendee-settings) | `pkg/dashboard/api_meetings_persona.go:164` | -| `PUT` | `/api/meetings/persona` | Owner or read-write | Save `meetings.persona` to `hive.yaml`: `{enabled?, preset?, clear_overrides?, name?, display_name?, aliases?, pronouns?, language?, voice?, voice_type?, accent?, intro_template?, leave_phrases?, humor?, unprompted?: {per_10m?, min_gap_s?, per_meeting?, answer_wait_s?, min_confidence?, queue_depth?, stale_s?}, voice_on_address?}`. An absent field is unchanged, an empty one (or 0 for a cap) clears the override, a value equal to the preset's is stored as no override, and `clear_overrides` drops every field override first. Validated as if the persona were on; returns the GET body. 400 invalid. Audited as `meetings_persona`. An optional `email: {calendar_invites?, mail_in_files?}` sets `meetings.email.invites_enabled` / `files_enabled` (owner only, else 403; [#18395](https://github.com/hivecommons/hive/issues/18395)) | `pkg/dashboard/api_meetings_persona.go:165` | -| `POST` | `/api/meetings/persona/sample` | Owner or read-write | Speak a short line through the meetings TTS service (Kokoro `POST /v1/audio/speech`) and return the audio (`audio/mpeg`, at most 2 MiB, `Cache-Control: no-store`). Optional `{voice?, language?, display_name?, text?}` (unsaved form values; `text` at most 200 characters, default the persona's sample line in its language). 503 "TTS not configured" until `meetings.tts_url` is set, 400 invalid, 502 the TTS service failed | `pkg/dashboard/api_meetings_persona.go:166` | -| `GET` | `/api/meetings/persona/sample` | Owner or read-write | The same sample with `voice`, `language`, `display_name` and `text` as query parameters, for the dashboard's `