diff --git a/docs/designs/data-directory-layout.md b/docs/designs/data-directory-layout.md index 794cc4ead..c09e4cedc 100644 --- a/docs/designs/data-directory-layout.md +++ b/docs/designs/data-directory-layout.md @@ -439,6 +439,9 @@ every checkout, so that is where they live now: ├── reports-wt/ (the side-branch locks sit beside them) ├── pending-learnings/ pendingLearningsDir → /pending-learnings └── workspaces// + ├── managed-mcp.json managedMcpManifestPath, one per checkout + ├── managed-mcp-files.json resolvedMcpFilesPath: project MCP configs teamai may have written a resolved ${VAR} to, and whether + │ the paths earlier teamai.yaml revisions mapped were read; one of those git tracks is marked tracked (#882) └── search-index.json getProjectSearchIndexPath, one per checkout /.teamai/ one per checkout: committed knowledge, knowledge-wt/ ``` @@ -655,8 +658,8 @@ the other repository, and `recall` rebuilds a missing index. `uninstall` lists, how many unpublished learnings each queue in the data home holds, set-aside ones included, so the member can publish or copy them first. -Every checkout keeps its `workspaces//` (search index, managed MCP, -resource cache) in the shared data home. A full `pull` removes those of +Every checkout keeps its `workspaces//` (search index, managed MCP and +the MCP configs it wrote a resolved value to, resource cache) in the shared data home. A full `pull` removes those of checkouts `git worktree list` no longer shows; the fast path does not list worktrees. @@ -905,3 +908,29 @@ user finds partitions safe to `rm -rf` by hand. `teamai migrate` / `gc` / `--revert` commands; cross-project shared team-repo clone. Downgrade to an older teamai after P1 migration is not supported (`.teamai.bak/` is the manual rollback path). + +## Team secret values (#875) + +A member's values for their teams' declared secrets live in `~/.teamai/secrets/`, +a class-A2 (machine-level) directory: `teams/.json`, one file per +team repo, named by the full SHA-256 hex digest (64 characters, never shortened) of the team repo URL in `~/.teamai/config.yaml` (never +`teamai.yaml`'s `repo:`), without the team name, so renaming `team:` keeps the values; +the hash covers the URL's scheme (the ssh forms count as one; https and http are two), +ssh user, host, non-default port and path (an scp-style path not starting with `/` or `~` +is in the user's home, as `ssh://host/~/path`; `ssh://host/path` is from the root), so an +scp path in the home and the `ssh://` path from the root, two ssh users' repos on one host, two +repos on one host with different ports, or repos behind http and https, get different files, +and `machine.json`, the values set with `teamai env set --global` for every team. +Every scope that uses the same team, and every worktree of it, reads the same file. +Each entry records whether it is a secret's value or the member's value for an +`env.yaml` variable (`kind`), so one is never used as the other. +It never goes to a partition or to ``, which in single-repo mode is inside +the business repo, and it is not `~/.teamai/env`, which is already the user scope's +env backup file. Files are written atomically with mode `0600`. Uninstalling a +project scope removes only its partition, so the values stay; uninstalling the user +scope removes `~/.teamai` and them with it. See [Team secrets](team-secrets.md#storage). + +Beside each scope's `env.sh`, in ``, `env.sh.exports.json` records what +that `env.sh` has exported: per key, a SHA-256 of `KEY=VALUE` for the last 20 +values, never a value, mode `0600`. It is machine data like `env.sh` and is +removed with it. See [Team secrets](team-secrets.md#resolution). diff --git a/docs/designs/management-backend.md b/docs/designs/management-backend.md index 2f6409b84..99206bd5f 100644 --- a/docs/designs/management-backend.md +++ b/docs/designs/management-backend.md @@ -50,7 +50,8 @@ plane accepts identity-bound events with separate retention and write permission | `skills/` | SkillsHandler, namespaces and marketplace metadata | Resource bundles with immutable files, dependencies and generated marketplace views | | `rules/` | RulesHandler and enforced-rule selection | Versioned rules with separately enforced policy constraints | | `docs/` | DocsHandler and indexed documentation | Versioned documents, authorized materialization and recall indexing | -| `env/env.yaml` | EnvHandler, local overrides and environment injection | Non-secret templates plus secret references; secret resolution has separate authorization | +| `env/env.yaml` | EnvHandler, local overrides and environment injection; values in plaintext | Non-secret templates plus secret references; secret resolution has separate authorization | +| `env/secrets.yaml` | Secret declarations with no value ([team secrets](team-secrets.md)); v1 resolves each member's value on their machine | Secret references; the encrypted secret service below stays future work | | `agents/` | AgentsHandler and tool-format conversion | Versioned agent definitions rendered through the existing tool adapters | | `hooks/hooks.yaml` | HooksHandler plus hook reconciliation; no general per-item push | Reviewed declarative hooks, client consent and typed validation | | `mcp/mcp.yaml` | McpHandler plus MCP reconciliation; direct YAML editing for contributions | Reviewed server definitions, transport policy and secret references | diff --git a/docs/designs/management-backend.zh-CN.md b/docs/designs/management-backend.zh-CN.md index 4a2ae9b19..aec8fb6be 100644 --- a/docs/designs/management-backend.zh-CN.md +++ b/docs/designs/management-backend.zh-CN.md @@ -43,7 +43,8 @@ | `skills/` | SkillsHandler、命名空间及 marketplace 元数据 | 不可变文件和依赖组成的资源包，派生 marketplace 视图 | | `rules/` | RulesHandler 及强制规则选择 | 版本化规则，并单独执行强制策略约束 | | `docs/` | DocsHandler 及文档索引 | 版本化文档、授权物化及召回索引 | -| `env/env.yaml` | EnvHandler、本地覆盖和环境注入 | 非密钥模板及密钥引用，解析密钥时单独授权 | +| `env/env.yaml` | EnvHandler、本地覆盖和环境注入；值为明文 | 非密钥模板及密钥引用，解析密钥时单独授权 | +| `env/secrets.yaml` | 只声明、不含值的密钥（[团队密钥](team-secrets.zh-CN.md)）；v1 在每个成员的机器上解析其值 | 密钥引用；下文的加密密钥服务仍属未来工作 | | `agents/` | AgentsHandler 及工具格式转换 | 版本化 agent 定义，复用现有工具适配器渲染 | | `hooks/hooks.yaml` | HooksHandler 及 hook 协调，不支持通用逐项 push | 可审核的声明式 hooks、客户端同意及类型验证 | | `mcp/mcp.yaml` | McpHandler 及 MCP 协调，贡献通过直接编辑 YAML 完成 | 可审核的服务定义、transport 策略及密钥引用 | diff --git a/docs/designs/model-profiles.md b/docs/designs/model-profiles.md index ad0883475..55853e531 100644 --- a/docs/designs/model-profiles.md +++ b/docs/designs/model-profiles.md @@ -20,6 +20,8 @@ Model profiles let a team publish one gateway catalog that every supported agent A key is either stored or referenced as an environment variable; it is never accepted as a command-line argument. `0600` is not encryption. The team-key file name is a hash of the repository identity; the sanitized `teamai.yaml` team name plays no part in it, so renaming the team never orphans the keys. When no repository identity exists at all (no usable remote, URL, or `repo:` claim), the name hashes the team slug with the path instead — there is nothing repository-shaped to key on, and the slug keeps differently named teams that share a checkout path apart. While the hash-only file does not exist yet, a legacy `-.json` from an older version is read where it lies — nothing is renamed — and the next save writes the hash-only name. A legacy file under a repository-bound digest (a URL, a URL-shaped `repo:` claim or remote) carries its host in the identity, so it is read by digest alone; a legacy file under a **provider-ambiguous** digest (a path-shaped `repo:` claim, a provider-relative remote, a bare alias, or no repository identity) names no single repository — the old name scheme never encoded the provider, so two providers' same-named teams hash the same file, and the slug cannot prove ownership across providers. Such a file is never read by a silent rule: the CLI surfaces it once and the user explicitly adopts the exact `-` identity for this checkout, after which the read happens and the next save migrates the keys to the provider-qualified hash-only name. Non-interactive and `--dry-run` runs never adopt: they report the file and leave it unread. Switch records under a provider-ambiguous identity are never claimed either — the old name never encoded the provider, so no slug (not even this checkout's exact slug) proves ownership: a GitHub and a GitCode team both named `Alpha` on the bare claim `acme/widgets` share the identical `alpha-` form. No machine-global record is ever used as proof, because a store shared by every checkout on the machine would equally belong to a foreign team; the explicit adoption of the values file re-establishes this team's presence, and its switched agents are re-recorded by the next `models switch` under the provider-qualified identity. Repository-bound switch records still match by digest alone. The identity is recorded with each `team:` switch so `pull` only re-applies the current team's profiles. Inside it, each key is stored under `team:@`; see [Namespaces and key binding](#namespaces-and-key-binding). +Model profile keys are not [team secrets](team-secrets.md): `teamai env set` does not configure them, `env/secrets.yaml` cannot declare one, and the two stores share code, not files. + ## Catalog and protocols ```yaml diff --git a/docs/designs/model-profiles.zh-CN.md b/docs/designs/model-profiles.zh-CN.md index 2c0632cad..f55cab947 100644 --- a/docs/designs/model-profiles.zh-CN.md +++ b/docs/designs/model-profiles.zh-CN.md @@ -20,6 +20,8 @@ 密钥要么保存在本地，要么引用环境变量，不接受命令行参数传入。`0600` 并非加密。团队密钥文件名只是仓库身份的哈希，`teamai.yaml` 中的团队名不参与其中，因此重命名团队不会导致密钥失效。当完全不存在仓库身份（可用的 remote、URL 或 `repo:` claim 都没有）时，文件名改为哈希团队 slug 与路径的组合——此时没有任何仓库形态的信息可以依赖，靠 slug 区分共享同一检出路径、名字不同的团队；旧版本遗留的 `<团队名>-<哈希>.json` 会在纯哈希文件尚不存在时被原位读取，不做任何改名，下一次保存才会写入纯哈希文件名。在 provider 可确定的 digest（URL 或 URL 形态的 `repo:` claim/remote）下写出的遗留文件，identity 自带主机，直接按 digest 读取；在 **provider 不明确** 的 digest（形如 `owner/repo` 的 `repo:` claim、provider 相对的 remote、裸 alias，或没有仓库身份）下写出的遗留文件既不指向唯一仓库——旧命名从未编码 provider，两个 provider 的同名团队会哈希出同一个文件，slug 无法在 provider 之间证明归属。这类文件绝不按静默规则读取：CLI 会展示一次，由用户显式确认采用这个确切的 `<团队名>-<哈希>` identity 后才读取，随后下一次保存会把密钥迁移到 provider 限定的纯哈希文件名下。非交互与 `--dry-run` 运行一律不采用：只报告该文件并保持不读。provider 不明确 identity 下的 switch 记录同样绝不采用——旧命名从未编码 provider，任何 slug（即使与本 checkout 完全相同的 slug）都无法证明归属：GitHub 与 GitCode 上都叫 `Alpha`、裸 claim 同为 `acme/widgets` 的两支团队会共享完全相同的 `alpha-` 形式。绝不把任何机器级记录当作归属证明，因为同一机器上被各 checkout 共享的记录对别的团队同样成立；只有对遗留 values 文件的显式采用才能重新确立本团队的存在，其已切换的 agent 随后通过下一次 `models switch` 以 provider 限定的 identity 重新记录。provider 可确切的 switch 记录仍按 digest 直接匹配。每次切换 `team:` 配置时也会记录这个身份，`pull` 只会重新应用当前团队的配置。文件内每个密钥保存在 `team:@` 下，见 [Namespace 与密钥绑定](#namespace-与密钥绑定)。 +模型配置的密钥不是[团队密钥](team-secrets.zh-CN.md)：`teamai env set` 不配置它们，`env/secrets.yaml` 也不能声明它们，两者只共享代码、不共享文件。 + ## 目录与协议 ```yaml diff --git a/docs/designs/multi-project-management.md b/docs/designs/multi-project-management.md index c7aec32d3..be5259dac 100644 --- a/docs/designs/multi-project-management.md +++ b/docs/designs/multi-project-management.md @@ -241,6 +241,7 @@ directory's projects list under `resources.`. | Type | `resources:` key | Replaced by name | Two active namespaces, one name | |---|---|---|---| | env | `env` | variable `key` | conflict | +| secrets (`env//secrets.yaml`, [#875](team-secrets.md)) | `env` | secret `key` | conflict | | hooks | `hooks` | hook `id` | conflict | | mcp | `mcp` | server `name` (`command`, `args`, `env` and `tools:` together) | conflict | | models | `models` | profile `id` | conflict | @@ -266,7 +267,8 @@ active, the next pull delivers the root item again and removes items that only the namespace had; for env, hooks and MCP that happens on an `Already synced` pull too, and `env.sh` is regenerated from the resolved set even when `env/env.yaml` is missing or declares nothing. MCP `${VAR}` lookup reads the same -resolved env set. +resolved env set, with the member's value for this team (`teamai env set KEY`) +first; the environment no longer overrides a team variable ([Team secrets](team-secrets.md#variables)). Skills keep one difference: in role/project mode the root `skills/` stays the tag catalog and is not delivered by default. A root skill that arrives through a @@ -320,13 +322,14 @@ copy is not a duplicate: each copy that passes the role filter is delivered, as | Type | Effect of a failure | |---|---| | env | `env.sh` and the shell profile keep what they had | +| secrets | the declared secrets are not resolved, and `env.sh`, the env backup and the MCP servers keep what they had; `teamai env list` and `teamai doctor` fail naming the file | | hooks | installed team hooks and the managed-hooks record stay as they are. The built-in hooks are still installed in each tool that misses one (a tool with all of them is not rewritten), so a first install gets the session-start pull that heals it: with the root file's `builtin:` overrides whenever `hooks/hooks.yaml` parses (a broken namespace file or a clash does not hide them), and when the root file itself does not parse, with their defaults and only in a tool that has no teamai hook yet. `teamai init` and bootstrap say the team hooks were not installed; `teamai hooks inject` exits 1 | | mcp | no tool's MCP config changes | | models | no switched agent is updated; `teamai models` commands fail with the same message; `teamai push` refuses any invalid models file, active or not | | skills, agents | that type is neither installed nor swept; the other types and the search index still sync | | rules, claudemd, docs, learnings | no conflict case | -For env, hooks, MCP and models the warning is also written to +For env, secrets, hooks, MCP and models the warning is also written to `~/.teamai/debug.log`, so a silent session-start pull leaves a trace. This replaces two earlier behaviours: an invalid hooks or MCP file reconciled to the empty set and removed every managed entry, and a skills or agents collision @@ -450,8 +453,9 @@ a root one is written to its namespace file, never to the root. The agents source order is active namespace, then this machine's placement record, then the shared root; in role/project mode a same-stem root file no longer withdraws the placement record (legacy mode still does). The skills push scan uses role ∪ -project namespaces, and `push` picks up a change to any `env//env.yaml`. -`teamai env add|remove` take `--role` / `--project`. `teamai remove mcp ` +project namespaces, and `push` picks up a change to any `env//env.yaml` or `env//secrets.yaml`. +`teamai env add|remove` take `--role` / `--project`, and `--secret` for that namespace's `secrets.yaml`. +`teamai remove mcp ` removes from the root file when it defines the name, otherwise from the one namespace file that does, and asks for `--role` / `--project` only when several namespace files and not the root define it, and removes nothing by a bare name diff --git a/docs/designs/team-secrets.md b/docs/designs/team-secrets.md new file mode 100644 index 000000000..7f97fe547 --- /dev/null +++ b/docs/designs/team-secrets.md @@ -0,0 +1,230 @@ +# Team secrets + +[简体中文](team-secrets.zh-CN.md) + +Proposal: [#875](https://github.com/Tencent/teamai-cli/issues/875). Plan: [#879](https://github.com/Tencent/teamai-cli/issues/879). + +A team declares which secrets its members need, in the team repo, with no value. Each member supplies the value on their own machine. No secret value is written to the team repo. + +This document grows with the implementation and describes only what the current version does. Today that is declaring secrets, a member's value for each team or for every team on the machine, `${VAR}` in MCP servers, keeping an MCP entry when a pull can't find a declared secret, telling the member what to run for it, running a CLI with the team's env and secrets through `teamai env exec`, and telling the agent which secrets exist. The same order resolves the team's plain `env.yaml` variables: the member's value for this team, then `env.yaml`; the environment no longer overrides either (see [Variables](#variables)). + +## Declaring secrets + +Secrets live next to the env variables, in a file of their own: + +```yaml +# env/secrets.yaml +secrets: + - key: GITHUB_TOKEN + description: GitHub token with repo scope, for the github MCP server and gh # optional + url: https://github.com/settings/tokens # optional: where a member gets one + - key: GITLAB_TOKEN +``` + +- `key` is required and must be a shell variable name (letters, digits and underscores, not starting with a digit). +- An entry with any other key, `value:` included, is not declared, and `pull` and `teamai doctor` name the file, the secret and the key. A value does not belong in this file. +- A file that does not parse, that has no top-level `secrets:` key, or that defines a key twice is never read as "no secrets": the secrets are not resolved this run, and `env.sh`, the env backup and the MCP servers keep what they had, as for an `env.yaml` that cannot be used; `env exec` applies no variables and no secrets, since any `env.yaml` key may be one the file declares. `pull` warns, `env list` and `mcp list` exit non-zero (`mcp list` shows the servers' variables as `not resolved`), and `teamai doctor` fails the `Team secrets can be resolved` check, each naming the file and the fix. +- An empty file or `secrets: []` declares none. + +It is a separate file so a member on an older CLI, which reads only `env.yaml`, ignores it, and an older `teamai env add` or `env remove`, which rewrite `env.yaml`, cannot drop it. + +An admin declares a secret with `teamai env add --secret`, which takes no value, and publishes it with `teamai push`, which lists a changed `env/secrets.yaml` or `env//secrets.yaml` like an env file, in single-repo mode too. Editing the file directly works as well. + +```text +teamai env add GITHUB_TOKEN --secret -d "GitHub token with repo scope" --url https://github.com/settings/tokens +teamai env add GITHUB_TOKEN --secret --role checkout # or --project : env//secrets.yaml +teamai env remove GITHUB_TOKEN # removes the declaration (same --role / --project) +teamai push +``` + +- `env add KEY --secret` declares the key, or updates the `description` and `url` of a key already declared in that file; an option not passed leaves its field as it was. A value after the key is rejected and not stored, and no output of `env add` or `env remove` names a value. +- `env remove KEY` removes a variable from `env.yaml` when that file sets the key, and otherwise the declaration from the `secrets.yaml` next to it. `env remove KEY --secret` removes only the declaration, for a key both files carry. +- A key the file declares twice, which fails every read of it, is left declared once by `env add KEY --secret` (it updates the first declaration) and not at all by `env remove`; each says how many duplicate declarations it removed. +- Neither command edits a secrets file that does not parse. `--role` and `--project` pick the namespace as they do for variables. + +## Namespaces + +A namespace declares its own secrets in `env//secrets.yaml`. It is active where `env//env.yaml` is: a role or project that lists `` under `resources.env`. The rules are the env rules (see [Env, hooks and MCP servers by namespace](../usage-guide.md#env-hooks-and-mcp-servers-by-namespace)): + +- An active namespace entry replaces the root entry with the same key, whole. +- The same key in two active namespaces, or twice in one file, fails the secrets. +- Legacy mode (a member with no role and a team without `projects.yaml`) reads `env/secrets.yaml` only, and `teamai doctor` notes a key it repeats. + +`teamai doctor` notes each override (`secrets: "GITHUB_TOKEN" from env/checkout/secrets.yaml replaces env/secrets.yaml`). + +## States + +`teamai env list` and `teamai list env` show each declared secret this directory receives, where it comes from, and its state. They never show a value, `--reveal` included; `--reveal` reveals only the env variables. + +| State | Meaning | +|---|---| +| `team` | The member set a value for this team with `teamai env set`. | +| `global` | The member set a value for every team on the machine with `teamai env set --global`, and none for this team. | +| `environment` | The member's own environment has a non-empty value for the key (see [Resolution](#resolution)). | +| `missing` | No value is available. | +| `unreadable` | The member's values file for this team or the machine can't be read, so nobody can tell. | + +```text +Team env variables (2): + + GITLAB_HOST=gi**** team (root) + API_URL=ht**** env.yaml (checkout) + +Team secrets (3): + + GITHUB_TOKEN team (root) + SENTRY_AUTH_TOKEN environment (root) + GITLAB_TOKEN missing (checkout) +``` + +Both commands print this same listing. Each variable shows the value it resolves to (masked unless `--reveal`) and where that comes from: `team` for the member's value (see [Variables](#variables)), `env.yaml` for the team's. While the declarations can't be used, the variables are listed without a value, `--reveal` included, since any of them may be a secret whose repo value is ignored; while the values file can't be read, a variable shows `unreadable` too. With `--verbose`, both print the description and the `url`. + +## Setting a value + +A member keeps their value for a secret the scope declares, or for an env variable it receives (see [Variables](#variables)), for this directory's team: + +```text +teamai env set GITHUB_TOKEN prompts, without echo +printf '%s' "$TOKEN" | teamai env set GITHUB_TOKEN --stdin for the member's own scripts +teamai env set GITHUB_TOKEN --from-env WORK_GITHUB_TOKEN reads WORK_GITHUB_TOKEN each time the value is used; no copy is stored +teamai env set GITHUB_TOKEN --global for every team on this machine; a value set for a team still wins +teamai env unset GITHUB_TOKEN [--global] +``` + +- The value is never taken from an argument, so it stays out of shell history. `--stdin` refuses a terminal. +- `env set` accepts a key the scope declares as a secret or, without `--global`, an `env.yaml` variable it receives; `--global` is for secrets only. On Windows the key may be typed in any case: `env set` and `env unset` use the name the scope declares, `env add --secret` and `env remove --secret` the name already declared, and a value stored under another case of the name is the key's. When the declarations or `env.yaml` cannot be read it changes nothing, since it cannot tell. A project config that exists but can't be read makes `env set`, `env unset` and `env list` fail with its path and why, rather than use the user scope, whose team may not be this project's. +- Outside any scope (no project here and no user scope), `env set --global` accepts any valid key name and notes that no team declares it yet, so a member can set a token they reuse across teams ahead of time. `env unset` accepts any key that has a value. +- `--from-env` warns when the variable is not set in the current shell. While it is unset, the secret is `missing`: the next source in the [order](#resolution) is not used instead, since that could be another account's token. +- Run `teamai pull` afterwards to update the MCP servers, and `env.sh` for a variable. + +## Storage + +- One file per team repo: `~/.teamai/secrets/teams/.json`, named by the SHA-256 hex digest (all 64 characters) of the team repo URL in the member's own `~/.teamai/config.yaml` alone, so renaming `team:` in `teamai.yaml` keeps every member's values. `teamai.yaml`'s `repo:` is not used: a copied or hostile team repo could claim another team's `repo:` and receive that team's values, and the digest is not shortened, since a shorter one lets a hostile team search for another URL whose name matches. Every project and worktree that uses the same team reads the same file, so a member sets a value once per team. +- The URL names the file by what says which repo it is: scheme (the ssh forms `ssh://`, `git+ssh://`, `ssh+git://` and scp-style are one; `https` and `http` are two), the ssh user, host in any case, a port other than the scheme's default (22, 443, 80), and the path, query and fragment as written. An scp-style path that starts with neither `/` nor `~` is in the ssh user's home, so it counts as `~/` followed by the path, the path `ssh://host/~/…` names; an scp-style path that starts with `/`, and every other `ssh://` path, is from the root. Only http(s) credentials, trailing slashes on the path and, for a repo served over ssh or http(s), a trailing `.git` are dropped (`file:///srv/team` and `file:///srv/team.git` are two directories, so two files), so `git@host:acme/team.git` and `ssh://git@host:22/~/acme/team` share one file, and so do `git@host:/acme/team` and `ssh://git@host/acme/team`, while `git@host:acme/team` and `ssh://git@host/acme/team` (one in the user's home, one from the root), `alice@host:team.git` and `bob@host:team.git` (each a path in that user's home), `ssh://host:2222/acme/team` and `ssh://host:2223/acme/team`, and `https://host/team?tenant=a` and `https://host/team?tenant=b` never share values. The ssh, https and http URLs of one repo name different files, so an http and an https endpoint on one host never share values. +- One file for the machine: `~/.teamai/secrets/machine.json`, in the same format. Every scope reads it for the secrets it declares. +- Always under `~/.teamai`, never in the scope's data directory, which in single-repo mode sits inside the business repo. `~/.teamai/env` is not used: it is the user scope's env backup file. +- Written atomically with mode `0600`. That is not encryption: anyone who can read the member's files can read the value. +- Each entry is exactly one of `{"value": "..."}` or `{"env": "VAR"}`, with a `kind`, `secret` or `variable`: what the scope declared the key as when `env set` wrote it. An entry without `kind` is a secret's. An entry is used only as its kind, so a secret's value is never exported as a variable after the team stops declaring the key while `env.yaml` still sets it, and a member's value for a variable never becomes a secret's. `env list` shows an entry of the other kind under its key as not used, with the fix: `teamai env unset KEY`, then `teamai env set KEY`. A file that does not parse is reported by its path only, one that holds any other entry by its path and the entry number, never with its content, and every secret of that team (of every team, for `machine.json`) is `unreadable` until it is fixed. +- Lifetime: uninstalling a project scope leaves the per-team and machine values in place, since another scope may use them; `teamai uninstall` of the user scope removes `~/.teamai`, and the values with it. +- Model profile keys stay where they are ([Model profiles](model-profiles.md)): `env set` does not configure them, and `env/secrets.yaml` cannot declare one. + +## Resolution + +`${VAR}` in `mcp/mcp.yaml` and [`env exec`](#running-a-cli-with-env-exec) resolve a declared secret in this order: + +```text +the member's value for this team teamai env set KEY [--from-env VAR] +> the member's value for the machine teamai env set KEY --global +> the member's own environment not a value a teamai env.sh exported +> missing the server is skipped; env exec runs the command without it +``` + +A team value wins over the environment because it is an explicit choice for that team: otherwise a personal `GITHUB_TOKEN` exported in `.zshrc` would override the token a member set for their work team. A machine value suits a token the member uses with every team; a team that needs another account sets its own value, which wins. + +**The member's own environment.** The shell profile loads the `env.sh` of whichever scope pulled, so the environment also carries values teamai exported. For a key, a value in the environment does not count when it equals what any teamai `env.sh` on the machine exports for that key (`~/.teamai/env.sh`, `~/.teamai/projects/*/env.sh`) or has exported for it before, or, for a declared secret, this scope's `env.yaml` value for it. On Windows, where environment names are case-insensitive, `token` exported by one scope is `TOKEN` for another, so these comparisons, and the `KEY` the markers and records below hash, ignore the key's case there. A non-git project keeps its `env.sh` at `/.teamai/env.sh`, which no scan of known paths finds, so each `env.sh` also exports one marker, `TEAMAI_ENV_SH_`, whose value lists the first 12 hex of the SHA-256 of `KEY=VALUE` for each of its exports and each value its record below keeps, never a value, so a shell that sources a rewritten `env.sh` keeps marking a value an earlier one exported; a value a marker in the environment lists does not count either. A shell opened before a pull keeps the old values in every command it runs, so each `env.sh` keeps a record beside it, `env.sh.exports.json`: for each key, a SHA-256 of `KEY=VALUE` for each of the last 20 values it exported, never the value, so the record adds no copy of a team value or token to the machine (mode `0600`). A command run under `env exec` keeps the markers, so a nested teamai reads the same provenance; an MCP server never gets one. Not covered: a value a shell got from an `env.sh` no scan finds, written by a CLI without the marker; a value older than the last 20 of its key; and one dropped from an `env.sh` by a CLI that kept no record. + +### Variables + +An `env.yaml` variable that isn't a declared secret resolves in one order, in MCP servers, `env exec` and `env.sh`: + +```text +the member's value for this team teamai env set KEY [--from-env VAR] +> env.yaml the root file, or the active namespace file that replaces it +``` + +- The environment no longer overrides it, so a value exported for one team doesn't reach another team's servers. This changes existing teams: a member who exported a variable to override `env.yaml` sets it with `teamai env set KEY` instead. A machine value doesn't apply to a variable. +- An interactive `pull` and `teamai doctor` (as a note) say so when the member's own environment (below) has another value: `` GITLAB_HOST in your environment differs from the value in env/env.yaml, which this team uses. To use yours for this team, run `teamai env set GITLAB_HOST`. `` `doctor` lists it because it runs in the member's shell and explains why an MCP server doesn't use their export. `mcp list` and `env list` don't, and the silent pull prints nothing. No line is printed once the member set a value for the key. +- `env.sh` exports the member's literal value, so a new shell follows the same order. A value stored with `--from-env` is left out of `env.sh`, which holds no copy of it, so a new shell has no value for the key at all, neither the member's nor the team's: the shell has the variable the entry reads, not the key. MCP servers and `env exec` still resolve it. While that variable is unset, the `env.yaml` value is used: unlike a secret's next source, it is the value every other member gets. +- A `${VAR}` the team sets nothing for still resolves from the environment. +- When the member's value file can't be read, MCP servers keep the values the last pull wrote, `pull` leaves `env.sh` as it is, and `teamai doctor` fails the `Your team secret values can be read` check with the reason. + +**Same key twice.** A key declared as a secret and also set as a variable in `env.yaml` (in any case on Windows, where `token` and `TOKEN` are one variable) resolves as the secret, and the repo value is ignored everywhere: it is left out of `env.sh` and the env backup (on every pull, `Already synced` included), out of `env list` and `list env`, `--reveal` included, and out of MCP servers. An older CLI keeps using the variable while the team removes the value. + +**Not bound to a host.** A secret reaches whatever server `mcp.yaml` names, as `${VAR}` always has. Unlike model profile keys, it is not tied to a gateway, so whoever can change `mcp.yaml` or add a namespace decides where members' tokens go. Whoever can push to the team repo already ships hooks that run on every member's machine. + +**Still reachable.** The resolved value is written in plaintext to each tool's MCP config, as before. A config that holds a resolved `${VAR}` value is written `0600`, an existing wider one (`.mcp.json` is often `0644`) included, and a pull that changes nothing in it still tightens it to `0600` without rewriting it, as it does any project config it keeps out of git for holding such a value (a disabled or moved tool's included); one without such a value keeps its mode, and a new one is created `0600`. A command run under `env exec` gets it in its environment, and so does every process it starts: an agent that runs `teamai env exec -- env` can read it. The agent skills forbid that, but nothing enforces it. This keeps secrets out of git, not away from the member's machine or the agent running on it. + +**Out of git.** A resolved value lands in a project-scope MCP config only once the clone's `.git/info/exclude` lists the file (#882). An exclude rule does not stop a file git already tracks, so no resolved value, declared secret or not, is written into a project config `git ls-files` tracks: pull leaves that file as it was (an entry an earlier pull wrote stays), and `pull` (a warning), `teamai mcp list` (`withheld:`) and `teamai doctor` (`MCP servers delivered to ` fails) name the file and the fix: `git rm --cached `, and rotate the token if it was ever committed. An exclusion that fails for another reason (`.git/info` or the exclude file not writable, another teamai command holding it, a git error) leaves the file as it was the same way, with that reason and its fix. + +## A missing secret keeps the MCP entry + +`${VAR}` in `mcp/mcp.yaml` can name a declared secret. The session-start pull runs in the agent's environment, which often lacks the member's shell exports (a GUI-launched tool, or a zsh export under `bash -lc`), so a secret can be there for one pull and gone for the next. When a pull finds no value for a server's declared secret: + +- A server an earlier pull wrote keeps its entry in each tool's config, as it is, and teamai still manages it: a later pull that finds a value updates it. +- A server no pull has written yet is skipped, as before. +- The entry is removed when its server leaves `mcp.yaml`, and by `teamai mcp remove`, `teamai uninstall`, and `teamai init` when it moves the Claude Code root. +- A server that also misses a variable not declared as a secret is removed, as before. Variables that aren't declared as secrets keep today's behaviour. + +A kept entry holds the value the earlier pull wrote. After a secret is rotated or revoked, the server keeps the old value until a pull finds the new one. + +While the declarations fail, `pull` and `teamai mcp inject` change no MCP server: nothing is added, updated or removed, and `mcp inject` exits 1. `mcp remove` and uninstall still remove every managed server. + +## A missing secret tells the member what to run + +An interactive `pull`, `teamai mcp list`, `teamai env list`, `teamai doctor` and `teamai env exec` (on stderr) print one line for each declared secret with no value: the MCP servers that use it, if any, the command that sets it, and the declared `url`. + +```text +github: GITHUB_TOKEN is not set. Run `teamai env set GITHUB_TOKEN` (https://github.com/settings/tokens). +GITLAB_TOKEN is not set. Run `teamai env set GITLAB_TOKEN`. +``` + +- The line comes from the declarations, so it appears for a secret no MCP server uses, with no `mcp.yaml`, with no tool to write to, and with `sharing.mcp.autoApply` off. +- `doctor` prints it as a note (`notes` in `doctor --json`) and exits as it would without it: a server skipped only because a declared secret has no value doesn't fail `MCP servers delivered to `. Any other problem in that tool's servers still fails it. +- The silent session-start pull prints nothing. +- `pull` and `doctor` also say when an entry is kept and may hold an old value (`github: the entry an earlier pull wrote stays in claude and may hold an old GITHUB_TOKEN until a pull finds its value.`), and warn about a key declared as a secret and also set in `env.yaml`, whose value is ignored, naming the file to remove it from. +- A secret stored with `--from-env` whose variable is unset reads as missing too, and its line says so: ``GITHUB_TOKEN reads WORK_GITHUB_TOKEN, which is not set. Set WORK_GITHUB_TOKEN, or run `teamai env set GITHUB_TOKEN` to replace the reference.`` (`--global` in the command for a machine value). +- When the declarations or the member's value file can't be read, no line is printed: the command reports that failure instead. + +## Running a CLI with `env exec` + +A CLI such as `gh`, `glab` or one the company ships reads its token from its environment. `teamai env exec` runs it with this directory's team env: + +```text +teamai env exec -- gh pr create +teamai env exec -- glab mr list GITLAB_HOST from env.yaml and GITLAB_TOKEN from the member, for this directory's team +``` + +- **Scope.** The directory's scope: the project teamai is set up for there, found through git, so every worktree of a project resolves to that project, else the user scope. +- **Environment.** The command inherits teamai's environment, overlaid with the scope's variables in the [variable order](#variables) (a scope variable wins over an inherited one), then with its secrets in the [resolution order](#resolution). A key declared as a secret that has no value for this scope is removed from the command's environment, so the command never sees a value `teamai env list` doesn't show for this scope: another team's export, or the member's own export when this team's value names another variable with `--from-env`. On Windows, where environment names are case-insensitive, a key in any case is the same variable: overlaying one replaces an inherited `api_url` or `API_URL` rather than adding a second name, and removing one removes every case of it. +- **Missing secret.** The [line](#a-missing-secret-tells-the-member-what-to-run) goes to stderr, and the command runs anyway: `gh` and `glab` can still use their own login. +- **Failures.** When the declarations fail, no variable and no secret is applied, and the command runs with the inherited environment: any `env.yaml` key may be a secret the file declares, so its repo value is not passed on, and every inherited value that is not the member's own (one a teamai `env.sh` exports or exported, by its file, its record or its marker) is removed, named on stderr by key only; when `env.yaml` fails, the secrets are applied and no variable is; when the value file can't be read, every declared key is removed and no variable is applied. In both, every inherited value that is not the member's own is removed too, named on stderr by key only, since a shell that sourced another team's `env.sh` would otherwise pass that team's variables on. Each says so on stderr. A project config that exists but can't be read is named on stderr, and the command runs with the inherited environment, without every value that is not the member's own, as when the declarations fail (the `env.sh` beside that config included), named by key only: it is not taken for "no scope", nor for the user scope. +- **No scope.** With no project or user config, the command runs with the inherited environment and a notice on stderr. Machine values are not applied there, since no team declares which keys the command needs. An HTTP team repo delivers no env here either. In both, every inherited value that is not the member's own is removed, as when the declarations fail, named on stderr by key only: a shell that sourced another team's `env.sh` doesn't pass that team's values on. +- **Output.** Everything teamai prints goes to stderr, so the command's stdout can be piped. The exit code is the command's; a command ended by a signal ends teamai with the same signal, or exits 128 + its number for one Node doesn't end on (SIGPIPE, SIGUSR1). A SIGTERM or SIGHUP sent to teamai is passed on. `Ctrl-C` and `Ctrl-\` are not: the terminal already sent them to the command, and a second SIGINT makes tools such as terraform force-quit, so while teamai runs in its terminal's foreground process group it ignores SIGINT and SIGQUIT and waits for the command. It checks once, before starting the command (`ps -o pgid=,tpgid=`). Anywhere else (a background job, a process without a terminal) a SIGINT or SIGQUIT (`kill -INT `) was sent to teamai alone and is passed on, as it is when `ps` can't say. On Windows the console sends `Ctrl-C` to every process attached to it, so teamai ignores it. Node doesn't say who sent a signal, so while teamai runs in its terminal's foreground a SIGINT or SIGQUIT sent to teamai's PID alone (`kill -INT ` from another shell) is not passed on either and the command keeps running: send teamai SIGTERM, which is passed on, or signal the command's PID. A command that can't be started exits 127. +- **Nothing written.** No value is written to disk or to `debug.log`. Finding the scope does what every command that finds one does: it may adopt a project partition, save the user scope's role migration, or set up a freshly cloned single-repo project; none of these writes a value. +- **Inherited as is, with three exceptions.** Without a terminal (every agent), teamai sets `GIT_TERMINAL_PROMPT=0`, `GIT_ASKPASS=echo` and `GCM_INTERACTIVE=never` where they are unset, so a git child never waits for a credential prompt. The command inherits them. +- **Not for agents.** A variable or secret named like one a model profile writes (`ANTHROPIC_*`) overrides that profile for the command. `env exec` is for CLIs, not for starting an agent. +- Put `--` before the command: without it, teamai would read the command's own options as its own (`teamai env exec gh pr list --dry-run` would run nothing), so it prints `Put -- before the command: teamai env exec -- ` and exits 2. teamai's own options may come before `--`. + +## Telling the agent + +An agent that runs `gh` without `env exec` silently uses whatever account its environment has. When the scope declares secrets, the session-start hook adds one line to the agent's context (`additionalContext`, beside the MR and package hints): + +```text +Team secrets in this scope: GITHUB_TOKEN (gh and the github MCP server), SENTRY_AUTH_TOKEN. Run the CLIs that need them through `teamai env exec -- ` so they get this team's values. Never ask for, read or print a secret value; if one is missing, ask the member to run `teamai env set KEY` in their own terminal. +``` + +- The line lists each declared key with its `description`, so the description should say which tool or server uses the key. It carries no value and no state. +- No line when the scope declares no secrets, when its secrets files can't be used (`pull` and `doctor` report that), or in a directory without teamai. +- Hosts that run SessionStart but discard its output (Hermes, Pi, OpenCode, OpenClaw), and JoyCode, which has no hooks, get the same rule from the teamai core skill. +- The skills say an agent never asks for a secret value in chat, never passes one through `--stdin`, never reads the value files and never prints a secret (`teamai env exec -- env` included). On a missing secret it asks the member to run `teamai env set KEY` in their own terminal. Declaring a secret with `env add --secret` takes no value, so an agent can run it. + +## Rotation + +A token that was ever committed to the team repo stays in its git history: rotate it, then declare it here and have each member set the new value. After `teamai env set` with a new value, `teamai pull` writes it to the MCP servers. + +## Declarations are absent, valid or failed + +The declarations a member reads have three outcomes, and consumers keep them apart: `absent` (no secrets file this member reads exists), `valid` (possibly declaring none), and `failed`. A failed file is never read as "no secrets": a consumer that did would act on a team having no secrets while it has some. + +## Workflows (#818) + +Workflows are a future consumer. Recorded here so the two fit ([#818](https://github.com/Tencent/teamai-cli/issues/818)): + +- A step's `requires.env` names keys declared here or in `env.yaml`; there is no second list in the workflow. +- A step gets only the secrets it lists, not every secret the scope declares. +- A missing required key fails the run before it starts, naming the key. +- Unattended runs take secrets from the environment only, never as flags. +- Resolved values are masked before anything is stored (outputs, `result.json`, run events). +- Inputs passed as environment variables cannot shadow a declared key. +- Headless agent steps get the step's environment. +- `run-step` carries no secret values; a remote executor maps `env/secrets.yaml` to its own secret store. diff --git a/docs/designs/team-secrets.zh-CN.md b/docs/designs/team-secrets.zh-CN.md new file mode 100644 index 000000000..596be9c5d --- /dev/null +++ b/docs/designs/team-secrets.zh-CN.md @@ -0,0 +1,230 @@ +# 团队密钥 + +[English](team-secrets.md) + +提案：[#875](https://github.com/Tencent/teamai-cli/issues/875)。实施计划：[#879](https://github.com/Tencent/teamai-cli/issues/879)。 + +团队在团队仓库中声明成员需要哪些密钥，但不写值。每个成员在自己的机器上提供值。密钥的值不会写入团队仓库。 + +本文档随实现逐步补充，只描述当前版本已有的行为。目前包括声明密钥、成员为每个团队或为本机所有团队设置的值、MCP server 中的 `${VAR}`，pull 找不到已声明的密钥时保留 MCP 条目，告诉成员该运行什么命令，通过 `teamai env exec` 用团队的 env 和密钥运行 CLI，以及告诉 agent 有哪些密钥。团队普通的 `env.yaml` 变量也按同样的顺序解析：先取成员为该团队设置的值，再取 `env.yaml`；环境不再覆盖二者（见[变量](#变量)）。 + +## 声明密钥 + +密钥与 env 变量放在一起，但使用单独的文件： + +```yaml +# env/secrets.yaml +secrets: + - key: GITHUB_TOKEN + description: GitHub token with repo scope, for the github MCP server and gh # 可选 + url: https://github.com/settings/tokens # 可选：成员获取 token 的地址 + - key: GITLAB_TOKEN +``` + +- `key` 必填，且必须是 shell 变量名（字母、数字和下划线，不以数字开头）。 +- 条目带有其他任何键（包括 `value:`）时不会被声明，`pull` 和 `teamai doctor` 会指出文件、密钥和该键。值不应该写在这个文件里。 +- 文件无法解析、没有顶层 `secrets:` 键，或同一个 key 定义了两次时，绝不会被当作"没有密钥"：本次不解析密钥，`env.sh`、env 备份和 MCP server 保持原样，与 `env.yaml` 无法使用时相同；`env exec` 不应用任何变量和密钥，因为 `env.yaml` 中的任何 key 都可能是该文件声明的密钥。`pull` 会警告，`env list` 和 `mcp list` 以非零状态退出（`mcp list` 把 server 的变量显示为 `not resolved`），`teamai doctor` 的 `Team secrets can be resolved` 检查失败，它们都会指出文件和修复方法。 +- 空文件或 `secrets: []` 表示没有声明任何密钥。 + +使用单独的文件，是为了让旧版 CLI（只读取 `env.yaml`）忽略它，旧版的 `teamai env add` 或 `env remove`（会重写 `env.yaml`）也不会把它丢掉。 + +管理员用 `teamai env add --secret` 声明密钥（不接受值），再用 `teamai push` 发布：`push` 会像 env 文件一样列出改动过的 `env/secrets.yaml` 或 `env//secrets.yaml`，单仓库模式也一样。也可以直接编辑该文件。 + +```text +teamai env add GITHUB_TOKEN --secret -d "GitHub token with repo scope" --url https://github.com/settings/tokens +teamai env add GITHUB_TOKEN --secret --role checkout # 或 --project ：env//secrets.yaml +teamai env remove GITHUB_TOKEN # 删除该声明（同样支持 --role / --project） +teamai push +``` + +- `env add KEY --secret` 声明该 key；若该文件已声明这个 key，则更新它的 `description` 和 `url`，未传的选项保留原值。key 后面带值会被拒绝且不会保存，`env add` 与 `env remove` 的任何输出都不会出现值。 +- `env remove KEY` 在 `env.yaml` 设置了该 key 时删除这个变量，否则删除同目录 `secrets.yaml` 中的声明。`env remove KEY --secret` 只删除声明，用于两个文件都有该 key 的情况。 +- 文件中声明了两次的 key 会让该文件每次读取都失败：`env add KEY --secret` 更新第一个声明并只保留它，`env remove` 删除它的全部声明；两者都会说明删除了几个重复声明。 +- 两个命令都不会编辑无法解析的密钥文件。`--role` 与 `--project` 选择 namespace 的方式与变量相同。 + +## Namespace + +namespace 在 `env//secrets.yaml` 中声明自己的密钥，生效条件与 `env//env.yaml` 相同：某个角色或项目在 `resources.env` 中列出了 ``。规则与 env 一致（见 [Env、hooks 与 MCP server 按 namespace 划分](../usage-guide.zh-CN.md#envhooks-与-mcp-server-按-namespace-划分)）： + +- 生效 namespace 中的条目整体替换根文件中同 key 的条目。 +- 同一个 key 出现在两个生效的 namespace 中，或在同一文件中出现两次，密钥解析失败。 +- 旧模式（成员没有角色、团队也没有 `projects.yaml`）只读取 `env/secrets.yaml`，`teamai doctor` 会提示其中重复的 key。 + +`teamai doctor` 会把每个覆盖列为提示（`secrets: "GITHUB_TOKEN" from env/checkout/secrets.yaml replaces env/secrets.yaml`）。 + +## 状态 + +`teamai env list` 和 `teamai list env` 显示当前目录收到的每个已声明密钥、它的来源和状态。它们从不显示值，`--reveal` 也一样；`--reveal` 只显示 env 变量的明文。 + +| 状态 | 含义 | +|---|---| +| `team` | 成员用 `teamai env set` 为该团队设置了值。 | +| `global` | 成员用 `teamai env set --global` 为本机所有团队设置了值，且没有为该团队设置值。 | +| `environment` | 成员自己的环境中该 key 有非空值（见[解析顺序](#解析顺序)）。 | +| `missing` | 没有可用的值。 | +| `unreadable` | 成员为该团队或本机保存值的文件无法读取，因此无从判断。 | + +```text +Team env variables (2): + + GITLAB_HOST=gi**** team (root) + API_URL=ht**** env.yaml (checkout) + +Team secrets (3): + + GITHUB_TOKEN team (root) + SENTRY_AUTH_TOKEN environment (root) + GITLAB_TOKEN missing (checkout) +``` + +两个命令打印同一份列表。每个变量显示它解析出的值（不加 `--reveal` 时打码）及其来源：`team` 表示成员自己的值（见[变量](#变量)），`env.yaml` 表示团队的值。声明无法使用时，变量只列出名字、不显示值，`--reveal` 也一样，因为其中任何一个都可能是 repo 值被忽略的密钥；值文件无法读取时，变量同样显示 `unreadable`。加 `--verbose` 时两者都会打印 description 和 `url`。 + +## 设置值 + +成员为当前目录的团队保存某个已声明密钥的值，或它收到的某个 env 变量的值（见[变量](#变量)）： + +```text +teamai env set GITHUB_TOKEN 提示输入，不回显 +printf '%s' "$TOKEN" | teamai env set GITHUB_TOKEN --stdin 供成员自己的脚本使用 +teamai env set GITHUB_TOKEN --from-env WORK_GITHUB_TOKEN 每次使用时读取 WORK_GITHUB_TOKEN，不保存副本 +teamai env set GITHUB_TOKEN --global 对本机所有团队生效；为某个团队设置的值仍然优先 +teamai env unset GITHUB_TOKEN [--global] +``` + +- 值从不通过命令行参数传入，因此不会进入 shell 历史。`--stdin` 拒绝终端输入。 +- `env set` 接受该 scope 声明为密钥的 key，不加 `--global` 时也接受它收到的 `env.yaml` 变量；`--global` 只用于密钥。在 Windows 上 key 可以用任意大小写输入：`env set` 和 `env unset` 使用该 scope 声明的名字，`env add --secret` 和 `env remove --secret` 使用已声明的名字，以其他大小写保存的值也属于该 key。声明或 `env.yaml` 无法读取时它不做任何修改，因为无法判断。项目配置存在但无法读取时，`env set`、`env unset` 和 `env list` 会报出该文件路径和原因并失败，而不是改用用户 scope，因为用户 scope 的团队未必是这个项目的团队。 +- 不在任何 scope 中时（当前目录没有项目，也没有用户 scope），`env set --global` 接受任何合法的 key 名，并提示目前还没有团队声明它，方便成员提前设置在多个团队间复用的 token。`env unset` 接受任何已有值的 key。 +- `--from-env` 指定的变量在当前 shell 中未设置时会警告。变量未设置期间该密钥为 `missing`：不会改用[解析顺序](#解析顺序)中的下一个来源，因为那可能是另一个账号的 token。 +- 之后运行 `teamai pull` 更新 MCP server，变量还会更新 `env.sh`。 + +## 存储 + +- 每个团队仓库一个文件：`~/.teamai/secrets/teams/.json`，只由成员自己 `~/.teamai/config.yaml` 中团队仓库 URL 的 SHA-256 十六进制摘要（完整 64 个字符）命名，所以修改 `teamai.yaml` 中的 `team:` 不会丢失成员的值。不使用 `teamai.yaml` 的 `repo:`：复制来的或恶意的团队仓库可以声称另一个团队的 `repo:`，从而拿到那个团队的值；摘要也不截短，因为截短后恶意团队可以搜索出另一个文件名相同的 URL。使用同一团队的每个项目和 worktree 读取同一个文件，所以成员每个团队只需设置一次。 +- 文件由 URL 中标识仓库的部分命名：协议（ssh 的各种写法 `ssh://`、`git+ssh://`、`ssh+git://` 和 scp 形式算同一个；`https` 与 `http` 是两个）、ssh 用户名、不区分大小写的主机、非协议默认值（22、443、80）的端口，以及原样的路径、查询串（query）和片段（fragment）。scp 形式中既不以 `/` 也不以 `~` 开头的路径位于 ssh 用户的主目录下，因此按 `~/` 加该路径计算，即 `ssh://host/~/…` 所指的路径；以 `/` 开头的 scp 路径以及其他所有 `ssh://` 路径都从根目录算起。只有 http(s) 凭据、路径结尾的斜杠，以及通过 ssh 或 http(s) 提供的仓库路径结尾的 `.git` 会被去掉（`file:///srv/team` 与 `file:///srv/team.git` 是两个目录，因此是两个文件），所以 `git@host:acme/team.git` 与 `ssh://git@host:22/~/acme/team` 共用一个文件，`git@host:/acme/team` 与 `ssh://git@host/acme/team` 也共用一个文件，而 `git@host:acme/team` 与 `ssh://git@host/acme/team`（一个在用户主目录下，一个从根目录算起）、`alice@host:team.git` 与 `bob@host:team.git`（各自是该用户主目录下的路径），`ssh://host:2222/acme/team` 与 `ssh://host:2223/acme/team`，以及 `https://host/team?tenant=a` 与 `https://host/team?tenant=b`，都绝不共享值。同一仓库的 ssh、https 和 http URL 对应不同的文件，因此同一主机上的 http 与 https 端点绝不共享值。 +- 本机一个文件：`~/.teamai/secrets/machine.json`，格式相同。每个 scope 都从中读取自己声明的密钥。 +- 始终位于 `~/.teamai` 下，绝不放在 scope 的数据目录中（单仓模式下该目录在业务仓库内）。不使用 `~/.teamai/env`：它是用户 scope 的 env 备份文件。 +- 以原子方式写入，权限 `0600`。这不是加密：能读取成员文件的人都能读到值。 +- 每个条目恰好是 `{"value": "..."}` 或 `{"env": "VAR"}` 之一，并带有 `kind`（`secret` 或 `variable`）：`env set` 写入时该 scope 把这个 key 声明为什么。没有 `kind` 的条目按密钥的值处理。条目只按它的 `kind` 使用：团队不再把某个 key 声明为密钥、而 `env.yaml` 仍设置它时，密钥的值绝不会作为变量导出；成员为变量设置的值也绝不会成为密钥的值。`env list` 会在该 key 下把另一种 `kind` 的条目显示为未使用，并给出修复方法：`teamai env unset KEY`，然后 `teamai env set KEY`。文件无法解析时只报告路径，含有其他条目时报告路径和条目序号，绝不输出其内容；修复之前，该团队的每个密钥（对 `machine.json` 而言是所有团队的每个密钥）都是 `unreadable`。 +- 生命周期：卸载项目 scope 不会删除按团队保存的值和本机的值，因为其他 scope 可能使用它们；卸载用户 scope（`teamai uninstall`）会删除 `~/.teamai`，值也随之删除。 +- 模型配置的密钥保持原位（[模型配置](model-profiles.zh-CN.md)）：`env set` 不配置它们，`env/secrets.yaml` 也不能声明它们。 + +## 解析顺序 + +`mcp/mcp.yaml` 中的 `${VAR}` 和 [`env exec`](#用-env-exec-运行-cli) 按以下顺序解析已声明的密钥： + +```text +成员为该团队设置的值 teamai env set KEY [--from-env VAR] +> 成员为本机设置的值 teamai env set KEY --global +> 成员自己的环境 不包括 teamai env.sh 导出的值 +> missing 跳过该 server；env exec 不带它运行命令 +``` + +团队值优先于环境，因为它是针对该团队的明确选择：否则 `.zshrc` 中导出的个人 `GITHUB_TOKEN` 会覆盖成员为工作团队设置的 token。本机值适合成员在所有团队中都使用的 token；需要另一个账号的团队设置自己的值，该值优先。 + +**成员自己的环境。** shell profile 加载最近一次 pull 的 scope 的 `env.sh`，因此环境中也带有 teamai 导出的值。对某个 key，环境中的值若等于本机任一 teamai `env.sh`（`~/.teamai/env.sh`、`~/.teamai/projects/*/env.sh`）当前或以前为该 key 导出的值，或者对已声明的密钥而言等于本 scope 的 `env.yaml` 值，则不计入。Windows 的环境变量名不区分大小写，一个 scope 导出的 `token` 就是另一个 scope 的 `TOKEN`，因此在 Windows 上这些比较以及下文标记和记录所哈希的 `KEY` 都忽略 key 的大小写。非 git 项目的 `env.sh` 位于 `/.teamai/env.sh`，扫描已知路径找不到它，因此每个 `env.sh` 还会导出一个标记变量 `TEAMAI_ENV_SH_<其 data home 的 SHA-256，64 位十六进制>`，其值列出它每个导出以及下文记录中保留的每个值的 `KEY=VALUE` SHA-256 前 12 位十六进制，从不包含值本身，因此重新 source 改写后的 `env.sh` 的 shell 仍会标记旧 `env.sh` 导出过的值；环境中某个标记列出的值同样不计入。在 pull 之前打开的 shell 在之后运行的每条命令中都带着旧值，因此每个 `env.sh` 旁边都有一份记录 `env.sh.exports.json`：对每个 key，记录它最近导出的 20 个值各自的 `KEY=VALUE` SHA-256，从不记录值本身，因此这份记录不会在本机多存一份团队的值或令牌（权限 `0600`）。在 `env exec` 下运行的命令保留这些标记，因此嵌套运行的 teamai 读到相同的来源信息；MCP 服务器永远拿不到标记。未覆盖的情况：shell 从扫描找不到、且由不写标记的 CLI 生成的 `env.sh` 得到的值，早于该 key 最近 20 个值的值，以及由不保留记录的 CLI 从 `env.sh` 中去掉的值。 + +### 变量 + +未声明为密钥的 `env.yaml` 变量在 MCP server、`env exec` 和 `env.sh` 中按同一顺序解析： + +```text +成员为该团队设置的值 teamai env set KEY [--from-env VAR] +> env.yaml 根文件，或替换它的活动 namespace 文件 +``` + +- 环境不再覆盖它，因此为一个团队导出的值不会进入另一个团队的 server。这会改变现有团队的行为：原先通过导出变量来覆盖 `env.yaml` 的成员，改用 `teamai env set KEY`。本机值不适用于变量。 +- 当成员自己的环境（见下文）中有不同的值时，交互式 `pull` 和 `teamai doctor`（作为备注）会指出：`` GITLAB_HOST in your environment differs from the value in env/env.yaml, which this team uses. To use yours for this team, run `teamai env set GITLAB_HOST`. `` `doctor` 列出它，因为它在成员的 shell 中运行，可以解释 MCP server 为什么没有使用成员导出的值。`mcp list` 和 `env list` 不列出，静默 pull 什么也不输出。成员为该 key 设置了值之后不再输出。 +- `env.sh` 导出成员设置的字面值，因此新 shell 遵循同一顺序。用 `--from-env` 保存的值不写入 `env.sh`，`env.sh` 中不保存它的副本，因此新 shell 中该 key 完全没有值，既没有成员的值也没有团队的值：shell 里有的是该条目读取的那个变量，而不是这个 key。MCP server 和 `env exec` 仍会解析它。该变量未设置期间使用 `env.yaml` 的值：与密钥的下一个来源不同，这是其他每个成员都拿到的值。 +- 团队没有设置的 `${VAR}` 仍从环境解析。 +- 成员的值文件无法读取时，MCP server 保留上一次 pull 写入的值，`pull` 保持 `env.sh` 不变，`teamai doctor` 的 `Your team secret values can be read` 检查失败并给出原因。 + +**同一个 key 出现两次。** 某个 key 既声明为密钥、又在 `env.yaml` 中设置为变量时（在 Windows 上不区分大小写，`token` 和 `TOKEN` 是同一个变量），按密钥解析，仓库中的值在所有地方都被忽略：不写入 `env.sh` 和 env 备份（每次 pull 都如此，包括 `Already synced`），不出现在 `env list` 和 `list env` 中（`--reveal` 也一样），也不进入 MCP server。旧版 CLI 在团队删除该值之前继续使用该变量。 + +**不绑定主机。** 密钥会发往 `mcp.yaml` 中指定的任何 server，与 `${VAR}` 一贯的行为相同。与模型配置的密钥不同，它不绑定网关，因此能修改 `mcp.yaml` 或添加 namespace 的人决定成员的 token 发往哪里。能推送到团队仓库的人本来就能下发在每个成员机器上运行的 hooks。 + +**仍可访问。** 解析后的值仍以明文写入各工具的 MCP 配置。含有已解析 `${VAR}` 值的配置以 `0600` 写入，已有的更宽权限文件（`.mcp.json` 常为 `0644`）也会收紧，即使 pull 没有改动其中任何内容，也会在不重写文件的情况下收紧为 `0600`，teamai 因含有这类值而排除在 git 之外的任何项目配置（包括已禁用或已移动的工具的配置）也一样；不含这类值的配置保持原权限，新文件以 `0600` 创建。在 `env exec` 下运行的命令会在环境变量中拿到它，它启动的每个进程也一样：agent 运行 `teamai env exec -- env` 就能读到。agent skill 禁止这样做，但没有任何机制强制。这让密钥不进入 git，而不是让它远离成员的机器或在上面运行的 agent。 + +**不进入 git。** 只有在本地克隆的 `.git/info/exclude` 列出某个项目级 MCP 配置之后，解析后的值才会写入该文件（#882）。exclude 规则挡不住 git 已跟踪的文件，因此无论是否为已声明密钥，解析后的值都不会写入 `git ls-files` 已跟踪的项目配置：pull 保持该文件原样（之前 pull 写入的条目保留），`pull`（警告）、`teamai mcp list`（`withheld:`）和 `teamai doctor`（`MCP servers delivered to ` 失败）会指出该文件和修复方法：`git rm --cached `，如果它曾随 token 一起提交过，还要轮换 token。因其他原因无法排除时（`.git/info` 或 exclude 文件不可写、另一个 teamai 命令占用它、git 出错），同样保持该文件原样，并给出对应的原因与修复方法。 + +## 缺少密钥时保留 MCP 条目 + +`mcp/mcp.yaml` 中的 `${VAR}` 可以引用已声明的密钥。会话开始时的 pull 运行在 agent 的环境里，而这个环境常常没有成员 shell 中导出的变量（从图形界面启动的工具，或在 `bash -lc` 下读不到的 zsh export），所以一个密钥可能这次 pull 能找到、下次就找不到。pull 找不到某个 server 所需的已声明密钥时： + +- 之前某次 pull 写入过的 server 会在每个工具的配置中原样保留，并仍由 teamai 管理：之后某次 pull 找到值时会更新它。 +- 还没有任何 pull 写入过的 server 照旧跳过。 +- 该 server 从 `mcp.yaml` 中移除时，条目随之删除；`teamai mcp remove`、`teamai uninstall`，以及 `teamai init` 迁移 Claude Code 根目录时，也会删除它。 +- 同时缺少某个未声明为密钥的变量的 server 照旧删除。未声明为密钥的变量保持现有行为。 + +保留下来的条目里是之前那次 pull 写入的值。密钥轮换或吊销后，server 会一直使用旧值，直到某次 pull 找到新值。 + +声明解析失败期间，`pull` 与 `teamai mcp inject` 不会改动任何 MCP server：不新增、不更新、不删除，`mcp inject` 以 1 退出。`mcp remove` 与 uninstall 仍会删除所有受管理的 server。 + +## 缺少密钥时告诉成员该运行什么 + +交互式 `pull`、`teamai mcp list`、`teamai env list`、`teamai doctor` 和 `teamai env exec`（输出到 stderr）会为每个没有值的已声明密钥打印一行：用到它的 MCP server（如果有）、设置它的命令，以及声明中的 `url`。 + +```text +github: GITHUB_TOKEN is not set. Run `teamai env set GITHUB_TOKEN` (https://github.com/settings/tokens). +GITLAB_TOKEN is not set. Run `teamai env set GITLAB_TOKEN`. +``` + +- 这一行来自声明本身，所以没有 MCP server 用到的密钥、没有 `mcp.yaml`、没有可写入的工具、`sharing.mcp.autoApply` 关闭时也会打印。 +- `doctor` 把它作为备注打印（`doctor --json` 中的 `notes`），退出码与没有这个缺失密钥时相同：只因已声明的密钥没有值而跳过的 server 不会让 `MCP servers delivered to ` 失败。该工具的 server 有其他问题时仍会失败。 +- 会话开始时的静默 pull 不打印任何内容。 +- `pull` 和 `doctor` 还会在条目被保留、可能含有旧值时说明（`github: the entry an earlier pull wrote stays in claude and may hold an old GITHUB_TOKEN until a pull finds its value.`），并在某个 key 既声明为密钥、又在 `env.yaml` 中设置时发出警告：该值被忽略，并指出应从哪个文件删除它。 +- 用 `--from-env` 保存、但对应变量未设置的密钥同样视为缺失，提示行会说明这一点：``GITHUB_TOKEN reads WORK_GITHUB_TOKEN, which is not set. Set WORK_GITHUB_TOKEN, or run `teamai env set GITHUB_TOKEN` to replace the reference.``（机器级的值在命令中带 `--global`）。 +- 声明或成员的值文件无法读取时不打印这一行：命令会改为报告该失败。 + +## 用 `env exec` 运行 CLI + +`gh`、`glab` 或公司发布的 CLI 从自己的环境变量读取 token。`teamai env exec` 用当前目录的团队 env 运行它： + +```text +teamai env exec -- gh pr create +teamai env exec -- glab mr list GITLAB_HOST 来自 env.yaml，GITLAB_TOKEN 来自成员，都按当前目录的团队 +``` + +- **Scope。** 当前目录的 scope：teamai 在此处配置的项目（通过 git 查找，因此项目的每个 worktree 都解析到该项目），否则是用户 scope。 +- **环境。** 命令继承 teamai 的环境，先按[变量顺序](#变量)叠加该 scope 的变量（scope 变量覆盖继承的同名变量），再按[解析顺序](#解析顺序)叠加它的密钥。声明为密钥、但在该 scope 下没有值的 key 会从命令的环境中移除，因此命令永远拿不到 `teamai env list` 不会显示为该 scope 的值：另一个团队导出的值，或者该团队的值用 `--from-env` 指向另一个变量时成员自己导出的值。Windows 上环境变量名不区分大小写，任意大小写的 key 都是同一个变量：叠加时替换继承的 `api_url` 或 `API_URL`，而不是再加一个同名变量；移除时移除它的所有大小写形式。 +- **缺少密钥。** 那一[行提示](#缺少密钥时告诉成员该运行什么)输出到 stderr，命令照常运行：`gh` 和 `glab` 仍可以使用它们自己的登录。 +- **失败。** 声明失败时，不应用任何变量和密钥，命令以继承的环境运行：`env.yaml` 中的任何 key 都可能是该文件声明的密钥，因此不传递它在仓库中的值，并移除继承环境中每个不属于成员自己的值（teamai `env.sh` 导出或曾经导出的值，按文件、记录或标记判断），在 stderr 上只列出 key 名；`env.yaml` 失败时，只应用密钥，不应用任何变量；值文件无法读取时，移除所有已声明的 key，也不应用任何变量。这两种情况下，每个不属于成员自己的继承值也会被移除，并在 stderr 上只列出 key 名，否则 sourced 过另一个团队 `env.sh` 的 shell 会把该团队的变量传下去。每种情况都会在 stderr 上说明。项目配置存在但无法读取时，会在 stderr 上指出该文件，并以继承的环境运行命令，与声明失败时一样移除每个不属于成员自己的值（包括该配置旁 `env.sh` 导出的值），只列出 key 名：既不当作"没有 scope"，也不回退到用户 scope。 +- **没有 scope。** 既没有项目配置也没有用户配置时，命令以继承的环境运行，并在 stderr 上给出提示。这里不应用本机值，因为没有团队声明命令需要哪些 key。HTTP 团队仓库在这里同样不提供 env。这两种情况下，与声明失败时一样，每个不属于成员自己的继承值都会被移除，并在 stderr 上只按 key 名列出：sourced 过另一个团队 `env.sh` 的 shell 不会把该团队的值传下去。 +- **输出。** teamai 打印的所有内容都输出到 stderr，因此命令的 stdout 可以直接接管道。退出码就是命令的退出码；命令被信号终止时，teamai 以同一信号结束；对于 Node 不会因之退出的信号（SIGPIPE、SIGUSR1），则以 128 + 信号编号退出。发给 teamai 的 SIGTERM 或 SIGHUP 会转发给命令。`Ctrl-C` 和 `Ctrl-\` 不转发：终端已经把它们发给了命令，第二个 SIGINT 会让 terraform 等工具强制退出，因此当 teamai 运行在其终端的前台进程组中时，它忽略 SIGINT 和 SIGQUIT 并等待命令结束。它在启动命令之前检查一次（`ps -o pgid=,tpgid=`）。在其他情况下（后台作业、没有终端的进程），SIGINT 或 SIGQUIT（`kill -INT `）是只发给 teamai 的，会转发给命令；`ps` 无法判断时也一样转发。在 Windows 上，控制台会把 `Ctrl-C` 发给所有附着在它上面的进程，因此 teamai 忽略它。Node 无法得知信号的发送者，因此当 teamai 运行在其终端的前台时，只发给 teamai PID 的 SIGINT 或 SIGQUIT（例如从另一个 shell 执行 `kill -INT `）同样不会转发，命令会继续运行：请向 teamai 发送 SIGTERM（会转发），或直接向命令的 PID 发信号。无法启动的命令以 127 退出。 +- **不写入值。** 不会把任何值写入磁盘或 `debug.log`。查找 scope 的行为与其他查找 scope 的命令相同：可能接管项目分区、保存用户 scope 的角色迁移，或为刚克隆的单仓项目完成配置；这些写入都不包含值。 +- **原样继承，有三个例外。** 没有终端时（所有 agent 都是这种情况），teamai 会在 `GIT_TERMINAL_PROMPT=0`、`GIT_ASKPASS=echo` 和 `GCM_INTERACTIVE=never` 未设置时设置它们，让 git 子进程不会等待凭据提示。命令会继承它们。 +- **不用于启动 agent。** 与模型配置写入的变量同名的变量或密钥（`ANTHROPIC_*`）会为该命令覆盖那个模型配置。`env exec` 用于 CLI，而不是用来启动 agent。 +- 在命令前加 `--`：否则 teamai 会把命令自己的选项当作 teamai 的选项（`teamai env exec gh pr list --dry-run` 什么也不会运行），因此它会输出 `Put -- before the command: teamai env exec -- ` 并以退出码 2 结束。teamai 自己的选项可以放在 `--` 之前。 + +## 告诉 agent + +不经过 `env exec` 运行 `gh` 的 agent 会悄无声息地使用它环境里恰好有的账号。scope 声明了密钥时，session-start hook 会在 agent 的上下文中加一行（`additionalContext`，与 MR 提示和 package 提示并列）： + +```text +Team secrets in this scope: GITHUB_TOKEN (gh and the github MCP server), SENTRY_AUTH_TOKEN. Run the CLIs that need them through `teamai env exec -- ` so they get this team's values. Never ask for, read or print a secret value; if one is missing, ask the member to run `teamai env set KEY` in their own terminal. +``` + +- 这一行列出每个已声明的 key 及其 `description`，因此 description 应写明哪个工具或 server 使用该 key。它不含任何值，也不含状态。 +- scope 没有声明密钥、密钥文件无法使用（`pull` 和 `doctor` 会报告）或目录中没有 teamai 时，不加这一行。 +- 运行 SessionStart 但丢弃其输出的宿主（Hermes、Pi、OpenCode、OpenClaw），以及没有 hook 的 JoyCode，从 teamai core skill 获得同样的规则。 +- skill 规定 agent 从不在对话中索要密钥值，从不通过 `--stdin` 传入值，从不读取值文件，也从不打印密钥（包括 `teamai env exec -- env`）。缺少密钥时，它请成员在自己的终端运行 `teamai env set KEY`。用 `env add --secret` 声明密钥不带值，因此 agent 可以运行它。 + +## 轮换 + +曾经提交到团队仓库的 token 会保留在 git 历史中：先轮换它，再在这里声明，并让每个成员设置新值。用 `teamai env set` 设置新值后，`teamai pull` 会把它写入 MCP server。 + +## 声明结果分为不存在、有效和失败 + +成员读取的声明有三种结果，使用方必须区分：`absent`（该成员读取的密钥文件都不存在）、`valid`（可能一个都没声明）和 `failed`。失败的文件绝不会被当作"没有密钥"：否则使用方会在团队明明有密钥时按没有密钥处理。 + +## 工作流（#818） + +工作流是未来的使用方。这里记录约定，保证两者衔接（[#818](https://github.com/Tencent/teamai-cli/issues/818)）： + +- 步骤的 `requires.env` 引用这里或 `env.yaml` 中声明的 key，工作流中不再另列一份。 +- 步骤只拿到它列出的密钥，而不是该 scope 声明的全部密钥。 +- 缺少必需的 key 时，运行在开始前失败，并指出该 key。 +- 无人值守的运行只从环境变量获取密钥，绝不通过命令行参数传入。 +- 解析出的值在存储任何内容（outputs、`result.json`、运行事件）之前都会被遮盖。 +- 以环境变量形式传入的 inputs 不能覆盖已声明的 key。 +- 无头（headless）Agent 步骤获得该步骤的环境。 +- `run-step` 不携带任何密钥值；远程执行器把 `env/secrets.yaml` 映射到它自己的密钥存储。 diff --git a/docs/product-overview.md b/docs/product-overview.md index ccd2a6a9a..66b2b2fef 100644 --- a/docs/product-overview.md +++ b/docs/product-overview.md @@ -61,7 +61,7 @@ Each resource is delivered to every agent: | **Agents** | `agents/.yaml`, `agents//.yaml` | | | **Culture** | `culture.md` | Team mission, values, and working principles — injected into each agent's CLAUDE.md / AGENTS.md so every session inherits them | | **CLAUDE.md** | `claudemd/*.md` | | -| **Env** | `env/env.yaml`, `env//env.yaml` | Shared team-level environment variables and switches; do not put secrets here | +| **Env** | `env/env.yaml`, `env//env.yaml` | Shared team-level environment variables and switches; do not put secret values here: declare a secret without its value in `env/secrets.yaml` | | **Hooks** | `hooks/hooks.yaml`, `hooks//hooks.yaml` | | | **MCP** | `mcp/mcp.yaml`, `mcp//mcp.yaml` | | | **Packages** | `teamai.yaml` | Currently npm packages and Claude Code plugins only | diff --git a/docs/product-overview.zh-CN.md b/docs/product-overview.zh-CN.md index c09f9f49c..38c39c93a 100644 --- a/docs/product-overview.zh-CN.md +++ b/docs/product-overview.zh-CN.md @@ -61,7 +61,7 @@ teamai push → 创建分支 + MR → reviewer 审批合并 | **Agents** | `agents/.yaml`、`agents//.yaml` | | | **Culture** | `culture.md` | 团队使命、价值观与协作准则——注入各 Agent 的 CLAUDE.md / AGENTS.md，成为每次会话的行事底色 | | **CLAUDE.md** | `claudemd/*.md` | | -| **Env** | `env/env.yaml`、`env//env.yaml` | 通用环境变量、团队级开关；不建议直接放密钥 | +| **Env** | `env/env.yaml`、`env//env.yaml` | 通用环境变量、团队级开关；不要直接放密钥的值：密钥在 `env/secrets.yaml` 中只声明、不写值 | | **Hooks** | `hooks/hooks.yaml`、`hooks//hooks.yaml` | | | **MCP** | `mcp/mcp.yaml`、`mcp//mcp.yaml` | | | **Packages** | `teamai.yaml` | 目前只支持 npm 包和 Claude 插件 | diff --git a/docs/usage-guide.md b/docs/usage-guide.md index 53a109f8c..d52549332 100644 --- a/docs/usage-guide.md +++ b/docs/usage-guide.md @@ -515,7 +515,7 @@ knowledge on main is left exactly in place). - `.teamai/hooks/hooks.yaml` — team hooks - `.teamai/mcp/mcp.yaml` — shared MCP servers -> **Heads-up on `env`.** In single-repo mode `.teamai/env/env.yaml` **is committed to main** (unlike standalone mode's per-machine env), so it travels to everyone who clones the repo. `env.yaml` stores plaintext key/value pairs — put only non-secret shared config there, and keep real secrets in your own untracked environment. +> **Heads-up on `env`.** In single-repo mode `.teamai/env/env.yaml` **is committed to main** (unlike standalone mode's per-machine env), so it travels to everyone who clones the repo. `env.yaml` stores plaintext key/value pairs — put only non-secret shared config there. Declare a secret without its value in `.teamai/env/secrets.yaml` (see [Team secrets](designs/team-secrets.md)) and keep the value in your own untracked environment. > **Limitation.** Single-repo mode ties one team setup to one business repo. If you need to share one team knowledge base across many business repos, use a standalone team repo (`teamai init `) instead. @@ -997,7 +997,11 @@ projects: `teamai hooks list` and `teamai list --source repo` show each entry's namespace and whether it overrides the root, and name every entry that is not delivered, with why; `teamai status` counts per namespace and names - them too; `teamai doctor` lists each override as a note. + them too; `teamai doctor` lists each override as a note. A variable takes your + value for this team when you set one with `teamai env set KEY`, else the file's; + the environment doesn't override either, and `env.sh` exports that value (not one + set with `--from-env`). `teamai env list` and `teamai list env` show that value + with where it comes from, `team` or `env.yaml`. - **Upgrade every member first.** teamai 0.25.0 and the 0.26.0 betas reject a `resources:` key they do not know, so declaring `env`, `hooks` or `mcp` breaks their pull. From this version on, an unknown `resources:` key only warns, and @@ -1055,6 +1059,86 @@ variables: description: Team API endpoint # optional ``` +**Secrets.** A secret the team needs is declared with no value, in +`env/secrets.yaml` or a namespace's `env//secrets.yaml` (active like +`env//env.yaml`, and a namespace entry replaces the root entry with the same +key). Each member keeps the value on their own machine. + +```yaml +secrets: + - key: GITHUB_TOKEN + description: GitHub token with repo scope # optional + url: https://github.com/settings/tokens # optional: where a member gets one +``` + +```bash +teamai env add GITHUB_TOKEN --secret -d "GitHub token with repo scope" --url https://github.com/settings/tokens +teamai env remove GITHUB_TOKEN # a key env.yaml does not set; --secret for one both files carry +teamai push +``` + +`teamai env add KEY --secret` declares a key, or updates its description and url, +in the root file or, with `--role` / `--project`, the namespace's; it takes no +value and prints none. + +Each member sets their value for this directory's team, never as an argument: + +```bash +teamai env set GITHUB_TOKEN # prompts, without echo +teamai env set GITHUB_TOKEN --stdin # from a pipe +teamai env set GITHUB_TOKEN --from-env WORK_GITHUB_TOKEN # read from that variable when used +teamai env set GITHUB_TOKEN --global # for every team on this machine +teamai env unset GITHUB_TOKEN [--global] +``` + +`env set` accepts a key the scope declares as a secret or, without `--global`, an +`env.yaml` variable it receives, and stores the value in +`~/.teamai/secrets/teams/.json` (mode `0600`), one file per team +repo, named by the team repo URL in your `~/.teamai/config.yaml` (not `teamai.yaml`'s `repo:`) so renaming `team:` keeps it; with `--global`, in `~/.teamai/secrets/machine.json`, for every team on the +machine, and a value set for a team still wins. Outside any scope, `--global` +accepts any valid key and notes that no team declares it yet. A value stays the +kind the key had when you set it: once the team stops declaring a secret that +`env.yaml` also sets, your value is not used for the variable, and `env list` +says to run `teamai env unset KEY`, then `teamai env set KEY`. +`teamai env list` and `teamai list env` show each declared secret as +`team` (you set it for this team), `global` (you set it for the machine), +`environment` (your own environment has a value for it), `missing`, or +`unreadable` (your values file can't be read), and never show a value, `--reveal` included. A key declared as a +secret and also set in `env.yaml` is a secret: its `env.yaml` value is not +exported to `env.sh` or listed. A secrets file that cannot be used is not read +as "no secrets": `env.sh` and the MCP servers keep what they had, `pull` warns, +`env list` and `mcp list` exit non-zero (`env list` then shows no variable +value, since any of them may be a secret), and `teamai doctor` fails a check +naming the file. A values file that can't be read fails +`Your team secret values can be read`. `teamai push` picks up a +change to any secrets file. See [Team secrets](designs/team-secrets.md). + +A CLI such as `gh` or `glab` gets this directory's variables and secrets when it +runs under `teamai env exec`, which finds the scope the same way for every +worktree of a project: + +```bash +teamai env exec -- gh pr create +teamai env exec -- glab mr list +``` + +The command inherits your environment, overlaid with the scope's `env.yaml` +variables and its secrets in the [resolution order](designs/team-secrets.md#resolution); +a declared secret with no value for this scope is removed from it. Put `--` +before the command: without it, teamai would read the command's flags as its +own, so it says so and exits 2. A missing secret prints the `teamai env set` +line on stderr and the command runs anyway. Everything teamai prints goes to +stderr, and the exit code is the command's. With no teamai config here, the +command runs with your environment and a notice. No value is written to disk. +See [Running a CLI with `env exec`](designs/team-secrets.md#running-a-cli-with-env-exec). + +When the scope declares secrets, the session-start hook tells the agent which +keys exist, with their `description`, and to run the CLIs that need them through +`teamai env exec --`. Agents whose tool discards hook output get the same rule +from the teamai core skill. An agent never asks for a secret value: when one is +missing, it asks you to run `teamai env set KEY` in your own terminal. See +[Telling the agent](designs/team-secrets.md#telling-the-agent). + A variable that no longer reaches this directory is removed from `env.sh` on the next pull, even one that reports `Already synced` because the team repo has not moved. Until that pull runs, `teamai doctor` reports a variable that @@ -1162,11 +1246,11 @@ Claude Code also reads the root `.mcp.json`, so this file is shared by both tool Copilot uses its native `mcpServers` schema: `stdio` becomes `type: "local"`, remote transports keep `http` or `sse`, and every managed entry gets the required `tools: ["*"]` allowlist. TeamAI honors `COPILOT_HOME`; project configuration uses Copilot CLI's documented `.github/mcp.json` repository location. See [Adding MCP servers for GitHub Copilot CLI](https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/add-mcp-servers). Codex supports `stdio` and `http`; `sse` is skipped. Qoder supports the Claude-compatible `mcpServers` format in its scope-specific `.qoder/settings.json`. Kiro supports the same `mcpServers` format in its dedicated, mcpServers-only `.kiro/settings/mcp.json` (see [Kiro's MCP configuration docs](https://kiro.dev/docs/mcp/configuration/)). OpenCode supports `stdio` (written as its `type:"local"` shape) and `http` (`type:"remote"`); `sse` is skipped, and its servers live under the `mcp` key of the shared `opencode.json`. Ownership is tracked in `~/.teamai/managed-mcp.json` — hand-added servers are left alone; name collisions skip unless `--force`. -**Secrets.** Write `${VAR}`, never a literal, in `mcp.yaml`. Values resolve from the environment, then from the team env variables this directory receives (`env/env.yaml` and the active `env//env.yaml`). Unresolved variables skip the server with a hint. +**Secrets.** Write `${VAR}`, never a literal, in `mcp.yaml`. A key the team declares in `env/secrets.yaml` resolves from your value for this team (`teamai env set`), then your value for the machine (`teamai env set --global`), then your own environment, which leaves out values a teamai `env.sh` exported (see [Team secrets](designs/team-secrets.md#resolution)). Any other variable resolves from your value for this team (`teamai env set KEY`), then from the team env variables this directory receives (`env/env.yaml` and the active `env//env.yaml`); the environment fills only a key the team sets nothing for, and no longer overrides a team variable (see [Team secrets](designs/team-secrets.md#variables)). An interactive `pull` and `teamai doctor` say when your export differs from the team's value and is ignored. Unresolved variables skip the server with a hint. A declared secret is different: when a pull can't find it, the entry an earlier pull wrote stays as it is, so it may hold a value that was since rotated, until a pull finds the new one (see [Team secrets](designs/team-secrets.md#a-missing-secret-keeps-the-mcp-entry)). An interactive `pull`, `teamai mcp list`, `teamai env list`, `teamai doctor` and `teamai env exec` name a declared secret with no value, the servers that use it and the command that sets it: `` github: GITHUB_TOKEN is not set. Run `teamai env set GITHUB_TOKEN` (). `` -teamai **resolves every `${VAR}` to its value and writes it verbatim** into each tool's config (new files are created `0600`). It does not rely on any tool's own env-var expansion: that expansion is fragile — most decisively, IDEs launched from the GUI (Dock/Launchpad) never inherit your shell's exported variables, so a `${VAR}` placeholder expands to empty and the server 401s. Resolving to plaintext makes the token present no matter how the tool is started. +teamai **resolves every `${VAR}` to its value and writes it verbatim** into each tool's config, which is then written `0600`, an existing `0644` one included (a config without a resolved value keeps its mode; new files are created `0600`). It does not rely on any tool's own env-var expansion: that expansion is fragile — most decisively, IDEs launched from the GUI (Dock/Launchpad) never inherit your shell's exported variables, so a `${VAR}` placeholder expands to empty and the server 401s. Resolving to plaintext makes the token present no matter how the tool is started. -> ⚠️ **The resolved token lands on disk.** Project-scope MCP configs (`.mcp.json`, `.github/mcp.json`, `.cursor/mcp.json`, `.codex/config.toml`, `opencode.json`) then contain the literal secret — add them to `.gitignore` and never commit them. +> ⚠️ **The resolved token lands on disk.** Project-scope MCP configs (`.mcp.json`, `.github/mcp.json`, `.cursor/mcp.json`, `.codex/config.toml`, `opencode.json`) then contain the literal secret. Whenever such a file would hold a value teamai resolved and git would track it, teamai lists the path in the clone's `.git/info/exclude`, inside a `# [teamai:mcp-exclude:start]` block (the worktrees of a repo share it), before it writes the value. A config reached through a symlinked directory (say `.cursor/` linking to `config/`) is judged where the write lands: that path (`/config/mcp.json`) is the one listed, checked and reported, and a tracked one is named with both paths. A symlink at the file itself is replaced by the write, so there the file's own path counts. That covers a file this pull did not write: one written earlier for a tool since disabled, one at the built-in location of a tool the team has dropped from `toolPaths` or moved elsewhere (it counts while it holds any MCP server, since teamai's record for that tool describes another file or none; one another tool maps today, such as CodeBuddy's `.mcp.json`, which Claude maps, while it holds a server that tool did not write, as below), one written under a `toolPaths` mapping the team has since changed (each worktree records the files it wrote a resolved value to in `managed-mcp-files.json`, beside its `managed-mcp.json`; for one an older teamai wrote before it kept that record, the first pull reads each `mcpProject` path in the team repo's history of `teamai.yaml`, and the built-in ones teamai has since changed (CodeBuddy's `.codebuddy/mcp.json`), once, as far as the clone has it, inside the project only, skipping a path the same tool maps today; such a file counts while it holds any MCP server, since teamai's record for the tool describes only today's path (one another tool maps today, while it holds a server that tool did not write, as below), and `teamai doctor` checks the same files until that pull; one git tracks is not listed, since a line does nothing for it, but is recorded as tracked whatever it holds, judged as the others once git no longer tracks it (`git rm --cached`), and forgotten once it is gone from both the disk and git), or one still holding a server since removed from `mcp.yaml`. An entry a pull wrote with a resolved value counts while it is unchanged, even after the team makes its `${VAR}` a literal. While the worktree has no `managed-mcp.json` at all (lost, or before its first pull), a config git does not track counts while it holds a server no record claims, one of your own included: the pull notes those servers in `managed-mcp-files.json`, as when it rebuilds a lost record, and they keep its path until they leave the file; `teamai doctor` checks the same way. So does a config a pull writes a tool's first record for while `managed-mcp.json` holds none for that tool (lost, or teamai's first delivery to it). A path git cannot say it ignores is listed all the same once `git ls-files` shows the file untracked; when git cannot say that either, it counts as git failing. When it cannot — `.git/info` or the exclude file is not writable, another teamai command holds the exclude file past a short wait, git already tracks the file, a rule in your own git ignore files re-includes it (say `!/.mcp.json`; the warning names it), or git fails — it leaves that file as it was (an entry an earlier pull wrote stays), warns with the reason and the fix, and `teamai mcp list` and `teamai doctor` report the server as withheld from each tool a pull would write it to; make the file writable (or `git rm --cached` the tracked file, or remove the rule that re-includes it) and run `teamai pull` again. A tracked file is reported first, and listed nowhere. The committed `.gitignore` is left alone, a path git already ignores adds nothing, and a pull, `teamai mcp remove` and `teamai uninstall` remove a path from the block (the block with its last path) once that file is gone, holds no MCP server, or holds none of: a team server with a resolved value, an entry of teamai's that cleanup left, a server that was in the file when teamai rebuilt a lost `managed-mcp.json`, or the value (8+ characters) of a variable still set in the environment, with teamai's record of what it wrote there (`managed-mcp.json`) present before the command ran, readable, and holding an entry for that file's tool (for a file two tools map, such as Claude and CodeBuddy on `.mcp.json`: for each tool `managed-mcp-files.json` says wrote a resolved value there, or for each tool mapping it when it names none; an empty, unreadable or truncated record proves nothing, and neither does one written by a pull that rebuilt it or found no record for its tool in `managed-mcp.json`, while that pull could not note the file's other servers in `managed-mcp-files.json`, until a later pull notes them). A file written under a mapping since changed, one at the built-in location of a tool the team dropped or moved (unless another tool maps it today), or one in a linked worktree of a nested repository, needs to be gone or hold no MCP server. One written for a tool the team has since moved elsewhere (recorded, found in that history, or at the tool's built-in location), that another tool's mapping still reaches, also keeps its path while it holds a server the tools now mapping it did not write (by their `managed-mcp.json` record); as in any file under a changed mapping, a server of your own there keeps it too. `teamai uninstall` applies that to the file in every worktree of the repository; a pull and `teamai mcp remove` apply it only to the current worktree's file, and keep the path while the file in any other worktree still holds an MCP server: an entry that worktree's last pull wrote (say, a `${VAR}` the team has since made a literal) is judged only by a pull there. A path a pull listed and then wrote no value into (the file does not parse, or holds a server of your own under the team's name) comes out again at the end of that pull, and so does its record in `managed-mcp-files.json`. Otherwise, or for a file it cannot check (for example one that does not parse), the path stays, and `teamai uninstall` warns, naming the file and why: remove teamai's servers from it, then delete that line yourself (with its last line, the block's markers). `teamai doctor` reports such a file git would still commit or cannot answer for — for example one already tracked: `git rm --cached` it and rotate the token. Claude Code may show project `.mcp.json` servers as pending approval until you accept them once in an interactive session. @@ -1938,7 +2022,7 @@ Besides the provider, clone, config and hook checks, `doctor` verifies what reac Two tools do not read a rules directory, so a per-file check cannot speak for them and each gets one of its own. `Team rules are active in opencode` checks that `opencode.json` still lists the glob the pull owns under `instructions`: OpenCode does not auto-scan `.opencode/rules`, so without it every delivered `.md` is inert while the per-file check keeps passing. `Team rules are inlined in Hermes SOUL.md` compares the teamai-managed block of `SOUL.md` with what the team rules inline to, since Hermes reads standing instructions from that one file rather than from a directory — a deleted block, or one left on an older rule set, is a tool reading the wrong rules with nothing on disk to show for it. -`MCP servers delivered to ` compares each server the team's `mcp.yaml` resolves for that tool against the entry in the tool's own config, and names any the reconcile skipped with its reason. The comparison is the entry, not the name: reconciliation leaves an entry teamai does not own alone, so a server of your own under a team name holds the key while the team's definition never arrives, and a stale copy is just as undelivered. Both are reported as `not the team's definition`, and only `teamai pull --force` replaces an entry teamai did not write. An unresolved `${VAR}` is reported here with the variable's name, which is otherwise said once during a pull and never again. An `mcp.yaml` that does not parse is not a team without MCP: it is reported as `Team MCP servers can be read` with the parse error, since it injects nothing into any tool and every run after the first is silent about it. Team hooks and team model profiles that cannot be resolved (a file that does not parse, a name defined twice in one file, or one name in two active namespaces) fail `Team hooks can be resolved` and `Team model profiles can be resolved` with the reason pull logs once; `teamai status` points here when it counts them as 0. `Env variables injected in shell profile` no longer stops at finding the marker comment: it checks that `env/env.yaml` parses and declares its variables under the `variables:` key (a plain `KEY: value` mapping parses as none, while an explicit `variables: []` is a configuration with nothing to deliver and fails nothing), that each one reached `env.sh` with the value `env.yaml` declares — a key left over from an older value exports it to every shell and MCP server until the next pull, and the comparison reads `env.sh` back through the generator's own inverse, so a multiline value quoted across several lines is matched rather than called stale — and that this scope's injected block (the one sourcing its own `env.sh`, since a profile can also carry another scope's) would actually load it — an unquoted Windows path degrades to something a POSIX shell cannot read, so `source` never runs and nothing says so. `No stale env blocks left behind` is a separate check: which file `pull` prefers has changed over time (Windows Git Bash's login shell reads `.bash_profile`/`.bash_login`/`.profile`, never `.bashrc`), and a pull only ever adds a block, never migrates an old one away, so a dead block from an earlier install or platform change can sit in another candidate file indefinitely. It names every such file (checking `.zshrc`, `.bashrc`, `.bash_profile`, `.bash_login` and `.profile`, current and legacy spellings alike) and points at `teamai uninstall` to remove them — separately from delivery, so a working env block never reads as broken just because an old one is still lying around. +`MCP servers delivered to ` compares each server the team's `mcp.yaml` resolves for that tool against the entry in the tool's own config, and names any the reconcile skipped with its reason. The comparison is the entry, not the name: reconciliation leaves an entry teamai does not own alone, so a server of your own under a team name holds the key while the team's definition never arrives, and a stale copy is just as undelivered. Both are reported as `not the team's definition`, and only `teamai pull --force` replaces an entry teamai did not write. An unresolved `${VAR}` is reported here with the variable's name, which is otherwise said once during a pull and never again. A declared secret with no value is not a failure: doctor prints it as a note (`notes` in `--json`) with the command that sets it, and the exit code stays as it would be without it; a note also says when an entry kept for it may hold an old value, and when a key is declared as a secret and also set in `env.yaml`. An `mcp.yaml` that does not parse is not a team without MCP: it is reported as `Team MCP servers can be read` with the parse error, since it injects nothing into any tool and every run after the first is silent about it. Team hooks and team model profiles that cannot be resolved (a file that does not parse, a name defined twice in one file, or one name in two active namespaces) fail `Team hooks can be resolved` and `Team model profiles can be resolved` with the reason pull logs once; `teamai status` points here when it counts them as 0. `Env variables injected in shell profile` no longer stops at finding the marker comment: it checks that `env/env.yaml` parses and declares its variables under the `variables:` key (a plain `KEY: value` mapping parses as none, while an explicit `variables: []` is a configuration with nothing to deliver and fails nothing), that each one reached `env.sh` with the value `env.yaml` declares, or your value for this team (one set with `--from-env` is not written there) — a key left over from an older value exports it to every shell and MCP server until the next pull, and the comparison reads `env.sh` back through the generator's own inverse, so a multiline value quoted across several lines is matched rather than called stale — and that this scope's injected block (the one sourcing its own `env.sh`, since a profile can also carry another scope's) would actually load it — an unquoted Windows path degrades to something a POSIX shell cannot read, so `source` never runs and nothing says so. `No stale env blocks left behind` is a separate check: which file `pull` prefers has changed over time (Windows Git Bash's login shell reads `.bash_profile`/`.bash_login`/`.profile`, never `.bashrc`), and a pull only ever adds a block, never migrates an old one away, so a dead block from an earlier install or platform change can sit in another candidate file indefinitely. It names every such file (checking `.zshrc`, `.bashrc`, `.bash_profile`, `.bash_login` and `.profile`, current and legacy spellings alike) and points at `teamai uninstall` to remove them — separately from delivery, so a working env block never reads as broken just because an old one is still lying around. `Contributed learnings are published` fails while `teamai contribute` has notes queued that could not be pushed. A manual `teamai pull` does not repeat it at the end when the pull has already said it: the pull tries to publish the queue and reports the outcome itself, with the push error that made it fail — more than this check can tell you. If the pull never got that far, because the team repo failed to refresh, the check is printed as usual. diff --git a/docs/usage-guide.zh-CN.md b/docs/usage-guide.zh-CN.md index 961809ff2..3f15c252e 100644 --- a/docs/usage-guide.zh-CN.md +++ b/docs/usage-guide.zh-CN.md @@ -450,7 +450,7 @@ main 的团队知识 —— `git status` 保持干净。旧版单仓装升级后 - `.teamai/hooks/hooks.yaml` —— 团队 hooks - `.teamai/mcp/mcp.yaml` —— 共享 MCP servers -> **关于 `env` 的提醒。** 单仓模式下 `.teamai/env/env.yaml` **会被提交到 main**（不同于独立模式的每机本地 env），因此会随 clone 分发给所有人。`env.yaml` 存的是明文键值对 —— 只放非敏感的共享配置，真正的密钥请留在你自己未追踪的环境里。 +> **关于 `env` 的提醒。** 单仓模式下 `.teamai/env/env.yaml` **会被提交到 main**（不同于独立模式的每机本地 env），因此会随 clone 分发给所有人。`env.yaml` 存的是明文键值对 —— 只放非敏感的共享配置。密钥请在 `.teamai/env/secrets.yaml` 中只声明、不写值（见[团队密钥](designs/team-secrets.zh-CN.md)），值留在你自己未追踪的环境里。 > **限制。** 单仓模式把一套团队配置绑定到一个业务仓。如果需要一套团队知识库被多个业务仓共享，请改用独立团队仓（`teamai init `）。 @@ -913,6 +913,9 @@ projects: `teamai list --source repo` 会给出每个条目的 namespace、是否覆盖了 根条目，并指出每个未下发的条目及其原因；`teamai status` 按 namespace 计数并同样 指出它们；`teamai doctor` 以提示信息列出每一处覆盖。 + 你用 `teamai env set KEY` 为该团队设置了值时，变量取你的值，否则取文件中的值；环境 + 不覆盖二者，`env.sh` 导出的就是这个值（用 `--from-env` 设置的除外）。`teamai env list` 与 + `teamai list env` 显示这个值及其来源：`team` 或 `env.yaml`。 - **先让所有成员升级。** teamai 0.25.0 与 0.26.0 beta 会拒绝不认识的 `resources:` key， 声明 `env`、`hooks` 或 `mcp` 会让这些版本的 pull 失败。从本版本起，未知的 `resources:` key 只会给出警告，`teamai roles` 与 `teamai projects` 保存 manifest 时也会保留它。 @@ -963,6 +966,66 @@ variables: description: 团队 API 地址 # 可选 ``` +**密钥。** 团队需要的密钥只声明、不写值，写在 `env/secrets.yaml` 或某个 namespace 的 +`env//secrets.yaml` 中（生效条件与 `env//env.yaml` 相同，namespace 条目替换根文件中同 key +的条目）。每个成员在自己的机器上保存值。 + +```yaml +secrets: + - key: GITHUB_TOKEN + description: GitHub token with repo scope # 可选 + url: https://github.com/settings/tokens # 可选：成员获取 token 的地址 +``` + +```bash +teamai env add GITHUB_TOKEN --secret -d "GitHub token with repo scope" --url https://github.com/settings/tokens +teamai env remove GITHUB_TOKEN # env.yaml 未设置的 key；两个文件都有时加 --secret +teamai push +``` + +`teamai env add KEY --secret` 在根文件中（或用 `--role` / `--project` 在对应 namespace 的文件中）声明一个 key， +或更新它的描述和 url；它不接受值，也不会输出值。 + +每个成员为当前目录的团队设置自己的值，从不通过命令行参数传入： + +```bash +teamai env set GITHUB_TOKEN # 提示输入，不回显 +teamai env set GITHUB_TOKEN --stdin # 从管道读取 +teamai env set GITHUB_TOKEN --from-env WORK_GITHUB_TOKEN # 使用时从该变量读取 +teamai env set GITHUB_TOKEN --global # 对本机所有团队生效 +teamai env unset GITHUB_TOKEN [--global] +``` + +`env set` 接受已声明的密钥，不加 `--global` 时也接受该目录收到的 `env.yaml` 变量，并把值保存在 `~/.teamai/secrets/teams/.json` +（权限 `0600`），每个团队仓库一个文件，按你的 `~/.teamai/config.yaml` 中的团队仓库 URL 命名（不使用 `teamai.yaml` 的 `repo:`），修改 `team:` 不影响它；加 `--global` 时保存在 `~/.teamai/secrets/machine.json`， +对本机所有团队生效，为某个团队设置的值仍然优先。不在任何 scope 中时，`--global` 接受任何合法的 key， +并提示目前还没有团队声明它。值保持设置时该 key 的类型：团队不再声明某个同时在 `env.yaml` 中设置的密钥后，你的值不会用于该变量，`env list` 会提示先运行 `teamai env unset KEY`，再运行 `teamai env set KEY`。`teamai env list` 和 `teamai list env` 会把每个已声明的密钥 +显示为 `team`（你为该团队设置了它）、`global`（你为本机设置了它）、`environment`（你自己的环境中有它的值）、`missing`， +或 `unreadable`（你的值文件无法读取），从不显示值， +`--reveal` 也一样。既声明为密钥、又在 `env.yaml` 中设置的 key 按密钥处理：它的 `env.yaml` 值不会 +导出到 `env.sh`，也不会列出。密钥文件无法使用时不会被当作"没有密钥"：`env.sh` 和 MCP server 保持原样， +`pull` 会警告，`env list` 和 `mcp list` 以非零状态退出（此时 `env list` 不显示任何变量的值，因为其中任何一个都可能是密钥）， +`teamai doctor` 的检查失败并指出该文件。值文件无法读取时，`Your team secret values can be read` 检查失败。`teamai push` 会带上任何密钥文件的改动。 +见[团队密钥](designs/team-secrets.zh-CN.md)。 + +`gh`、`glab` 等 CLI 在 `teamai env exec` 下运行时，会拿到当前目录的变量和密钥；它对项目的每个 worktree +都以同样的方式找到 scope： + +```bash +teamai env exec -- gh pr create +teamai env exec -- glab mr list +``` + +命令继承你的环境，并叠加该 scope 的 `env.yaml` 变量和按[解析顺序](designs/team-secrets.zh-CN.md#解析顺序)解析的密钥；在该 scope 下没有值的已声明密钥 +会从中移除。命令前要加 `--`：否则 teamai 会把命令的参数当作自己的，因此它会提示并以退出码 2 结束。缺少密钥时，会在 stderr 上打印 `teamai env set` 那一行提示，命令照常运行。teamai 打印的所有内容 +都输出到 stderr，退出码就是命令的退出码。这里没有 teamai 配置时，命令以你的环境运行，并给出提示。 +不会把任何值写入磁盘。见[用 `env exec` 运行 CLI](designs/team-secrets.zh-CN.md#用-env-exec-运行-cli)。 + +scope 声明了密钥时，session-start hook 会告诉 agent 有哪些 key 及其 `description`，并让它通过 +`teamai env exec --` 运行需要这些 key 的 CLI。工具会丢弃 hook 输出的 agent 从 teamai core skill 获得同样的规则。 +agent 从不索要密钥值：缺少密钥时，它会请你在自己的终端运行 `teamai env set KEY`。见 +[告诉 agent](designs/team-secrets.zh-CN.md#告诉-agent)。 + 不再下发到该目录的变量会在下一次 pull 时从 `env.sh` 中移除，即使这次 pull 因团队仓库 未变化而提示 `Already synced` 也一样。在那次 pull 之前，`teamai doctor` 会报告 `env.sh` 中仍在导出的这类变量，前一个项目的密钥不会悄无声息地继续生效。 @@ -1064,11 +1127,11 @@ TeamAI 不会迁移或删除旧文件。Claude Code 也读取根目录的 `.mcp. Copilot 使用原生 `mcpServers` 结构：`stdio` 写成 `type: "local"`，远程传输保留 `http` 或 `sse`，每个 TeamAI 管理的条目都会带上必需的 `tools: ["*"]` 允许列表。TeamAI 遵循 `COPILOT_HOME`，项目配置使用 Copilot CLI 官方文档指定的 `.github/mcp.json` 仓库路径。详见 [GitHub Copilot CLI 添加 MCP Server](https://docs.github.com/zh/copilot/how-tos/copilot-cli/customize-copilot/add-mcp-servers)。Codex 支持 `stdio` 与 `http`，`sse` 会被跳过。Qoder 使用对应作用域 `.qoder/settings.json` 中与 Claude 兼容的 `mcpServers` 格式。Kiro 在专用的、只含 `mcpServers` 的 `.kiro/settings/mcp.json` 中使用同一格式（见 [Kiro MCP 配置文档](https://kiro.dev/docs/mcp/configuration/)）。OpenCode 支持 `stdio`（写成其 `type:"local"` 形态）与 `http`（`type:"remote"`），`sse` 会被跳过，其 server 位于共享 `opencode.json` 的 `mcp` 键下。归属记录在 `~/.teamai/managed-mcp.json`——手动添加的 server 不动；与手写同名则跳过，除非 `--force`。 -**密钥**：在 `mcp.yaml` 里写 `${VAR}`，不要写明文。取值优先来自环境变量，其次是该目录收到的团队环境变量（`env/env.yaml` 与活动的 `env//env.yaml`）。变量无法解析则跳过并提示。 +**密钥**：在 `mcp.yaml` 里写 `${VAR}`，不要写明文。团队在 `env/secrets.yaml` 中声明的 key 优先取你为该团队设置的值（`teamai env set`），其次取你为本机设置的值（`teamai env set --global`），再次取你自己的环境，不包括 teamai `env.sh` 导出的值（见[团队密钥](designs/team-secrets.zh-CN.md#解析顺序)）。其他变量优先取你为该团队设置的值（`teamai env set KEY`），其次是该目录收到的团队环境变量（`env/env.yaml` 与活动的 `env//env.yaml`）；环境只补充团队没有设置的 key，不再覆盖团队变量（见[团队密钥](designs/team-secrets.zh-CN.md#变量)）。你导出的值与团队的值不同而被忽略时，交互式 `pull` 和 `teamai doctor` 会指出。变量无法解析则跳过并提示。已声明的密钥不同：pull 找不到它时，之前某次 pull 写入的条目原样保留，因此里面可能是已经轮换掉的旧值，直到某次 pull 找到新值（见[团队密钥](designs/team-secrets.zh-CN.md#缺少密钥时保留-mcp-条目)）。交互式 `pull`、`teamai mcp list`、`teamai env list`、`teamai doctor` 和 `teamai env exec` 会指出没有值的已声明密钥、用到它的 server 以及设置它的命令：`` github: GITHUB_TOKEN is not set. Run `teamai env set GITHUB_TOKEN` (). `` -teamai 会**把每个 `${VAR}` 解析成取值后原样写入**各工具的配置文件（新建文件权限为 `0600`）。它不依赖任何工具自身的环境变量展开——因为那种展开很脆弱：最典型的是，以 GUI 方式（Dock/Launchpad）启动的 IDE 不会继承你 shell 中 `export` 的变量，`${VAR}` 占位符会展开为空、导致服务端 401。解析成明文可以保证无论工具如何启动，token 都在。 +teamai 会**把每个 `${VAR}` 解析成取值后原样写入**各工具的配置文件，并以 `0600` 写入该文件，已有的 `0644` 文件也会收紧（不含已解析值的配置保持原权限；新建文件权限为 `0600`）。它不依赖任何工具自身的环境变量展开——因为那种展开很脆弱：最典型的是，以 GUI 方式（Dock/Launchpad）启动的 IDE 不会继承你 shell 中 `export` 的变量，`${VAR}` 占位符会展开为空、导致服务端 401。解析成明文可以保证无论工具如何启动，token 都在。 -> ⚠️ **解析后的 token 会落盘。** 项目级 MCP 配置（`.mcp.json`、`.github/mcp.json`、`.cursor/mcp.json`、`.codex/config.toml`、`opencode.json`）因此含有明文密钥——请把它们加入 `.gitignore`，切勿提交。 +> ⚠️ **解析后的 token 会落盘。** 项目级 MCP 配置（`.mcp.json`、`.github/mcp.json`、`.cursor/mcp.json`、`.codex/config.toml`、`opencode.json`）因此含有明文密钥。只要这类文件将含有 teamai 解析出的值且 git 会跟踪它，teamai 就会在写入该值之前把路径写入本地克隆的 `.git/info/exclude`，放在 `# [teamai:mcp-exclude:start]` 块中（同一仓库的各 worktree 共用该文件）。经由符号链接目录访问的配置（例如 `.cursor/` 指向 `config/`）按写入实际落到的位置判断：写入 exclude、检查和报告的都是该路径（`/config/mcp.json`），已被跟踪时会同时给出两个路径。文件本身是符号链接时，写入会替换该链接，因此以文件自身的路径为准。本次 pull 未写入的文件同样适用：之前为某个现已禁用的工具写入的文件，团队已从 `toolPaths` 移除或改到别处的工具的内置位置上的文件（只要含有任何 MCP server 就算数，因为 teamai 对该工具的记录描述的是另一个文件或没有文件；当前由另一个工具映射的文件，例如 Claude 映射的 CodeBuddy 的 `.mcp.json`，则在含有该工具未写入的 server 时算数，见下文），在团队此后改动的 `toolPaths` 映射下写入的文件（每个 worktree 会把写入过解析值的文件记录在其 `managed-mcp.json` 旁的 `managed-mcp-files.json` 中；对于旧版 teamai 在有这份记录之前写入的文件，第一次 pull 会读取一次团队仓库中 `teamai.yaml` 历史里的每个 `mcpProject` 路径，以及 teamai 此后改掉的内置路径（CodeBuddy 的 `.codebuddy/mcp.json`），以克隆中现有的历史为限，且只看项目内的文件，跳过同一工具当前仍映射的路径；这类文件只要含有任何 MCP server 就算数，因为 teamai 对该工具的记录只描述当前路径（当前由另一个工具映射的文件，则在含有该工具未写入的 server 时算数，见下文），在那次 pull 之前 `teamai doctor` 也会检查这些文件；被 git 跟踪的文件不会写入 exclude（写入也不起作用），但无论其内容如何都会记为已跟踪，待 git 不再跟踪它（`git rm --cached`）后按其他此类文件的规则判断，直到它从磁盘和 git 中都消失才会被遗忘），或仍含已从 `mcp.yaml` 删除的 server 的文件。pull 写入的带解析值的条目只要未被改动就一直算数，即使团队后来把其中的 `${VAR}` 改成了字面值。worktree 中完全没有 `managed-mcp.json` 时（记录丢失，或在其第一次 pull 之前），未被 git 跟踪的配置只要含有任何记录都未认领的 server 就算数，你自己的 server 也包括在内：pull 会像重建丢失的记录时那样把这些 server 记入 `managed-mcp-files.json`，在它们离开该文件之前该路径一直保留；`teamai doctor` 也按同样方式检查。`managed-mcp.json` 中没有某个工具的记录时（记录丢失，或这是 teamai 对该工具的第一次投递），pull 为该工具写入第一份记录的配置也按此处理。git 无法判断是否忽略的路径，只要 `git ls-files` 显示该文件未被跟踪，也会照样写入；若连这一点也无法判断，则按 git 出错处理。若无法写入——`.git/info` 或 exclude 文件不可写、另一个 teamai 命令在短暂等待后仍占用 exclude 文件、git 已跟踪该文件、你自己的 git 忽略文件中有规则重新包含了它（例如 `!/.mcp.json`；警告会指出该规则），或 git 出错——teamai 会保持该文件原样（之前 pull 写入的条目保留），给出原因与修复方法的警告，`teamai mcp list` 和 `teamai doctor` 也会针对 pull 会写入它的每个工具，把该 server 报告为未写入（withheld）；请让文件可写（或对已跟踪的文件执行 `git rm --cached`，或删除重新包含它的规则），再运行 `teamai pull`。已被跟踪的文件会优先报告，且不会写入任何路径。不会改动已提交的 `.gitignore`，git 已忽略的路径不会重复添加，pull、`teamai mcp remove` 和 `teamai uninstall` 会从块中移除某个路径（移除最后一个路径时连同整个块），前提是该文件已不存在、不含任何 MCP server，或在命令运行前 teamai 的写入记录（`managed-mcp.json`）就已存在、可以解析且记有该文件所属工具的条目的情况下（对于两个工具共用的文件，例如 Claude 和 CodeBuddy 共用的 `.mcp.json`：需记有 `managed-mcp-files.json` 中写入过解析值的每个工具的条目；若其中没有列出任何工具，则需记有映射到它的每个工具的条目；空的、无法读取或被截断的记录不能作为依据；pull 重建记录时、或在 `managed-mcp.json` 中没有该工具的记录时写入记录时，若无法把文件中的其他 server 记入 `managed-mcp-files.json`，该记录在之后某次 pull 记下它们之前也不能作为依据）不含以下任何一项：带解析值的团队 server、清理后仍残留的 teamai 条目、teamai 重建丢失的 `managed-mcp.json` 时文件中已有的 server、仍在环境中设置的变量的值（8 个字符以上）。在已改动的映射下写入的文件、团队已移除或改到别处的工具的内置位置上的文件（当前有另一个工具映射到它的除外），或位于嵌套仓库某个关联 worktree 中的文件，须已不存在或不含任何 MCP server。为团队此后改到别处的工具写入（有记录、在上述历史中找到，或位于该工具的内置位置）、但仍被另一个工具的映射指向的文件，只要含有当前映射到它的工具未写入的 server（以它们的 `managed-mcp.json` 记录为准），也会保留该路径；与其他在已改动映射下写入的文件一样，你自己的 server 也会让它保留。`teamai uninstall` 对仓库每个 worktree 中的该文件都按此判断；pull 和 `teamai mcp remove` 只对当前 worktree 的文件按此判断，只要其他任一 worktree 中的该文件仍含 MCP server，就保留该路径：那个 worktree 上次 pull 写入的条目（例如团队后来改成字面值的 `${VAR}`）只能由在那里运行的 pull 判断。某次 pull 写入了路径、随后却没有把值写进该文件（文件无法解析，或其中有你自己的同名 server）时，该路径会在这次 pull 结束时移除，它在 `managed-mcp-files.json` 中的记录也会一并移除。否则，或对无法检查的文件（例如无法解析），会保留该路径，`teamai uninstall` 会给出警告，说明文件及原因：请先从中移除 teamai 的 server，再自行删除那一行（删到最后一行时连同块的首尾标记）。`teamai doctor` 会报告 git 仍会提交或无法判断的这类文件——例如已被跟踪的文件：请 `git rm --cached` 并轮换 token。 Claude Code 可能把来自仓库的 `.mcp.json` 标为待批准，需在交互式会话中确认一次。 @@ -1825,7 +1888,7 @@ teamai remove rules --force # 跳过确认，用于脚本和 CI 有两个工具并不读取 rules 目录，按文件比对的检查无法代表它们，因此各自单列一项。`Team rules are active in opencode` 检查 `opencode.json` 的 `instructions` 中是否仍列着 teamai 所拥有的那条 glob：OpenCode 不会自动扫描 `.opencode/rules`，缺了它，已送达的每个 `.md` 都不会生效，而按文件比对的检查依旧通过。`Team rules are inlined in Hermes SOUL.md` 把 `SOUL.md` 中 teamai 管理的代码块与团队 rule 内联后的内容比对——Hermes 的常驻指令来自这一个文件而非某个目录，因此代码块被删除或停留在旧版规则集上，都意味着该工具读到的是错误的规则，而磁盘上看不出任何异常。 -`MCP servers delivered to ` 将团队 `mcp.yaml` 为该工具解析出的每个 server 与该工具自己配置文件中的条目逐一比对，并列出 reconcile 跳过的 server 及原因。比对的是条目内容而非名字：reconcile 不会覆盖不属于 teamai 的条目，因此你自己写的同名 server 会占住这个名字，团队的定义从未真正送达；过期的旧副本同样等于没送达。两者都报告为 `not the team's definition`，而覆盖非 teamai 写入的条目只有 `teamai pull --force` 能做到。未解析的 `${VAR}` 会在这里连同变量名一起报告——否则它只在 pull 时出现一次，之后再无提示。无法解析的 `mcp.yaml` 并不等于团队没有 MCP：它会作为 `Team MCP servers can be read` 连同解析错误一起报告，因为这种文件不会向任何工具注入内容，而且除第一次之外的每次运行都对此保持沉默。无法解析的团队 hooks 与团队模型配置（文件无法解析、同一文件内重复的名字，或两个活动 namespace 中的同名条目）会让 `Team hooks can be resolved` 与 `Team model profiles can be resolved` 失败，并给出 pull 只记录一次的原因；`teamai status` 把它们计为 0 时会指向这里。`Env variables injected in shell profile` 不再只查标记注释：它会检查 `env/env.yaml` 能否解析、以及是否在 `variables:` 键下声明了变量（写成普通的 `KEY: value` 映射等于没有声明；而显式写成 `variables: []` 属于没有内容要下发的配置，不会判为失败）、每个变量是否以 `env.yaml` 声明的值写进了 `env.sh`（残留的旧值会一直被导出到每个 shell 和 MCP server，直到下次 pull；比对时会用生成器自身的逆运算读回 `env.sh`，因此跨多行引用的多行值能够正确匹配，而不会被误判为过期），以及本作用域注入的代码块（即 source 本作用域 `env.sh` 的那一块，因为同一个 profile 里还可能有其他作用域的代码块）是否真的能加载它——未加引号的 Windows 路径在 POSIX shell 中会被转义破坏，`source` 从不执行，而且没有任何提示。`No stale env blocks left behind` 是独立的一项检查：pull 优先选用哪个文件会随时间变化（Windows 上 Git Bash 的登录 shell 读取的是 `.bash_profile`/`.bash_login`/`.profile`，从不读取 `.bashrc`），而 pull 只会新增代码块，从不迁移旧的，因此早期安装或平台变化留下的失效代码块可能一直留在另一个候选文件里。它会列出每一个这样的文件（检查 `.zshrc`、`.bashrc`、`.bash_profile`、`.bash_login` 和 `.profile`，新旧写法都算），并指向 `teamai uninstall` 来清除它们——这与投递检查分开进行，因此不会因为还留着一个旧副本，就让一个正常工作的 env 代码块被判成故障。 +`MCP servers delivered to ` 将团队 `mcp.yaml` 为该工具解析出的每个 server 与该工具自己配置文件中的条目逐一比对，并列出 reconcile 跳过的 server 及原因。比对的是条目内容而非名字：reconcile 不会覆盖不属于 teamai 的条目，因此你自己写的同名 server 会占住这个名字，团队的定义从未真正送达；过期的旧副本同样等于没送达。两者都报告为 `not the team's definition`，而覆盖非 teamai 写入的条目只有 `teamai pull --force` 能做到。未解析的 `${VAR}` 会在这里连同变量名一起报告——否则它只在 pull 时出现一次，之后再无提示。没有值的已声明密钥不算失败：doctor 把它作为备注打印（`--json` 中的 `notes`），并附上设置它的命令，退出码与没有它时相同；备注还会说明为它保留的条目可能含有旧值，以及某个 key 既声明为密钥、又在 `env.yaml` 中设置的情况。无法解析的 `mcp.yaml` 并不等于团队没有 MCP：它会作为 `Team MCP servers can be read` 连同解析错误一起报告，因为这种文件不会向任何工具注入内容，而且除第一次之外的每次运行都对此保持沉默。无法解析的团队 hooks 与团队模型配置（文件无法解析、同一文件内重复的名字，或两个活动 namespace 中的同名条目）会让 `Team hooks can be resolved` 与 `Team model profiles can be resolved` 失败，并给出 pull 只记录一次的原因；`teamai status` 把它们计为 0 时会指向这里。`Env variables injected in shell profile` 不再只查标记注释：它会检查 `env/env.yaml` 能否解析、以及是否在 `variables:` 键下声明了变量（写成普通的 `KEY: value` 映射等于没有声明；而显式写成 `variables: []` 属于没有内容要下发的配置，不会判为失败）、每个变量是否以 `env.yaml` 声明的值（或你为该团队设置的值；用 `--from-env` 设置的不会写入）写进了 `env.sh`（残留的旧值会一直被导出到每个 shell 和 MCP server，直到下次 pull；比对时会用生成器自身的逆运算读回 `env.sh`，因此跨多行引用的多行值能够正确匹配，而不会被误判为过期），以及本作用域注入的代码块（即 source 本作用域 `env.sh` 的那一块，因为同一个 profile 里还可能有其他作用域的代码块）是否真的能加载它——未加引号的 Windows 路径在 POSIX shell 中会被转义破坏，`source` 从不执行，而且没有任何提示。`No stale env blocks left behind` 是独立的一项检查：pull 优先选用哪个文件会随时间变化（Windows 上 Git Bash 的登录 shell 读取的是 `.bash_profile`/`.bash_login`/`.profile`，从不读取 `.bashrc`），而 pull 只会新增代码块，从不迁移旧的，因此早期安装或平台变化留下的失效代码块可能一直留在另一个候选文件里。它会列出每一个这样的文件（检查 `.zshrc`、`.bashrc`、`.bash_profile`、`.bash_login` 和 `.profile`，新旧写法都算），并指向 `teamai uninstall` 来清除它们——这与投递检查分开进行，因此不会因为还留着一个旧副本，就让一个正常工作的 env 代码块被判成故障。 `Contributed learnings are published` 会在 `teamai contribute` 写下、但尚未推送成功的笔记仍在队列中时失败。当本次 pull 已经说过时，手动 `teamai pull` 结束时不会再重复它：pull 会尝试发布队列并自行报告结果，还会带上导致失败的推送错误——这是该检查本身给不出的信息。如果 pull 因为团队仓库刷新失败而根本没走到那一步，该检查会照常打印。 diff --git a/skill-data/core/SKILL.md b/skill-data/core/SKILL.md index af0bd6f5b..6cad8fcae 100644 --- a/skill-data/core/SKILL.md +++ b/skill-data/core/SKILL.md @@ -90,6 +90,16 @@ says so and why.) session, use the name of **this** tool — do not assume Claude Code or Cursor. Some hosts need extra manual steps for hooks — see the troubleshooting reference ("Agent-specific caveats"). +5. **Team secrets: the user types the value, you run the CLI.** When the team + declares secrets (the session-start context lists them; `teamai env list` + shows them), run the CLIs that use them through `teamai env exec -- `, + `--` first, so they get this team's value. It is for CLIs, not for starting + an agent: a secret named like a model profile's (`ANTHROPIC_*`) overrides it. + When a secret is missing, ask the user to run `teamai env set KEY` in their + own terminal. Never ask for a value in chat, pass one to `--stdin` or + `--secret`, read the files under `~/.teamai/secrets/`, or print one + (`teamai env exec -- env` and `printenv` do). Declaring a secret with + `teamai env add KEY --secret` takes no value, so you can run it. ## Daily commands @@ -100,6 +110,7 @@ teamai status # Show local vs team differences teamai doctor # Diagnose configuration and hook problems teamai list # List resources (skills|rules|docs|env|agents|hooks|mcp) teamai recall # Search what the team has already learned +teamai env exec -- # Run a CLI with this directory's team env and secrets ``` Every other command, every flag, and the flags `--help` hides live in the diff --git a/skill-data/core/references/commands.md b/skill-data/core/references/commands.md index 198f1a73b..6adbbac26 100644 --- a/skill-data/core/references/commands.md +++ b/skill-data/core/references/commands.md @@ -193,13 +193,23 @@ Generated: do not edit by hand. Regenerate with - `--reveal` — Show env variable values in plaintext (default: masked) - `teamai env list` — List team environment variables - `--reveal` — Show env variable values in plaintext (default: masked) - - `teamai env add ` — Add or update a team environment variable - - `-d, --description ` — Description for the variable - - `--role ` — Write to env//env.yaml instead of env/env.yaml + - `teamai env add [value]` — Add or update a team environment variable, or declare a secret with --secret + - `-d, --description ` — Description for the variable or secret + - `--secret` — Declare a secret in env/secrets.yaml: no value, each member sets their own + - `--url ` — Where a member gets a value for the secret (with --secret) + - `--role ` — Write to env// instead of env/ (env.yaml, or secrets.yaml with --secret) - `--project ` — Write to the project's env namespace (resources.env in manifest/projects.yaml) - - `teamai env remove ` — Remove a team environment variable - - `--role ` — Remove from env//env.yaml instead of env/env.yaml + - `teamai env remove ` — Remove a team environment variable or declared secret + - `--secret` — Remove the declared secret only (env/secrets.yaml), for a key env.yaml also sets + - `--role ` — Remove from env// instead of env/ - `--project ` — Remove from the project's env namespace (resources.env in manifest/projects.yaml) + - `teamai env set ` — Set your value for a secret the team declares, or an env variable it sets, for this directory's team, on this machine (prompts without echo) + - `--stdin` — Read the value from piped stdin + - `--from-env ` — Read the value from this environment variable each time it is used; no copy is stored + - `--global` — Set a secret for every team on this machine; a value set for a team still wins + - `teamai env unset ` — Remove your value for a secret or env variable, for this directory's team, from this machine + - `--global` — Remove the value set for every team on this machine instead + - `teamai env exec ` — Run a command with this directory's team env variables and secrets (put -- before the command) ## hooks diff --git a/skill-data/core/references/troubleshooting.md b/skill-data/core/references/troubleshooting.md index 89eafa8ea..6fb169cc1 100644 --- a/skill-data/core/references/troubleshooting.md +++ b/skill-data/core/references/troubleshooting.md @@ -64,6 +64,36 @@ This is the #1 onboarding issue. In order: `recall` refuses the same way with `Nothing was searched: : `: no team knowledge was searched, so do not report that the team has none. +## "KEY is not set. Run `teamai env set KEY`" + +`pull`, `teamai mcp list`, `teamai env list`, `teamai doctor` and +`teamai env exec` (on stderr) print this for a secret the team declares in +`env/secrets.yaml` that has no value on this machine, naming the MCP servers +that need it and where to get one. It is a note, not a failure: `doctor` exits +as it would without it. The value is the user's: ask them to run +`teamai env set KEY` in their own terminal (it prompts without echo), then +`teamai pull` to update the MCP servers; a CLI run through `teamai env exec` +gets it on its next run. Never ask for the value in chat or pipe one to +`teamai env set --stdin`. A note that an entry "may hold an old" value means an earlier pull wrote +it and it stays until a pull finds the value. + +`KEY reads VAR, which is not set` means the user's value for KEY is a +reference to VAR (`--from-env`) and VAR is unset in this environment. Ask the +user whether to set VAR in their shell or replace the reference with the +command in the line; do not choose for them. + +## "Did not write 's MCP servers to " / `withheld:` + +`pull` prints this, and `teamai mcp list` (`withheld:`) and `teamai doctor` +report it, when a project MCP config would get a resolved `${VAR}` value that +git would commit: the file could not be kept out of git first. It is left as +it was, and an entry an earlier pull wrote stays. The line names the reason and the +fix. For `git already tracks `, tell the user: `git rm --cached ` +(the file stays on disk), commit that, and rotate the token if the file was +ever committed with it; then `teamai pull`. Do not run `git rm` or commit for +them. For an exclude file that is not writable, one another teamai command +held, or a git error, relay the fix the line gives. + ## Permission / access denied `init`, `pull`, or `push` failing with a permission error usually means the user diff --git a/skill-data/setup/references/manage-admin.md b/skill-data/setup/references/manage-admin.md index 6de25dd8c..50c04a8f9 100644 --- a/skill-data/setup/references/manage-admin.md +++ b/skill-data/setup/references/manage-admin.md @@ -37,6 +37,42 @@ teamai mcp remove # remove teamai-managed MCP servers MCP definitions travel with the team repo like skills/rules — edit, then the members pick them up on sync. +A server with a `${VAR}` the tool cannot expand itself gets the resolved value +written into its project config (`.mcp.json`, `.cursor/mcp.json`, ...). Before +that write, teamai lists the file in the clone's `.git/info/exclude`, inside a +`# [teamai:mcp-exclude:start]` block; the committed `.gitignore` is never touched. +A file under a symlinked directory is listed and checked where the write lands +(`.cursor/` linking to `config/`: `/config/mcp.json`); a symlink at the file +itself is replaced by the write. +When it cannot (git already tracks the file, a rule in the member's git ignore +files re-includes it, `.git/info` is not writable, the exclude file is held by +another teamai command, or git errors), it leaves the file as it was, warns, and +`teamai mcp list` shows `withheld: — . `. Apply the fix it +names (a tracked file: `git rm --cached ` and rotate the token; a +re-including rule such as `!/.mcp.json`: remove it), then run `teamai pull`. A pull or `teamai mcp remove` takes a line out +once its file no longer holds a resolved value; `teamai uninstall` does so in +every worktree. A file written under a `toolPaths..mcpProject` the team +later changes or removes stays listed until it is deleted or holds no server; +for one an older teamai wrote, the first pull finds the path in the team repo's +history of `teamai.yaml`, or among the built-in paths teamai has since changed +(not one the same tool maps today), and lists it while it holds any server; one +git tracks is recorded instead and listed once the member runs `git rm --cached` +on it. `teamai doctor` checks those paths until that pull. A file written for a +tool the team moved elsewhere (recorded, or found in that history), that another +tool still maps, stays listed while it holds a server that tool did not write, +one of the member's own included. The built-in location of a tool the team drops +from `toolPaths` or moves elsewhere stays listed while it holds any server; one +another tool maps today (CodeBuddy's `.mcp.json`, which Claude maps) while it +holds a server that tool did not write. A file two tools map, with no pull on +this version having recorded it, needs a `managed-mcp.json` record from each of +them. While a worktree has no `managed-mcp.json` at all (lost, or before its +first pull), an untracked config holding a server no record claims is listed, +and that server noted: it keeps the line until it leaves the file. So is the +file of a tool `managed-mcp.json` has no record for, when a pull writes that +tool's first record (its record lost, or teamai's first delivery to it). While that +note cannot be written (another teamai command holds the record), the line stays +until a later pull writes it. + ## Invite a member There is **no CLI invite flag.** Inviting is done on the Git platform's website: @@ -150,10 +186,12 @@ teamai push # share the updated teamai.yaml ```bash teamai env list # what reaches this directory, each with its namespace (values masked) -teamai env list --reveal # show values in plaintext +teamai env list --reveal # show variable values in plaintext (never a secret's) teamai env add # add or update in env/env.yaml teamai env add --project # or --role : in that namespace's env//env.yaml (warns if nothing declares ) teamai env remove # remove (same --role / --project) +teamai env add --secret -d "" --url # declare a secret in env/secrets.yaml, no value (same --role / --project) +teamai env remove --secret # remove a declared secret (plain `env remove` does too when env.yaml does not set ) teamai remove mcp # root mcp/mcp.yaml if it has the name, else the one namespace file; --role / --project pick a namespace ``` @@ -187,6 +225,22 @@ and push it with git. `teamai doctor` lists each override. have `` active. A member's API key is bound to the profile's gateway origin: when an override points at another host, their pull leaves the agent alone and asks them to run `teamai models switch team:` to set the key for it. +- Secrets are declared with no value in `env/secrets.yaml` or `env//secrets.yaml` + (active through `resources.env`; a namespace entry replaces the root entry with the + same key): a `secrets:` list of `key`, optional `description` and optional `url` + (where a member gets one). Never put a value there: `teamai env add --secret` + takes none and rejects one. Declare with it or edit the file in the team repo; + `teamai push` picks it up. Each member sets their own value with `teamai env set KEY` + in their terminal (`--global` for every team on their machine; a team value still + wins). `teamai env list` shows each secret as `team`, `global`, `environment`, + `missing` or `unreadable` and never shows a value, `--reveal` included. The `description` is what + agents see: the session-start hook lists each declared key with it and tells the + agent to run the CLIs that need them through `teamai env exec --`, so say which + tool or server uses the key. As the agent, run `teamai env add --secret` + yourself and leave the value to each member's own terminal. A key declared as a secret + and also set in `env.yaml` is a secret: its `env.yaml` value is not delivered. A + secrets file that does not parse keeps `env.sh` and MCP servers as they were, and + `teamai doctor` fails a check naming the file. - Have every member upgrade before declaring `env`, `hooks`, `mcp`, `models` or `docs` in a manifest: teamai 0.25.0 and the 0.26.0 betas reject those keys and their pull stops. diff --git a/skill-data/setup/references/uninstall.md b/skill-data/setup/references/uninstall.md index 3f526b93c..069f637bb 100644 --- a/skill-data/setup/references/uninstall.md +++ b/skill-data/setup/references/uninstall.md @@ -55,3 +55,16 @@ and give it your team repo URL."* and neither should you. - If the user only wants to stop auto-sync for one tool but keep TeamAI otherwise, that is the `--agent ` form, not a full uninstall. +- In a project, uninstall also takes teamai's lines out of `.git/info/exclude` + (the `# [teamai:mcp-exclude:start]` block) for MCP configs it proves hold no + resolved `${VAR}` value. A line names the path a write lands in: for a config + under a symlinked directory, the link's target (`/config/mcp.json` for + `.cursor/` linking to `config/`). For one it cannot prove clean (including one written + under a `toolPaths` mapping since changed, at the built-in location of a tool + the team dropped or moved that no other tool maps, or in a nested repository's + linked worktree, that still holds servers, and one written for a tool since moved + (or at its built-in location) that another tool maps, holding a server that tool + did not write) it keeps the line + and warns, naming the file and why: have the user remove teamai's servers from + that file, then delete the line (with the last one, the block's markers). Do not + delete a kept line while its file still holds a token. diff --git a/src/__tests__/anchors.test.ts b/src/__tests__/anchors.test.ts index 160bdebff..c84d95055 100644 --- a/src/__tests__/anchors.test.ts +++ b/src/__tests__/anchors.test.ts @@ -234,4 +234,8 @@ describe('listWorktrees', () => { expect(await listWorktrees(plain)).toEqual([]); fs.rmSync(plain, { recursive: true, force: true }); }); + + it('returns [] for a directory that no longer exists', async () => { + expect(await listWorktrees(path.join(os.tmpdir(), 'teamai-gone-', String(process.pid), 'project'))).toEqual([]); + }); }); diff --git a/src/__tests__/doctor-entry-namespaces.test.ts b/src/__tests__/doctor-entry-namespaces.test.ts index 9c326f810..80fdea378 100644 --- a/src/__tests__/doctor-entry-namespaces.test.ts +++ b/src/__tests__/doctor-entry-namespaces.test.ts @@ -147,6 +147,37 @@ describe('doctor — env, hooks and MCP namespaces', () => { expect(checks[0]?.fix).toContain('models/models.yaml'); }); + // #875: the secrets file is its own set, named as such, and a broken one fails a check. + it('fails a check naming the secrets file when it does not parse', async () => { + await fse.outputFile(path.join(repoPath, 'env', 'env.yaml'), 'variables:\n - { key: A, value: x }\n'); + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets: [\n'); + + const checks = await buildEntryResolutionChecks(ctx()); + + expect(checks.map((check) => check.name)).toEqual(['Team secrets can be resolved']); + expect(await checks[0]?.check()).toBe(false); + expect(checks[0]?.fix).toMatch(/^env\/secrets\.yaml is not valid YAML: .*Team secrets were not resolved this run/s); + }); + + it('lists a secret override as a secrets note', async () => { + await fse.outputFile(path.join(repoPath, 'manifest', 'projects.yaml'), + 'version: 1\nprojects:\n - id: checkout\n resources: { env: [checkout] }\n'); + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - { key: GITHUB_TOKEN }\n'); + await fse.outputFile(path.join(repoPath, 'env', 'checkout', 'secrets.yaml'), 'secrets:\n - { key: GITHUB_TOKEN }\n'); + + expect(await entryNamespaceNotes(ctx({ projects: ['checkout'] }))).toEqual([ + 'secrets: 1 received here (1 checkout)', + 'secrets: "GITHUB_TOKEN" from env/checkout/secrets.yaml replaces env/secrets.yaml', + ]); + }); + + it('names env/secrets.yaml for a key it repeats in legacy mode', async () => { + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - { key: A }\n - { key: A }\n'); + + expect(await entryNamespaceNotes(ctx())) + .toEqual(['secrets: "A" is defined more than once in env/secrets.yaml (legacy mode does not check this; keep one of them)']); + }); + it('adds no resolution check when hooks and model profiles resolve', async () => { await fse.outputFile(path.join(repoPath, 'hooks', 'hooks.yaml'), 'hooks:\n - { id: lint, description: x, event: Stop, command: echo }\n'); expect(await buildEntryResolutionChecks(ctx())).toEqual([]); diff --git a/src/__tests__/doctor-env-delivery.test.ts b/src/__tests__/doctor-env-delivery.test.ts index a29fbf66f..de75e69e1 100644 --- a/src/__tests__/doctor-env-delivery.test.ts +++ b/src/__tests__/doctor-env-delivery.test.ts @@ -21,6 +21,7 @@ import { loadLocalConfig, loadTeamConfig } from '../config.js'; import { buildChecks, resolveDoctorContext, type Check } from '../doctor.js'; import { EnvHandler } from '../resources/env.js'; import type { LocalConfig, TeamaiConfig } from '../types.js'; +import { getTeamSecretsPath, writeSecretStore } from '../secret-store.js'; /** * The env half of the delivery check (#624). The plumbing version asked only @@ -507,4 +508,46 @@ describe('doctor — env variables reach a shell', () => { expect(await check.check()).toBe(false); expect(check.fix).toContain('variable "API_BASE" is defined in both env/checkout/env.yaml and env/billing/env.yaml'); }); + + // #875 (#879 Conflict 13): pull leaves the env.yaml value of a key declared as a secret out of env.sh. + it('does not owe env.sh a key the team also declares as a secret, and reports one it still exports', async () => { + await writeEnvYaml('variables:\n - key: JIRA_PASSWORD\n value: "s3cret"\n - key: API_URL\n value: "u"\n'); + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: JIRA_PASSWORD\n'); + await writeProfile(`[ -f ${envShPath} ] && source ${envShPath}`); + + await writeEnvSh("export API_URL='u'\n"); + expect(await (await envCheck()).check()).toBe(true); + + await writeEnvSh("export API_URL='u'\nexport JIRA_PASSWORD='s3cret'\n"); + const check = await envCheck(); + expect(await check.check()).toBe(false); + expect(check.fix).toContain('still exports JIRA_PASSWORD'); + }); + + // #875 (#879 S9): pull writes the member's value for this team, and leaves a --from-env one out. + it("expects the member's value for a variable in env.sh, and no --from-env one", async () => { + await writeEnvYaml('variables:\n - key: GITLAB_HOST\n value: "gitlab.team.example"\n - key: API_URL\n value: "u"\n'); + await writeProfile(`[ -f ${envShPath} ] && source ${envShPath}`); + await writeSecretStore(getTeamSecretsPath(localConfig), { GITLAB_HOST: { value: 'gitlab.mine.example', kind: 'variable' }, API_URL: { env: 'MY_API_URL', kind: 'variable' } }); + + await writeEnvSh("export GITLAB_HOST='gitlab.mine.example'\n"); + expect(await (await envCheck()).check()).toBe(true); + + await writeEnvSh("export GITLAB_HOST='gitlab.team.example'\nexport API_URL='u'\n"); + const check = await envCheck(); + expect(await check.check()).toBe(false); + expect(check.fix).toContain('has a stale value for GITLAB_HOST'); + expect(check.fix).toContain('still exports API_URL'); + }); + + // #879 Conflict 14: a failed declaration keeps env.sh as it is, so it cannot be checked against env.yaml. + it('names the secrets file when the declarations cannot be read', async () => { + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets: [\n'); + await writeEnvSh("export JIRA_PASSWORD='s3cret'\n"); + await writeProfile(`[ -f ${envShPath} ] && source ${envShPath}`); + + const check = await envCheck(); + expect(await check.check()).toBe(false); + expect(check.fix).toContain('env/secrets.yaml is not valid YAML'); + }); }); diff --git a/src/__tests__/doctor-mcp-delivery.test.ts b/src/__tests__/doctor-mcp-delivery.test.ts index b170dc5a9..9cc9da8c4 100644 --- a/src/__tests__/doctor-mcp-delivery.test.ts +++ b/src/__tests__/doctor-mcp-delivery.test.ts @@ -2,6 +2,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import fse from 'fs-extra'; import os from 'node:os'; import path from 'node:path'; +import { execFileSync } from 'node:child_process'; vi.mock('../config.js', async (importOriginal) => ({ ...(await importOriginal()), @@ -19,7 +20,7 @@ vi.mock('../utils/logger.js', () => ({ import { loadLocalConfig, loadTeamConfig } from '../config.js'; import { buildChecks, resolveDoctorContext, type Check } from '../doctor.js'; -import type { LocalConfig, TeamaiConfig } from '../types.js'; +import { getDataHome, managedMcpManifestKey, managedMcpManifestPath, type LocalConfig, type TeamaiConfig } from '../types.js'; /** * The MCP half of the delivery check (#624). A server lands as an entry inside @@ -229,4 +230,521 @@ describe('doctor — MCP servers delivered on disk', () => { expect(await fse.readFile(file, 'utf8')).toBe(before); }); + + describe('project MCP config holding a resolved value (#882)', () => { + const NAME = 'Project MCP configs with resolved values are kept out of git'; + let projectRoot: string; + + beforeEach(async () => { + projectRoot = path.join(tempDir, 'business-repo'); + await fse.ensureDir(path.join(projectRoot, '.claude', 'skills')); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + Object.assign(localConfig, { scope: 'project', projectRoot }); + teamConfig.toolPaths = { claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: '.mcp.json' } }; + await writeTeamMcp( + 'servers:\n - name: jira\n transport: http\n url: https://jira.example/mcp\n' + + ' headers:\n Authorization: "Bearer ${JIRA_TOKEN}"\n', + ); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { + mcpServers: { jira: { type: 'http', url: 'https://jira.example/mcp', headers: { Authorization: 'Bearer t0ken' } } }, + }); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'jira', hash: 'h' }], + }); + }); + + async function excludeCheck(): Promise { + return (await checks()).find((c) => c.name === NAME); + } + + it('fails while git would track the file, and names it', async () => { + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect(check.fix).toContain(path.join(projectRoot, '.mcp.json')); + expect(check.fix).toContain('teamai pull'); + }); + + it('passes once git ignores the file', async () => { + await fse.appendFile(path.join(projectRoot, '.git', 'info', 'exclude'), '/.mcp.json\n'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(true); + }); + + it.each([ + ['its tool is disabled', async () => { localConfig.disabledAgents = ['claude', 'tclaude']; }], + ['its tool is no longer detected', async () => { await fse.remove(path.join(projectRoot, '.claude')); }], + ['it does not parse', async () => { + await fse.writeFile(path.join(projectRoot, '.mcp.json'), '{ "mcpServers": { "jira": { "headers": { "Authorization": "Bearer t0ken" } } },\n'); + }], + ['git cannot say whether it would commit it', async () => { + await fse.writeFile(path.join(projectRoot, '.git', 'config'), '[core\nbroken\n'); + }], + ])('still fails while git would track the file when %s', async (_label, arrange) => { + await arrange(); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect(check.fix).toContain(path.join(projectRoot, '.mcp.json')); + }); + + it.each([ + ['its server has left mcp.yaml and its tool is disabled', async () => { + await writeTeamMcp('servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + localConfig.disabledAgents = ['claude', 'tclaude']; + }], + ['the team dropped its tool from toolPaths', async () => { + teamConfig.toolPaths = { cursor: { skills: '.cursor/skills', mcp: '.cursor/mcp.json', mcpProject: '.cursor/mcp.json' } }; + }], + ['the team\'s mcp.yaml does not parse', async () => { + await writeTeamMcp('servers: [unclosed\n'); + }], + ])('still fails, naming the file once, when %s', async (_label, arrange) => { + await arrange(); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(path.join(projectRoot, '.mcp.json'))).toHaveLength(2); + }); + + it.each([ + ['notes it wrote a resolved value', { resolved: true }], + ['is an older teamai\'s, without that note', {}], + ])('still fails when the server\'s ${VAR} became a literal, its tool is disabled and the record %s', async (_label, note) => { + const { entryHash } = await import('../resources/mcp-format.js'); + const { mcpServers } = await fse.readJson(path.join(projectRoot, '.mcp.json')) as { mcpServers: Record }; + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'jira', hash: entryHash(mcpServers.jira), ...note }], + }); + await writeTeamMcp( + 'servers:\n - name: jira\n transport: http\n url: https://jira.example/mcp\n' + + ' headers:\n Authorization: "Bearer published-literal"\n', + ); + localConfig.disabledAgents = ['claude', 'tclaude']; + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect(check.fix).toContain(path.join(projectRoot, '.mcp.json')); + }); + + it('fails, naming it once, for a config a pull wrote under a mcpProject the team has since changed', async () => { + const { trackResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + const old = path.join(projectRoot, '.cursor', 'team-mcp.json'); + await fse.outputJson(old, { + mcpServers: { jira: { type: 'http', url: 'https://jira.example/mcp', headers: { Authorization: 'Bearer t0ken' } } }, + }); + expect(await trackResolvedMcpFiles(localConfig, [{ tool: 'cursor', file: old }])).toBe('written'); + await fse.appendFile(path.join(projectRoot, '.git', 'info', 'exclude'), '/.mcp.json\n'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(old)).toHaveLength(2); + expect(check.fix).not.toContain(path.join(projectRoot, '.mcp.json')); + }); + + describe('a config an older teamai wrote under a mapping an earlier teamai.yaml made, before a pull on this version', () => { + const old = (): string => path.join(projectRoot, '.cursor', 'team-mcp.json'); + const commitTeamYaml = (toolPaths: object): void => { + fse.writeFileSync(path.join(repoPath, 'teamai.yaml'), JSON.stringify({ team: 't', toolPaths })); + execFileSync('git', ['add', '-A'], { cwd: repoPath }); + execFileSync('git', ['-c', 'user.name=t', '-c', 'user.email=t@t', '-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'toolPaths'], { cwd: repoPath }); + }; + + beforeEach(async () => { + execFileSync('git', ['init', '-q'], { cwd: repoPath }); + commitTeamYaml({ ...teamConfig.toolPaths, cursor: { skills: '.cursor/skills', mcp: '.cursor/mcp.json', mcpProject: '.cursor/team-mcp.json' } }); + commitTeamYaml(teamConfig.toolPaths ?? {}); + // Its server left mcp.yaml since, and no record names the file. + await fse.outputJson(old(), { + mcpServers: { gone: { type: 'http', url: 'https://gone.example/mcp', headers: { Authorization: 'Bearer t0ken' } } }, + }); + await fse.appendFile(path.join(projectRoot, '.git', 'info', 'exclude'), '/.mcp.json\n'); + }); + + it('fails, naming it, without writing managed-mcp-files.json', async () => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(old())).toHaveLength(2); + expect(check.fix).not.toContain(path.join(projectRoot, '.mcp.json')); + expect(await fse.pathExists(resolvedMcpFilesPath(localConfig) ?? '')).toBe(false); + }); + + it('passes once it is kept out of git', async () => { + await fse.appendFile(path.join(projectRoot, '.git', 'info', 'exclude'), '/.cursor/team-mcp.json\n'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(true); + }); + + it.each([ + ['a pull on this version has read those mappings', async () => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + await fse.outputJson(resolvedMcpFilesPath(localConfig) ?? '', { version: 1, files: {}, earlierMappingsRead: true }); + }], + ['git tracks it', async () => { + execFileSync('git', ['add', '-f', '.cursor/team-mcp.json'], { cwd: projectRoot }); + }], + ['git cannot read the team repo\'s history', async () => { + await fse.emptyDir(path.join(repoPath, '.git', 'objects')); + }], + ])('does not name it when %s', async (_label, arrange) => { + await arrange(); + + const check = await excludeCheck(); + if (check) expect(check.fix).not.toContain(old()); + // .mcp.json is listed, so nothing is left to fail on. + if (check) expect(await check.check()).toBe(true); + }); + + describe('recorded as tracked by a pull that found git tracking it', () => { + beforeEach(async () => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + await fse.outputJson(resolvedMcpFilesPath(localConfig) ?? '', { + version: 1, files: { [old()]: { tools: ['cursor'], tracked: true } }, earlierMappingsRead: true, + }); + }); + + it('does not name it while git tracks it', async () => { + execFileSync('git', ['add', '-f', '.cursor/team-mcp.json'], { cwd: projectRoot }); + + const check = await excludeCheck(); + if (check) expect(check.fix).not.toContain(old()); + if (check) expect(await check.check()).toBe(true); + }); + + it('fails, naming it, once git no longer tracks it', async () => { + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(old())).toHaveLength(2); + }); + }); + }); + + describe('a tool\'s built-in location, once the team moved the tool, and its records describe the file it maps now', () => { + const old = (): string => path.join(projectRoot, '.cursor', 'mcp.json'); + + beforeEach(async () => { + teamConfig.toolPaths = { + claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: '.mcp.json' }, + cursor: { skills: '.cursor/skills', mcp: '.cursor/mcp.json', mcpProject: '.cursor/team-mcp.json' }, + }; + // Its server left mcp.yaml since. + await fse.outputJson(old(), { + mcpServers: { gone: { type: 'http', url: 'https://gone.example/mcp', headers: { Authorization: 'Bearer t0ken' } } }, + }); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'jira', hash: 'h' }], + [managedMcpManifestKey('cursor', true)]: [], + }); + await fse.appendFile(path.join(projectRoot, '.git', 'info', 'exclude'), '/.mcp.json\n'); + }); + + it('fails, naming it once', async () => { + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(old())).toHaveLength(2); + expect(check.fix).not.toContain(path.join(projectRoot, '.mcp.json')); + }); + + it('passes once it is kept out of git', async () => { + await fse.appendFile(path.join(projectRoot, '.git', 'info', 'exclude'), '/.cursor/mcp.json\n'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(true); + }); + }); + + // The toolPaths here drop CodeBuddy, whose built-in location is Claude's .mcp.json. + describe('a tool\'s built-in location another tool maps today, once the team moved or dropped the tool', () => { + const file = (): string => path.join(projectRoot, '.mcp.json'); + + beforeEach(async () => { + await writeTeamMcp('servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + // Claude's own entry, and one an older teamai wrote there for CodeBuddy, whose record is lost. + await fse.writeJson(file(), { + mcpServers: { + docs: { type: 'http', url: 'https://docs.example/mcp' }, + gone: { type: 'http', url: 'https://gone.example/mcp', headers: { Authorization: 'Bearer t0ken' } }, + }, + }); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'docs', hash: 'h', resolved: false }], + }); + }); + + it('fails, naming it once, while it holds a server none of the tools mapping it own', async () => { + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(file())).toHaveLength(2); + }); + + it('emits no check once only their servers are left', async () => { + await fse.writeJson(file(), { mcpServers: { docs: { type: 'http', url: 'https://docs.example/mcp' } } }); + + expect(await excludeCheck()).toBeUndefined(); + }); + }); + + describe('a config written for a tool the team has since moved, that another tool\'s mapping still reaches', () => { + const file = (): string => path.join(projectRoot, '.mcp.json'); + + beforeEach(async () => { + const { trackResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + teamConfig.toolPaths = { + claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: '.mcp.json' }, + cursor: { skills: '.cursor/skills', mcp: '.cursor/mcp.json', mcpProject: '.cursor/mcp.json' }, + }; + await writeTeamMcp('servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + // Claude's own entry, and the one a pull wrote there for Cursor with a token before the team moved it. + await fse.writeJson(file(), { + mcpServers: { + docs: { type: 'http', url: 'https://docs.example/mcp' }, + gone: { type: 'http', url: 'https://gone.example/mcp', headers: { Authorization: 'Bearer t0ken' } }, + }, + }); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'docs', hash: 'h', resolved: false }], + }); + expect(await trackResolvedMcpFiles(localConfig, [{ tool: 'cursor', file: file() }])).toBe('written'); + }); + + it('fails, naming it once, while it holds a server none of the tools mapping it own', async () => { + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(file())).toHaveLength(2); + }); + + it('emits no check once only their servers are left', async () => { + await fse.writeJson(file(), { mcpServers: { docs: { type: 'http', url: 'https://docs.example/mcp' } } }); + + expect(await excludeCheck()).toBeUndefined(); + }); + + describe('written by an older teamai under a mapping only an earlier teamai.yaml made, before a pull on this version', () => { + beforeEach(async () => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + await fse.remove(resolvedMcpFilesPath(localConfig) ?? ''); + execFileSync('git', ['init', '-q'], { cwd: repoPath }); + for (const cursorFile of ['.mcp.json', '.cursor/mcp.json']) { + const toolPaths = { ...teamConfig.toolPaths, cursor: { skills: '.cursor/skills', mcp: '.cursor/mcp.json', mcpProject: cursorFile } }; + fse.writeFileSync(path.join(repoPath, 'teamai.yaml'), JSON.stringify({ team: 't', toolPaths })); + execFileSync('git', ['add', '-A'], { cwd: repoPath }); + execFileSync('git', ['-c', 'user.name=t', '-c', 'user.email=t@t', '-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'toolPaths'], { cwd: repoPath }); + } + }); + + it('fails, naming it once, while it holds a server none of the tools mapping it own', async () => { + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(file())).toHaveLength(2); + }); + + it('emits no check once only their servers are left', async () => { + await fse.writeJson(file(), { mcpServers: { docs: { type: 'http', url: 'https://docs.example/mcp' } } }); + + expect(await excludeCheck()).toBeUndefined(); + }); + }); + }); + + it('fails for a server that was in the file when a pull rebuilt the lost record, after it left mcp.yaml', async () => { + const { trackResolvedMcpFiles, recordUnverifiedMcpServers } = await import('../mcp-resolved-files.js'); + const file = path.join(projectRoot, '.mcp.json'); + await trackResolvedMcpFiles(localConfig, [{ tool: 'claude', file }]); + expect(await recordUnverifiedMcpServers(localConfig, [{ file, names: ['jira'] }])).toBe('written'); + await writeTeamMcp('servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'docs', hash: 'h' }], + }); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect(check.fix).toContain(file); + }); + + it('names a file two tools share once', async () => { + teamConfig.toolPaths = { + claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: '.mcp.json' }, + codebuddy: { skills: '.codebuddy/skills', mcp: '.codebuddy/mcp.json', mcpProject: '.mcp.json' }, + }; + vi.stubEnv('JIRA_TOKEN', 'long-t0ken-value-7c1'); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { + mcpServers: { jira: { type: 'http', url: 'https://jira.example/mcp', headers: { Authorization: 'Bearer long-t0ken-value-7c1' } } }, + }); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect((check.fix ?? '').split(path.join(projectRoot, '.mcp.json'))).toHaveLength(2); + }); + + it('still fails when the manifest is gone but the resolved value is in the file', async () => { + vi.stubEnv('JIRA_TOKEN', 'long-t0ken-value-7c1'); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { + mcpServers: { jira: { type: 'http', url: 'https://jira.example/mcp', headers: { Authorization: 'Bearer long-t0ken-value-7c1' } } }, + }); + await fse.remove(managedMcpManifestPath(getDataHome(localConfig), projectRoot)); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + }); + + it('fails, naming it, while this worktree has no managed-mcp.json and the file holds a server since dropped from mcp.yaml', async () => { + teamConfig.toolPaths = { ...teamConfig.toolPaths, codebuddy: { skills: '.codebuddy/skills', mcp: '.codebuddy/mcp.json', mcpProject: '.mcp.json' } }; + await fse.remove(managedMcpManifestPath(getDataHome(localConfig), projectRoot)); + await writeTeamMcp('servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(path.join(projectRoot, '.mcp.json'))).toHaveLength(2); + }); + + it('fails the same way while an installed tool mapping the file has no record, though another tool\'s is there', async () => { + teamConfig.toolPaths = { ...teamConfig.toolPaths, codebuddy: { skills: '.codebuddy/skills', mcp: '.codebuddy/mcp.json', mcpProject: '.mcp.json' } }; + await fse.ensureDir(path.join(projectRoot, '.codebuddy', 'skills')); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'docs', hash: 'fixture-hash', resolved: false }], + }); + await writeTeamMcp('servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + }); + + it('fails the same way for the config an uninstalled tool left, its record lost, though another tool\'s is there', async () => { + teamConfig.toolPaths = { ...teamConfig.toolPaths, opencode: { skills: '.opencode/skills', mcp: '.config/opencode/opencode.json', mcpProject: 'opencode.json' } }; + await fse.outputJson(path.join(projectRoot, 'opencode.json'), { mcp: { stale: { type: 'remote', url: 'https://stale.example/mcp' } } }); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'docs', hash: 'fixture-hash', resolved: false }], + }); + await writeTeamMcp('servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect(check.fix ?? '').toContain(path.join(projectRoot, 'opencode.json')); + }); + + it('fails the same way while the record a pull wrote without managed-mcp.json is still marked unnoted', async () => { + teamConfig.toolPaths = { ...teamConfig.toolPaths, codebuddy: { skills: '.codebuddy/skills', mcp: '.codebuddy/mcp.json', mcpProject: '.mcp.json' } }; + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'docs', hash: 'fixture-hash', unnoted: true }], + }); + await writeTeamMcp('servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect((check.fix ?? '').split(path.join(projectRoot, '.mcp.json'))).toHaveLength(2); + }); + + it('emits no check when the server of that name is the member\'s own, not teamai\'s', async () => { + // CodeBuddy at its built-in .mcp.json: dropped, any server there Claude's records don't own would hold it. + teamConfig.toolPaths = { ...teamConfig.toolPaths, codebuddy: { skills: '.codebuddy/skills', mcp: '.codebuddy/mcp.json', mcpProject: '.mcp.json' } }; + // teamai owns nothing there. With no managed-mcp.json at all, any server would hold it. + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { [managedMcpManifestKey('claude', true)]: [] }); + + expect(await excludeCheck()).toBeUndefined(); + }); + + it.skipIf(process.getuid?.() === 0)('says a server was withheld because its file cannot be kept out of git, and the fix', async () => { + await fse.remove(path.join(projectRoot, '.mcp.json')); + vi.stubEnv('JIRA_TOKEN', 'long-t0ken-value-7c1'); + const excludeFile = path.join(projectRoot, '.git', 'info', 'exclude'); + await fse.chmod(excludeFile, 0o444); + + try { + const check = await mcpCheck(); + expect(await check.check()).toBe(false); + expect(check.fix).toMatch(/\.git\/info\/exclude is not writable/); + expect(check.fix).toContain('teamai pull'); + expect(check.fix).not.toContain('again..'); + } finally { + await fse.chmod(excludeFile, 0o644); + } + }); + + it('emits no check when the installed servers carry no resolved value', async () => { + await writeTeamMcp('servers:\n - name: jira\n transport: http\n url: https://jira.example/mcp\n'); + + expect(await excludeCheck()).toBeUndefined(); + }); + + it('fails the delivery check for a server withheld from a file git tracks, naming the file and the fix once', async () => { + vi.stubEnv('JIRA_TOKEN', 'fixture-jira-token'); + execFileSync('git', ['add', '.mcp.json'], { cwd: projectRoot }); + + const check = await mcpCheck(); + expect(await check.check()).toBe(false); + const file = path.join(projectRoot, '.mcp.json'); + expect(check.fix).toContain(`In ${file}, withheld: jira, as git would commit the file: git already tracks ${file}.`); + expect(check.fix).toContain(`git rm --cached ${file}\` (rotate any value a commit of it holds)`); + expect(check.fix).not.toContain('not the team\'s definition'); + expect(check.fix).not.toContain('pull --force'); + }); + // The appliers replace the file itself but follow its directories (#886). + describe('for a config under a symlinked directory, judged where the write lands', () => { + const logical = (): string => path.join(projectRoot, 'cfg', 'mcp.json'); + + beforeEach(async () => { + teamConfig.toolPaths = { claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: 'cfg/mcp.json' } }; + await fse.move(path.join(projectRoot, '.mcp.json'), path.join(projectRoot, 'config', 'mcp.json')); + await fse.symlink('config', path.join(projectRoot, 'cfg'), 'dir'); + }); + + it('fails while git tracks the file it lands in, naming both paths', async () => { + execFileSync('git', ['add', 'config/mcp.json'], { cwd: projectRoot }); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(false); + expect(check.fix).toContain(`${path.join(await fse.realpath(projectRoot), 'config', 'mcp.json')} (where ${logical()} is written)`); + }); + + it.each([ + ['passes once git ignores the file it lands in', '/config/mcp.json\n', true], + ['still fails when git ignores only the path it is reached by', '/cfg/mcp.json\n', false], + ])('%s', async (_label, line, ok) => { + await fse.appendFile(path.join(projectRoot, '.git', 'info', 'exclude'), line); + + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(ok); + }); + + it('passes when the directory links outside any repository', async () => { + const outside = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-no-repo-')); + await fse.move(path.join(projectRoot, 'config', 'mcp.json'), path.join(outside, 'mcp.json')); + await fse.remove(path.join(projectRoot, 'cfg')); + await fse.symlink(outside, path.join(projectRoot, 'cfg'), 'dir'); + + try { + const check = await excludeCheck(); + if (!check) throw new Error('no git exclude check'); + expect(await check.check()).toBe(true); + } finally { + await fse.remove(outside); + } + }); + }); + }); }); diff --git a/src/__tests__/doctor.test.ts b/src/__tests__/doctor.test.ts index 50083d36d..760d4fe2d 100644 --- a/src/__tests__/doctor.test.ts +++ b/src/__tests__/doctor.test.ts @@ -15,8 +15,9 @@ vi.mock('../config.js', async (importOriginal) => ({ vi.mock('../utils/fs.js', () => ({ pathExists: vi.fn(), readFileSafe: vi.fn(), - // Manifest loaders read through this one; no manifest exists on this machine. + // Manifest loaders read through these; no manifest exists on this machine. readFileIfExists: vi.fn().mockResolvedValue(null), + readJson: vi.fn().mockResolvedValue(null), // The delivery checks walk the team repo through resolveDesiredSkills, // resolveDesiredRules, resolveDesiredAgents and DocsHandler. This machine // has none of those; delivery on a real disk is covered by @@ -355,7 +356,11 @@ describe('doctor — hook checks', () => { copilot: { hooks: '.github/hooks/teamai.json' }, }, }); - mockedPathExists.mockImplementation(async (filePath: string) => filePath !== copilotHome); + // No project MCP config exists: one at a tool's built-in location that cannot be read would fail the git exclude check. + const { TeamaiConfigSchema } = await import('../types.js'); + const mcpConfigs = Object.values(TeamaiConfigSchema.shape.toolPaths.parse(undefined)) + .flatMap((paths) => paths.mcpProject ? [path.join(projectRoot, paths.mcpProject)] : []); + mockedPathExists.mockImplementation(async (filePath: string) => filePath !== copilotHome && !mcpConfigs.includes(filePath)); let allPassed: boolean; try { diff --git a/src/__tests__/dry-run-load-path.test.ts b/src/__tests__/dry-run-load-path.test.ts index c2c04bb63..a1091a0b1 100644 --- a/src/__tests__/dry-run-load-path.test.ts +++ b/src/__tests__/dry-run-load-path.test.ts @@ -33,6 +33,8 @@ import { codebaseCmd } from '../codebase-cmd.js'; import { contribute } from '../contribute.js'; import { generateDigest } from '../digest.js'; import { loadLocalConfigForScope } from '../config.js'; +import { envList, envUnset } from '../env-commands.js'; +import { envExec } from '../env-exec.js'; import { resolveDoctorContext } from '../doctor.js'; import { excludeList } from '../exclude.js'; import { hooksList } from '../hooks-cmd.js'; @@ -57,7 +59,7 @@ import { tagsAdd, tagsList, tagsRemove, tagsSubscribe, tagsUnsubscribe } from '. import { uninstall } from '../uninstall.js'; import { listWebhooks } from '../webhook.js'; import { updateReports } from '../utils/reports-branch.js'; -import { log } from '../utils/logger.js'; +import { log, setStderrOnly } from '../utils/logger.js'; import { legacyProjectSlug } from '../utils/partition.js'; const ROLES_YAML = @@ -158,6 +160,15 @@ function snapshotTree(root: string): Record { return files; } +/** `env exec` sends the logger to stderr for the rest of the process; put it back for the next case. */ +async function envExecDryRun(): Promise { + try { + await envExec(['--', 'true'], { dryRun: true }); + } finally { + setStderrOnly(false); + } +} + const FIXTURES: Array<[string, (root: string) => string]> = [ ['a fresh self-mode clone', setupSelfModeClone], ['a config pending the legacy role migration', setupLegacyRoleConfig], @@ -263,8 +274,8 @@ describe('--dry-run through the loaders the commands share (#850)', () => { // The command-level half of #850. Each of these reaches the legacy role // migration through a loader it used to call bare, so the fixture's // `config.yaml` gained `primaryRole` even though nothing had asked to write. - // `pull`/`push` carry `--dry-run`; `status`/`list` are read-only and pass it - // unconditionally (see the note at their `autoDetectInit` call site). + // `pull`/`push` carry `--dry-run`; `status`/`list`/`env list` are read-only + // and pass it unconditionally (see the note at their `autoDetectInit` call site). // // The positive control is the test directly above: the SAME fixture does gain // `primaryRole` when the flag is absent, so an unchanged tree here is a real @@ -274,6 +285,9 @@ describe('--dry-run through the loaders the commands share (#850)', () => { ['push --dry-run', () => push({ dryRun: true })], ['status', () => status({})], ['list', () => list(undefined, {})], + ['env list', () => envList({})], + ['env unset --dry-run', () => envUnset('TOKEN', { dryRun: true })], + ['env exec --dry-run', envExecDryRun], ['mcp inject --dry-run', () => mcpInject({ dryRun: true })], ['mcp list', () => mcpList({})], ['roles list', () => rolesList()], @@ -417,6 +431,8 @@ describe('--dry-run through the loaders the commands share (#850)', () => { ['pull --dry-run', () => pull({ dryRun: true })], ['status', () => status({})], ['list', () => list(undefined, {})], + ['env list', () => envList({})], + ['env exec --dry-run', envExecDryRun], // What the CLI's preAction hook runs before a command under --dry-run: // `maybeMigrate` before every write command (`pull`, `push`, ...), and // `queueKeptInCheckout` before one that queues a learning. Calling diff --git a/src/__tests__/e2e/env-exec-signals.test.ts b/src/__tests__/e2e/env-exec-signals.test.ts new file mode 100644 index 000000000..39da5a3b4 --- /dev/null +++ b/src/__tests__/e2e/env-exec-signals.test.ts @@ -0,0 +1,130 @@ +/** + * E2E (#875, #879 S8): `teamai env exec` passes signals through as a direct + * run would. A terminal's Ctrl-C (SIGINT) or Ctrl-\ (SIGQUIT) reaches the + * whole foreground process group, so the command already has it: teamai must + * not send it a second one, and must wait for the command to end. A signal + * sent to teamai alone (SIGTERM, or SIGINT without a terminal) is passed on. A + * command that dies of a signal Node survives (SIGPIPE) still gives 128 + its + * number. + */ +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import { spawn, spawnSync } from 'node:child_process'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const CLI = path.resolve(__dirname, '..', '..', '..', 'dist', 'index.js'); + +/** Counts `argv[2]` signals and exits with the count 500 ms after the first. */ +const COUNTER = [ + 'let n = 0;', + 'process.on(process.argv[2], () => { if (++n === 1) setTimeout(() => process.exit(n), 500); });', + 'require("fs").writeFileSync(process.argv[1], "ready");', + 'setInterval(() => {}, 1000);', +].join('\n'); + +/** + * Runs argv[2:] as the foreground job of a new terminal (os.forkpty), writes + * its pid to argv[1], and exits as it did. + */ +const IN_TERMINAL = [ + 'import os, sys', + 'pid, fd = os.forkpty()', + 'if pid == 0:', + ' os.execvp(sys.argv[2], sys.argv[2:])', + 'open(sys.argv[1], "w").write(str(pid))', + 'while True:', + ' try:', + ' if not os.read(fd, 4096): break', + ' except OSError:', + ' break', + 'sys.exit(os.waitstatus_to_exitcode(os.waitpid(pid, 0)[1]))', +].join('\n'); +const hasPython = spawnSync('python3', ['-c', 'import os; os.forkpty'], { stdio: 'ignore' }).status === 0; + +type Ended = { code: number | null; signal: NodeJS.Signals | null; stderr: string }; + +describe.skipIf(process.platform === 'win32')('teamai env exec: signals', () => { + let tmpDir: string; + let home: string; + + const cliEnv = (): NodeJS.ProcessEnv => { + const { CLAUDE_CONFIG_DIR: _ignored, ...env } = process.env; + return { ...env, HOME: home, USERPROFILE: home, FORCE_COLOR: '0', NO_COLOR: '1' }; + }; + + /** Run `env exec -- ` as the leader of a new session without a controlling terminal. */ + function start(command: string[], wrapper: string[] = []): { pid: number; ended: Promise } { + const [file = 'node', ...args] = [...wrapper, 'node', CLI, 'env', 'exec', '--', ...command]; + const child = spawn(file, args, { + cwd: tmpDir, env: cliEnv(), detached: true, stdio: ['ignore', 'ignore', 'pipe'], + }); + if (child.pid === undefined) throw new Error('teamai did not start'); + let stderr = ''; + child.stderr.on('data', (data: Buffer) => { stderr += data.toString(); }); + const ended = new Promise((resolve) => { child.on('close', (code, signal) => resolve({ code, signal, stderr })); }); + return { pid: child.pid, ended }; + } + + async function waitFor(file: string): Promise { + for (let waited = 0; !fs.existsSync(file); waited += 50) { + if (waited > 20_000) throw new Error('the command did not start within 20 s'); + await new Promise((resolve) => setTimeout(resolve, 50)); + } + } + + /** `env exec` running a command that counts `signal`; `pid` is teamai's. */ + async function counting(signal: NodeJS.Signals, where: 'no terminal' | 'terminal'): Promise<{ pid: number; ended: Promise }> { + const ready = path.join(tmpDir, `${signal}.ready`); + const pidFile = path.join(tmpDir, `${signal}.pid`); + const wrapper = where === 'terminal' ? ['python3', '-c', IN_TERMINAL, pidFile] : []; + const run = start([process.execPath, '-e', COUNTER, ready, signal], wrapper); + await waitFor(ready); + if (where === 'no terminal') return run; + await waitFor(pidFile); + return { pid: Number(fs.readFileSync(pidFile, 'utf8')), ended: run.ended }; + } + + beforeEach(() => { + tmpDir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'teamai-env-exec-signals-'))); + home = path.join(tmpDir, 'home'); + fs.mkdirSync(home); + }); + + afterEach(() => { fs.rmSync(tmpDir, { recursive: true, force: true }); }); + + it.skipIf(!hasPython).each(['SIGINT', 'SIGQUIT'] as const)( + "a %s to the terminal's foreground group reaches the command once, and teamai exits with it", + async (signal) => { + const run = await counting(signal, 'terminal'); + + process.kill(-run.pid, signal); + + expect(await run.ended).toMatchObject({ code: 1, signal: null }); + }, + ); + + it.each(['SIGINT', 'SIGQUIT'] as const)('passes a %s sent to teamai alone, without a terminal, on to the command, once', async (signal) => { + const run = await counting(signal, 'no terminal'); + + process.kill(run.pid, signal); + + expect(await run.ended).toMatchObject({ code: 1, signal: null }); + }); + + it('passes a SIGTERM sent to teamai alone on to the command, once', async () => { + const run = await counting('SIGTERM', 'no terminal'); + + process.kill(run.pid, 'SIGTERM'); + + expect(await run.ended).toMatchObject({ code: 1, signal: null }); + }); + + it('exits 141 when the command dies of SIGPIPE', async () => { + const { ended } = start(['sh', '-c', 'kill -PIPE $$']); + + expect(await ended).toMatchObject({ code: 141, signal: null }); + }); +}); diff --git a/src/__tests__/e2e/namespaced-entries.test.ts b/src/__tests__/e2e/namespaced-entries.test.ts index cc00f4d2d..f1be1b2da 100644 --- a/src/__tests__/e2e/namespaced-entries.test.ts +++ b/src/__tests__/e2e/namespaced-entries.test.ts @@ -208,8 +208,8 @@ describe('env, hooks and MCP by namespace via the real CLI (#707)', () => { expect(hookCommands()).not.toContain('echo root-lint'); const envList = await runCLI(['env', 'list'], projectRoot, home); - expect(envList.output).toContain('API_BASE=ht**** (checkout, overrides root)'); - expect(envList.output).toContain('SHARED=ev**** (root)'); + expect(envList.output).toContain('API_BASE=ht**** env.yaml (checkout, overrides root)'); + expect(envList.output).toContain('SHARED=ev**** env.yaml (root)'); const mcpList = await runCLI(['mcp', 'list'], projectRoot, home); expect(mcpList.output).toContain('from: mcp/checkout/mcp.yaml (checkout, overrides root)'); const hooksList = await runCLI(['hooks', 'list'], projectRoot, home); diff --git a/src/__tests__/e2e/project-scoped-delivery.test.ts b/src/__tests__/e2e/project-scoped-delivery.test.ts index 315cebd40..8aeca8777 100644 --- a/src/__tests__/e2e/project-scoped-delivery.test.ts +++ b/src/__tests__/e2e/project-scoped-delivery.test.ts @@ -342,7 +342,7 @@ describe('project-scoped hooks, MCP servers and env variables via the real CLI ( const envList = await runCLI(['env', 'list'], projectRoot, home); expect(envList.code, envList.output).toBe(0); - expect(envList.output).toMatch(/BILLING_URL=\S+ {2}\(billing\)/); + expect(envList.output).toMatch(/BILLING_URL=\S+ {2}env\.yaml {2}\(billing\)/); // The delivery notice for the withheld per-entry `roles:` key names // DEVOPS_ONLY in its warning; the variable itself must stay out of the // delivered list, where it would print as `DEVOPS_ONLY=`. diff --git a/src/__tests__/entry-file-read.test.ts b/src/__tests__/entry-file-read.test.ts index 1ce956c30..b04cf6415 100644 --- a/src/__tests__/entry-file-read.test.ts +++ b/src/__tests__/entry-file-read.test.ts @@ -2,6 +2,8 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import fse from 'fs-extra'; import os from 'node:os'; import path from 'node:path'; +import YAML from 'yaml'; +import { z } from 'zod'; vi.mock('../utils/logger.js', () => ({ log: { @@ -9,7 +11,9 @@ vi.mock('../utils/logger.js', () => ({ }, })); -import { entryFilePath, resolveEntriesFor, type EntryReader } from '../namespaced-entries.js'; +import { + describeEntryFailure, entryFilePath, entryLayout, readEntryFileText, resolveEntriesFor, type EntryReader, +} from '../namespaced-entries.js'; import { envEntryReader } from '../resources/env.js'; import { hooksEntryReader } from '../resources/hooks.js'; import { mcpEntryReader } from '../resources/mcp.js'; @@ -78,6 +82,54 @@ describe('reading the active entry files', () => { expect(resolution.entries.map((entry) => entry.source)).toEqual([entryFilePath(reader.type, 'Checkout')]); }); + // A second file under a type's directory, as `env/secrets.yaml` (#875): the + // reader declares its file, activation key and wording instead of its type's. + it('reads the file a reader declares, in the namespaces its activation key makes active', async () => { + const secrets = z.object({ secrets: z.array(z.object({ key: z.string(), description: z.string() })) }); + const reader: EntryReader<{ key: string; description: string }> = { + type: 'env', + layout: { ...entryLayout('env'), file: 'secrets.yaml', activation: 'env', noun: 'secret', kept: 'No secret was resolved this run.' }, + async read(absolutePath, relativePath) { + const file = await readEntryFileText(absolutePath, relativePath); + if (!file.ok) return file; + if (file.text === null) return null; + const parsed = secrets.safeParse(YAML.parse(file.text)); + return parsed.success ? { ok: true, entries: parsed.data.secrets } : { ok: false, reason: `${relativePath} is broken` }; + }, + nameOf: (secret) => secret.key, + scopeOf: () => ({}), + }; + await fse.outputFile(path.join(repoPath, 'manifest', 'projects.yaml'), + 'version: 1\nprojects:\n - id: checkout\n resources: { env: [checkout], mcp: [billing] }\n'); + await fse.outputFile(path.join(repoPath, 'env', 'env.yaml'), 'variables:\n - { key: PLAIN, value: x }\n'); + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), + 'secrets:\n - { key: GITHUB_TOKEN, description: root }\n - { key: NPM_TOKEN, description: root }\n'); + await fse.outputFile(path.join(repoPath, 'env', 'checkout', 'secrets.yaml'), 'secrets:\n - { key: GITHUB_TOKEN, description: checkout }\n'); + await fse.outputFile(path.join(repoPath, 'env', 'billing', 'secrets.yaml'), 'not: [a secrets file\n'); + const localConfig: LocalConfig = { + repo: { localPath: repoPath, remote: 'owner/repo' }, username: 't', scope: 'user', additionalRoles: [], projects: ['checkout'], + }; + + const resolution = await resolveEntriesFor(reader, localConfig); + + expect(resolution.kind).toBe('resolved'); + if (resolution.kind !== 'resolved') return; + expect(resolution.entries.map(({ name, entry, source, replaces }) => ({ name, description: entry.description, source, replaces }))) + .toEqual([ + { name: 'GITHUB_TOKEN', description: 'checkout', source: 'env/checkout/secrets.yaml', replaces: 'env/secrets.yaml' }, + { name: 'NPM_TOKEN', description: 'root', source: 'env/secrets.yaml', replaces: null }, + ]); + + await fse.outputFile(path.join(repoPath, 'env', 'checkout', 'secrets.yaml'), + 'secrets:\n - { key: A, description: x }\n - { key: A, description: y }\n'); + const failed = await resolveEntriesFor(reader, localConfig); + + expect(failed.kind).toBe('failed'); + if (failed.kind !== 'failed') return; + expect(describeEntryFailure(failed.failure)) + .toBe('env/checkout/secrets.yaml defines secret "A" more than once. No secret was resolved this run. Keep one of them in the team repo and push.'); + }); + it('does not take a model profile namespace directory for inactive when only its case differs', async () => { await fse.outputFile(path.join(repoPath, 'models', 'Checkout', 'models.yaml'), "profiles:\n - { id: gw, name: Gateway, base_url: 'https://gw.test', api_key: '${API_KEY}', model_groups: [{ protocols: [anthropic], models: [m] }] }\n"); diff --git a/src/__tests__/env-advisories.test.ts b/src/__tests__/env-advisories.test.ts new file mode 100644 index 000000000..7939219f8 --- /dev/null +++ b/src/__tests__/env-advisories.test.ts @@ -0,0 +1,354 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; +import { execFileSync } from 'node:child_process'; + +vi.mock('../config.js', async (importOriginal) => ({ + ...(await importOriginal()), + autoDetectInit: vi.fn(), + detectProjectConfig: vi.fn().mockResolvedValue(null), + loadLocalConfig: vi.fn(), + loadTeamConfig: vi.fn(), + requireInit: vi.fn(), +})); + +vi.mock('../utils/logger.js', () => ({ + log: { + debug: vi.fn(), error: vi.fn(), info: vi.fn(), success: vi.fn(), warn: vi.fn(), dim: vi.fn(), persist: vi.fn(), + }, + setStderrOnly: vi.fn(), +})); + +import { autoDetectInit, loadLocalConfig, loadTeamConfig, requireInit } from '../config.js'; +import { doctor, type DoctorReport } from '../doctor.js'; +import { envList } from '../env-commands.js'; +import { mcpList } from '../mcp-cmd.js'; +import { EnvHandler } from '../resources/env.js'; +import { getMachineSecretsPath, getTeamSecretsPath, writeSecretStore } from '../secret-store.js'; +import { log } from '../utils/logger.js'; +import { resetWarnOnce } from '../utils/warn-once.js'; +import type { LocalConfig, TeamaiConfig } from '../types.js'; + +const GITHUB_LINE = 'github: GITHUB_TOKEN is not set. Run `teamai env set GITHUB_TOKEN` (https://github.com/settings/tokens).'; +const GITHUB_SERVER = [ + 'servers:', + ' - name: github', + ' transport: http', + ' url: https://api.example.com/mcp/', + ' headers:', + ' Authorization: Bearer ${GITHUB_TOKEN}', +].join('\n'); +const GITHUB_SECRET = 'secrets:\n - key: GITHUB_TOKEN\n url: https://github.com/settings/tokens\n'; + +/** + * #875 (#879 S6): a declared secret with no value names the server that needs + * it and the command that fixes it, in `mcp list`, `env list` and `doctor`. + * Pull is covered in pull-env-advisories.test.ts. + */ +describe('a missing declared secret tells the member what to run', () => { + let tmpDir: string; + let homeDir: string; + let repoPath: string; + let localConfig: LocalConfig; + let teamConfig: TeamaiConfig; + + const write = (relativePath: string, content: string): Promise => + fse.outputFile(path.join(repoPath, ...relativePath.split('/')), content); + const warned = (): string[] => vi.mocked(log.warn).mock.calls.map(([message]) => String(message)); + + async function doctorReport(): Promise<{ allPassed: boolean; report: DoctorReport }> { + const spy = vi.spyOn(console, 'log').mockImplementation(() => {}); + try { + const allPassed = await doctor({ json: true }); + return { allPassed, report: JSON.parse(String(spy.mock.calls.at(-1)?.[0])) as DoctorReport }; + } finally { + spy.mockRestore(); + } + } + + async function quietly(run: () => Promise): Promise { + const spy = vi.spyOn(console, 'log').mockImplementation(() => {}); + try { + await run(); + } finally { + spy.mockRestore(); + } + } + + beforeEach(async () => { + resetWarnOnce(); + tmpDir = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-env-advisories-')); + homeDir = path.join(tmpDir, 'home'); + repoPath = path.join(tmpDir, 'team-repo'); + await fse.ensureDir(path.join(homeDir, '.claude', 'skills')); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('USERPROFILE', homeDir); + vi.stubEnv('GITHUB_TOKEN', undefined); + vi.stubEnv('GITLAB_TOKEN', undefined); + localConfig = { repo: { localPath: repoPath, remote: 'owner/repo' }, username: 'tester', scope: 'user', additionalRoles: [] }; + teamConfig = { + team: 'test', description: '', repo: 'owner/repo', provider: 'git', reviewers: [], + sharing: { skills: {}, rules: { enforced: [] }, docs: { localDir: '' }, env: { injectShellProfile: false } }, + toolPaths: { claude: { skills: '.claude/skills', mcp: '.claude.json' } }, + }; + await write('teamai.yaml', 'team: test\n'); + await write('mcp/mcp.yaml', GITHUB_SERVER); + await write('env/secrets.yaml', GITHUB_SECRET); + vi.mocked(loadLocalConfig).mockResolvedValue(localConfig); + vi.mocked(loadTeamConfig).mockResolvedValue(teamConfig); + vi.mocked(autoDetectInit).mockResolvedValue({ localConfig, teamConfig }); + vi.mocked(requireInit).mockResolvedValue({ localConfig, teamConfig }); + }); + + afterEach(async () => { + vi.unstubAllEnvs(); + vi.clearAllMocks(); + await fse.remove(tmpDir); + }); + + it('mcp list names the server, the key, the command and the url', async () => { + await quietly(() => mcpList({})); + + expect(warned()).toContain(GITHUB_LINE); + }); + + // #879 Conflict 14: a failed declaration can't mean "no secrets". + it('mcp list reports a broken secrets.yaml, exits 1 and does not call a secret from the environment set', async () => { + await write('env/secrets.yaml', 'secret:\n - key: GITHUB_TOKEN\n'); + vi.stubEnv('GITHUB_TOKEN', 'other-team-token'); + const spy = vi.spyOn(console, 'log').mockImplementation(() => {}); + try { + await mcpList({}); + const out = spy.mock.calls.map(([line]) => String(line)).join('\n'); + expect(out).toContain('secrets: GITHUB_TOKEN (not resolved)'); + expect(out).not.toContain('all set'); + expect(process.exitCode).toBe(1); + } finally { + spy.mockRestore(); + process.exitCode = undefined; + } + expect(vi.mocked(log.error)).toHaveBeenCalledWith(expect.stringContaining('env/secrets.yaml declares no secrets')); + }); + + it('on Windows, mcp list matches ${github_token} to the declared GITHUB_TOKEN, missing or set', async () => { + await write('mcp/mcp.yaml', GITHUB_SERVER.replace('${GITHUB_TOKEN}', '${github_token}')); + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + let spy: ReturnType | undefined; + try { + await quietly(() => mcpList({})); + expect(warned()).toContain(GITHUB_LINE); + vi.stubEnv('GITHUB_TOKEN', 'fixture-exported'); + spy = vi.spyOn(console, 'log').mockImplementation(() => {}); + await mcpList({}); + expect(spy.mock.calls.map(([line]) => String(line)).join('\n')).toContain('secrets: github_token (all set)'); + } finally { + spy?.mockRestore(); + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + }); + + it('env list prints the same line', async () => { + await quietly(() => envList({})); + + expect(warned()).toContain(GITHUB_LINE); + }); + + it('names no server for a secret no MCP server uses, with no mcp.yaml and no url', async () => { + await fse.remove(path.join(repoPath, 'mcp')); + await write('env/secrets.yaml', 'secrets:\n - key: GITLAB_TOKEN\n'); + + await quietly(() => envList({})); + await quietly(() => mcpList({})); + const { report } = await doctorReport(); + + const line = 'GITLAB_TOKEN is not set. Run `teamai env set GITLAB_TOKEN`.'; + expect(warned().filter((message) => message === line)).toHaveLength(2); + expect(report.notes).toContain(line); + }); + + it('says nothing once the member set a value', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'team-token' } }); + + await quietly(() => envList({})); + await quietly(() => mcpList({})); + const { report } = await doctorReport(); + + expect(warned().some((message) => message.includes('is not set'))).toBe(false); + expect(report.notes ?? []).not.toContain(GITHUB_LINE); + }); + + // `teamai env set KEY` there would replace the reference, which may be what the member wants, or not. + it('says to set the variable a --from-env value reads when it is unset, or to replace the reference', async () => { + vi.stubEnv('WORK_GITHUB_TOKEN', undefined); + vi.stubEnv('MY_GITHUB_TOKEN', undefined); + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN' } }); + + await quietly(() => mcpList({})); + + expect(warned()).toContain( + 'github: GITHUB_TOKEN reads WORK_GITHUB_TOKEN, which is not set. ' + + 'Set WORK_GITHUB_TOKEN, or run `teamai env set GITHUB_TOKEN` to replace the reference.', + ); + expect(warned()).not.toContain(GITHUB_LINE); + + vi.mocked(log.warn).mockClear(); + await writeSecretStore(getTeamSecretsPath(localConfig), {}); + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { env: 'MY_GITHUB_TOKEN' } }); + await quietly(() => envList({})); + + expect(warned()).toContain( + 'github: GITHUB_TOKEN reads MY_GITHUB_TOKEN, which is not set. ' + + 'Set MY_GITHUB_TOKEN, or run `teamai env set GITHUB_TOKEN --global` to replace the reference.', + ); + }); + + it('says nothing when the only value is the machine value', async () => { + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'machine-token' } }); + + await quietly(() => envList({})); + await quietly(() => mcpList({})); + const { report } = await doctorReport(); + + expect(warned().some((message) => message.includes('is not set'))).toBe(false); + expect(report.notes ?? []).not.toContain(GITHUB_LINE); + }); + + it('doctor reports it as a note and exits as it would without the secret', async () => { + await write('mcp/mcp.yaml', 'servers: []\n'); + await fse.remove(path.join(repoPath, 'env')); + const without = await doctorReport(); + + await write('mcp/mcp.yaml', GITHUB_SERVER); + await write('env/secrets.yaml', GITHUB_SECRET); + const withMissing = await doctorReport(); + + expect(withMissing.report.notes).toContain(GITHUB_LINE); + expect(withMissing.allPassed).toBe(without.allPassed); + expect(withMissing.report.ok).toBe(without.report.ok); + expect(withMissing.report.checks.filter((check) => !check.ok).map((check) => check.name)) + .toEqual(without.report.checks.filter((check) => !check.ok).map((check) => check.name)); + }); + + // Otherwise the MCP check passes and only a pull warning says why the secrets have no value. + it('doctor fails a check when the member\'s values file can\'t be read, naming the file and never a value', async () => { + await fse.outputFile(getTeamSecretsPath(localConfig), '{ "GITHUB_TOKEN": { "value": ghp_fixture_value } }'); + + const { allPassed, report } = await doctorReport(); + + const check = report.checks.find((candidate) => candidate.name === 'Your team secret values can be read'); + expect(check?.ok).toBe(false); + expect(check?.fix).toContain(`${getTeamSecretsPath(localConfig)} is not valid JSON`); + expect(allPassed).toBe(false); + expect(JSON.stringify(report)).not.toContain('ghp_fixture_value'); + }); + + it('doctor passes that check once the values file can be read', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'team-token' } }); + + const { report } = await doctorReport(); + + expect(report.checks.find((candidate) => candidate.name === 'Your team secret values can be read')?.ok).toBe(true); + }); + + it('doctor still fails an unrelated MCP delivery problem next to it', async () => { + await write('mcp/mcp.yaml', `${GITHUB_SERVER}\n - name: docs\n transport: stdio\n command: docs-server\n`); + + const { report } = await doctorReport(); + + const mcp = report.checks.find((check) => check.name === 'MCP servers delivered to claude'); + expect(mcp?.ok).toBe(false); + expect(mcp?.fix).toContain('not injected: docs'); + expect(mcp?.fix).not.toContain('GITHUB_TOKEN'); + expect(report.notes).toContain(GITHUB_LINE); + }); + + it('doctor notes that an entry kept for a missing secret may hold an old value', async () => { + await fse.writeJson(path.join(homeDir, '.claude.json'), { + mcpServers: { github: { type: 'http', url: 'https://api.example.com/mcp/', headers: { Authorization: 'Bearer old' } } }, + }); + await fse.outputJson(path.join(homeDir, '.teamai', 'managed-mcp.json'), { claude: [{ name: 'github', hash: 'h' }] }); + + const { report } = await doctorReport(); + + expect(report.notes).toContain( + 'github: the entry an earlier pull wrote stays in claude and may hold an old GITHUB_TOKEN until a pull finds its value.', + ); + expect(JSON.stringify(report)).not.toContain('Bearer old'); + }); + + it('doctor does not call an entry teamai never wrote a kept one', async () => { + await fse.writeJson(path.join(homeDir, '.claude.json'), { mcpServers: { github: { type: 'http', url: 'https://mine/' } } }); + + const { report } = await doctorReport(); + + expect((report.notes ?? []).some((note) => note.includes('earlier pull'))).toBe(false); + }); + + it('doctor notes a key declared as a secret and also set in env.yaml', async () => { + await write('env/env.yaml', 'variables:\n - key: GITHUB_TOKEN\n value: repo-token\n'); + + const { report } = await doctorReport(); + + expect(report.notes).toContain( + 'GITHUB_TOKEN is a team secret and is also set in env/env.yaml, whose value is ignored. ' + + 'Remove it from env/env.yaml and run `teamai push`.', + ); + expect(JSON.stringify(report)).not.toContain('repo-token'); + }); + + // #879 S9: doctor runs in the member's shell too, so it explains why an MCP + // server doesn't use their export; a note, like the rest. + it('doctor notes an ignored export for a team with no secrets, and env list and mcp list do not', async () => { + await fse.remove(path.join(repoPath, 'env', 'secrets.yaml')); + await write('env/env.yaml', 'variables:\n - key: GITLAB_HOST\n value: gitlab.team.example\n'); + vi.stubEnv('GITLAB_HOST', 'gitlab.dave.example'); + + await quietly(() => envList({})); + await quietly(() => mcpList({})); + const { report } = await doctorReport(); + + const line = 'GITLAB_HOST in your environment differs from the value in env/env.yaml, which this team uses. ' + + 'To use yours for this team, run `teamai env set GITLAB_HOST`.'; + expect(report.notes).toContain(line); + expect(warned().some((message) => message.includes('GITLAB_HOST'))).toBe(false); + expect(JSON.stringify(report)).not.toContain('gitlab.dave.example'); + }); + + // #879 Conflict 10: a shell opened before a team edit carries the old value + // through every later command, not only the pull that rewrote env.sh. + it('doctor does not call the value an earlier env.sh exported an ignored export', async () => { + await fse.remove(path.join(repoPath, 'env', 'secrets.yaml')); + const handler = new EnvHandler(); + await handler.writeResolvedEnv([{ key: 'GITLAB_HOST', value: 'gitlab.old.example' }], teamConfig, localConfig); + await handler.writeResolvedEnv([{ key: 'GITLAB_HOST', value: 'gitlab.new.example' }], teamConfig, localConfig); + await write('env/env.yaml', 'variables:\n - key: GITLAB_HOST\n value: gitlab.new.example\n'); + vi.stubEnv('GITLAB_HOST', 'gitlab.old.example'); + + const { report } = await doctorReport(); + + expect((report.notes ?? []).some((note) => note.includes('GITLAB_HOST'))).toBe(false); + }); + + it('mcp list names a declared secret withheld from a project config git tracks, with the file and the fix (#879)', async () => { + const projectRoot = path.join(tmpDir, 'business-repo'); + await fse.ensureDir(path.join(projectRoot, '.claude', 'skills')); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: {} }); + execFileSync('git', ['add', '.mcp.json'], { cwd: projectRoot }); + Object.assign(localConfig, { scope: 'project', projectRoot }); + teamConfig.toolPaths = { claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: '.mcp.json' } }; + vi.stubEnv('GITHUB_TOKEN', 'fixture-github-token'); + const spy = vi.spyOn(console, 'log').mockImplementation(() => {}); + try { + await mcpList({}); + const out = spy.mock.calls.map(([line]) => String(line)).join('\n'); + const file = path.join(projectRoot, '.mcp.json'); + expect(out).toContain(`withheld: claude — git already tracks ${file}. Run \`git rm --cached ${file}\` (rotate any value a commit of it holds)`); + expect(out.match(/withheld:/g)).toHaveLength(1); + } finally { + spy.mockRestore(); + } + }); +}); diff --git a/src/__tests__/env-commands.test.ts b/src/__tests__/env-commands.test.ts index ca50f0923..503b581ea 100644 --- a/src/__tests__/env-commands.test.ts +++ b/src/__tests__/env-commands.test.ts @@ -3,6 +3,7 @@ import path from 'node:path'; import os from 'node:os'; import fse from 'fs-extra'; import YAML from 'yaml'; +import { execFileSync } from 'node:child_process'; // Mock external dependencies before importing modules vi.mock('../config.js', async (importOriginal) => ({ @@ -11,7 +12,8 @@ vi.mock('../config.js', async (importOriginal) => ({ detectProjectConfig: vi.fn().mockResolvedValue(null), })); -vi.mock('../utils/git.js', () => ({ +vi.mock('../utils/git.js', async (importOriginal) => ({ + ...(await importOriginal()), pullRepo: vi.fn().mockResolvedValue('Already up to date.'), })); @@ -35,8 +37,19 @@ vi.mock('../utils/logger.js', () => ({ })), })); -import { envList, envAdd, envRemove } from '../env-commands.js'; -import { requireInit } from '../config.js'; +vi.mock('../utils/prompt.js', async (importOriginal) => ({ + ...(await importOriginal()), + askSecret: vi.fn(), + readStdin: vi.fn(), +})); + +import { envList, envAdd, envRemove, envSet, envUnset } from '../env-commands.js'; +import { askSecret, readStdin } from '../utils/prompt.js'; +import { getMachineSecretsPath, getTeamSecretsPath, writeSecretStore } from '../secret-store.js'; +import { NotInitializedError, detectProjectConfig, requireInit } from '../config.js'; +import { resolveAnchors } from '../utils/git.js'; +import { projectDataHome } from '../utils/partition.js'; +import { resolveSecretDeclarations } from '../resources/secrets.js'; import { log } from '../utils/logger.js'; import { resetWarnOnce } from '../utils/warn-once.js'; import { pullRepo } from '../utils/git.js'; @@ -155,9 +168,9 @@ scope: 'user', await envList({ reveal: true }); const allOutput = consoleSpy.mock.calls.map(c => c[0]).join('\n'); - expect(allOutput).toContain('API_BASE=checkout-value (checkout, overrides root)'); - expect(allOutput).toContain('SHARED=s (root)'); - expect(allOutput).toContain('CHECKOUT_ONLY=c (checkout)'); + expect(allOutput).toContain('API_BASE=checkout-value env.yaml (checkout, overrides root)'); + expect(allOutput).toContain('SHARED=s env.yaml (root)'); + expect(allOutput).toContain('CHECKOUT_ONLY=c env.yaml (checkout)'); expect(allOutput).not.toContain('BILLING_ONLY'); }); @@ -195,6 +208,531 @@ scope: 'user', expect(log.dim).toHaveBeenCalledWith(expect.stringContaining('My API endpoint')); }); + // #875: a declared secret is listed with where its value comes from, never the value. + it('lists each declared secret with its state and never its value, --reveal included', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), YAML.stringify({ + secrets: [ + { key: 'GITHUB_TOKEN', description: 'GitHub token', url: 'https://github.com/settings/tokens' }, + { key: 'GITLAB_TOKEN' }, + ], + })); + vi.stubEnv('GITHUB_TOKEN', 'fixture-github-value'); + vi.stubEnv('GITLAB_TOKEN', ''); + + await envList({ reveal: true, verbose: true }); + + const allOutput = consoleSpy.mock.calls.map(c => c[0]).join('\n'); + expect(allOutput).toContain('API_URL=u env.yaml (root)'); + expect(allOutput).toContain('Team secrets (2):'); + expect(allOutput).toContain('GITHUB_TOKEN environment (root)'); + expect(allOutput).toContain('GITLAB_TOKEN missing (root)'); + expect(allOutput).not.toContain('fixture-github-value'); + expect(vi.mocked(log.dim).mock.calls.map(c => c[0])).toEqual(expect.arrayContaining([ + expect.stringContaining('GitHub token'), + expect.stringContaining('https://github.com/settings/tokens'), + ])); + }); + + it('lists declared secrets when the team has no env variables', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: GITHUB_TOKEN\n'); + vi.stubEnv('GITHUB_TOKEN', ''); + + await envList({}); + + const allOutput = consoleSpy.mock.calls.map(c => c[0]).join('\n'); + expect(allOutput).toContain('GITHUB_TOKEN missing (root)'); + expect(allOutput).not.toContain('Team env variables'); + expect(log.info).not.toHaveBeenCalledWith('No env variables defined'); + }); + + // #879 Conflict 14: while the declarations fail, any variable may be a + // secret whose repo value is ignored, so no value is shown, --reveal included. + it('still lists the variables when the secrets file is broken, without their values, and says so in secret wording', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'fixture-url' }] })); + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets: [\n'); + + await envList({ reveal: true }); + + const allOutput = consoleSpy.mock.calls.map(c => c[0]).join('\n'); + expect(allOutput).toContain('API_URL (root)'); + expect(allOutput).not.toContain('fixture-url'); + expect(log.error).toHaveBeenCalledWith(expect.stringMatching( + /^env\/secrets\.yaml is not valid YAML: .*Team secrets were not resolved this run; env variables and MCP servers stay as they are\./s, + )); + expect(process.exitCode).toBe(1); + process.exitCode = undefined; + }); + }); + + // ─── envSet / envUnset (#875) ──────────────────────────── + + describe('envSet / envUnset', () => { + const logged = (): string => [log.info, log.success, log.warn, log.error, log.dim] + .flatMap((fn) => vi.mocked(fn).mock.calls.map((c) => String(c[0]))) + .concat(consoleSpy.mock.calls.map((c) => String(c[0]))) + .join('\n'); + const storeFile = (): string => getTeamSecretsPath(localConfig); + const stored = async (): Promise => fse.readJson(storeFile()); + + beforeEach(async () => { + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: GITHUB_TOKEN\n - key: GITLAB_TOKEN\n'); + vi.mocked(log.warn).mockClear(); + vi.mocked(askSecret).mockReset(); + vi.mocked(readStdin).mockReset(); + vi.stubEnv('GITHUB_TOKEN', ''); + process.exitCode = undefined; + }); + afterEach(() => { + process.exitCode = undefined; + }); + + it('keeps a value piped on stdin for this team, 0600, and never prints it', async () => { + vi.mocked(readStdin).mockResolvedValue('fixture-token-value'); + + await envSet('GITHUB_TOKEN', { stdin: true }); + + expect(await stored()).toEqual({ GITHUB_TOKEN: { value: 'fixture-token-value', kind: 'secret' } }); + expect(storeFile().startsWith(path.join(tmpDir, 'home', '.teamai', 'secrets', 'teams') + path.sep)).toBe(true); + if (process.platform !== 'win32') expect((await fse.stat(storeFile())).mode & 0o777).toBe(0o600); + await envList({ reveal: true }); + expect(logged()).toContain('GITHUB_TOKEN team (root)'); + expect(logged()).not.toContain('fixture-token-value'); + expect(process.exitCode).toBeUndefined(); + }); + + it('reads the value from the hidden prompt without a flag', async () => { + vi.mocked(askSecret).mockResolvedValue('fixture-prompt-value'); + + await envSet('GITHUB_TOKEN', {}); + + expect(askSecret).toHaveBeenCalledWith('Value for GITHUB_TOKEN: '); + expect(await stored()).toEqual({ GITHUB_TOKEN: { value: 'fixture-prompt-value', kind: 'secret' } }); + }); + + it('says how to pass a value when there is no terminal to prompt on', async () => { + vi.mocked(askSecret).mockRejectedValue(new Error('Cannot prompt for a secret in non-interactive mode')); + + await envSet('GITHUB_TOKEN', {}); + + expect(log.error).toHaveBeenCalledWith( + 'Cannot prompt for GITHUB_TOKEN without a terminal. Pipe the value with --stdin, or pass --from-env . Nothing was changed.', + ); + expect(process.exitCode).toBe(1); + expect(await fse.pathExists(storeFile())).toBe(false); + }); + + it('refuses --stdin from a terminal', async () => { + const descriptor = Object.getOwnPropertyDescriptor(process.stdin, 'isTTY'); + Object.defineProperty(process.stdin, 'isTTY', { value: true, configurable: true }); + try { + await envSet('GITHUB_TOKEN', { stdin: true }); + } finally { + if (descriptor) Object.defineProperty(process.stdin, 'isTTY', descriptor); + else delete (process.stdin as { isTTY?: boolean }).isTTY; + } + + expect(readStdin).not.toHaveBeenCalled(); + expect(log.error).toHaveBeenCalledWith(expect.stringContaining('--stdin expects piped stdin')); + expect(process.exitCode).toBe(1); + }); + + it('stores a --from-env reference, not a copy, and warns when that variable is unset', async () => { + vi.stubEnv('WORK_GITHUB_TOKEN', ''); + + await envSet('GITHUB_TOKEN', { fromEnv: 'WORK_GITHUB_TOKEN' }); + + expect(await stored()).toEqual({ GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN', kind: 'secret' } }); + expect(log.warn).toHaveBeenCalledWith('WORK_GITHUB_TOKEN is not set in this shell; GITHUB_TOKEN has no value until it is.'); + }); + + it('accepts only a key the scope declares as a secret or receives as a variable', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + vi.mocked(readStdin).mockResolvedValue('fixture-token-value'); + + await envSet('OTHER_URL', { stdin: true }); + + expect(log.error).toHaveBeenCalledWith( + "OTHER_URL is neither a secret nor an env variable this directory's team declares, so it was not set. " + + 'Its secrets: GITHUB_TOKEN, GITLAB_TOKEN. Its variables: API_URL. If the team added it recently, run `teamai pull` first.', + ); + expect(process.exitCode).toBe(1); + expect(readStdin).not.toHaveBeenCalled(); + expect(await fse.pathExists(storeFile())).toBe(false); + }); + + // #875 (#879 S9): a member overrides a variable for this team. + it('keeps a value for an env variable the scope receives, for this team', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + vi.mocked(readStdin).mockResolvedValue('https://mine.example'); + + await envSet('API_URL', { stdin: true }); + + expect(await stored()).toEqual({ API_URL: { value: 'https://mine.example', kind: 'variable' } }); + expect(log.info).toHaveBeenCalledWith('Run `teamai pull` to update MCP servers and env.sh.'); + expect(process.exitCode).toBeUndefined(); + }); + + it('on Windows, sets and unsets a key under the name the team declares it by, in any case', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + vi.mocked(readStdin).mockResolvedValue('fixture-token-value'); + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + try { + await envSet('github_token', { stdin: true }); + await envSet('api_url', { stdin: true }); + expect(await stored()).toEqual({ + GITHUB_TOKEN: { value: 'fixture-token-value', kind: 'secret' }, + API_URL: { value: 'fixture-token-value', kind: 'variable' }, + }); + + await envUnset('github_token', {}); + await envUnset('Api_Url', {}); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(await stored()).toEqual({}); + expect(process.exitCode).toBeUndefined(); + }); + + it('on Windows, replaces and removes every stored entry under another case of the key', async () => { + await writeSecretStore(storeFile(), { github_token: { value: 'fixture-old', kind: 'secret' }, Github_Token: { value: 'fixture-older', kind: 'secret' } }); + vi.mocked(readStdin).mockResolvedValue('fixture-new'); + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + try { + await envSet('GITHUB_TOKEN', { stdin: true }); + expect(await stored()).toEqual({ GITHUB_TOKEN: { value: 'fixture-new', kind: 'secret' } }); + + await writeSecretStore(storeFile(), { GITHUB_TOKEN: { value: 'fixture-new', kind: 'secret' }, github_token: { value: 'fixture-old', kind: 'secret' } }); + await envUnset('GITHUB_TOKEN', {}); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(await stored()).toEqual({}); + }); + + it('says env.sh needs a pull too after unsetting a value for an env variable, and not for a secret', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + await writeSecretStore(storeFile(), { API_URL: { value: 'https://mine.example', kind: 'variable' }, GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN' } }); + + await envUnset('API_URL', {}); + expect(log.info).toHaveBeenLastCalledWith('Run `teamai pull` to update MCP servers and env.sh.'); + + await envUnset('GITHUB_TOKEN', {}); + expect(log.info).toHaveBeenLastCalledWith('Run `teamai pull` to update MCP servers.'); + expect(await stored()).toEqual({}); + }); + + it("refuses to set a key it cannot tell is a variable when env.yaml can't be read", async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), 'variables: [\n'); + + await envSet('API_URL', { fromEnv: 'MY_API_URL' }); + + expect(log.error).toHaveBeenCalledWith('Cannot tell whether API_URL is an env variable this team sets. Nothing was changed.'); + expect(process.exitCode).toBe(1); + expect(await fse.pathExists(storeFile())).toBe(false); + }); + + it('refuses to set anything when the declarations cannot be read', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets: [\n'); + + await envSet('GITHUB_TOKEN', { fromEnv: 'WORK_GITHUB_TOKEN' }); + + expect(log.error).toHaveBeenCalledWith('Cannot tell whether GITHUB_TOKEN is a secret this team declares. Nothing was changed.'); + expect(process.exitCode).toBe(1); + expect(await fse.pathExists(storeFile())).toBe(false); + }); + + it('lets an unexpected failure reading stdin through, rather than report it as a user error', async () => { + vi.mocked(readStdin).mockRejectedValue(new Error('EIO: i/o error, read')); + + await expect(envSet('GITHUB_TOKEN', { stdin: true })).rejects.toThrow('EIO: i/o error, read'); + expect(await fse.pathExists(storeFile())).toBe(false); + }); + + it('rejects --stdin with --from-env, and an invalid key', async () => { + await envSet('GITHUB_TOKEN', { stdin: true, fromEnv: 'X' }); + await envSet('bad key', { fromEnv: 'X' }); + + expect(vi.mocked(log.error).mock.calls.map((c) => c[0])).toEqual([ + 'Pass either --stdin or --from-env, not both. Nothing was changed.', + expect.stringContaining('Invalid env variable name "bad key"'), + ]); + expect(await fse.pathExists(storeFile())).toBe(false); + }); + + it('leaves a store it cannot read as it is, and names it without its content', async () => { + const corrupt = '{"GITLAB_TOKEN": {"value": ghp_fixture_value}}'; + await fse.outputFile(storeFile(), corrupt); + + await envSet('GITHUB_TOKEN', { fromEnv: 'WORK_GITHUB_TOKEN' }); + await envUnset('GITLAB_TOKEN', {}); + + expect(await fse.readFile(storeFile(), 'utf8')).toBe(corrupt); + expect(vi.mocked(log.error).mock.calls.map((c) => c[0])).toEqual([ + expect.stringContaining(`${storeFile()} is not valid JSON.`), + expect.stringContaining(`${storeFile()} is not valid JSON.`), + ]); + expect(logged()).not.toContain('ghp_fixture_value'); + }); + + it('does not write on --dry-run, nor take the store lock', async () => { + await envSet('GITHUB_TOKEN', { fromEnv: 'WORK_GITHUB_TOKEN', dryRun: true }); + expect(await fse.pathExists(path.dirname(storeFile()))).toBe(false); + }); + + it.each([ + ['the hidden prompt', {}], + ['--stdin', { stdin: true }], + ])('previews a --dry-run set without reading a value from %s', async (_source, flags) => { + vi.mocked(askSecret).mockRejectedValue(new Error('Cannot prompt for a secret in non-interactive mode')); + vi.mocked(readStdin).mockRejectedValue(new Error('stdin was read')); + + await envSet('GITHUB_TOKEN', { ...flags, dryRun: true }); + + expect(askSecret).not.toHaveBeenCalled(); + expect(readStdin).not.toHaveBeenCalled(); + expect(log.info).toHaveBeenCalledWith(`[dry-run] Would set GITHUB_TOKEN for this team in ${storeFile()}`); + expect(process.exitCode).toBeUndefined(); + expect(await fse.pathExists(path.dirname(storeFile()))).toBe(false); + }); + + it('keeps every change when env set and env unset run at the same time', async () => { + await writeSecretStore(storeFile(), { OLD_TOKEN: { env: 'OLD' } }); + await fse.writeFile( + path.join(repoPath, 'env', 'secrets.yaml'), + 'secrets:\n - key: GITHUB_TOKEN\n - key: GITLAB_TOKEN\n - key: OLD_TOKEN\n', + ); + + await Promise.all([ + envSet('GITHUB_TOKEN', { fromEnv: 'WORK_GITHUB_TOKEN' }), + envSet('GITLAB_TOKEN', { fromEnv: 'WORK_GITLAB_TOKEN' }), + envUnset('OLD_TOKEN', {}), + ]); + + expect(await stored()).toEqual({ + GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN', kind: 'secret' }, GITLAB_TOKEN: { env: 'WORK_GITLAB_TOKEN', kind: 'secret' }, + }); + expect(await fse.pathExists(`${storeFile()}.lock`)).toBe(false); + expect(process.exitCode).toBeUndefined(); + }); + + it('unset removes the team value and keeps the others', async () => { + vi.mocked(readStdin).mockResolvedValue('fixture-token-value'); + await envSet('GITHUB_TOKEN', { stdin: true }); + await envSet('GITLAB_TOKEN', { fromEnv: 'WORK_GITLAB_TOKEN' }); + + await envUnset('GITHUB_TOKEN', {}); + await envUnset('GITHUB_TOKEN', {}); + + expect(await stored()).toEqual({ GITLAB_TOKEN: { env: 'WORK_GITLAB_TOKEN', kind: 'secret' } }); + expect(log.info).toHaveBeenCalledWith('GITHUB_TOKEN has no value for this team. Nothing was changed.'); + expect(process.exitCode).toBeUndefined(); + }); + + // #875: one value for every team on the machine. + it('--global keeps the value in machine.json, and env list shows it as global until a team value wins', async () => { + vi.mocked(readStdin).mockResolvedValue('fixture-machine-value'); + + await envSet('GITHUB_TOKEN', { stdin: true, global: true }); + + const machineFile = path.join(tmpDir, 'home', '.teamai', 'secrets', 'machine.json'); + expect(getMachineSecretsPath()).toBe(machineFile); + expect(await fse.readJson(machineFile)).toEqual({ GITHUB_TOKEN: { value: 'fixture-machine-value', kind: 'secret' } }); + if (process.platform !== 'win32') expect((await fse.stat(machineFile)).mode & 0o777).toBe(0o600); + expect(await fse.pathExists(storeFile())).toBe(false); + expect(log.success).toHaveBeenCalledWith(`Set GITHUB_TOKEN as your global value (every team on this machine) (${machineFile}).`); + + await envList({ reveal: true }); + expect(logged()).toContain('GITHUB_TOKEN global (root)'); + + await envSet('GITHUB_TOKEN', { fromEnv: 'WORK_GITHUB_TOKEN' }); + vi.stubEnv('WORK_GITHUB_TOKEN', 'fixture-work-value'); + consoleSpy.mockClear(); + await envList({ reveal: true }); + expect(logged()).toContain('GITHUB_TOKEN team (root)'); + expect(logged()).not.toContain('fixture-machine-value'); + expect(logged()).not.toContain('fixture-work-value'); + expect(process.exitCode).toBeUndefined(); + }); + + it('--global in a scope still accepts only a key the scope declares as a secret', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + + await envSet('API_URL', { fromEnv: 'MY_API_URL', global: true }); + + expect(log.error).toHaveBeenCalledWith(expect.stringContaining("API_URL is not a secret this directory's team declares, so it was not set.")); + expect(process.exitCode).toBe(1); + expect(await fse.pathExists(getMachineSecretsPath())).toBe(false); + }); + + it('--global outside any scope accepts any valid key and notes that no team declares it yet', async () => { + vi.mocked(requireInit).mockRejectedValue(new NotInitializedError('teamai is not initialized. Run `teamai init` first.')); + + await envSet('SENTRY_AUTH_TOKEN', { fromEnv: 'MY_SENTRY_TOKEN', global: true }); + + expect(await fse.readJson(getMachineSecretsPath())).toEqual({ SENTRY_AUTH_TOKEN: { env: 'MY_SENTRY_TOKEN', kind: 'secret' } }); + expect(log.info).toHaveBeenCalledWith( + 'No teamai scope here, so no team declares SENTRY_AUTH_TOKEN yet. The value applies to every team on this machine that declares it.', + ); + expect(process.exitCode).toBeUndefined(); + + await envUnset('SENTRY_AUTH_TOKEN', { global: true }); + expect(await fse.readJson(getMachineSecretsPath())).toEqual({}); + }); + + it.each([ + ['env set', () => envSet('GITHUB_TOKEN', { fromEnv: 'X' })], + ['env unset', () => envUnset('GITHUB_TOKEN', {})], + ['env list', () => envList({})], + ['env add', () => envAdd('API_URL', 'u', {})], + ['env remove', () => envRemove('API_URL', {})], + ])('%s outside any scope says it is not initialized and exits 1, without a stack trace', async (_name, run) => { + vi.mocked(requireInit).mockRejectedValue(new NotInitializedError('teamai is not initialized. Run `teamai init` first.')); + + await run(); + + expect(log.error).toHaveBeenCalledWith('teamai is not initialized. Run `teamai init` first.'); + expect(process.exitCode).toBe(1); + }); + + it('unset does not call a key a variable when the declarations can\'t be read', async () => { + await envSet('GITHUB_TOKEN', { fromEnv: 'WORK_GITHUB_TOKEN' }); + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets: [\n'); + vi.mocked(log.info).mockClear(); + + await envUnset('GITHUB_TOKEN', {}); + + expect(log.info).toHaveBeenCalledWith('Run `teamai pull` to apply it.'); + expect(log.info).not.toHaveBeenCalledWith(expect.stringContaining('env.sh')); + }); + + it('refuses set, unset and list in a project whose config cannot be read, and writes no store for any team', async () => { + const actual = await vi.importActual('../config.js'); + vi.mocked(detectProjectConfig).mockImplementation(actual.detectProjectConfig); + const root = path.join(tmpDir, 'api'); + await fse.ensureDir(root); + execFileSync('git', ['init', '-q'], { cwd: root, stdio: 'pipe' }); + const anchors = await resolveAnchors(root); + if (!anchors) throw new Error('no git anchors for the fixture project'); + const configPath = path.join(projectDataHome(anchors.projectAnchor), 'config.yaml'); + await fse.outputFile(configPath, 'repo: [not a config\n'); + vi.spyOn(process, 'cwd').mockReturnValue(root); + vi.mocked(readStdin).mockResolvedValue('fixture-work-token'); + + try { + await envSet('GITHUB_TOKEN', { stdin: true }); + await envUnset('GITHUB_TOKEN', {}); + await envList({}); + } finally { + vi.mocked(process.cwd).mockRestore(); + vi.mocked(detectProjectConfig).mockResolvedValue(null); + } + + expect(await fse.pathExists(path.join(tmpDir, 'home', '.teamai', 'secrets'))).toBe(false); + expect(vi.mocked(log.error).mock.calls.map((c) => c[0])).toEqual([ + expect.stringMatching(/^Cannot tell which team this directory belongs to: .*config\.yaml/), + expect.stringMatching(/^Cannot tell which team this directory belongs to: .*config\.yaml/), + expect.stringMatching(/^Cannot tell which team this directory belongs to: .*config\.yaml/), + ]); + expect(String(vi.mocked(log.error).mock.calls[0][0])).toContain(configPath); + expect(logged()).not.toContain('fixture-work-token'); + expect(process.exitCode).toBe(1); + }); + + it('unset --global removes only the machine value', async () => { + await envSet('GITHUB_TOKEN', { fromEnv: 'WORK_GITHUB_TOKEN' }); + await envSet('GITHUB_TOKEN', { fromEnv: 'PERSONAL_GITHUB_TOKEN', global: true }); + await envSet('GITLAB_TOKEN', { fromEnv: 'PERSONAL_GITLAB_TOKEN', global: true }); + + await envUnset('GITHUB_TOKEN', { global: true }); + await envUnset('GITHUB_TOKEN', { global: true }); + + expect(await fse.readJson(getMachineSecretsPath())).toEqual({ GITLAB_TOKEN: { env: 'PERSONAL_GITLAB_TOKEN', kind: 'secret' } }); + expect(await stored()).toEqual({ GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN', kind: 'secret' } }); + expect(log.success).toHaveBeenCalledWith(`Removed GITHUB_TOKEN's global value (every team on this machine) (${getMachineSecretsPath()}).`); + expect(log.info).toHaveBeenCalledWith('GITHUB_TOKEN has no global value (every team on this machine). Nothing was changed.'); + }); + + it('env list shows the member\'s value of an overridden variable, as team, and the team\'s value otherwise', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ + variables: [{ key: 'GITLAB_HOST', value: 'gitlab.team.example' }, { key: 'API_URL', value: 'u' }], + })); + vi.mocked(readStdin).mockResolvedValue('gitlab.dave.example'); + await envSet('GITLAB_HOST', { stdin: true }); + consoleSpy.mockClear(); + + await envList({ reveal: true }); + + expect(logged()).toContain('GITLAB_HOST=gitlab.dave.example team (root)'); + expect(logged()).toContain('API_URL=u env.yaml (root)'); + expect(logged()).not.toContain('gitlab.team.example'); + }); + + // #879: a value keeps the kind it was set as, so a former secret's value never becomes a variable override. + it('env list says a value set while the key was a secret is not used for the variable it is now, and how to fix it', async () => { + vi.mocked(readStdin).mockResolvedValue('fixture-old-secret'); + await envSet('GITLAB_TOKEN', { stdin: true }); + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: GITHUB_TOKEN\n'); + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'GITLAB_TOKEN', value: 'team-gitlab' }] })); + consoleSpy.mockClear(); + + await envList({ reveal: true }); + + expect(logged()).toContain('GITLAB_TOKEN=team-gitlab env.yaml (root)'); + expect(logged()).toContain( + ' Your value for this team was set while GITLAB_TOKEN was a secret, so it is not used. ' + + 'Run `teamai env unset GITLAB_TOKEN` to remove it, then `teamai env set GITLAB_TOKEN` to set one for the env variable.', + ); + expect(logged()).not.toContain('fixture-old-secret'); + expect(process.exitCode).toBeUndefined(); + }); + + it('env list says a variable override is not used for the secret the key is now', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + vi.mocked(readStdin).mockResolvedValue('https://mine.example'); + await envSet('API_URL', { stdin: true }); + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: API_URL\n'); + consoleSpy.mockClear(); + + await envList({ reveal: true }); + + expect(logged()).toContain('API_URL missing (root)'); + expect(logged()).toContain( + ' Your value for this team was set while API_URL was an env variable, so it is not used. ' + + 'Run `teamai env unset API_URL` to remove it, then `teamai env set API_URL` to set one for the secret.', + ); + }); + + it('env list shows unreadable, not missing, while the member\'s values file can\'t be read, and exits 1', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + await fse.outputFile(storeFile(), '{ "GITHUB_TOKEN": { "value": ghp_fixture_value } }'); + + await envList({ reveal: true }); + + expect(logged()).toContain('GITHUB_TOKEN unreadable (root)'); + expect(logged()).toContain('API_URL unreadable (root)'); + expect(logged()).toContain(`${storeFile()} is not valid JSON`); + expect(logged()).not.toContain('ghp_fixture_value'); + expect(logged()).not.toContain('GITHUB_TOKEN is not set'); + expect(process.exitCode).toBe(1); + }); + + // #879 Conflict 13: a key declared twice is listed only as a secret. + it('env list --reveal leaves out the env.yaml value of a key declared as a secret', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ + variables: [{ key: 'GITHUB_TOKEN', value: 'fixture-repo-value' }, { key: 'API_URL', value: 'u' }], + })); + + await envList({ reveal: true }); + + expect(logged()).toContain('Team env variables (1):'); + expect(logged()).toContain('API_URL=u env.yaml (root)'); + expect(logged()).toContain('GITHUB_TOKEN missing (root)'); + expect(logged()).not.toContain('fixture-repo-value'); + }); + it('names the variable an unknown key takes out of the delivered set (#822)', async () => { await fse.writeFile( path.join(repoPath, 'env', 'env.yaml'), @@ -241,6 +779,8 @@ scope: 'user', await envAdd('bad key', 'v', {}); expect(log.error).toHaveBeenCalledWith(expect.stringContaining('bad key')); + expect(process.exitCode).toBe(1); + process.exitCode = undefined; // Nothing written, and no env.yaml is created just to hold nothing. const envYamlPath = path.join(repoPath, 'env', 'env.yaml'); expect(await fse.pathExists(envYamlPath)).toBe(false); @@ -569,6 +1109,285 @@ scope: 'user', }); }); + // ─── Declaring secrets (#875) ──────────────────────────── + + describe('env add --secret / env remove of a secret (#875)', () => { + const rootSecrets = () => path.join(repoPath, 'env', 'secrets.yaml'); + const nsSecrets = (ns: string) => path.join(repoPath, 'env', ns, 'secrets.yaml'); + const secretsIn = async (file: string): Promise => YAML.parse(await fse.readFile(file, 'utf-8')).secrets; + /** Every string the command logged, to assert a value never appears in it. */ + const logged = (): string => [log.info, log.success, log.warn, log.error, log.dim] + .flatMap((fn) => vi.mocked(fn).mock.calls.flat()).join('\n'); + + beforeEach(() => { + vi.mocked(log.warn).mockClear(); + process.exitCode = 0; + }); + afterEach(() => { + process.exitCode = 0; + }); + + it('declares a secret in env/secrets.yaml with its description and url, and no value', async () => { + await envAdd('GITHUB_TOKEN', undefined, { + secret: true, description: 'GitHub token for gh', url: 'https://github.com/settings/tokens', + }); + + expect(await secretsIn(rootSecrets())).toEqual([ + { key: 'GITHUB_TOKEN', description: 'GitHub token for gh', url: 'https://github.com/settings/tokens' }, + ]); + expect(await fse.pathExists(path.join(repoPath, 'env', 'env.yaml'))).toBe(false); + expect(log.success).toHaveBeenCalledWith('Declared secret: GITHUB_TOKEN'); + expect(log.info).toHaveBeenCalledWith('Run `teamai push` to sync to team repo.'); + // What was written is what a member's CLI reads back. + const declarations = await resolveSecretDeclarations(localConfig); + expect(declarations.kind === 'resolved' && declarations.entries.map((s) => s.name)).toEqual(['GITHUB_TOKEN']); + }); + + it('on Windows, updates and removes an env.yaml variable typed in another case, before any secret of that name', async () => { + const envYaml = path.join(repoPath, 'env', 'env.yaml'); + await fse.outputFile(envYaml, YAML.stringify({ variables: [{ key: 'TOKEN', value: 'a' }] })); + await fse.outputFile(rootSecrets(), YAML.stringify({ secrets: [{ key: 'token' }] })); + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + try { + await envAdd('token', 'b', {}); + expect((YAML.parse(await fse.readFile(envYaml, 'utf-8')) as { variables: unknown[] }).variables).toEqual([{ key: 'TOKEN', value: 'b' }]); + + await envRemove('token', {}); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect((YAML.parse(await fse.readFile(envYaml, 'utf-8')) as { variables?: unknown[] }).variables ?? []).toEqual([]); + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'token' }]); + }); + + it('on Windows, updates and removes a declaration typed in another case, rather than add a second one', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ secrets: [{ key: 'TOKEN' }] })); + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + try { + await envAdd('token', undefined, { secret: true, description: 'the token' }); + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'TOKEN', description: 'the token' }]); + + await envRemove('Token', { secret: true }); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(await secretsIn(rootSecrets())).toEqual([]); + }); + + it('declares a secret with the key alone', async () => { + await envAdd('NPM_TOKEN', undefined, { secret: true }); + + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'NPM_TOKEN' }]); + }); + + it('--role declares it in env//secrets.yaml, and --role names that file when no one declares the namespace', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ secrets: [{ key: 'GITHUB_TOKEN' }] })); + + await envAdd('GITHUB_TOKEN', undefined, { secret: true, role: 'checkout', description: 'checkout token' }); + + expect(await secretsIn(nsSecrets('checkout'))).toEqual([{ key: 'GITHUB_TOKEN', description: 'checkout token' }]); + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'GITHUB_TOKEN' }]); + expect(log.success).toHaveBeenCalledWith('Declared secret in env/checkout/secrets.yaml: GITHUB_TOKEN'); + }); + + it('--role warns about the secrets file when no role or project declares the namespace', async () => { + await fse.outputFile(path.join(repoPath, 'manifest', 'projects.yaml'), YAML.stringify({ + version: 1, projects: [{ id: 'billing', resources: { env: ['billing'] } }], + })); + + await envAdd('GITHUB_TOKEN', undefined, { secret: true, role: 'checkout' }); + + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining( + 'No role or project declares env namespace "checkout", so env/checkout/secrets.yaml reaches nobody', + )); + }); + + it("--project declares it in the project's env namespace", async () => { + await fse.outputFile(path.join(repoPath, 'manifest', 'projects.yaml'), YAML.stringify({ + version: 1, projects: [{ id: 'checkout', resources: { env: ['checkout-env'] } }], + })); + + await envAdd('GITHUB_TOKEN', undefined, { secret: true, project: 'checkout' }); + + expect(await secretsIn(nsSecrets('checkout-env'))).toEqual([{ key: 'GITHUB_TOKEN' }]); + }); + + it('updates a declared secret: a new description replaces the old one, the url and other entries stay', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ + secrets: [ + { key: 'GITHUB_TOKEN', description: 'old', url: 'https://github.com/settings/tokens' }, + { key: 'NPM_TOKEN', owner: 'infra' }, + ], + })); + + await envAdd('GITHUB_TOKEN', undefined, { secret: true, description: 'new' }); + + expect(await secretsIn(rootSecrets())).toEqual([ + { key: 'GITHUB_TOKEN', description: 'new', url: 'https://github.com/settings/tokens' }, + { key: 'NPM_TOKEN', owner: 'infra' }, + ]); + expect(log.success).toHaveBeenCalledWith('Updated secret: GITHUB_TOKEN'); + expect(log.warn).not.toHaveBeenCalled(); + }); + + it('updates the first declaration of a key declared twice, removes the rest, and says how many', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ + secrets: [{ key: 'GITHUB_TOKEN', description: 'first' }, { key: 'NPM_TOKEN' }, { key: 'GITHUB_TOKEN' }, { key: 'GITHUB_TOKEN' }], + })); + + await envAdd('GITHUB_TOKEN', undefined, { secret: true, url: 'https://github.com/settings/tokens' }); + + expect(await secretsIn(rootSecrets())).toEqual([ + { key: 'GITHUB_TOKEN', description: 'first', url: 'https://github.com/settings/tokens' }, + { key: 'NPM_TOKEN' }, + ]); + expect(log.success).toHaveBeenCalledWith('Updated secret: GITHUB_TOKEN, and removed 2 duplicate declarations of it'); + }); + + it('updating a secret with an unknown key warns that it is still not declared, without printing a value', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ + secrets: [{ key: 'GITHUB_TOKEN', value: 'ghp_do_not_print', owner: 'infra' }], + })); + + await envAdd('GITHUB_TOKEN', undefined, { secret: true, description: 'new' }); + + expect(await secretsIn(rootSecrets())).toEqual([ + { key: 'GITHUB_TOKEN', value: 'ghp_do_not_print', owner: 'infra', description: 'new' }, + ]); + expect(log.warn).toHaveBeenCalledWith( + 'env/secrets.yaml: secret "GITHUB_TOKEN" has unknown keys `value:`, `owner:`, so it is not declared. ' + + 'Correct the keys or remove them in env/secrets.yaml.', + ); + expect(logged()).not.toContain('ghp_do_not_print'); + }); + + it('writes nothing on dry-run', async () => { + await envAdd('GITHUB_TOKEN', undefined, { secret: true, dryRun: true }); + + expect(await fse.pathExists(rootSecrets())).toBe(false); + expect(log.info).toHaveBeenCalledWith('[dry-run] Would declare secret: GITHUB_TOKEN'); + }); + + it('rejects a value with --secret, writes nothing and never prints the value', async () => { + await envAdd('GITHUB_TOKEN', 'ghp_do_not_print', { secret: true }); + + expect(await fse.pathExists(rootSecrets())).toBe(false); + expect(await fse.pathExists(path.join(repoPath, 'env', 'env.yaml'))).toBe(false); + expect(log.error).toHaveBeenCalledWith(expect.stringContaining('A secret has no value in the team repo')); + expect(logged()).not.toContain('ghp_do_not_print'); + expect(process.exitCode).toBe(1); + }); + + it('rejects a variable without a value', async () => { + await envAdd('API_BASE', undefined, {}); + + expect(await fse.pathExists(path.join(repoPath, 'env', 'env.yaml'))).toBe(false); + expect(log.error).toHaveBeenCalledWith(expect.stringContaining('No value for "API_BASE"')); + expect(process.exitCode).toBe(1); + }); + + it('rejects --url without --secret', async () => { + await envAdd('API_BASE', 'x', { url: 'https://example.com' }); + + expect(await fse.pathExists(path.join(repoPath, 'env', 'env.yaml'))).toBe(false); + expect(log.error).toHaveBeenCalledWith(expect.stringContaining('--url')); + expect(process.exitCode).toBe(1); + }); + + it('refuses to write into a secrets file that does not parse, and leaves it as it was', async () => { + const broken = 'GITHUB_TOKEN: x\n'; + await fse.outputFile(rootSecrets(), broken); + + await envAdd('NPM_TOKEN', undefined, { secret: true }); + + expect(log.error).toHaveBeenCalledWith(expect.stringContaining('env/secrets.yaml')); + expect(await fse.readFile(rootSecrets(), 'utf-8')).toBe(broken); + expect(process.exitCode).toBe(1); + }); + + it('env remove removes a declared secret', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ secrets: [{ key: 'GITHUB_TOKEN' }, { key: 'NPM_TOKEN' }] })); + + await envRemove('GITHUB_TOKEN', {}); + + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'NPM_TOKEN' }]); + expect(log.success).toHaveBeenCalledWith('Removed secret: GITHUB_TOKEN'); + expect(log.info).toHaveBeenCalledWith('Run `teamai push` to sync to team repo.'); + }); + + it('env remove --secret removes every declaration of a key declared twice, and says how many', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ secrets: [{ key: 'GITHUB_TOKEN' }, { key: 'NPM_TOKEN' }, { key: 'GITHUB_TOKEN' }] })); + + await envRemove('GITHUB_TOKEN', { secret: true }); + + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'NPM_TOKEN' }]); + expect(log.success).toHaveBeenCalledWith('Removed secret: GITHUB_TOKEN, and 1 duplicate declaration of it'); + }); + + it('env remove --role removes the secret from the namespace file only', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ secrets: [{ key: 'GITHUB_TOKEN' }] })); + await fse.outputFile(nsSecrets('checkout'), YAML.stringify({ secrets: [{ key: 'GITHUB_TOKEN' }] })); + + await envRemove('GITHUB_TOKEN', { role: 'checkout' }); + + expect(await secretsIn(nsSecrets('checkout'))).toEqual([]); + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'GITHUB_TOKEN' }]); + expect(log.success).toHaveBeenCalledWith('Removed secret in env/checkout/secrets.yaml: GITHUB_TOKEN'); + }); + + // An admin moving a token out of env.yaml declares it first, then removes the value. + it('with the key in both files, env remove removes the variable and --secret removes the secret', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'GITHUB_TOKEN', value: 'v' }] })); + await fse.outputFile(rootSecrets(), YAML.stringify({ secrets: [{ key: 'GITHUB_TOKEN' }] })); + + await envRemove('GITHUB_TOKEN', {}); + + expect(YAML.parse(await fse.readFile(path.join(repoPath, 'env', 'env.yaml'), 'utf-8')).variables).toEqual([]); + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'GITHUB_TOKEN' }]); + + await envRemove('GITHUB_TOKEN', { secret: true }); + + expect(await secretsIn(rootSecrets())).toEqual([]); + }); + + it('env remove --secret leaves a variable alone and says the secret is not declared', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'GITHUB_TOKEN', value: 'v' }] })); + + await envRemove('GITHUB_TOKEN', { secret: true }); + + expect(YAML.parse(await fse.readFile(path.join(repoPath, 'env', 'env.yaml'), 'utf-8')).variables) + .toEqual([{ key: 'GITHUB_TOKEN', value: 'v' }]); + expect(log.error).toHaveBeenCalledWith( + 'Secret "GITHUB_TOKEN" is not declared in env/secrets.yaml. Nothing was changed. For a namespace\'s file, pass ' + + '--role or --project ; `teamai env list` shows where each secret this directory receives comes from.', + ); + expect(process.exitCode).toBe(1); + }); + + it('env remove of a variable env.yaml lacks names the broken secrets file and still says the variable is not there', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables: [{ key: 'API_URL', value: 'u' }] })); + await fse.outputFile(rootSecrets(), 'secrets: [\n'); + + await envRemove('FOO', {}); + + expect(log.error).toHaveBeenCalledWith(expect.stringMatching(/^env\/secrets\.yaml is not valid YAML/)); + expect(log.error).toHaveBeenCalledWith('Env variable "FOO" not found'); + expect(process.exitCode).toBe(1); + process.exitCode = undefined; + }); + + it('env remove of a secret writes nothing on dry-run', async () => { + await fse.outputFile(rootSecrets(), YAML.stringify({ secrets: [{ key: 'GITHUB_TOKEN' }] })); + + await envRemove('GITHUB_TOKEN', { dryRun: true }); + + expect(await secretsIn(rootSecrets())).toEqual([{ key: 'GITHUB_TOKEN' }]); + expect(log.info).toHaveBeenCalledWith('[dry-run] Would remove secret: GITHUB_TOKEN'); + }); + }); + describe('envRemove', () => { it('should remove existing variable locally and show push hint', async () => { await fse.writeFile( diff --git a/src/__tests__/env-exec.test.ts b/src/__tests__/env-exec.test.ts new file mode 100644 index 000000000..0944070a7 --- /dev/null +++ b/src/__tests__/env-exec.test.ts @@ -0,0 +1,552 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { execFileSync } from 'node:child_process'; +import crypto from 'node:crypto'; +import fs from 'node:fs'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; +import YAML from 'yaml'; + +import { envExec, exitLike, inTerminalForeground } from '../env-exec.js'; +import { envShMarker } from '../env-sh-exports.js'; +import { getMachineSecretsPath, getTeamSecretsPath, writeSecretStore, type SecretStore } from '../secret-store.js'; +import { resolveAnchors } from '../utils/git.js'; +import { _resetState, _setLogFilePath, setStderrOnly } from '../utils/logger.js'; +import { projectDataHome } from '../utils/partition.js'; +import type { LocalConfig } from '../types.js'; + +/** + * `teamai env exec -- ` (#875, #879 S8): the command runs with the + * inherited environment overlaid with this directory's team env variables and + * its secrets in the resolution order. Everything teamai prints goes to stderr. + */ +describe('teamai env exec', () => { + let tmpDir: string; + let home: string; + let out: string; + let stdout: string[]; + let stderr: string[]; + + const GITHUB_SECRET = 'secrets:\n - key: GITHUB_TOKEN\n url: https://github.com/settings/tokens\n'; + const GITHUB_LINE = 'GITHUB_TOKEN is not set. Run `teamai env set GITHUB_TOKEN` (https://github.com/settings/tokens).'; + // Writes the child's environment to a file: the child's own stdout is the + // terminal the test runs in. + const DUMP = 'require("fs").writeFileSync(process.argv[1], JSON.stringify(process.env))'; + + const git = (cwd: string, ...args: string[]): void => { execFileSync('git', args, { cwd, stdio: 'pipe' }); }; + const text = (lines: string[]): string => lines.join('\n'); + + /** A team repo clone with these files, and a config for it. */ + async function team(name: string, files: Record): Promise<{ repoPath: string }> { + const repoPath = path.join(tmpDir, `${name}-repo`); + await fse.outputFile(path.join(repoPath, 'teamai.yaml'), `team: ${name}\nrepo: https://example.com/${name}.git\n`); + for (const [file, content] of Object.entries(files)) await fse.outputFile(path.join(repoPath, file), content); + return { repoPath }; + } + + async function userScope(repoPath: string, extra: Partial = {}): Promise { + const config: LocalConfig = { + repo: { localPath: repoPath, remote: 'https://example.com/user.git' }, username: 't', scope: 'user', additionalRoles: [], ...extra, + }; + await fse.outputFile(path.join(home, '.teamai', 'config.yaml'), YAML.stringify(config)); + return config; + } + + /** A git project with a linked worktree, set up as a teamai project in its partition. */ + async function project(repoPath: string): Promise<{ root: string; worktree: string; config: LocalConfig; partition: string }> { + const root = path.join(tmpDir, 'api'); + await fse.ensureDir(root); + git(root, 'init', '-q'); + git(root, 'config', 'user.email', 't@example.com'); + git(root, 'config', 'user.name', 't'); + git(root, 'commit', '--allow-empty', '-q', '-m', 'init'); + const worktree = path.join(tmpDir, 'api-feature'); + git(root, 'worktree', 'add', '-q', worktree, 'HEAD'); + const anchors = await resolveAnchors(root); + if (!anchors) throw new Error('no git anchors for the fixture project'); + const partition = projectDataHome(anchors.projectAnchor); + const config: LocalConfig = { + repo: { localPath: repoPath, remote: 'https://example.com/work.git' }, username: 't', scope: 'project', + projectRoot: root, additionalRoles: [], + }; + await fse.outputFile(path.join(partition, 'config.yaml'), YAML.stringify(config)); + return { root, worktree, config, partition }; + } + + async function exec(cwd: string, script = DUMP, args: string[] = [out]): ReturnType { + return envExec(['--', process.execPath, '-e', script, ...args], {}, cwd); + } + + async function childEnv(cwd: string): Promise> { + const outcome = await exec(cwd); + expect(outcome).toEqual({ kind: 'exited', code: 0 }); + return JSON.parse(await fse.readFile(out, 'utf8')) as Record; + } + + beforeEach(async () => { + tmpDir = fs.realpathSync(await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-env-exec-'))); + home = path.join(tmpDir, 'home'); + out = path.join(tmpDir, 'child-env.json'); + await fse.ensureDir(home); + vi.stubEnv('HOME', home); + vi.stubEnv('USERPROFILE', home); + for (const key of ['GITHUB_TOKEN', 'API_URL', 'WORK_GITHUB_TOKEN']) vi.stubEnv(key, undefined); + _setLogFilePath(path.join(home, '.teamai', 'debug.log')); + stdout = []; + stderr = []; + const record = (sink: string[]) => (...parts: unknown[]) => { sink.push(parts.map(String).join(' ')); }; + vi.spyOn(console, 'log').mockImplementation(record(stdout)); + vi.spyOn(console, 'info').mockImplementation(record(stdout)); + vi.spyOn(console, 'error').mockImplementation(record(stderr)); + vi.spyOn(console, 'warn').mockImplementation(record(stderr)); + }); + + afterEach(async () => { + setStderrOnly(false); + _resetState(); + vi.restoreAllMocks(); + vi.unstubAllEnvs(); + await fse.remove(tmpDir); + }); + + it('overlays the scope variables on the inherited environment, and resolves a secret team > machine > environment', async () => { + const { repoPath } = await team('personal', { 'env/env.yaml': 'variables:\n - key: API_URL\n value: https://team.example\n', 'env/secrets.yaml': GITHUB_SECRET }); + const config = await userScope(repoPath); + vi.stubEnv('API_URL', 'https://inherited.example'); + vi.stubEnv('UNRELATED', 'kept'); + vi.stubEnv('GITHUB_TOKEN', 'fixture-exported'); + + let env = await childEnv(home); + expect(env.API_URL).toBe('https://team.example'); + expect(env.UNRELATED).toBe('kept'); + expect(env.GITHUB_TOKEN).toBe('fixture-exported'); + + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'fixture-machine' } }); + env = await childEnv(home); + expect(env.GITHUB_TOKEN).toBe('fixture-machine'); + + await writeSecretStore(getTeamSecretsPath(config), { GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN' } }); + vi.stubEnv('WORK_GITHUB_TOKEN', 'fixture-from-env'); + env = await childEnv(home); + expect(env.GITHUB_TOKEN).toBe('fixture-from-env'); + expect(text(stderr)).not.toContain('is not set'); + }); + + // #875 (#879 S9): the same order as MCP for a variable. + it("gives a variable the member's value for this team over env.yaml and the inherited one", async () => { + const { repoPath } = await team('personal', { 'env/env.yaml': 'variables:\n - key: API_URL\n value: https://team.example\n' }); + const config = await userScope(repoPath); + vi.stubEnv('API_URL', 'https://inherited.example'); + await writeSecretStore(getMachineSecretsPath(), { API_URL: { value: 'https://machine.example' } }); + + expect((await childEnv(home)).API_URL).toBe('https://team.example'); + + await writeSecretStore(getTeamSecretsPath(config), { API_URL: { value: 'https://mine.example', kind: 'variable' } }); + expect((await childEnv(home)).API_URL).toBe('https://mine.example'); + + await writeSecretStore(getTeamSecretsPath(config), { API_URL: { env: 'MY_API_URL', kind: 'variable' } }); + vi.stubEnv('MY_API_URL', 'https://mine-from-env.example'); + expect((await childEnv(home)).API_URL).toBe('https://mine-from-env.example'); + }); + + // #879: a former secret's stored value never reaches the child as the variable it is now. + it('gives a variable its env.yaml value, never a value stored while it was a secret, and a secret never a variable override', async () => { + const { repoPath } = await team('personal', { + 'env/env.yaml': 'variables:\n - key: API_URL\n value: https://team.example\n', 'env/secrets.yaml': GITHUB_SECRET, + }); + const config = await userScope(repoPath); + await writeSecretStore(getTeamSecretsPath(config), { + API_URL: { value: 'fixture-old-secret' }, + GITHUB_TOKEN: { value: 'fixture-override', kind: 'variable' }, + }); + + const env = await childEnv(home); + expect(env.API_URL).toBe('https://team.example'); + expect(Object.hasOwn(env, 'GITHUB_TOKEN')).toBe(false); + expect(text(stderr)).toContain(GITHUB_LINE); + }); + + // `__proto__` is a valid env key; an ordinary object's inherited setter would drop it. + it('passes a secret and a variable named __proto__, and removes the secret when it has no value', async () => { + const { repoPath } = await team('personal', { 'env/secrets.yaml': 'secrets:\n - key: __proto__\n' }); + const config = await userScope(repoPath); + vi.stubEnv('__proto__', 'fixture-exported'); + + expect((await childEnv(home))['__proto__']).toBe('fixture-exported'); + + await writeSecretStore(getTeamSecretsPath(config), { ['__proto__']: { env: 'WORK_GITHUB_TOKEN' } }); + expect(Object.hasOwn(await childEnv(home), '__proto__')).toBe(false); + + await writeSecretStore(getTeamSecretsPath(config), { ['__proto__']: { value: 'fixture-secret' } }); + expect((await childEnv(home))['__proto__']).toBe('fixture-secret'); + + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets: []\n'); + await fse.outputFile(path.join(repoPath, 'env', 'env.yaml'), 'variables:\n - key: __proto__\n value: fixture-team\n'); + await writeSecretStore(getTeamSecretsPath(config), {}); + expect((await childEnv(home))['__proto__']).toBe('fixture-team'); + }); + + // Windows environment names are case-insensitive: a declared key in another case is the same variable. + it('on Windows, removes and overlays a declared key in any case, and elsewhere only in its own case', async () => { + const { repoPath } = await team('personal', { + 'env/env.yaml': 'variables:\n - key: api_url\n value: https://team.example\n', 'env/secrets.yaml': GITHUB_SECRET, + }); + const config = await userScope(repoPath); + await writeSecretStore(getTeamSecretsPath(config), { GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN' } }); + vi.stubEnv('API_URL', 'https://inherited.example'); + vi.stubEnv('github_token', 'fixture-exported'); + const named = (env: Record, key: string): Record => + Object.fromEntries(Object.entries(env).filter(([name]) => name.toUpperCase() === key)); + + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + let env: Record; + try { + env = await childEnv(home); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(named(env, 'API_URL')).toEqual({ api_url: 'https://team.example' }); + expect(named(env, 'GITHUB_TOKEN')).toEqual({}); + + env = await childEnv(home); + expect(named(env, 'API_URL')).toEqual({ API_URL: 'https://inherited.example', api_url: 'https://team.example' }); + expect(named(env, 'GITHUB_TOKEN')).toEqual({ github_token: 'fixture-exported' }); + }); + + it('resolves the project scope from a linked worktree of the project, and the user scope elsewhere', async () => { + const personal = await team('personal', { 'env/secrets.yaml': GITHUB_SECRET }); + const user = await userScope(personal.repoPath); + const work = await team('work', { 'env/secrets.yaml': GITHUB_SECRET, 'env/env.yaml': 'variables:\n - key: API_URL\n value: https://work.example\n' }); + const { root, worktree, config } = await project(work.repoPath); + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'fixture-personal' } }); + await writeSecretStore(getTeamSecretsPath(config), { GITHUB_TOKEN: { value: 'fixture-work' } }); + expect(getTeamSecretsPath(config)).not.toBe(getTeamSecretsPath(user)); + const sub = path.join(worktree, 'src'); + await fse.ensureDir(sub); + + expect((await childEnv(home)).GITHUB_TOKEN).toBe('fixture-personal'); + expect((await childEnv(root)).GITHUB_TOKEN).toBe('fixture-work'); + const fromWorktree = await childEnv(sub); + expect(fromWorktree.GITHUB_TOKEN).toBe('fixture-work'); + expect(fromWorktree.API_URL).toBe('https://work.example'); + }); + + it('prints the missing-secret line on stderr and still runs the command', async () => { + const { repoPath } = await team('personal', { 'env/secrets.yaml': GITHUB_SECRET }); + await userScope(repoPath); + + const env = await childEnv(home); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(text(stderr)).toContain(GITHUB_LINE); + expect(stdout).toEqual([]); + }); + + it('removes a declared key whose only environment value is one teamai exported for another scope', async () => { + const { repoPath } = await team('personal', { 'env/secrets.yaml': GITHUB_SECRET }); + await userScope(repoPath); + await fse.outputFile(path.join(home, '.teamai', 'projects', 'other-0123456789', 'env.sh'), "export GITHUB_TOKEN='fixture-other-team'\n"); + vi.stubEnv('GITHUB_TOKEN', 'fixture-other-team'); + + const env = await childEnv(home); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(text(stderr)).toContain(GITHUB_LINE); + }); + + it('removes a declared key whose team entry names an unset variable, rather than pass the inherited value', async () => { + const { repoPath } = await team('personal', { 'env/secrets.yaml': GITHUB_SECRET }); + const config = await userScope(repoPath); + await writeSecretStore(getTeamSecretsPath(config), { GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN' } }); + vi.stubEnv('GITHUB_TOKEN', 'fixture-personal-export'); + + expect((await childEnv(home)).GITHUB_TOKEN).toBeUndefined(); + }); + + it('applies no variables and no secrets on a failed declaration, and names the failure', async () => { + const { repoPath } = await team('personal', { + 'env/env.yaml': 'variables:\n - key: API_URL\n value: https://team.example\n', + 'env/secrets.yaml': 'secrets:\n - key: 1BAD\n', + }); + const config = await userScope(repoPath); + await writeSecretStore(getTeamSecretsPath(config), { GITHUB_TOKEN: { value: 'fixture-team' } }); + vi.stubEnv('GITHUB_TOKEN', 'fixture-exported'); + + const env = await childEnv(home); + + expect(env.API_URL).toBeUndefined(); + expect(env.GITHUB_TOKEN).toBe('fixture-exported'); + expect(text(stderr)).toContain('env/secrets.yaml'); + expect(text(stderr)).toContain('The command runs with the inherited environment, without team env variables or secrets.'); + }); + + it('removes what a teamai env.sh exported while the declarations fail, keeps the member\'s own exports, and names the keys', async () => { + const { repoPath } = await team('personal', { 'env/secrets.yaml': 'secrets: [not yaml\n' }); + await userScope(repoPath); + await fse.outputFile(path.join(home, '.teamai', 'env.sh'), "export GITHUB_TOKEN='fixture-repo-token'\nexport SENTRY_TOKEN='fixture-repo-sentry'\n"); + const [marker, digests] = envShMarker(path.join(tmpDir, 'unscanned', '.teamai'), [['GITLAB_TOKEN', 'fixture-marked']]) ?? []; + if (!marker || !digests) throw new Error('no marker for the fixture export'); + vi.stubEnv(marker, digests); + vi.stubEnv('GITHUB_TOKEN', 'fixture-repo-token'); + vi.stubEnv('GITLAB_TOKEN', 'fixture-marked'); + vi.stubEnv('SENTRY_TOKEN', 'fixture-hand-export'); + + const env = await childEnv(home); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(env.GITLAB_TOKEN).toBeUndefined(); + expect(env.SENTRY_TOKEN).toBe('fixture-hand-export'); + expect(text(stderr)).toContain('without GITHUB_TOKEN, GITLAB_TOKEN, whose values a teamai env.sh exported.'); + expect(text(stderr)).not.toMatch(/fixture-(repo|marked|hand)/); + }); + + it.each([ + ['env.yaml does not parse', { 'env/env.yaml': 'variables: [not yaml\n' }, false], + ['the values file cannot be read', { 'env/env.yaml': 'variables:\n - key: REGION\n value: eu\n' }, true], + ])('removes what a teamai env.sh exported when %s, keeps the member\'s own exports, and names the keys', async (_, files, corruptStore) => { + const { repoPath } = await team('personal', files); + const config = await userScope(repoPath); + if (corruptStore) await fse.outputFile(getTeamSecretsPath(config), '{"REGION": {"value": fixture-corrupt}}'); + await fse.outputFile(path.join(home, '.teamai', 'env.sh'), "export GITHUB_TOKEN='fixture-repo-token'\n"); + vi.stubEnv('GITHUB_TOKEN', 'fixture-repo-token'); + vi.stubEnv('SENTRY_TOKEN', 'fixture-hand-export'); + + const env = await childEnv(home); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(env.SENTRY_TOKEN).toBe('fixture-hand-export'); + expect(text(stderr)).toContain('without GITHUB_TOKEN, whose values a teamai env.sh exported.'); + expect(text(stderr)).not.toMatch(/fixture-(repo|hand|corrupt)/); + }); + + it('keeps a legacy env.yaml value of a key that may be a secret from the command while the declarations fail', async () => { + const { repoPath } = await team('personal', { + 'env/env.yaml': 'variables:\n - key: GITHUB_TOKEN\n value: fixture-legacy-repo\n', + 'env/secrets.yaml': 'secrets: [not yaml\n', + }); + await userScope(repoPath); + + const env = await childEnv(home); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(text(stderr)).not.toContain('fixture-legacy-repo'); + }); + + it('removes every declared key when the values file cannot be read, and says why', async () => { + const { repoPath } = await team('personal', { 'env/secrets.yaml': GITHUB_SECRET }); + const config = await userScope(repoPath); + await fse.outputFile(getTeamSecretsPath(config), '{"GITHUB_TOKEN": {"value": fixture-corrupt}}'); + vi.stubEnv('GITHUB_TOKEN', 'fixture-exported'); + + const env = await childEnv(home); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(text(stderr)).toContain(getTeamSecretsPath(config)); + expect(text(stderr)).not.toContain('fixture-corrupt'); + }); + + it('names a project config that cannot be read, applies no stored values, and runs the command', async () => { + const personal = await team('personal', { 'env/env.yaml': 'variables:\n - key: API_URL\n value: https://personal.example\n', 'env/secrets.yaml': GITHUB_SECRET }); + await userScope(personal.repoPath); + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'fixture-machine' } }); + const work = await team('work', {}); + const { root, partition } = await project(work.repoPath); + await fse.outputFile(path.join(partition, 'config.yaml'), 'repo: [not a config\n'); + + const env = await childEnv(root); + + expect(env.API_URL).toBeUndefined(); + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(text(stderr)).toContain(path.join(partition, 'config.yaml')); + expect(text(stderr)).not.toContain('No teamai config'); + }); + + it("removes what another scope's env.sh exported when a project config cannot be read, keeps the member's own exports, and names the keys", async () => { + const personal = await team('personal', { 'env/secrets.yaml': GITHUB_SECRET }); + await userScope(personal.repoPath); + const work = await team('work', {}); + const { root, partition } = await project(work.repoPath); + await fse.outputFile(path.join(partition, 'config.yaml'), 'repo: [not a config\n'); + await fse.outputFile(path.join(home, '.teamai', 'env.sh'), "export GITHUB_TOKEN='fixture-user-scope'\n"); + await fse.outputFile(path.join(partition, 'env.sh'), "export API_URL='fixture-work-scope'\n"); + const [marker, digests] = envShMarker(path.join(tmpDir, 'unscanned', '.teamai'), [['GITLAB_TOKEN', 'fixture-marked']]) ?? []; + if (!marker || !digests) throw new Error('no marker for the fixture export'); + vi.stubEnv(marker, digests); + vi.stubEnv('GITHUB_TOKEN', 'fixture-user-scope'); + vi.stubEnv('API_URL', 'fixture-work-scope'); + vi.stubEnv('GITLAB_TOKEN', 'fixture-marked'); + vi.stubEnv('SENTRY_TOKEN', 'fixture-hand-export'); + + const env = await childEnv(root); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(env.API_URL).toBeUndefined(); + expect(env.GITLAB_TOKEN).toBeUndefined(); + expect(env.SENTRY_TOKEN).toBe('fixture-hand-export'); + expect(text(stderr)).toContain(path.join(partition, 'config.yaml')); + expect(text(stderr)).toMatch(/without (?=.*GITHUB_TOKEN)(?=.*API_URL)(?=.*GITLAB_TOKEN)[A-Z_, ]+, whose values a teamai env\.sh exported/); + expect(text(stderr)).not.toMatch(/fixture-(user|work|marked|hand)/); + }); + + it('with no config at all, runs with the inherited environment, applies no machine value, and says so', async () => { + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'fixture-machine' } }); + vi.stubEnv('UNRELATED', 'kept'); + const nowhere = path.join(tmpDir, 'nowhere'); + await fse.ensureDir(nowhere); + + const env = await childEnv(nowhere); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(env.UNRELATED).toBe('kept'); + expect(text(stderr)).toContain('No teamai config'); + expect(stdout).toEqual([]); + }); + + it("with no config, removes what a teamai env.sh exported, keeps the member's own exports, and names the keys", async () => { + await fse.outputFile(path.join(home, '.teamai', 'projects', 'other-0123456789', 'env.sh'), "export GITHUB_TOKEN='fixture-other-team'\n"); + vi.stubEnv('GITHUB_TOKEN', 'fixture-other-team'); + vi.stubEnv('SENTRY_TOKEN', 'fixture-hand-export'); + const nowhere = path.join(tmpDir, 'nowhere'); + await fse.ensureDir(nowhere); + + const env = await childEnv(nowhere); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(env.SENTRY_TOKEN).toBe('fixture-hand-export'); + expect(text(stderr)).toContain('No teamai config'); + expect(text(stderr)).toContain('without GITHUB_TOKEN, whose values a teamai env.sh exported'); + expect(text(stderr)).not.toMatch(/fixture-(other|hand)/); + }); + + it("in an HTTP-backed scope, removes what another team's env.sh exported, keeps the member's own exports, and names the keys", async () => { + const { repoPath } = await team('http', {}); + await userScope(repoPath, { repo: { localPath: repoPath, remote: 'https://team.example/api', kind: 'http', url: 'https://team.example/api' } }); + await fse.outputFile(path.join(home, '.teamai', 'projects', 'other-0123456789', 'env.sh'), "export GITHUB_TOKEN='fixture-other-team'\n"); + vi.stubEnv('GITHUB_TOKEN', 'fixture-other-team'); + vi.stubEnv('SENTRY_TOKEN', 'fixture-hand-export'); + + const env = await childEnv(home); + + expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(env.SENTRY_TOKEN).toBe('fixture-hand-export'); + expect(text(stderr)).toContain('HTTP team repo'); + expect(text(stderr)).toContain('without GITHUB_TOKEN, whose values a teamai env.sh exported'); + expect(text(stderr)).not.toMatch(/fixture-(other|hand)/); + }); + + // Without `--`, a flag of the command (`gh pr list --dry-run`) would be read as teamai's. + it('rejects a command without -- before it, with exit code 2, and runs nothing', async () => { + const marker = path.join(tmpDir, 'ran'); + const outcome = await envExec([process.execPath, '-e', `require("fs").writeFileSync(${JSON.stringify(marker)}, "")`, '--dry-run'], {}, tmpDir); + + expect(outcome).toEqual({ kind: 'exited', code: 2 }); + expect(text(stderr)).toContain('Put -- before the command: teamai env exec -- '); + expect(await fse.pathExists(marker)).toBe(false); + }); + + it('accepts teamai options before --, and passes everything after it to the command', async () => { + const outcome = await envExec(['--verbose', '--', process.execPath, '-e', 'process.exit(process.argv[1] === "--dry-run" ? 4 : 5)', '--', '--dry-run'], {}, tmpDir); + + expect(outcome).toEqual({ kind: 'exited', code: 4 }); + }); + + it('passes the exit code and the signal through', async () => { + const nowhere = path.join(tmpDir, 'nowhere'); + await fse.ensureDir(nowhere); + + expect(await exec(nowhere, 'process.exit(3)', [])).toEqual({ kind: 'exited', code: 3 }); + expect(await exec(nowhere, 'process.kill(process.pid, "SIGTERM"); setTimeout(() => {}, 5000)', [])) + .toEqual({ kind: 'signaled', signal: 'SIGTERM' }); + }); + + it('exits 128 + the signal number for a signal that does not end teamai (SIGPIPE, SIGUSR1)', () => { + try { + exitLike({ kind: 'signaled', signal: 'SIGPIPE' }); + expect(process.exitCode).toBe(141); + exitLike({ kind: 'signaled', signal: 'SIGUSR1' }); + expect(process.exitCode).toBe(128 + os.constants.signals.SIGUSR1); + } finally { + process.exitCode = undefined; + } + }); + + // #879: a terminal sends Ctrl-C to its foreground group, the command included; any other SIGINT is teamai's alone. + it.each([ + ['in the foreground group of its terminal', '77167 77167\n', true], + ['in a background job of its terminal', ' 77167 80012\n', false], + ['without a controlling terminal (macOS)', '77167 0\n', false], + ['without a controlling terminal (Linux)', '77167 -1\n', false], + ['when ps printed nothing', '', false], + ['when ps printed something else', 'PGID TPGID\n', false], + ] as const)('takes teamai to be %s from `ps -o pgid=,tpgid=`', (_name, ps, foreground) => { + expect(inTerminalForeground(ps)).toBe(foreground); + }); + + it('reports a command that cannot be started, with exit code 127', async () => { + const nowhere = path.join(tmpDir, 'nowhere'); + await fse.ensureDir(nowhere); + + expect(await envExec(['--', 'teamai-no-such-command-875'], {}, nowhere)).toEqual({ kind: 'exited', code: 127 }); + expect(text(stderr)).toContain('teamai-no-such-command-875'); + }); + + it('prints nothing on stdout, the scope lookup included', async () => { + const { repoPath } = await team('personal', { + 'env/secrets.yaml': GITHUB_SECRET, + 'manifest/roles.yaml': 'version: 1\nroles:\n - id: hai\n description: default\n resources:\n knowledge: []\n skills: []\n', + }); + await userScope(repoPath); + + await childEnv(home); + + expect(stdout).toEqual([]); + expect(text(stderr)).toContain('Migrated legacy teamai config'); + }); + + it('writes no member value to disk or debug.log', async () => { + const personal = await team('personal', { 'env/secrets.yaml': GITHUB_SECRET }); + const user = await userScope(personal.repoPath); + const work = await team('work', { 'env/secrets.yaml': GITHUB_SECRET }); + const { root, config } = await project(work.repoPath); + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'fixture-machine' } }); + await writeSecretStore(getTeamSecretsPath(user), { GITHUB_TOKEN: { value: 'fixture-team' } } satisfies SecretStore); + await writeSecretStore(getTeamSecretsPath(config), { GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN' } }); + vi.stubEnv('WORK_GITHUB_TOKEN', 'fixture-parent-env-only'); + const fixtures = ['fixture-machine', 'fixture-team', 'fixture-parent-env-only']; + const holding = async (): Promise> => { + const found = new Map(); + for (const file of await filesUnder([home, root, personal.repoPath, work.repoPath])) { + const content = await fse.readFile(file); + if (fixtures.some((fixture) => content.includes(fixture))) { + found.set(file, crypto.createHash('sha256').update(content).digest('hex')); + } + } + return found; + }; + const before = await holding(); + + expect((await childEnv(home)).GITHUB_TOKEN).toBe('fixture-team'); + expect((await childEnv(root)).GITHUB_TOKEN).toBe('fixture-parent-env-only'); + await fse.remove(out); + + expect(await holding()).toEqual(before); + for (const log of ['debug.log', 'debug.log.1']) { + const content = await fse.readFile(path.join(home, '.teamai', log), 'utf8').catch(() => ''); + for (const fixture of fixtures) expect(content).not.toContain(fixture); + } + }); +}); + +async function filesUnder(roots: string[]): Promise { + const files: string[] = []; + const walk = async (dir: string): Promise => { + for (const entry of await fse.readdir(dir, { withFileTypes: true }).catch(() => [])) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) await walk(full); + else if (entry.isFile()) files.push(full); + } + }; + for (const root of roots) await walk(root); + return files; +} diff --git a/src/__tests__/env-handler.test.ts b/src/__tests__/env-handler.test.ts index c83bc8885..16bc7d908 100644 --- a/src/__tests__/env-handler.test.ts +++ b/src/__tests__/env-handler.test.ts @@ -4,7 +4,7 @@ import os from 'node:os'; import fse from 'fs-extra'; import YAML from 'yaml'; import { execFileSync } from 'node:child_process'; -import { EnvHandler, describeEnvYamlShapeProblem } from '../resources/env.js'; +import { EnvHandler, describeEnvYamlShapeProblem, parseEnvFile } from '../resources/env.js'; import { resetWarnOnce } from '../utils/warn-once.js'; import { TEAMAI_ENV_START, TEAMAI_ENV_END } from '../types.js'; import type { TeamaiConfig, LocalConfig, ResourceItem } from '../types.js'; @@ -118,6 +118,20 @@ scope: 'user', expect(items.map((item) => item.name)).toEqual(['billing/env.yaml', 'checkout/env.yaml']); }); + // #875: a declared secret is published by push like a variable. + it('reports a changed secrets file, root and namespace', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'env.yaml'), 'variables: []\n'); + run(['init', '-q', '-b', 'main']); + run(['add', '-A']); + run(['commit', '-q', '-m', 'seed']); + + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: GITHUB_TOKEN\n'); + await fse.outputFile(path.join(repoPath, 'env', 'checkout', 'secrets.yaml'), 'secrets:\n - key: NPM_TOKEN\n'); + + const items = await handler.scanLocalForPush(teamConfig, localConfig); + expect(items.map((item) => item.relativePath)).toEqual(['env/secrets.yaml', 'env/checkout/secrets.yaml']); + }); + // git quotes a non-ASCII path in its default output, so it never matched. it('reports a changed namespace file whose name is not ASCII', async () => { await fse.outputFile(path.join(repoPath, 'env', 'café', 'env.yaml'), 'variables: []\n'); @@ -650,7 +664,9 @@ scope: 'user', expect(await envSh()).toContain('CHECKOUT_ONLY'); await handler.pullItem(item, teamConfig, { ...localConfig, projects: ['billing'] }); - expect((await envSh()).trim()).toBe(''); + // Only the marker of what it exported before is left (env-sh-exports.ts). + expect(await envSh()).not.toContain('CHECKOUT_ONLY'); + expect([...parseEnvFile(await envSh()).keys()]).toEqual([]); const backup = await fse.readFile(path.join(homeDir, '.teamai', 'env'), 'utf-8'); expect(backup).not.toContain('CHECKOUT_ONLY'); }); diff --git a/src/__tests__/fs-atomic-mode.test.ts b/src/__tests__/fs-atomic-mode.test.ts new file mode 100644 index 000000000..502fcb264 --- /dev/null +++ b/src/__tests__/fs-atomic-mode.test.ts @@ -0,0 +1,48 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; + +import { writeFileAtomic, writeJsonAtomic } from '../utils/fs.js'; + +/** + * The secret and model key stores write through these helpers (#879): the + * temp file must never be readable by group or other, not even between its + * creation and the chmod that sets the target's mode. + */ +describe.skipIf(process.platform === 'win32')('atomic writes create the temp file with the target mode', () => { + let tmpDir: string; + let previousUmask: number; + + beforeEach(async () => { + tmpDir = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-fs-atomic-')); + previousUmask = process.umask(0o022); + }); + + afterEach(async () => { + process.umask(previousUmask); + vi.restoreAllMocks(); + await fse.remove(tmpDir); + }); + + const writers = [ + ['writeFileAtomic', (file: string) => writeFileAtomic(file, 'fixture-secret\n', { mode: 0o600 })], + ['writeJsonAtomic', (file: string) => writeJsonAtomic(file, { KEY: { value: 'fixture-secret' } }, { mode: 0o600 })], + ] as const; + + it.each(writers)('%s: the temp file has no group or other bits before its chmod', async (_name, write) => { + const seen: number[] = []; + const chmod = fse.chmod.bind(fse); + vi.spyOn(fse, 'chmod').mockImplementation(async (file: fse.PathLike, mode: fse.Mode) => { + seen.push((await fse.stat(file)).mode & 0o777); + return chmod(file, mode); + }); + const file = path.join(tmpDir, 'store.json'); + + await write(file); + + expect(seen).toHaveLength(1); + expect(seen[0] & 0o077).toBe(0); + expect((await fse.stat(file)).mode & 0o777).toBe(0o600); + }); +}); diff --git a/src/__tests__/hook-handlers.test.ts b/src/__tests__/hook-handlers.test.ts index 55622efde..410dbf7a5 100644 --- a/src/__tests__/hook-handlers.test.ts +++ b/src/__tests__/hook-handlers.test.ts @@ -123,6 +123,11 @@ vi.mock('../pkg/pkg-hint.js', () => ({ takePendingPackageHint: mockTakePendingPackageHint, })); +const mockMrHintOutput = vi.fn().mockResolvedValue(null); +vi.mock('../mr-hint.js', () => ({ + computeMrHintOutput: mockMrHintOutput, +})); + vi.mock('../transcript-parser.js', () => ({ parseTranscriptForVotes: mockParseTranscriptForVotes, })); @@ -1380,6 +1385,69 @@ describe('post-tool-use dispatch — local-agent runs detached, never blocks hos }); }); +describe('session-start secrets hint (#875)', () => { + let teamRepo: string; + + beforeEach(() => { + vi.clearAllMocks(); + teamRepo = fs.mkdtempSync(path.join(os.tmpdir(), 'teamai-secrets-hint-')); + fs.writeFileSync(path.join(teamRepo, 'teamai.yaml'), 'team: acme\nrepo: https://example.test/acme/team.git\n'); + const sessionStart = (context: string) => JSON.stringify({ hookSpecificOutput: { hookEventName: 'SessionStart', additionalContext: context } }); + mockMrHintOutput.mockResolvedValueOnce(sessionStart('MR context')); + mockClaimPackageHint.mockResolvedValueOnce(sessionStart('Package context')); + }); + + afterEach(() => { + fs.rmSync(teamRepo, { recursive: true, force: true }); + }); + + async function sessionContext(): Promise { + const localConfig: LocalConfig = { ...scope, repo: { localPath: teamRepo, remote: '' } }; + const dispatcher = createDispatcher({ handlers: filterHandlersForConfig(buildHandlerRegistry(), localConfig), localConfig }); + const result = await dispatcher.dispatch('session-start', '*', { session_id: 'sid-secrets', cwd: teamRepo }, 'claude', 'foreground'); + expect(result.errors).toEqual([]); + return (JSON.parse(result.output ?? '{}').hookSpecificOutput.additionalContext as string).split('\n'); + } + + it('tells the agent once which secrets the scope declares and to run their CLIs through env exec', async () => { + fs.mkdirSync(path.join(teamRepo, 'env')); + fs.writeFileSync(path.join(teamRepo, 'env', 'secrets.yaml'), [ + 'secrets:', + ' - { key: GITHUB_TOKEN, description: "gh and the github MCP server" }', + ' - { key: SENTRY_AUTH_TOKEN }', + '', + ].join('\n')); + + const lines = await sessionContext(); + + const secretLines = lines.filter((line) => line.includes('GITHUB_TOKEN')); + expect(secretLines).toHaveLength(1); + expect(secretLines[0]).toContain('GITHUB_TOKEN (gh and the github MCP server)'); + expect(secretLines[0]).toContain('SENTRY_AUTH_TOKEN'); + expect(secretLines[0]).toContain('teamai env exec --'); + expect(secretLines[0]).toContain('teamai env set KEY'); + expect(lines.filter((line) => line !== secretLines[0])).toEqual(['MR context', 'Package context']); + }); + + it('adds nothing when the scope declares no secrets', async () => { + expect(await sessionContext()).toEqual(['MR context', 'Package context']); + }); + + it('adds nothing when the secrets file cannot be read', async () => { + fs.mkdirSync(path.join(teamRepo, 'env')); + fs.writeFileSync(path.join(teamRepo, 'env', 'secrets.yaml'), 'secrets: [\n'); + + expect(await sessionContext()).toEqual(['MR context', 'Package context']); + }); + + it('is a foreground team handler, so a directory without teamai never gets the line', () => { + const registration = buildHandlerRegistry().find((r) => r.handler.name === 'secrets-hint'); + expect(registration).toMatchObject({ event: 'session-start', matcher: '*', requiresConfig: true }); + expect(registration?.background).not.toBe(true); + expect(filterHandlersForConfig(buildHandlerRegistry(), null).map((r) => r.handler.name)).not.toContain('secrets-hint'); + }); +}); + describe('dashboard-report team correction keywords', () => { const handler = () => buildHandlerRegistry().find( (r) => r.event === 'prompt-submit' && r.handler.name === 'dashboard-report', diff --git a/src/__tests__/mcp-cmd.test.ts b/src/__tests__/mcp-cmd.test.ts index 64debaf59..97c9fd533 100644 --- a/src/__tests__/mcp-cmd.test.ts +++ b/src/__tests__/mcp-cmd.test.ts @@ -8,22 +8,43 @@ vi.mock('../namespaced-entries.js', async (importOriginal) => ({ ...(await importOriginal()), resolveEntriesFor: vi.fn(), })); -vi.mock('../mcp-reconcile.js', () => ({ - reconcileMcpForConfig: vi.fn(), - resolveMcpTargets: vi.fn().mockResolvedValue([]), - buildVarTable: vi.fn().mockResolvedValue({}), +// The per-target delivery filters stay real: `withheld` must name only where a pull would write. +vi.mock('../mcp-reconcile.js', async (importOriginal) => { + const actual = await importOriginal(); + return { + desiredMcpForTarget: actual.desiredMcpForTarget, + mcpTargetExcluded: actual.mcpTargetExcluded, + reconcileMcpForConfig: vi.fn(), + releaseCleanMcpGitExcludes: vi.fn(), + resolveMcpTargets: vi.fn().mockResolvedValue([]), + buildDesiredMcpContext: vi.fn().mockResolvedValue({ + sharing: { autoApply: true, allowedCommands: [], allowedHosts: [] }, + excluded: new Set(), + vars: { JIRA_TOKEN: 'jira-token-value' }, + secrets: { kind: 'absent' }, + }), + }; +}); +vi.mock('../mcp-git-exclude.js', async (importOriginal) => ({ + ...(await importOriginal()), + ensureExcludedFromGit: vi.fn(), })); vi.mock('../utils/fs.js', () => ({ readJson: vi.fn().mockResolvedValue(null), + // No teamai env.sh on this machine (member-env.ts, via the env advisories). + readFileSafe: vi.fn().mockResolvedValue(null), })); vi.mock('../utils/logger.js', () => ({ log: { info: vi.fn(), success: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn(), persist: vi.fn() }, })); import { autoDetectInit } from '../config.js'; -import { resolveEntriesFor } from '../namespaced-entries.js'; -import { mcpInject, mcpList } from '../mcp-cmd.js'; -import { reconcileMcpForConfig } from '../mcp-reconcile.js'; +import { entryLayout, resolveEntriesFor } from '../namespaced-entries.js'; +import { mcpInject, mcpList, mcpRemove } from '../mcp-cmd.js'; +import { reconcileMcpForConfig, releaseCleanMcpGitExcludes, resolveMcpTargets } from '../mcp-reconcile.js'; +import { ensureExcludedFromGit } from '../mcp-git-exclude.js'; +import { readJson } from '../utils/fs.js'; +import { managedMcpManifestKey } from '../types.js'; import { resetWarnOnce } from '../utils/warn-once.js'; const mockedAutoDetectInit = autoDetectInit as Mock; @@ -89,6 +110,69 @@ describe('mcpList', () => { expect(text.match(/roles:/g)).toHaveLength(1); }); + it('says where a server needing a resolved value is withheld because git would commit the file, and the fix (#882)', async () => { + mockedResolve.mockResolvedValue(resolved([ + [{ name: 'jira', transport: 'http', url: 'https://jira.example/mcp', headers: { Authorization: 'Bearer ${JIRA_TOKEN}' } }, 'mcp/mcp.yaml', null], + ])); + (resolveMcpTargets as Mock).mockResolvedValueOnce([ + { tool: 'claude', format: 'claude', file: '/work/app/.mcp.json', projectScope: true }, + ]); + (ensureExcludedFromGit as Mock).mockResolvedValueOnce({ + kind: 'failed', + reason: '/work/app/.git/info/exclude is not writable', + fix: 'Make it writable, then run `teamai pull` again.', + }); + + const text = await listOutput(); + + expect(ensureExcludedFromGit).toHaveBeenCalledWith('/work/app/.mcp.json', { dryRun: true }); + expect(text).toContain('withheld: claude — /work/app/.git/info/exclude is not writable. Make it writable, then run `teamai pull` again.'); + }); + + it('still says a server is withheld from a file an earlier pull installed it in (#882)', async () => { + mockedAutoDetectInit.mockResolvedValue({ + localConfig: { repo: { localPath: '/repo' }, scope: 'project', projectRoot: '/work/app', additionalRoles: [] }, + teamConfig: { toolPaths: {} }, + }); + mockedResolve.mockResolvedValue(resolved([ + [{ name: 'jira', transport: 'http', url: 'https://jira.example/mcp', headers: { Authorization: 'Bearer ${JIRA_TOKEN}' } }, 'mcp/mcp.yaml', null], + ])); + (resolveMcpTargets as Mock).mockResolvedValueOnce([ + { tool: 'claude', format: 'claude', file: '/work/app/.mcp.json', projectScope: true }, + ]); + (readJson as Mock).mockResolvedValueOnce({ [managedMcpManifestKey('claude', true)]: [{ name: 'jira', hash: 'h' }] }); + (ensureExcludedFromGit as Mock).mockResolvedValueOnce({ + kind: 'failed', + reason: 'git already tracks /work/app/.mcp.json', + fix: 'Run `git rm --cached /work/app/.mcp.json` (rotate any value a commit of it holds), then `teamai pull` again.', + }); + + const text = await listOutput(); + + expect(text).toContain('installed: claude'); + expect(text).toContain('withheld: claude — git already tracks /work/app/.mcp.json. Run `git rm --cached /work/app/.mcp.json`'); + }); + + it('does not say a server is withheld from a tool delivery never writes it to (#882)', async () => { + mockedResolve.mockResolvedValue(resolved([ + [{ name: 'jira', transport: 'http', url: 'https://jira.example/mcp', headers: { Authorization: 'Bearer ${JIRA_TOKEN}' }, tools: ['cursor'] }, 'mcp/mcp.yaml', null], + ])); + (resolveMcpTargets as Mock).mockResolvedValueOnce([ + { tool: 'claude', format: 'claude', file: '/work/app/.mcp.json', projectScope: true }, + ]); + (ensureExcludedFromGit as Mock).mockResolvedValue({ + kind: 'failed', + reason: 'git already tracks /work/app/.mcp.json', + fix: 'Run `git rm --cached /work/app/.mcp.json`, then `teamai pull` again.', + }); + + try { + expect(await listOutput()).not.toContain('withheld'); + } finally { + (ensureExcludedFromGit as Mock).mockReset(); + } + }); + it('reports a set that cannot be resolved instead of listing part of it', async () => { mockedResolve.mockResolvedValue({ kind: 'failed', @@ -96,7 +180,10 @@ describe('mcpList', () => { kind: 'unknown-key', message: 'mcp/mcp.yaml: server "hidden" has unknown key `role:`, so this entry is not delivered.', }], - failure: { kind: 'two-namespaces', type: 'mcp', name: 'db', first: 'mcp/checkout/mcp.yaml', second: 'mcp/billing/mcp.yaml' }, + failure: { + kind: 'two-namespaces', type: 'mcp', name: 'db', first: 'mcp/checkout/mcp.yaml', second: 'mcp/billing/mcp.yaml', + layout: entryLayout('mcp'), + }, }); const { log } = await import('../utils/logger.js'); await listOutput(); @@ -167,3 +254,26 @@ describe('mcpInject', () => { } }); }); + +describe('mcpRemove', () => { + it('takes out the .git/info/exclude lines of the configs it leaves clean, after removing the servers (#882)', async () => { + const init = { localConfig: { repo: { localPath: '/repo' }, scope: 'project', projectRoot: '/work/app' }, teamConfig: { toolPaths: {} } }; + mockedAutoDetectInit.mockResolvedValue(init); + const order: string[] = []; + (reconcileMcpForConfig as Mock).mockImplementationOnce(async () => { + order.push('reconcile'); + return { changes: [], wrote: false }; + }); + (releaseCleanMcpGitExcludes as Mock).mockImplementationOnce(async () => { order.push('release'); }); + const spy = vi.spyOn(console, 'log').mockImplementation(() => undefined); + try { + await mcpRemove({}); + } finally { + spy.mockRestore(); + } + + expect(reconcileMcpForConfig).toHaveBeenCalledWith(init.teamConfig, init.localConfig, { removeAll: true }); + expect(releaseCleanMcpGitExcludes).toHaveBeenCalledWith(init.teamConfig, init.localConfig); + expect(order).toEqual(['reconcile', 'release']); + }); +}); diff --git a/src/__tests__/mcp-git-exclude.test.ts b/src/__tests__/mcp-git-exclude.test.ts new file mode 100644 index 000000000..e17bb895b --- /dev/null +++ b/src/__tests__/mcp-git-exclude.test.ts @@ -0,0 +1,351 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; +import { execFileSync } from 'node:child_process'; + +vi.mock('../utils/logger.js', () => ({ + log: { debug: vi.fn(), error: vi.fn(), info: vi.fn(), success: vi.fn(), warn: vi.fn(), dim: vi.fn() }, +})); + +// Git's own failure modes (unsafe repository, bad config) are hard to stage for one subcommand alone. +const failCheckIgnore = vi.hoisted(() => ({ on: false })); +const failLsFiles = vi.hoisted(() => ({ on: false })); +const failVerboseCheckIgnore = vi.hoisted(() => ({ on: false })); +vi.mock('../utils/exec.js', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + execCommand: (cmd: string, args: string[], opts?: Parameters[2]) => + (failCheckIgnore.on || (failVerboseCheckIgnore.on && args.includes('-v'))) && args[0] === 'check-ignore' + ? Promise.resolve({ code: 128, stdout: '', stderr: 'fatal: detected dubious ownership in repository' }) + : failLsFiles.on && args.includes('ls-files') + ? Promise.resolve({ code: 128, stdout: '', stderr: 'fatal: index file corrupt' }) + : actual.execCommand(cmd, args, opts), + }; +}); + +// Widens the read-modify-write window on the exclude file, as a slow disk or a second process would. +const slowExcludeRead = vi.hoisted(() => ({ on: false })); +vi.mock('../utils/fs.js', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + readFileSafe: async (file: string) => { + const content = await actual.readFileSafe(file); + if (slowExcludeRead.on && file.endsWith(path.join('info', 'exclude'))) await new Promise((r) => setTimeout(r, 30)); + return content; + }, + }; +}); + +import { MCP_EXCLUDE_END, MCP_EXCLUDE_START, ensureExcludedFromGit, excludeFromGit, removeMcpGitExclude } from '../mcp-git-exclude.js'; +import { acquireLock, releaseLock } from '../update.js'; +import { log } from '../utils/logger.js'; + +describe('teamai block in .git/info/exclude (#882)', () => { + let repo: string; + let excludeFile: string; + + beforeEach(async () => { + repo = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-mcp-exclude-')); + execFileSync('git', ['init', '-q'], { cwd: repo }); + excludeFile = path.join(repo, '.git', 'info', 'exclude'); + }); + + afterEach(async () => { + failCheckIgnore.on = false; + failLsFiles.on = false; + failVerboseCheckIgnore.on = false; + slowExcludeRead.on = false; + vi.mocked(log.warn).mockClear(); + await fse.remove(repo); + }); + + describe('when git cannot say whether it would commit the file', () => { + it('still excludes it while the exclude file is reachable', async () => { + await fse.writeJson(path.join(repo, '.mcp.json'), {}); + failCheckIgnore.on = true; + + await excludeFromGit(path.join(repo, '.mcp.json')); + + expect(await fse.readFile(excludeFile, 'utf8')).toMatch(/^\/\.mcp\.json$/m); + }); + + it('excludes a file git answers it does not track', async () => { + failCheckIgnore.on = true; + + expect(await ensureExcludedFromGit(path.join(repo, '.mcp.json'))).toEqual({ kind: 'excluded', added: true }); + expect(await fse.readFile(excludeFile, 'utf8')).toMatch(/^\/\.mcp\.json$/m); + }); + + it('fails for a file git tracks, and writes nothing', async () => { + const file = path.join(repo, '.mcp.json'); + await fse.writeJson(file, {}); + execFileSync('git', ['add', '.mcp.json'], { cwd: repo }); + failCheckIgnore.on = true; + + expect(await ensureExcludedFromGit(file)).toMatchObject({ kind: 'failed', reason: `git already tracks ${file}` }); + expect(await fse.pathExists(excludeFile) ? await fse.readFile(excludeFile, 'utf8') : '').not.toContain('teamai'); + }); + + it('fails with git\'s error, and writes nothing, when git cannot say whether it tracks the file either', async () => { + failCheckIgnore.on = true; + failLsFiles.on = true; + + expect(await ensureExcludedFromGit(path.join(repo, '.mcp.json'))).toMatchObject({ + kind: 'failed', + reason: expect.stringContaining('fatal: index file corrupt'), + }); + expect(await fse.pathExists(excludeFile) ? await fse.readFile(excludeFile, 'utf8') : '').not.toContain('teamai'); + }); + + it('warns with the file and git\'s error when it is not', async () => { + await fse.writeJson(path.join(repo, '.mcp.json'), {}); + await fse.writeFile(path.join(repo, '.git', 'config'), '[core\nbroken\n'); + + await excludeFromGit(path.join(repo, '.mcp.json')); + + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining(path.join(repo, '.mcp.json'))); + expect(log.warn).toHaveBeenCalledWith(expect.stringMatching(/config/)); + }); + }); + + it('keeps every pattern when several writers add to the same exclude file at once', async () => { + const files = ['a', 'b', 'c', 'd', 'e'].map((name) => path.join(repo, `${name}.json`)); + for (const file of files) await fse.writeJson(file, {}); + slowExcludeRead.on = true; + + await Promise.all(files.map((file) => excludeFromGit(file))); + + const content = await fse.readFile(excludeFile, 'utf8'); + for (const name of ['a', 'b', 'c', 'd', 'e']) expect(content).toMatch(new RegExp(`^/${name}\\.json$`, 'm')); + }); + + describe('while another command holds the exclude file\'s lock', () => { + beforeEach(async () => { + expect(await acquireLock(`${excludeFile}.teamai-lock`)).toBe(true); + }); + + afterEach(async () => { + await releaseLock(`${excludeFile}.teamai-lock`); + }); + + it('does not write, and warns that the file is not excluded yet and to pull again', async () => { + await fse.outputFile(excludeFile, 'scratch/\n'); + await fse.writeJson(path.join(repo, '.mcp.json'), {}); + + await excludeFromGit(path.join(repo, '.mcp.json')); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe('scratch/\n'); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining(path.join(repo, '.mcp.json'))); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining('teamai pull')); + }); + + it('does not remove patterns', async () => { + const content = `${MCP_EXCLUDE_START}\n/.mcp.json\n${MCP_EXCLUDE_END}\n`; + await fse.outputFile(excludeFile, content); + + expect(await removeMcpGitExclude(excludeFile, ['/.mcp.json'])).toBe('locked'); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(content); + }); + }); + + it('removes only the patterns asked for, and the block with its last one', async () => { + await fse.outputFile(excludeFile, `mine/\n${MCP_EXCLUDE_START}\n/a.json\n/b.json\n${MCP_EXCLUDE_END}\n`); + + expect(await removeMcpGitExclude(excludeFile, ['/a.json'])).toBe('written'); + expect(await fse.readFile(excludeFile, 'utf8')).toBe(`mine/\n${MCP_EXCLUDE_START}\n/b.json\n${MCP_EXCLUDE_END}\n`); + + expect(await removeMcpGitExclude(excludeFile, ['/b.json'])).toBe('written'); + expect(await fse.readFile(excludeFile, 'utf8')).toBe('mine/\n'); + }); + + describe('for a file git already tracks', () => { + beforeEach(async () => { + await fse.writeJson(path.join(repo, '.mcp.json'), {}); + execFileSync('git', ['add', '.mcp.json'], { cwd: repo }); + }); + + it('says so on a dry run before any pull has listed it, and writes nothing', async () => { + const file = path.join(repo, '.mcp.json'); + + expect(await ensureExcludedFromGit(file, { dryRun: true })).toEqual({ + kind: 'failed', + reason: `git already tracks ${file}`, + fix: `Run \`git rm --cached ${file}\` (rotate any value a commit of it holds), then \`teamai pull\` again.`, + }); + expect(await fse.pathExists(excludeFile) ? await fse.readFile(excludeFile, 'utf8') : '').not.toContain('teamai'); + }); + + it.skipIf(process.getuid?.() === 0).each([ + ['a pull', {}], + ['a dry run', { dryRun: true }], + ])('names the tracked file first on %s when .git/info is not writable either', async (_label, options) => { + const info = path.join(repo, '.git', 'info'); + await fse.chmod(info, 0o555); + + try { + const exclusion = await ensureExcludedFromGit(path.join(repo, '.mcp.json'), options); + expect(exclusion).toMatchObject({ kind: 'failed', reason: `git already tracks ${path.join(repo, '.mcp.json')}` }); + } finally { + await fse.chmod(info, 0o755); + } + }); + }); + + // A rule after teamai's line, or in a .gitignore, which git reads first, can re-include the file. + describe('for a file a rule of the member\'s re-includes', () => { + beforeEach(async () => { + await fse.writeFile(path.join(repo, '.gitignore'), 'node_modules/\n!/.mcp.json\n'); + }); + + it('names the rule, and says to remove it rather than untrack the file', async () => { + const file = path.join(repo, '.mcp.json'); + const gitignore = path.join(await fse.realpath(repo), '.gitignore'); + + expect(await ensureExcludedFromGit(file)).toEqual({ + kind: 'failed', + reason: `a rule in your git ignore files re-includes ${file}: \`!/.mcp.json\` (${gitignore}:2)`, + fix: `Remove \`!/.mcp.json\` from ${gitignore}, then run \`teamai pull\` again.`, + }); + }); + + it('names the rule on a dry run too, before any line is written', async () => { + const file = path.join(repo, '.mcp.json'); + const gitignore = path.join(await fse.realpath(repo), '.gitignore'); + + expect(await ensureExcludedFromGit(file, { dryRun: true })).toEqual({ + kind: 'failed', + reason: `a rule in your git ignore files re-includes ${file}: \`!/.mcp.json\` (${gitignore}:2)`, + fix: `Remove \`!/.mcp.json\` from ${gitignore}, then run \`teamai pull\` again.`, + }); + expect(await fse.readFile(path.join(repo, '.git', 'info', 'exclude'), 'utf-8').catch(() => '')).not.toContain('teamai'); + }); + + it('says so when git cannot name the rule', async () => { + failVerboseCheckIgnore.on = true; + const file = path.join(repo, '.mcp.json'); + + expect(await ensureExcludedFromGit(file)).toEqual({ + kind: 'failed', + reason: `a rule in your git ignore files re-includes ${file}`, + fix: 'Remove the rule in .gitignore, .git/info/exclude or core.excludesFile that re-includes it (`git check-ignore -v` names it), then run `teamai pull` again.', + }); + }); + }); + + it('stays quiet outside any repository', async () => { + const outside = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-no-repo-')); + await fse.writeJson(path.join(outside, '.mcp.json'), {}); + + await excludeFromGit(path.join(outside, '.mcp.json')); + + expect(log.warn).not.toHaveBeenCalled(); + await fse.remove(outside); + }); + + it('never takes the member\'s lines when a start marker has lost its end marker', async () => { + await fse.writeFile(excludeFile, `${MCP_EXCLUDE_START}\n/old.json\nscratch/\n`); + await fse.writeJson(path.join(repo, '.mcp.json'), {}); + + await excludeFromGit(path.join(repo, '.mcp.json')); + expect(await removeMcpGitExclude(excludeFile, ['/.mcp.json'])).toBe('written'); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(`${MCP_EXCLUDE_START}\n/old.json\nscratch/\n`); + }); + // The appliers replace the file itself (tmp + rename) but follow its directories (#886). + describe('for a file under a symlinked directory, judged where the write lands', () => { + let real: string; + const commit = (...files: string[]): void => { + execFileSync('git', ['add', ...files], { cwd: repo }); + execFileSync('git', ['-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '-m', 'config'], { cwd: repo }); + }; + + beforeEach(async () => { + real = await fse.realpath(repo); + await fse.outputFile(path.join(repo, 'config', 'README.md'), 'cursor config\n'); + await fse.symlink('config', path.join(repo, '.cursor'), 'dir'); + }); + + it('fails for a file git tracks there, naming both paths and the one to untrack', async () => { + await fse.writeJson(path.join(repo, 'config', 'mcp.json'), {}); + commit('config', '.cursor'); + const file = path.join(repo, '.cursor', 'mcp.json'); + const landed = path.join(real, 'config', 'mcp.json'); + + expect(await ensureExcludedFromGit(file)).toEqual({ + kind: 'failed', + reason: `git already tracks ${landed} (where ${file} is written)`, + fix: `Run \`git rm --cached ${landed}\` (rotate any value a commit of it holds), then \`teamai pull\` again.`, + }); + expect(await fse.pathExists(excludeFile) ? await fse.readFile(excludeFile, 'utf8') : '').not.toContain('teamai'); + }); + + it('lists the file it lands in, in a tracked directory', async () => { + commit('config', '.cursor'); + + expect(await ensureExcludedFromGit(path.join(repo, '.cursor', 'mcp.json'))).toEqual({ kind: 'excluded', added: true }); + await fse.writeJson(path.join(repo, 'config', 'mcp.json'), {}); + + expect(await fse.readFile(excludeFile, 'utf8')).toMatch(/^\/config\/mcp\.json$/m); + expect(await fse.readFile(excludeFile, 'utf8')).not.toContain('/.cursor/'); + expect(execFileSync('git', ['status', '--porcelain', '--untracked-files=all'], { cwd: repo, encoding: 'utf8' })).not.toContain('config/mcp.json'); + }); + + it('lists a file whose directory does not exist yet under the one it will be created in', async () => { + expect(await ensureExcludedFromGit(path.join(repo, '.cursor', 'sub', 'mcp.json'))).toEqual({ kind: 'excluded', added: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toMatch(/^\/config\/sub\/mcp\.json$/m); + }); + + it('protects it in the repository the directory links into, not this one', async () => { + const other = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-other-repo-')); + execFileSync('git', ['init', '-q'], { cwd: other }); + await fse.ensureDir(path.join(other, 'cfg')); + await fse.symlink(path.join(other, 'cfg'), path.join(repo, '.tool'), 'dir'); + + try { + expect(await ensureExcludedFromGit(path.join(repo, '.tool', 'mcp.json'))).toEqual({ kind: 'excluded', added: true }); + expect(await fse.readFile(path.join(other, '.git', 'info', 'exclude'), 'utf8')).toMatch(/^\/cfg\/mcp\.json$/m); + expect(await fse.pathExists(excludeFile) ? await fse.readFile(excludeFile, 'utf8') : '').not.toContain('teamai'); + } finally { + await fse.remove(other); + } + }); + + it('lists nothing and stays quiet when the directory links outside any repository', async () => { + const outside = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-no-repo-')); + await fse.symlink(outside, path.join(repo, '.tool'), 'dir'); + await fse.writeJson(path.join(outside, 'mcp.json'), {}); + + try { + expect(await ensureExcludedFromGit(path.join(repo, '.tool', 'mcp.json'))).toEqual({ kind: 'excluded', added: false }); + await excludeFromGit(path.join(repo, '.tool', 'mcp.json')); + expect(log.warn).not.toHaveBeenCalled(); + expect(await fse.pathExists(excludeFile) ? await fse.readFile(excludeFile, 'utf8') : '').not.toContain('teamai'); + } finally { + await fse.remove(outside); + } + }); + + it('judges a directory that links nowhere from the closest one that exists: no write lands through it', async () => { + await fse.symlink('missing', path.join(repo, '.dangling'), 'dir'); + + expect(await ensureExcludedFromGit(path.join(repo, '.dangling', 'mcp.json'))).toEqual({ kind: 'excluded', added: true }); + expect(await fse.readFile(excludeFile, 'utf8')).toMatch(/^\/\.dangling\/mcp\.json$/m); + await expect(fse.ensureDir(path.join(repo, '.dangling'))).rejects.toThrow(); + }); + + it('judges a symlink at the file itself as the file: the write replaces it', async () => { + await fse.writeJson(path.join(repo, 'config', 'mcp.json'), {}); + commit('config'); + await fse.symlink(path.join('config', 'mcp.json'), path.join(repo, '.mcp.json')); + + expect(await ensureExcludedFromGit(path.join(repo, '.mcp.json'))).toEqual({ kind: 'excluded', added: true }); + expect(await fse.readFile(excludeFile, 'utf8')).toMatch(/^\/\.mcp\.json$/m); + expect(await fse.readFile(excludeFile, 'utf8')).not.toContain('/config/'); + }); + }); +}); diff --git a/src/__tests__/mcp-reconcile.test.ts b/src/__tests__/mcp-reconcile.test.ts index 9ad27eb4f..38964d1ee 100644 --- a/src/__tests__/mcp-reconcile.test.ts +++ b/src/__tests__/mcp-reconcile.test.ts @@ -1,7 +1,9 @@ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import fs from 'node:fs'; import path from 'node:path'; import os from 'node:os'; import fse from 'fs-extra'; +import { execFileSync } from 'node:child_process'; vi.mock('../utils/logger.js', () => ({ log: { @@ -20,7 +22,29 @@ vi.mock('../utils/logger.js', () => ({ })), })); -import { reconcileMcpForConfig, resolveMcpTargets, spliceCodexBlock, codexServerNames } from '../mcp-reconcile.js'; +// What .git/info/exclude held at the moment each JSON config was written (#882). +const excludeAtWrite = vi.hoisted(() => new Map()); +// Runs just before each JSON config write, as a concurrent command would. +const beforeJsonWrite = vi.hoisted(() => ({ run: null as null | ((file: string) => Promise) })); +vi.mock('../utils/fs.js', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + writeJsonAtomic: async (...args: Parameters) => { + const [file] = args; + await beforeJsonWrite.run?.(String(file)); + const gitDir = path.join(path.dirname(String(file)), '.git'); + if (await fse.pathExists(gitDir)) { + excludeAtWrite.set(String(file), await actual.readFileSafe(path.join(gitDir, 'info', 'exclude'))); + } + return actual.writeJsonAtomic(...args); + }, + }; +}); + +import { reconcileMcpForConfig, releaseCleanMcpGitExcludes, resolveMcpTargets, spliceCodexBlock, codexServerNames, writeCodexAtomic } from '../mcp-reconcile.js'; +import { acquireLock, releaseLock } from '../update.js'; +import { log } from '../utils/logger.js'; import { resetWarnOnce } from '../utils/warn-once.js'; import { TeamaiConfigSchema, type TeamaiConfig, type LocalConfig } from '../types.js'; @@ -31,6 +55,9 @@ const TOOL_PATHS = { codex: { skills: '.codex/skills', settings: '.codex/hooks.json', mcp: '.codex/config.toml' }, tclaude: { skills: '.tclaude/skills', settings: '.tclaude/settings.json', mcp: '.tclaude/.claude.json' }, }; +// CodeBuddy at its built-in .mcp.json, beside Claude. TOOL_PATHS moves it, which makes .mcp.json a location of +// CodeBuddy's that no record of it describes: any server there that Claude's records don't own holds a line (#882). +const UNMOVED_TOOL_PATHS = { ...TOOL_PATHS, codebuddy: { ...TOOL_PATHS.codebuddy, mcpProject: '.mcp.json' } }; describe('MCP reconcile', () => { let tmpDir: string; @@ -864,6 +891,1995 @@ servers: delete process.env.SECRET_TOKEN; }); + describe('project MCP configs holding a resolved value stay out of git (#882)', () => { + let projectRoot: string; + let projectConfig: LocalConfig; + const git = (cwd: string, ...args: string[]): string => + execFileSync('git', args, { cwd, encoding: 'utf-8' }); + const excludeOf = (root: string): Promise => + fse.readFile(path.join(root, '.git', 'info', 'exclude'), 'utf-8'); + const withSecret = ` +servers: + - name: with-secret + transport: http + url: https://example.com/mcp + headers: + Authorization: Bearer \${SECRET_TOKEN} +`; + + beforeEach(async () => { + projectRoot = path.join(tmpDir, 'business-repo'); + for (const d of ['.claude', '.cursor']) await fse.ensureDir(path.join(projectRoot, d, 'skills')); + git(projectRoot, 'init', '-q'); + projectConfig = { ...localConfig, scope: 'project', projectRoot } as unknown as LocalConfig; + vi.stubEnv('SECRET_TOKEN', 'super-secret-value'); + }); + const unmovedConfig = (): TeamaiConfig => ({ ...teamConfig, toolPaths: UNMOVED_TOOL_PATHS } as TeamaiConfig); + // A worktree an earlier pull ran in, holding nothing of teamai's. With no managed-mcp.json at all, + // any server in a config may be one teamai wrote, and the pull notes it (#882). + const pulledBefore = async (): Promise => { + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.outputJson(managedMcpManifestPath(getDataHome(projectConfig), projectRoot), { 'claude:project': [], 'cursor:project': [] }); + }; + + it('lists again the config of a tool whose record alone is lost, while it holds a server no record claims', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, projectConfig); + expect(await fse.readFile(path.join(projectRoot, '.cursor', 'mcp.json'), 'utf-8')).toContain('super-secret-value'); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record; + delete manifest['cursor:project']; + await fse.writeJson(manifestFile, manifest); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await fse.readFile(path.join(projectRoot, '.cursor', 'mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + it('lists again the config an uninstalled tool left, its record lost, while it holds a server no record claims', async () => { + // OpenCode's config sits outside its root: uninstalled (.opencode gone), opencode.json stays. + const withOpencode = { ...teamConfig, toolPaths: { ...TOOL_PATHS, opencode: { skills: '.opencode/skills', mcp: '.config/opencode/opencode.json', mcpProject: 'opencode.json' } } } as TeamaiConfig; + await fse.ensureDir(path.join(projectRoot, '.opencode', 'skills')); + // Claude's record stays: only OpenCode's is lost. + await writeMcpYaml(`${withSecret} tools: [opencode]\n - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n`); + await reconcileMcpForConfig(withOpencode, projectConfig); + const opencodeFile = path.join(projectRoot, 'opencode.json'); + expect(await fse.readFile(opencodeFile, 'utf-8')).toContain('super-secret-value'); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record; + delete manifest['opencode:project']; + await fse.writeJson(manifestFile, manifest); + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + await fse.remove(resolvedMcpFilesPath(projectConfig) ?? ''); + await fse.remove(path.join(projectRoot, '.opencode')); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(withOpencode, projectConfig); + + expect(await fse.readFile(opencodeFile, 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/opencode\.json$/m); + }); + + it('keeps suspect a tool managed-mcp-files.json lists as a writer, uninstalled since, though an installed tool maps the file', async () => { + const unmoved = { ...teamConfig, toolPaths: UNMOVED_TOOL_PATHS } as TeamaiConfig; + await fse.ensureDir(path.join(projectRoot, '.codebuddy', 'skills')); + await writeMcpYaml(`${withSecret} tools: [codebuddy]\n - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n`); + await reconcileMcpForConfig(unmoved, projectConfig); + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record; + delete manifest['codebuddy:project']; + await fse.writeJson(manifestFile, manifest); + await fse.remove(path.join(projectRoot, '.codebuddy')); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n'); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(unmoved, projectConfig); + + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('never lets one format\'s record claim a server of the same name under another format\'s key', async () => { + const toolPaths = { + ...UNMOVED_TOOL_PATHS, + cursor: { ...TOOL_PATHS.cursor, mcpProject: '.mcp.json' }, + opencode: { skills: '.opencode/skills', mcp: '.config/opencode/opencode.json', mcpProject: '.mcp.json' }, + }; + const shared = { ...teamConfig, toolPaths } as TeamaiConfig; + await fse.ensureDir(path.join(projectRoot, '.opencode', 'skills')); + // Cursor owns x under mcpServers, now a literal; OpenCode's x under mcp still holds a token, its record lost. + await writeMcpYaml('servers:\n - name: x\n transport: http\n url: https://example.com/x\n tools: [cursor]\n'); + await reconcileMcpForConfig(shared, projectConfig); + const doc = await fse.readJson(path.join(projectRoot, '.mcp.json')) as Record; + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { + ...doc, mcp: { x: { type: 'remote', url: 'https://example.com/x', headers: { Authorization: 'Bearer stale-token-value' } } }, + }); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + + await reconcileMcpForConfig(shared, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('notes the unclaimed servers under each format of a file tools of different formats share', async () => { + const toolPaths = { + ...UNMOVED_TOOL_PATHS, + cursor: { ...TOOL_PATHS.cursor, mcpProject: '.mcp.json' }, + opencode: { skills: '.opencode/skills', mcp: '.config/opencode/opencode.json', mcpProject: '.mcp.json' }, + }; + const shared = { ...teamConfig, toolPaths } as TeamaiConfig; + await fse.ensureDir(path.join(projectRoot, '.opencode', 'skills')); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { + mcpServers: { 'stale-cursor': { type: 'http', url: 'https://a.example/mcp' } }, + mcp: { 'stale-opencode': { type: 'remote', url: 'https://b.example/mcp' } }, + }); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n'); + + await reconcileMcpForConfig(shared, projectConfig); + + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + const unverified = (await readResolvedMcpFiles(projectConfig)).files[path.join(projectRoot, '.mcp.json')]?.unverified ?? []; + expect(unverified).toEqual(expect.arrayContaining(['stale-cursor', 'stale-opencode'])); + }); + + it('keeps the line of a file tools of different formats share while a stale entry sits under any of their keys', async () => { + // Cursor (mcpServers) and OpenCode (mcp) both on .mcp.json, OpenCode last: judged in one format, the other hides. + const toolPaths = { + ...UNMOVED_TOOL_PATHS, + cursor: { ...TOOL_PATHS.cursor, mcpProject: '.mcp.json' }, + opencode: { skills: '.opencode/skills', mcp: '.config/opencode/opencode.json', mcpProject: '.mcp.json' }, + }; + const shared = { ...teamConfig, toolPaths } as TeamaiConfig; + await writeMcpYaml(`${withSecret} tools: [cursor]\n - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n`); + await reconcileMcpForConfig(shared, projectConfig); + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n'); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(shared, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + describe('a config two tools share (Claude and CodeBuddy on .mcp.json)', () => { + const shared = { ...teamConfig, toolPaths: { ...TOOL_PATHS, codebuddy: { ...TOOL_PATHS.codebuddy, mcpProject: '.mcp.json' } } } as TeamaiConfig; + const open = ' - name: open\n transport: http\n url: https://example.com/open\n'; + + it('keeps its line while a tool that wrote a resolved value there has lost its record, though the other tool\'s is intact', async () => { + await fse.ensureDir(path.join(projectRoot, '.codebuddy', 'skills')); + await writeMcpYaml(`${withSecret} tools: [codebuddy]\n${open} tools: [claude]\n`); + await reconcileMcpForConfig(shared, projectConfig); + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record; + delete manifest['codebuddy:project']; + await fse.writeJson(manifestFile, manifest); + await writeMcpYaml(`servers:\n${open} tools: [claude]\n`); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(shared, projectConfig); + + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('releases its line once clean when only one of them ever wrote a resolved value there', async () => { + await writeMcpYaml(`${withSecret} tools: [claude]\n${open}`); + await reconcileMcpForConfig(shared, projectConfig); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + await writeMcpYaml(`servers:\n${open}`); + + await reconcileMcpForConfig(shared, projectConfig); + + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).not.toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).not.toMatch(/^\/\.mcp\.json$/m); + }); + + describe('without managed-mcp-files.json, as an install from before it has none', () => { + const withoutSidecar = async (): Promise => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + await fse.remove(resolvedMcpFilesPath(projectConfig) ?? ''); + }; + + it('keeps its line while a tool that wrote a resolved value there has lost its record, though the other tool\'s is intact', async () => { + await fse.ensureDir(path.join(projectRoot, '.codebuddy', 'skills')); + await writeMcpYaml(`${withSecret} tools: [codebuddy]\n${open} tools: [claude]\n`); + await reconcileMcpForConfig(shared, projectConfig); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record; + delete manifest['codebuddy:project']; + await fse.writeJson(manifestFile, manifest); + await withoutSidecar(); + await writeMcpYaml(`servers:\n${open} tools: [claude]\n`); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(shared, projectConfig); + + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + // Nothing says which of the two wrote there, so a tool mapping it with no record at all holds it too. + it('keeps its line while a tool mapping it has no record, until a pull records the file', async () => { + await writeMcpYaml(`${withSecret} tools: [claude]\n${open}`); + await reconcileMcpForConfig(shared, projectConfig); + await withoutSidecar(); + await writeMcpYaml(`servers:\n${open}`); + + await reconcileMcpForConfig(shared, projectConfig); + + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).not.toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + }); + }); + + it('adds every such config to .git/info/exclude once, inside a teamai block', async () => { + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig, { force: true }); + + const exclude = await excludeOf(projectRoot); + expect(exclude.match(/^\/\.mcp\.json$/gm)).toHaveLength(1); + expect(exclude.match(/^\/\.cursor\/mcp\.json$/gm)).toHaveLength(1); + expect(exclude).toContain('# [teamai:mcp-exclude:start]'); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/ (\.mcp\.json|\.cursor\/)/); + expect(await fse.pathExists(path.join(projectRoot, '.gitignore'))).toBe(false); + }); + + it('adds nothing for a config that carries no resolved value', async () => { + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }); + + it('adds nothing for a path git already ignores, and leaves .gitignore as it is', async () => { + await fse.writeFile(path.join(projectRoot, '.gitignore'), '.mcp.json\n.cursor/\n'); + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + expect(await fse.readFile(path.join(projectRoot, '.gitignore'), 'utf-8')).toBe('.mcp.json\n.cursor/\n'); + }); + + it('writes to the repository git dir from a linked worktree', async () => { + git(projectRoot, '-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '--allow-empty', '-m', 'init'); + const worktree = path.join(tmpDir, 'business-wt'); + git(projectRoot, 'worktree', 'add', '-q', worktree); + for (const d of ['.claude', '.cursor']) await fse.ensureDir(path.join(worktree, d, 'skills')); + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(teamConfig, { ...projectConfig, projectRoot: worktree } as LocalConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect(git(worktree, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/ (\.mcp\.json|\.cursor\/)/); + }); + + describe('a config an earlier pull wrote is protected even when this pull delivers nothing to it', () => { + beforeEach(async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, projectConfig); + // As if written before this release: the token is on disk, nothing excludes it. + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + }); + + it('when its tool is disabled', async () => { + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + it('when the team turned automatic MCP delivery off', async () => { + const manual = { ...teamConfig, sharing: { ...teamConfig.sharing, mcp: { autoApply: false } } } as TeamaiConfig; + + await reconcileMcpForConfig(manual, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + it('when its tool is no longer detected', async () => { + await fse.remove(path.join(projectRoot, '.claude')); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it.skipIf(process.getuid?.() === 0)('when writing another tool\'s config fails', async () => { + await writeMcpYaml(`${withSecret} - name: added-later\n transport: http\n url: https://example.com/later\n`); + await fse.chmod(path.join(projectRoot, '.cursor'), 0o555); + + await expect(reconcileMcpForConfig(teamConfig, projectConfig)).rejects.toThrow(); + await fse.chmod(path.join(projectRoot, '.cursor'), 0o755); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + it('when its ownership manifest is gone', async () => { + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.remove(managedMcpManifestPath(getDataHome(projectConfig), projectRoot)); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('when its server has left mcp.yaml and its tool is disabled', async () => { + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + // Claude's copy was cleaned by this pull, so nothing of teamai's is left to protect there. + expect(await excludeOf(projectRoot)).not.toMatch(/^\/\.mcp\.json$/m); + }); + + it.each([ + ['drops the tool', { claude: TOOL_PATHS.claude }], + ['moves its project MCP file', { ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '.cursor/team-mcp.json' } }], + ])('when the team %s', async (_label, toolPaths) => { + await reconcileMcpForConfig({ ...teamConfig, toolPaths } as TeamaiConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + it('when the team\'s mcp.yaml does not parse', async () => { + await writeMcpYaml('servers: [unclosed\n'); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + describe('when its server\'s ${VAR} has since become a literal and the variable is gone', () => { + beforeEach(async () => { + await writeMcpYaml(withSecret.replace('${SECRET_TOKEN}', 'published-literal')); + vi.stubEnv('SECRET_TOKEN', ''); + }); + + it('and its tool is disabled', async () => { + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await fse.readFile(path.join(projectRoot, '.cursor', 'mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + // Claude's copy now holds the literal: nothing resolved is left there. + expect(await excludeOf(projectRoot)).not.toMatch(/^\/\.mcp\.json$/m); + }); + + it('and the team turned automatic MCP delivery off', async () => { + const manual = { ...teamConfig, sharing: { ...teamConfig.sharing, mcp: { autoApply: false } } } as TeamaiConfig; + + await reconcileMcpForConfig(manual, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('and its tool is disabled, recorded by an older teamai that did not note resolved values', async () => { + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record>>; + await fse.writeJson(manifestFile, Object.fromEntries(Object.entries(manifest).map(([key, records]) => + [key, records.map(({ name, hash }) => ({ name, hash }))]))); + + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + }); + }); + + describe('a config an earlier pull wrote under a custom mcpProject the team has since changed', () => { + const custom = { ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '.cursor/team-mcp.json' } }; + const customFile = (): string => path.join(projectRoot, '.cursor', 'team-mcp.json'); + const sidecar = async (): Promise> => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + return (await readResolvedMcpFiles(projectConfig)).files; + }; + + beforeEach(async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig({ ...teamConfig, toolPaths: custom } as TeamaiConfig, projectConfig); + expect(await fse.readFile(customFile(), 'utf-8')).toContain('super-secret-value'); + }); + + it.each([ + ['restores the built-in path', TOOL_PATHS], + ['drops the tool', { claude: TOOL_PATHS.claude }], + ['moves it again', { ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '.cursor/other-mcp.json' } }], + ])('is listed again when the team %s', async (_label, toolPaths) => { + // As if written before this release, or listed and since dropped: nothing excludes it. + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + + await reconcileMcpForConfig({ ...teamConfig, toolPaths } as TeamaiConfig, projectConfig); + + expect(await fse.readFile(customFile(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/team-mcp\.json/); + }); + + it('keeps its line while it holds a server of the member\'s own', async () => { + await fse.writeJson(customFile(), { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + }); + + it.each([ + ['it is deleted', () => fse.remove(customFile())], + ['it holds no server', () => fse.writeJson(customFile(), { mcpServers: {} })], + ])('lets its line go, and forgets it, once %s', async (_label, arrange) => { + await arrange(); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/team-mcp\.json/); + expect(Object.keys(await sidecar())).not.toContain(customFile()); + }); + + it('lets its line go when `teamai mcp remove` finds it holding no server', async () => { + await fse.writeJson(customFile(), { mcpServers: {} }); + + await releaseCleanMcpGitExcludes(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/team-mcp\.json/); + expect(Object.keys(await sidecar())).not.toContain(customFile()); + }); + }); + + it('keeps excluding a config whose entry a pull kept for a missing declared secret', async () => { + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: SECRET_TOKEN\n'); + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, projectConfig); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + vi.stubEnv('SECRET_TOKEN', undefined); + await writeMcpYaml(`${withSecret} - name: open\n transport: http\n url: https://example.com/open\n`); + + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('https://example.com/open'); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const manifest = await fse.readJson(managedMcpManifestPath(getDataHome(projectConfig), projectRoot)); + expect(manifest['claude:project']).toEqual(expect.arrayContaining([expect.objectContaining({ name: 'with-secret', resolved: true })])); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + it('adds nothing for a disabled tool\'s config whose server never held a resolved value, after its definition changed', async () => { + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + await reconcileMcpForConfig(teamConfig, projectConfig); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/v2\n'); + + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await fse.readFile(path.join(projectRoot, '.cursor', 'mcp.json'), 'utf-8')).toContain('https://example.com/open'); + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }); + + it('keeps listing a Codex project config after its server\'s ${VAR} became a literal and Codex was disabled', async () => { + const withCodex = { ...teamConfig, toolPaths: { ...TOOL_PATHS, codex: { ...TOOL_PATHS.codex, mcpProject: '.codex/config.toml' } } } as TeamaiConfig; + await fse.ensureDir(path.join(projectRoot, '.codex', 'skills')); + await writeMcpYaml(`${withSecret} tools: [codex]\n`); + await reconcileMcpForConfig(withCodex, projectConfig); + expect(await fse.readFile(path.join(projectRoot, '.codex', 'config.toml'), 'utf-8')).toContain('super-secret-value'); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + await writeMcpYaml(`${withSecret.replace('${SECRET_TOKEN}', 'published-literal')} tools: [codex]\n`); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(withCodex, { ...projectConfig, disabledAgents: ['codex'] } as LocalConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.codex\/config\.toml$/m); + }); + + it('writes a symlinked Codex project config at its own path, never into the tracked file it links to', async () => { + const withCodex = { ...teamConfig, toolPaths: { ...TOOL_PATHS, codex: { ...TOOL_PATHS.codex, mcpProject: '.codex/config.toml' } } } as TeamaiConfig; + const tracked = path.join(projectRoot, 'config', 'codex.toml'); + const link = path.join(projectRoot, '.codex', 'config.toml'); + await fse.outputFile(tracked, 'model = "gpt-5"\n'); + git(projectRoot, 'add', 'config/codex.toml'); + git(projectRoot, '-c', 'user.name=t', '-c', 'user.email=t@t', '-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'codex'); + await fse.ensureDir(path.join(projectRoot, '.codex', 'skills')); + await fse.symlink(path.join('..', 'config', 'codex.toml'), link); + await writeMcpYaml(`${withSecret} tools: [codex]\n`); + + await reconcileMcpForConfig(withCodex, projectConfig); + + expect(await fse.readFile(tracked, 'utf-8')).toBe('model = "gpt-5"\n'); + expect((await fse.lstat(link)).isSymbolicLink()).toBe(false); + expect(await fse.readFile(link, 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.codex\/config\.toml$/m); + }); + + describe('a config an older teamai wrote under a mapping an earlier teamai.yaml made', () => { + const custom = { ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '.cursor/team-mcp.json' } }; + const customFile = (): string => path.join(projectRoot, '.cursor', 'team-mcp.json'); + const commitTeamYaml = async (toolPaths: object, cwd = repoPath): Promise => { + // JSON is YAML. + await fse.writeFile(path.join(cwd, 'teamai.yaml'), JSON.stringify({ team: 't', toolPaths })); + git(cwd, 'add', '-A'); + git(cwd, '-c', 'user.name=t', '-c', 'user.email=t@t', '-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'toolPaths'); + }; + const ledger = async (cfg = projectConfig): Promise<{ files: Record; earlierMappingsRead?: true }> => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + return readResolvedMcpFiles(cfg); + }; + // What a teamai from before managed-mcp-files.json leaves: no record of the path, nothing in the exclude. + const asOlderTeamai = async (): Promise => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.remove(resolvedMcpFilesPath(projectConfig) ?? ''); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record>>; + await fse.writeJson(manifestFile, Object.fromEntries(Object.entries(manifest).map(([key, records]) => + [key, records.map(({ name, hash }) => ({ name, hash }))]))); + }; + + beforeEach(async () => { + await writeMcpYaml(withSecret); + git(repoPath, 'init', '-q'); + await commitTeamYaml(custom); + await reconcileMcpForConfig({ ...teamConfig, toolPaths: custom } as TeamaiConfig, projectConfig); + expect(await fse.readFile(customFile(), 'utf-8')).toContain('super-secret-value'); + await commitTeamYaml(TOOL_PATHS); + await asOlderTeamai(); + }); + + it('is listed and recorded by the first pull on this version', async () => { + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/team-mcp\.json/); + expect((await ledger()).files[customFile()]).toEqual({ tools: ['cursor'] }); + }); + + it('keeps its line on the pulls after, from the record', async () => { + await reconcileMcpForConfig(teamConfig, projectConfig); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + }); + + it('reads the team repo\'s history once per worktree', async () => { + await reconcileMcpForConfig(teamConfig, projectConfig); + expect((await ledger()).earlierMappingsRead).toBe(true); + const { updateResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + await updateResolvedMcpFiles(projectConfig, (files) => delete files[customFile()]); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/team-mcp\.json/); + }); + + it.each([ + ['removed its server', async () => { + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + }], + ['renamed its server, whose variable is no longer set', async () => { + await writeMcpYaml(withSecret.replace('with-secret', 'renamed')); + vi.stubEnv('SECRET_TOKEN', ''); + }], + ])('is listed and recorded when the team also %s', async (_label, arrange) => { + await arrange(); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(customFile(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + expect((await ledger()).files[customFile()]).toEqual({ tools: ['cursor'] }); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + }); + + // No record describes that path any more, so a server of the member's own cannot be told from an older teamai's. + it('lists and records a file holding only a server of the member\'s own', async () => { + await fse.writeJson(customFile(), { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + expect((await ledger()).files[customFile()]).toEqual({ tools: ['cursor'] }); + }); + + it('lists nothing for a file git tracks, and records it as tracked: an exclude line does nothing for it', async () => { + await fse.writeJson(customFile(), { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + git(projectRoot, 'add', '-f', '.cursor/team-mcp.json'); + git(projectRoot, '-c', 'user.name=t', '-c', 'user.email=t@t', '-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'mine'); + vi.mocked(log.warn).mockClear(); + + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/team-mcp\.json/); + expect((await ledger()).files[customFile()]).toEqual({ tools: ['cursor'], tracked: true }); + expect(vi.mocked(log.warn).mock.calls.flat().join('\n')).not.toMatch(/team-mcp\.json/); + expect((await ledger()).earlierMappingsRead).toBe(true); + }); + + describe('once git tracks it', () => { + const commitIt = (): void => { + git(projectRoot, 'add', '-f', '.cursor/team-mcp.json'); + git(projectRoot, '-c', 'user.name=t', '-c', 'user.email=t@t', '-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'old'); + }; + + it('is listed and recorded as any other on the first pull after the member stops git tracking it', async () => { + commitIt(); + await reconcileMcpForConfig(teamConfig, projectConfig); + git(projectRoot, 'rm', '-q', '--cached', '.cursor/team-mcp.json'); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(customFile(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\?\? .*team-mcp\.json/); + expect((await ledger()).files[customFile()]).toEqual({ tools: ['cursor'] }); + }); + + // A checkout brings back what git holds, so only a file gone from both is forgotten. + it('keeps its record while git tracks it, whatever the file holds, and forgets it once it is gone from git and disk', async () => { + commitIt(); + await reconcileMcpForConfig(teamConfig, projectConfig); + await fse.writeJson(customFile(), { mcpServers: {} }); + await reconcileMcpForConfig(teamConfig, projectConfig); + await fse.remove(customFile()); + await reconcileMcpForConfig(teamConfig, projectConfig); + expect((await ledger()).files[customFile()]).toEqual({ tools: ['cursor'], tracked: true }); + git(projectRoot, 'rm', '-q', '--cached', '.cursor/team-mcp.json'); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(Object.keys((await ledger()).files)).not.toContain(customFile()); + expect(await excludeOf(projectRoot)).not.toMatch(/team-mcp\.json/); + }); + }); + + it('leaves a file that holds no server alone', async () => { + await fse.writeJson(customFile(), { mcpServers: {} }); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/team-mcp\.json/); + expect(Object.keys((await ledger()).files)).not.toContain(customFile()); + }); + + it('also finds one under a built-in default teamai has since changed (CodeBuddy\'s .codebuddy/mcp.json)', async () => { + const today = { ...TOOL_PATHS, codebuddy: { ...TOOL_PATHS.codebuddy, mcpProject: '.mcp.json' } }; + await fse.remove(path.join(repoPath, '.git')); + git(repoPath, 'init', '-q'); + await commitTeamYaml(today); + const oldDefault = path.join(projectRoot, '.codebuddy', 'mcp.json'); + await fse.outputFile(oldDefault, await fse.readFile(customFile(), 'utf-8')); + + await reconcileMcpForConfig({ ...teamConfig, toolPaths: today } as TeamaiConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.codebuddy\/mcp\.json$/m); + expect((await ledger()).files[oldDefault]).toEqual({ tools: ['codebuddy'] }); + }); + + it('leaves a mapped path outside the project root alone', async () => { + const outside = path.join(tmpDir, 'outside', 'mcp.json'); + await fse.outputFile(outside, await fse.readFile(customFile(), 'utf-8')); + await commitTeamYaml({ ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '../outside/mcp.json' } }); + await commitTeamYaml(TOOL_PATHS); + await fse.remove(customFile()); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(Object.keys((await ledger()).files)).not.toContain(outside); + expect(await excludeOf(projectRoot)).not.toMatch(/outside/); + }); + + it.each([ + ['the team repo is a shallow clone without that revision', true, async (): Promise => { + const shallow = path.join(tmpDir, 'team-shallow'); + execFileSync('git', ['clone', '-q', '--depth', '1', `file://${repoPath}`, shallow]); + return { ...projectConfig, repo: { ...projectConfig.repo, localPath: shallow } } as LocalConfig; + }], + ['teamai.yaml was never committed', true, async (): Promise => { + await fse.remove(path.join(repoPath, '.git')); + git(repoPath, 'init', '-q'); + git(repoPath, 'add', 'mcp'); + git(repoPath, '-c', 'user.name=t', '-c', 'user.email=t@t', '-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'mcp'); + return projectConfig; + }], + ['the team repo has no commits', false, async (): Promise => { + await fse.remove(path.join(repoPath, '.git')); + git(repoPath, 'init', '-q'); + return projectConfig; + }], + ['the team repo is not a git repository', false, async (): Promise => { + await fse.remove(path.join(repoPath, '.git')); + return projectConfig; + }], + ['git fails reading it', false, async (): Promise => { + await fse.emptyDir(path.join(repoPath, '.git', 'objects')); + return projectConfig; + }], + ])('protects as before when %s', async (_label, read, arrange) => { + const cfg = await arrange(); + + await expect(reconcileMcpForConfig(teamConfig, cfg)).resolves.toBeDefined(); + + expect(await excludeOf(projectRoot)).not.toMatch(/team-mcp\.json/); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + // Read as far as git could: a failure is tried again on the next pull. + expect((await ledger(cfg)).earlierMappingsRead).toBe(read ? true : undefined); + }); + }); + + describe('a config written for a tool the team has since moved, that another tool\'s mapping still reaches', () => { + const shared = { ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '.mcp.json' } }; + const mcpJson = (): string => path.join(projectRoot, '.mcp.json'); + const ledger = async (): Promise> => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + return (await readResolvedMcpFiles(projectConfig)).files; + }; + const setServers = async (servers: Record): Promise => { + const doc = await fse.readJson(mcpJson()) as { mcpServers: Record }; + await fse.writeJson(mcpJson(), { mcpServers: { open: doc.mcpServers.open, ...servers } }); + }; + + beforeEach(async () => { + // Cursor's own server, with the token, lands in the file Claude maps too. + await writeMcpYaml(`${withSecret} tools: [cursor]\n - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n`); + await reconcileMcpForConfig({ ...teamConfig, toolPaths: shared } as TeamaiConfig, projectConfig); + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + // Then the team moves Cursor back to its own file and drops that server; the token is no longer set. + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n'); + vi.stubEnv('SECRET_TOKEN', ''); + }); + + it('keeps its line while the file holds that tool\'s server', async () => { + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\.mcp\.json/); + }); + + // As for any recorded file: nothing tells the member's server from one teamai wrote there for Cursor. + it('keeps it while the file holds a server of the member\'s own', async () => { + await reconcileMcpForConfig(teamConfig, projectConfig); + await setServers({ mine: { type: 'http', url: 'https://mine.example/mcp' } }); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('lets the line go, and takes that tool off the record, once only servers the tools mapping it own are left', async () => { + await reconcileMcpForConfig(teamConfig, projectConfig); + await setServers({}); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/^\/\.mcp\.json$/m); + expect(Object.keys(await ledger())).not.toContain(mcpJson()); + }); + + describe('written by an older teamai, under a mapping only an earlier teamai.yaml made', () => { + const commitTeamYaml = (toolPaths: object): void => { + // JSON is YAML. + fse.writeFileSync(path.join(repoPath, 'teamai.yaml'), JSON.stringify({ team: 't', toolPaths })); + git(repoPath, 'add', '-A'); + git(repoPath, '-c', 'user.name=t', '-c', 'user.email=t@t', '-c', 'commit.gpgsign=false', 'commit', '-q', '-m', 'toolPaths'); + }; + const sidecarState = async (): Promise<{ files: Record; earlierMappingsRead?: true }> => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + return readResolvedMcpFiles(projectConfig); + }; + + beforeEach(async () => { + git(repoPath, 'init', '-q'); + commitTeamYaml(shared); + commitTeamYaml(TOOL_PATHS); + // What a teamai from before managed-mcp-files.json leaves: no record of the path, nothing in the exclude. + // The manifest keeps its resolved notes, so only the moved tool's server can hold the line. + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + await fse.remove(resolvedMcpFilesPath(projectConfig) ?? ''); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + }); + + it('is listed, and recorded for that tool, by the first pull on this version', async () => { + await reconcileMcpForConfig(unmovedConfig(), projectConfig); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\.mcp\.json/); + expect((await sidecarState()).files[mcpJson()]).toEqual({ tools: ['cursor'] }); + }); + + it('keeps its line on the pulls after, from the record', async () => { + await reconcileMcpForConfig(teamConfig, projectConfig); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('adds that tool to the record the file already has', async () => { + const { trackResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + await trackResolvedMcpFiles(projectConfig, [{ tool: 'claude', file: mcpJson() }]); + + await reconcileMcpForConfig(unmovedConfig(), projectConfig); + + expect((await sidecarState()).files[mcpJson()]).toEqual({ tools: ['claude', 'cursor'] }); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('leaves it to the tools mapping it when only their servers are left', async () => { + await setServers({}); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/^\/\.mcp\.json$/m); + expect(Object.keys((await sidecarState()).files)).not.toContain(mcpJson()); + expect((await sidecarState()).earlierMappingsRead).toBe(true); + }); + + it('leaves a file the same tool still maps to that tool\'s own rules', async () => { + const { earlierMappedMcpTargets } = await import('../mcp-reconcile.js'); + const today = { ...teamConfig, toolPaths: shared } as TeamaiConfig; + + const found = await earlierMappedMcpTargets(projectConfig, await resolveMcpTargets(today, projectConfig, { includeUndetected: true })); + + expect(found?.map((target) => target.file)).not.toContain(mcpJson()); + const moved = await earlierMappedMcpTargets(projectConfig, await resolveMcpTargets(teamConfig, projectConfig, { includeUndetected: true })); + expect(moved?.map(({ tool, file }) => ({ tool, file }))).toContainEqual({ tool: 'cursor', file: mcpJson() }); + }); + }); + }); + + describe('a tool\'s built-in location, once the team moves or drops the tool', () => { + const moved = { ...teamConfig, toolPaths: { ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '.cursor/team-mcp.json' } } } as TeamaiConfig; + const dropped = { ...teamConfig, toolPaths: { claude: TOOL_PATHS.claude, codebuddy: TOOL_PATHS.codebuddy } } as TeamaiConfig; + const cursorJson = (): string => path.join(projectRoot, '.cursor', 'mcp.json'); + const open = 'servers:\n - name: open\n transport: http\n url: https://example.com/open\n'; + const ledger = async (): Promise> => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + return (await readResolvedMcpFiles(projectConfig)).files; + }; + // What a teamai from before managed-mcp-files.json leaves: no record of the path, nothing in the exclude. + const asOlderTeamai = async (): Promise => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.remove(resolvedMcpFilesPath(projectConfig) ?? ''); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record>>; + await fse.writeJson(manifestFile, Object.fromEntries(Object.entries(manifest).map(([key, records]) => + [key, records.map(({ name, hash }) => ({ name, hash }))]))); + }; + + beforeEach(async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, projectConfig); + expect(await fse.readFile(cursorJson(), 'utf-8')).toContain('super-secret-value'); + // Then the team drops that server; the token is no longer set. + await writeMcpYaml(open); + vi.stubEnv('SECRET_TOKEN', ''); + }); + + // The pull that writes the moved file replaces Cursor's records: from the next one they describe that file. + it('keeps the line a pull on this version added when the team moves the tool', async () => { + await reconcileMcpForConfig(moved, projectConfig); + await reconcileMcpForConfig(moved, projectConfig); + + expect(await fse.readFile(cursorJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\.cursor\/mcp\.json/); + }); + + it('is listed and recorded by the first pull on this version when an older teamai wrote it', async () => { + await asOlderTeamai(); + + await reconcileMcpForConfig(moved, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\.cursor\/mcp\.json/); + expect((await ledger())[cursorJson()]).toEqual({ tools: ['cursor'] }); + }); + + it('keeps its line on the pulls after, from the record', async () => { + await asOlderTeamai(); + await reconcileMcpForConfig(moved, projectConfig); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + + await reconcileMcpForConfig(moved, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + it('keeps its line when the team then drops the tool', async () => { + await asOlderTeamai(); + await reconcileMcpForConfig(moved, projectConfig); + await asOlderTeamai(); + + await reconcileMcpForConfig(dropped, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + // No record describes that path any more, so a server of the member's own cannot be told from an older teamai's. + it.each([ + ['moves', moved], + ['drops', dropped], + ])('lists and records it while it holds only a server of the member\'s own, when the team %s the tool', async (_label, config) => { + await reconcileMcpForConfig(moved, projectConfig); + await fse.writeJson(cursorJson(), { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + await asOlderTeamai(); + + await reconcileMcpForConfig(config, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect((await ledger())[cursorJson()]).toEqual({ tools: ['cursor'] }); + }); + + it('lets its line go, and forgets it, once it holds no server', async () => { + await reconcileMcpForConfig(moved, projectConfig); + await fse.writeJson(cursorJson(), { mcpServers: {} }); + + await reconcileMcpForConfig(moved, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/\.cursor\/mcp\.json/); + expect(Object.keys(await ledger())).not.toContain(cursorJson()); + }); + + }); + + // CodeBuddy's built-in location is .mcp.json, which Claude maps; TOOL_PATHS moves CodeBuddy to .codebuddy/mcp.json. + describe('a tool\'s built-in location another tool maps today, once the team moves or drops the tool', () => { + const builtin = (): TeamaiConfig => ({ ...teamConfig, toolPaths: { ...TOOL_PATHS, codebuddy: { ...TOOL_PATHS.codebuddy, mcpProject: '.mcp.json' } } } as TeamaiConfig); + const dropped = (): TeamaiConfig => ({ ...teamConfig, toolPaths: { claude: TOOL_PATHS.claude, cursor: TOOL_PATHS.cursor } } as TeamaiConfig); + const mcpJson = (): string => path.join(projectRoot, '.mcp.json'); + const open = ' - name: open\n transport: http\n url: https://example.com/open\n tools: [claude]\n'; + const ledger = async (): Promise> => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + return (await readResolvedMcpFiles(projectConfig)).files; + }; + // An older teamai: no record of the path, nothing in the exclude; and CodeBuddy's record is lost. + const asOlderTeamai = async (): Promise => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.remove(resolvedMcpFilesPath(projectConfig) ?? ''); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record; + delete manifest['codebuddy:project']; + await fse.writeJson(manifestFile, manifest); + }; + const setServers = async (servers: Record): Promise => { + const doc = await fse.readJson(mcpJson()) as { mcpServers: Record }; + await fse.writeJson(mcpJson(), { mcpServers: { open: doc.mcpServers.open, ...servers } }); + }; + + beforeEach(async () => { + // CodeBuddy's server, with the token, lands in .mcp.json beside Claude's. + await fse.ensureDir(path.join(projectRoot, '.codebuddy', 'skills')); + await writeMcpYaml(`${withSecret} tools: [codebuddy]\n${open}`); + await reconcileMcpForConfig(builtin(), projectConfig); + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + // Then the team drops that server; the token is no longer set. + await writeMcpYaml(`servers:\n${open}`); + vi.stubEnv('SECRET_TOKEN', ''); + }); + + it.each([ + ['moves', () => teamConfig], + ['drops', dropped], + ])('lists it, and records it for that tool, when an older teamai wrote it and the team %s the tool', async (_label, config) => { + await asOlderTeamai(); + + await reconcileMcpForConfig(config(), projectConfig); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\.mcp\.json/); + expect((await ledger())[mcpJson()]).toEqual({ tools: ['codebuddy'] }); + }); + + it('keeps its line on the pulls after, from the record', async () => { + await asOlderTeamai(); + await reconcileMcpForConfig(teamConfig, projectConfig); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + // The accepted cost: nothing tells a member's own server there from one teamai wrote for CodeBuddy. + it('keeps a line while it holds a server of the member\'s own', async () => { + await setServers({ mine: { type: 'http', url: 'https://mine.example/mcp' } }); + await asOlderTeamai(); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect((await ledger())[mcpJson()]).toEqual({ tools: ['codebuddy'] }); + }); + + it('leaves it to the tools mapping it once only their servers are left', async () => { + await reconcileMcpForConfig(teamConfig, projectConfig); + await setServers({}); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/^\/\.mcp\.json$/m); + expect(Object.keys(await ledger())).not.toContain(mcpJson()); + }); + }); + + it('lists the config in .git/info/exclude before writing the value into it', async () => { + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(excludeAtWrite.get(path.join(projectRoot, '.mcp.json'))).toMatch(/^\/\.mcp\.json$/m); + expect(await fse.readFile(path.join(projectRoot, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + }); + + describe('the line of a config left without a resolved value goes', () => { + const mcpJson = (): string => path.join(projectRoot, '.mcp.json'); + const claudeOnly = (): LocalConfig => ({ ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + const open = 'servers:\n - name: open\n transport: http\n url: https://example.com/open\n'; + + beforeEach(() => { + vi.mocked(log.info).mockClear(); + vi.mocked(log.debug).mockClear(); + }); + + afterEach(() => { + beforeJsonWrite.run = null; + }); + + it.each([ + ['it does not parse', () => fse.writeFile(mcpJson(), '{ "mcpServers": ')], + ['it holds a server of the member\'s own under the team\'s name', () => fse.writeJson(mcpJson(), { + mcpServers: { 'with-secret': { type: 'http', url: 'https://mine.example/mcp' } }, + })], + ])('when this pull listed it and then wrote nothing, as %s', async (_label, arrange) => { + await pulledBefore(); + await arrange(); + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(unmovedConfig(), claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).not.toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + // The member never saw the line go in, so its rollback is not news. + expect(vi.mocked(log.info).mock.calls.flat().join('\n')).not.toMatch(/Removed/); + expect(vi.mocked(log.debug).mock.calls.flat().join('\n')).toMatch(/\/\.mcp\.json/); + }); + + it('but one this pull listed stays when it wrote the value and then failed to record it', async () => { + // Shorter than eight characters: no scan of the file can find it again. + vi.stubEnv('SECRET_TOKEN', 'short'); + await writeMcpYaml(withSecret); + beforeJsonWrite.run = async (file) => { + if (path.basename(file) === 'managed-mcp.json') throw new Error('disk full'); + }; + + await expect(reconcileMcpForConfig(teamConfig, claudeOnly())).rejects.toThrow('disk full'); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('Bearer short'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('but one an earlier pull listed stays while the config cannot be proven clean', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.remove(managedMcpManifestPath(getDataHome(projectConfig), projectRoot)); + await fse.writeFile(mcpJson(), '{ "mcpServers": '); + vi.stubEnv('SECRET_TOKEN', 'rotated-secret-value'); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('but one an earlier pull listed stays when this pull rewrote the manifest it had lost', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.remove(managedMcpManifestPath(getDataHome(projectConfig), projectRoot)); + await writeMcpYaml(open); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await fse.pathExists(managedMcpManifestPath(getDataHome(projectConfig), projectRoot))).toBe(true); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('and every pull after the one that rewrote the manifest it had lost', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.remove(managedMcpManifestPath(getDataHome(projectConfig), projectRoot)); + await writeMcpYaml(open); + vi.stubEnv('SECRET_TOKEN', ''); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('and every pull after one that rewrote the manifest it had lost while another command held managed-mcp-files.json', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + await fse.remove(managedMcpManifestPath(getDataHome(projectConfig), projectRoot)); + await writeMcpYaml(open); + vi.stubEnv('SECRET_TOKEN', ''); + const lock = `${resolvedMcpFilesPath(projectConfig)}.teamai-lock`; + expect(await acquireLock(lock)).toBe(true); + try { + await reconcileMcpForConfig(teamConfig, claudeOnly()); + expect(await fse.readJson(mcpJson())).toMatchObject({ mcpServers: { open: expect.anything() } }); + // Still held: the note is still missing, and so the line stays. + await reconcileMcpForConfig(teamConfig, claudeOnly()); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + } finally { + await releaseLock(lock); + } + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + + // Noted at last: once the member takes the stale server out, the line goes. + const doc = await fse.readJson(mcpJson()) as { mcpServers: Record }; + delete doc.mcpServers['with-secret']; + await fse.writeJson(mcpJson(), doc); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }, 30_000); + + describe.each([ + ['empty', ''], + ['truncated', '{ "claude:project": [ { "name": "with-sec'], + ['recording nothing for this tool', '{ "cursor:project": [ { "name": "with-secret", "hash": "h" } ] }'], + ])('but one an earlier pull listed stays while managed-mcp.json is %s', (_label, content) => { + beforeEach(async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.writeFile(managedMcpManifestPath(getDataHome(projectConfig), projectRoot), content); + vi.stubEnv('SECRET_TOKEN', ''); + }); + + it('and a pull finds its server gone from mcp.yaml', async () => { + await writeMcpYaml(open); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('and `teamai mcp remove` runs after its server left mcp.yaml', async () => { + await writeMcpYaml('servers: []\n'); + + await reconcileMcpForConfig(teamConfig, claudeOnly(), { removeAll: true }); + await releaseCleanMcpGitExcludes(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('and a later pull runs after one rebuilt the record for another server', async () => { + await writeMcpYaml(open); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('and `teamai mcp remove` runs after a pull rebuilt the record for another server', async () => { + await writeMcpYaml(open); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + await reconcileMcpForConfig(teamConfig, claudeOnly(), { removeAll: true }); + await releaseCleanMcpGitExcludes(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + it('until the member takes that server out of the config', async () => { + await writeMcpYaml(open); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + const doc = await fse.readJson(mcpJson()) as { mcpServers: Record }; + delete doc.mcpServers['with-secret']; + await fse.writeJson(mcpJson(), doc); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }); + }); + + it('when a server of the member\'s own was in the config before teamai first wrote to it', async () => { + await pulledBefore(); + await fse.writeJson(mcpJson(), { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(unmovedConfig(), claudeOnly()); + await writeMcpYaml(open); + + await reconcileMcpForConfig(unmovedConfig(), claudeOnly()); + + expect(await fse.readJson(mcpJson())).toEqual({ + mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' }, open: expect.anything() }, + }); + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }); + + it('when `teamai mcp remove` takes teamai\'s servers out of a config that also holds the member\'s own', async () => { + await pulledBefore(); + await fse.writeJson(mcpJson(), { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(unmovedConfig(), claudeOnly()); + + await reconcileMcpForConfig(unmovedConfig(), claudeOnly(), { removeAll: true }); + await releaseCleanMcpGitExcludes(unmovedConfig(), claudeOnly()); + + expect(await fse.readJson(mcpJson())).toEqual({ mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }); + + it('when the last server with a resolved value leaves mcp.yaml', async () => { + await writeMcpYaml(`${withSecret} - name: open\n transport: http\n url: https://example.com/open\n`); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + await fse.appendFile(path.join(projectRoot, '.git', 'info', 'exclude'), 'mine/\n'); + await writeMcpYaml(open); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('https://example.com/open'); + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + expect(await excludeOf(projectRoot)).toMatch(/^mine\/$/m); + expect(log.info).toHaveBeenCalledWith(expect.stringMatching(/^Removed \/\.mcp\.json from /)); + }); + + it('when `teamai mcp remove` takes teamai\'s servers out', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + await reconcileMcpForConfig(teamConfig, claudeOnly(), { removeAll: true }); + await releaseCleanMcpGitExcludes(teamConfig, claudeOnly()); + + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }); + + it('when `teamai mcp remove` finds a nested repository\'s linked worktree holding no server', async () => { + const cursorDir = path.join(projectRoot, '.cursor'); + git(cursorDir, 'init', '-q'); + git(cursorDir, '-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '--allow-empty', '-m', 'init'); + const linked = path.join(tmpDir, 'cursor-linked'); + git(cursorDir, 'worktree', 'add', '-q', linked); + await fse.writeJson(path.join(linked, 'mcp.json'), { mcpServers: {} }); + await fse.writeFile(path.join(cursorDir, '.git', 'info', 'exclude'), [ + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/mcp.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n')); + await writeMcpYaml(withSecret); + + await releaseCleanMcpGitExcludes(teamConfig, projectConfig); + + expect(await excludeOf(cursorDir)).not.toContain('teamai'); + }); + + it('but not while another worktree\'s copy of the config still holds one', async () => { + git(projectRoot, '-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '--allow-empty', '-m', 'init'); + const worktree = path.join(await fse.realpath(tmpDir), 'business-wt'); + git(projectRoot, 'worktree', 'add', '-q', worktree); + await fse.ensureDir(path.join(worktree, '.claude', 'skills')); + const { resolveProjectDataHome } = await import('../config.js'); + const other = { ...claudeOnly(), projectRoot: worktree, dataHome: await resolveProjectDataHome(worktree) } as LocalConfig; + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + await reconcileMcpForConfig(teamConfig, other); + await writeMcpYaml(open); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(path.join(worktree, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + describe('after a server\'s ${VAR} became a literal, judged from another worktree', () => { + const literal = withSecret.replace('${SECRET_TOKEN}', 'published-literal'); + let worktree: string; + + beforeEach(async () => { + git(projectRoot, '-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '--allow-empty', '-m', 'init'); + // As git lists it (macOS /var is a symlink), so its manifest is found under the same key. + worktree = path.join(await fse.realpath(tmpDir), 'business-wt'); + git(projectRoot, 'worktree', 'add', '-q', worktree); + await fse.ensureDir(path.join(worktree, '.claude', 'skills')); + const { resolveProjectDataHome } = await import('../config.js'); + const other = { ...claudeOnly(), projectRoot: worktree, dataHome: await resolveProjectDataHome(worktree) } as LocalConfig; + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + await reconcileMcpForConfig(teamConfig, other); + await writeMcpYaml(literal); + vi.stubEnv('SECRET_TOKEN', ''); + }); + + it('keeps the shared line while that worktree\'s config still holds the stale entry', async () => { + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('published-literal'); + expect(await fse.readFile(path.join(worktree, '.mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect(git(worktree, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\.mcp\.json/); + }); + + it('removes the line once that worktree\'s config is gone', async () => { + await fse.remove(path.join(worktree, '.mcp.json')); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }); + }); + }); + + it('lists the config again when a concurrent uninstall drops its line between the check and the write', async () => { + const { findMcpGitExcludes, removeMcpGitExclude } = await import('../mcp-git-exclude.js'); + beforeJsonWrite.run = async () => { + for (const [excludeFile, entries] of await findMcpGitExcludes([projectRoot])) { + await removeMcpGitExclude(excludeFile, entries.map((entry) => entry.pattern)); + } + }; + await writeMcpYaml(withSecret); + + try { + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + } finally { + beforeJsonWrite.run = null; + } + + expect(excludeAtWrite.get(path.join(projectRoot, '.mcp.json'))).not.toContain('teamai'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + }); + + describe('when the config cannot be kept out of git first', () => { + const mcpJson = (): string => path.join(projectRoot, '.mcp.json'); + const infoDir = (): string => path.join(projectRoot, '.git', 'info'); + const claudeOnly = (): LocalConfig => ({ ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + beforeEach(() => { + vi.mocked(log.warn).mockClear(); + }); + + afterEach(async () => { + await fse.chmod(infoDir(), 0o755); + await fse.chmod(path.join(infoDir(), 'exclude'), 0o644); + }); + + it.skipIf(process.getuid?.() === 0).each([ + ['.git/info/exclude is read-only', () => fse.chmod(path.join(infoDir(), 'exclude'), 0o444)], + ['.git/info is read-only', () => fse.chmod(infoDir(), 0o555)], + ])('writes no value when %s, and warns with the fix', async (_label, lockDown) => { + await writeMcpYaml(withSecret); + await lockDown(); + + const { changes } = await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.pathExists(mcpJson())).toBe(false); + expect(changes).toContainEqual(expect.objectContaining({ tool: 'claude', server: 'with-secret', action: 'skipped' })); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining(mcpJson())); + expect(log.warn).toHaveBeenCalledWith(expect.stringMatching(/not writable[\s\S]*teamai pull/)); + }); + + it.skipIf(process.getuid?.() === 0)('keeps an earlier entry as it was', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + await fse.writeFile(path.join(infoDir(), 'exclude'), ''); + const before = await fse.readFile(mcpJson(), 'utf-8'); + await writeMcpYaml(withSecret.replace('https://example.com/mcp', 'https://example.com/v2')); + vi.stubEnv('SECRET_TOKEN', 'rotated-secret-value'); + await fse.chmod(infoDir(), 0o555); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toBe(before); + }); + + it('writes no value while another command holds the exclude file\'s lock', async () => { + const lock = path.join(infoDir(), 'exclude.teamai-lock'); + expect(await acquireLock(lock)).toBe(true); + await writeMcpYaml(withSecret); + + try { + await reconcileMcpForConfig(teamConfig, claudeOnly()); + } finally { + await releaseLock(lock); + } + + expect(await fse.pathExists(mcpJson())).toBe(false); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining(mcpJson())); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining('teamai pull')); + }); + + it('writes no value into a file git already tracks, names the fix once, and still writes an untracked config', async () => { + await fse.writeJson(mcpJson(), { mcpServers: {} }); + git(projectRoot, 'add', '.mcp.json'); + await writeMcpYaml(withSecret); + + const { changes } = await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(mcpJson(), 'utf-8')).not.toContain('super-secret-value'); + expect(changes).toContainEqual(expect.objectContaining({ + tool: 'claude', server: 'with-secret', action: 'skipped', reason: expect.stringContaining(`git already tracks ${mcpJson()}`), + })); + const warnings = vi.mocked(log.warn).mock.calls.map(([line]) => String(line)).filter((line) => line.includes(mcpJson())); + expect(warnings).toHaveLength(1); + expect(warnings[0]).toMatch(new RegExp(`git already tracks[\\s\\S]*git rm --cached ${mcpJson()}[\\s\\S]*rotate`)); + expect((await fse.readJson(path.join(projectRoot, '.cursor', 'mcp.json'))).mcpServers['with-secret'].headers.Authorization) + .toBe('Bearer super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }); + + it('keeps the entry an earlier pull wrote to a file git now tracks, and updates it once the file is untracked (#879)', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, claudeOnly()); + git(projectRoot, 'add', '-f', '.mcp.json'); + const before = await fse.readFile(mcpJson(), 'utf-8'); + vi.stubEnv('SECRET_TOKEN', 'rotated-secret-value'); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + expect(await fse.readFile(mcpJson(), 'utf-8')).toBe(before); + + git(projectRoot, 'rm', '-q', '--cached', '.mcp.json'); + const { changes } = await reconcileMcpForConfig(teamConfig, claudeOnly()); + expect(changes).toContainEqual({ tool: 'claude', server: 'with-secret', action: 'updated' }); + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('rotated-secret-value'); + }); + + it('still writes a config that carries no resolved value', async () => { + await fse.chmod(infoDir(), 0o555); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('https://example.com/open'); + }); + }); + + it('leaves .git/info/exclude alone on a dry run', async () => { + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(teamConfig, projectConfig, { dryRun: true }); + + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + expect(await fse.pathExists(resolvedMcpFilesPath(projectConfig) ?? '')).toBe(false); + }); + + // The appliers replace the file itself but follow its directories (#886). + describe('a config under a symlinked directory is judged where the write lands', () => { + const cursorOnly = (): LocalConfig => ({ ...projectConfig, disabledAgents: ['claude'] } as LocalConfig); + const landed = async (): Promise => path.join(await fse.realpath(projectRoot), 'config', 'mcp.json'); + + beforeEach(async () => { + vi.mocked(log.warn).mockClear(); + await fse.remove(path.join(projectRoot, '.cursor')); + await fse.outputFile(path.join(projectRoot, 'config', 'skills', 'README.md'), 'cursor skills\n'); + await fse.symlink('config', path.join(projectRoot, '.cursor'), 'dir'); + }); + + it('withholds the servers from a file git tracks there, naming both paths', async () => { + await fse.writeJson(path.join(projectRoot, 'config', 'mcp.json'), { mcpServers: {} }); + git(projectRoot, 'add', 'config', '.cursor'); + git(projectRoot, '-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '-m', 'cursor config'); + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(teamConfig, cursorOnly()); + + expect(await fse.readFile(path.join(projectRoot, 'config', 'mcp.json'), 'utf-8')).not.toContain('super-secret-value'); + const warning = vi.mocked(log.warn).mock.calls.map(([m]) => String(m)).find((m) => m.includes('git already tracks')); + expect(warning).toContain(path.join(projectRoot, '.cursor', 'mcp.json')); + expect(warning).toContain(`git rm --cached ${await landed()}\``); + }); + + it('lists the file it lands in, and releases that line once it holds no resolved value', async () => { + git(projectRoot, 'add', 'config', '.cursor'); + git(projectRoot, '-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '-m', 'cursor config'); + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(teamConfig, cursorOnly()); + + expect(await fse.readFile(await landed(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/config\/mcp\.json$/m); + expect(await excludeOf(projectRoot)).not.toContain('/.cursor/'); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toContain('config/mcp.json'); + + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + await reconcileMcpForConfig(teamConfig, cursorOnly()); + + expect(await fse.readFile(await landed(), 'utf-8')).not.toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).not.toContain('/config/mcp.json'); + }); + + it('writes, listing nothing and warning of nothing, when the directory links outside any repository', async () => { + const outside = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-no-repo-')); + await fse.copy(path.join(projectRoot, 'config'), outside); + await fse.remove(path.join(projectRoot, '.cursor')); + await fse.symlink(outside, path.join(projectRoot, '.cursor'), 'dir'); + await writeMcpYaml(withSecret); + + try { + await reconcileMcpForConfig(teamConfig, cursorOnly()); + + expect(await fse.readFile(path.join(outside, 'mcp.json'), 'utf-8')).toContain('super-secret-value'); + expect(log.warn).not.toHaveBeenCalled(); + expect(await excludeOf(projectRoot)).not.toContain('/.cursor/'); + } finally { + await fse.remove(outside); + } + }); + }); + + it('records each config it writes a resolved value to, by path, before writing it', async () => { + const { readResolvedMcpFiles, resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + const sidecarAtWrite = new Map(); + beforeJsonWrite.run = async (file) => { + sidecarAtWrite.set(file, file in (await readResolvedMcpFiles(projectConfig)).files); + }; + await writeMcpYaml(withSecret); + + try { + await reconcileMcpForConfig(unmovedConfig(), projectConfig); + } finally { + beforeJsonWrite.run = null; + } + + expect(sidecarAtWrite.get(path.join(projectRoot, '.mcp.json'))).toBe(true); + expect((await readResolvedMcpFiles(projectConfig)).files).toEqual({ + [path.join(projectRoot, '.mcp.json')]: { tools: ['claude'] }, + [path.join(projectRoot, '.cursor', 'mcp.json')]: { tools: ['cursor'] }, + }); + expect((await fse.stat(resolvedMcpFilesPath(projectConfig) ?? '')).mode & 0o777).toBe(0o600); + }); + + // An empty record says teamai owns nothing left in the file, which a pull that could not read it cannot say. + it('keeps a tool\'s record as it was when this pull could not read its config, so a stale entry keeps its line once repaired', async () => { + const cursorJson = path.join(projectRoot, '.cursor', 'mcp.json'); + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, projectConfig); + const repaired = await fse.readFile(cursorJson, 'utf-8'); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + await fse.writeFile(cursorJson, '{ "mcpServers": '); + vi.stubEnv('SECRET_TOKEN', ''); + await reconcileMcpForConfig(teamConfig, projectConfig); + + await fse.writeFile(cursorJson, repaired); + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await fse.readFile(cursorJson, 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\.cursor\/mcp\.json/); + }); + + it('takes back a tool it recorded before a write it then skipped, in a file another tool wrote this pull', async () => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + const mcpJson = path.join(projectRoot, '.mcp.json'); + await fse.ensureDir(path.join(projectRoot, '.codebuddy', 'skills')); + await writeMcpYaml(withSecret); + + // Claude writes with-secret first; CodeBuddy finds it there, not its own, and skips it. + const result = await reconcileMcpForConfig(unmovedConfig(), projectConfig); + + expect(result.changes).toContainEqual(expect.objectContaining({ tool: 'codebuddy', server: 'with-secret', action: 'skipped' })); + expect((await readResolvedMcpFiles(projectConfig)).files[mcpJson]).toEqual({ tools: ['claude'] }); + }); + + it('keeps a tool it records again whose entry with a resolved value is already in the file, with no write', async () => { + const { readResolvedMcpFiles, resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + const mcpJson = path.join(projectRoot, '.mcp.json'); + await fse.ensureDir(path.join(projectRoot, '.codebuddy', 'skills')); + await writeMcpYaml(`${withSecret} tools: [codebuddy]\n${withSecret.replace('servers:\n', '').replace('with-secret', 'other-secret')} tools: [claude]\n`); + await reconcileMcpForConfig(unmovedConfig(), projectConfig); + expect((await readResolvedMcpFiles(projectConfig)).files[mcpJson]?.tools.sort()).toEqual(['claude', 'codebuddy']); + await fse.writeJson(resolvedMcpFilesPath(projectConfig) ?? '', { version: 1, files: { [mcpJson]: { tools: ['claude'] } } }); + + const result = await reconcileMcpForConfig(unmovedConfig(), projectConfig); + + expect(result.wrote).toBe(false); + expect((await readResolvedMcpFiles(projectConfig)).files[mcpJson]?.tools.sort()).toEqual(['claude', 'codebuddy']); + }); + + it('takes back a tool it recorded before a write that did not happen, in a file another tool recorded', async () => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + const mcpJson = path.join(projectRoot, '.mcp.json'); + await writeMcpYaml(`${withSecret} tools: [claude]\n`); + await reconcileMcpForConfig(unmovedConfig(), projectConfig); + expect((await readResolvedMcpFiles(projectConfig)).files[mcpJson]).toEqual({ tools: ['claude'] }); + await fse.ensureDir(path.join(projectRoot, '.codebuddy', 'skills')); + await writeMcpYaml(withSecret); + await fse.writeFile(mcpJson, '{ "mcpServers": '); + + await reconcileMcpForConfig(unmovedConfig(), projectConfig); + + expect((await readResolvedMcpFiles(projectConfig)).files[mcpJson]).toEqual({ tools: ['claude'] }); + }); + + describe('forgets a config it recorded before a write that did not happen', () => { + const custom = { ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '.cursor/team-mcp.json' } }; + const customFile = (): string => path.join(projectRoot, '.cursor', 'team-mcp.json'); + const mine = { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }; + + afterEach(async () => { + await fse.chmod(path.join(projectRoot, '.cursor'), 0o755); + }); + + // Root writes into a read-only directory. + it.skipIf(process.getuid?.() === 0).each([ + ['its write fails', async () => { + await fse.writeJson(customFile(), mine); + await fse.chmod(path.join(projectRoot, '.cursor'), 0o555); + await expect(reconcileMcpForConfig({ ...teamConfig, toolPaths: custom } as TeamaiConfig, projectConfig)).rejects.toThrow(); + await fse.chmod(path.join(projectRoot, '.cursor'), 0o755); + }], + ['it does not parse', async () => { + await fse.writeFile(customFile(), '{ "mcpServers": '); + await reconcileMcpForConfig({ ...teamConfig, toolPaths: custom } as TeamaiConfig, projectConfig); + await fse.writeJson(customFile(), mine); + }], + ])('so a config of the member\'s own there is not kept listed once the mapping changes, when %s', async (_label, arrange) => { + await pulledBefore(); + await writeMcpYaml(withSecret); + await arrange(); + expect(await fse.readJson(customFile())).toEqual(mine); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/team-mcp\.json/); + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + expect(Object.keys((await readResolvedMcpFiles(projectConfig)).files)).not.toContain(customFile()); + }); + }); + + describe('without a usable managed-mcp-files.json, as before it existed', () => { + const mcpJson = (): string => path.join(projectRoot, '.mcp.json'); + const claudeOnly = (): LocalConfig => ({ ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + const open = 'servers:\n - name: open\n transport: http\n url: https://example.com/open\n'; + const sidecarFile = async (): Promise => { + const { resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + return resolvedMcpFilesPath(projectConfig) ?? ''; + }; + const loseManifest = async (): Promise => { + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + await fse.remove(managedMcpManifestPath(getDataHome(projectConfig), projectRoot)); + }; + + it('keeps the line of a config under a changed mapping it can no longer find, even holding no server', async () => { + const custom = { ...TOOL_PATHS, cursor: { ...TOOL_PATHS.cursor, mcpProject: '.cursor/team-mcp.json' } }; + await writeMcpYaml(withSecret); + await reconcileMcpForConfig({ ...teamConfig, toolPaths: custom } as TeamaiConfig, projectConfig); + await fse.remove(await sidecarFile()); + await fse.writeJson(path.join(projectRoot, '.cursor', 'team-mcp.json'), { mcpServers: {} }); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + // No tool reads it any more, and nothing says teamai wrote it: the line stays, as before. + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/team-mcp\.json$/m); + }); + + it('rewrites one that does not parse', async () => { + await fse.outputFile(await sidecarFile(), '{ "version": 1, "files": '); + await writeMcpYaml(withSecret); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + expect(Object.keys((await readResolvedMcpFiles(projectConfig)).files)).toEqual([mcpJson()]); + }); + + it('still writes the config while another command holds its lock, and records it on the next pull', async () => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + const lock = `${await sidecarFile()}.teamai-lock`; + await fse.ensureDir(path.dirname(lock)); + expect(await acquireLock(lock)).toBe(true); + await writeMcpYaml(withSecret); + try { + await reconcileMcpForConfig(teamConfig, claudeOnly()); + } finally { + await releaseLock(lock); + } + expect(await fse.readFile(mcpJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.mcp\.json$/m); + expect((await readResolvedMcpFiles(projectConfig)).files).toEqual({}); + + await reconcileMcpForConfig(teamConfig, claudeOnly()); + + expect(Object.keys((await readResolvedMcpFiles(projectConfig)).files)).toEqual([mcpJson()]); + }, 30_000); + + // Cursor's file: CodeBuddy's built-in location is Claude's .mcp.json, which TOOL_PATHS moves CodeBuddy off. + describe('while this worktree has no managed-mcp.json at all either', () => { + const cursorJson = (): string => path.join(projectRoot, '.cursor', 'mcp.json'); + const lost = async (): Promise => { + await loseManifest(); + await fse.remove(await sidecarFile()); + await fse.writeFile(path.join(projectRoot, '.git', 'info', 'exclude'), ''); + }; + + beforeEach(async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, projectConfig); + await lost(); + vi.stubEnv('SECRET_TOKEN', ''); + }); + + it.each([ + ['still delivers to it', open], + ['delivers nothing to it', `${open} tools: [claude]\n`], + ])('lists a config holding a stale entry, and keeps it on the pulls after, when the team %s', async (_label, yaml) => { + await writeMcpYaml(yaml); + + await reconcileMcpForConfig(teamConfig, projectConfig); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(cursorJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect(git(projectRoot, 'status', '--porcelain', '--untracked-files=all')).not.toMatch(/\.cursor\/mcp\.json/); + }); + + it('keeps the line of a config holding a stale entry on the pulls after one that could not note it, and notes it once it can', async () => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + await writeMcpYaml(open); + const lock = `${await sidecarFile()}.teamai-lock`; + await fse.ensureDir(path.dirname(lock)); + expect(await acquireLock(lock)).toBe(true); + try { + await reconcileMcpForConfig(teamConfig, projectConfig); + } finally { + await releaseLock(lock); + } + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(cursorJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect((await readResolvedMcpFiles(projectConfig)).files[cursorJson()]?.unverified).toEqual(['with-secret']); + const doc = await fse.readJson(cursorJson()) as { mcpServers: Record }; + delete doc.mcpServers['with-secret']; + await fse.writeJson(cursorJson(), doc); + await reconcileMcpForConfig(teamConfig, projectConfig); + expect(await excludeOf(projectRoot)).not.toMatch(/\.cursor\/mcp\.json/); + }, 30_000); + + it('keeps that line through a pull that rewrites the record while the note still cannot land', async () => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + await writeMcpYaml(open); + const lock = `${await sidecarFile()}.teamai-lock`; + await fse.ensureDir(path.dirname(lock)); + expect(await acquireLock(lock)).toBe(true); + try { + await reconcileMcpForConfig(teamConfig, projectConfig); + await writeMcpYaml(`${open} - name: more\n transport: http\n url: https://example.com/more\n`); + await reconcileMcpForConfig(teamConfig, projectConfig); + } finally { + await releaseLock(lock); + } + + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(cursorJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect((await readResolvedMcpFiles(projectConfig)).files[cursorJson()]?.unverified).toEqual(['with-secret']); + }, 30_000); + + it('keeps that line through a pull that empties the record while the note still cannot land', async () => { + await writeMcpYaml(open); + const lock = `${await sidecarFile()}.teamai-lock`; + await fse.ensureDir(path.dirname(lock)); + expect(await acquireLock(lock)).toBe(true); + try { + await reconcileMcpForConfig(teamConfig, projectConfig); + await writeMcpYaml(`${open} tools: [claude]\n`); + await reconcileMcpForConfig(teamConfig, projectConfig); + } finally { + await releaseLock(lock); + } + + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(cursorJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + }, 30_000); + + // The cost: a first pull in a new worktree cannot tell a member's own server from a stale one of teamai's. + it('lists a config holding only a server of the member\'s own at the first pull in a worktree, until it leaves', async () => { + await fse.writeJson(cursorJson(), { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + await writeMcpYaml(open); + + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + const doc = await fse.readJson(cursorJson()) as { mcpServers: Record }; + delete doc.mcpServers.mine; + await fse.writeJson(cursorJson(), doc); + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/\.cursor\/mcp\.json/); + }); + + it('lets the line go once the member takes the stale entry out', async () => { + await writeMcpYaml(open); + await reconcileMcpForConfig(teamConfig, projectConfig); + const doc = await fse.readJson(cursorJson()) as { mcpServers: Record }; + delete doc.mcpServers['with-secret']; + await fse.writeJson(cursorJson(), doc); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/\.cursor\/mcp\.json/); + }); + + it('leaves one git tracks as it is, and says nothing', async () => { + git(projectRoot, 'add', '.cursor/mcp.json'); + vi.mocked(log.warn).mockClear(); + await writeMcpYaml(open); + + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/\.cursor\/mcp\.json/); + expect(vi.mocked(log.warn).mock.calls.flat().join('\n')).not.toContain(cursorJson()); + }); + }); + + // Cursor's file, as above; Claude's record keeps managed-mcp.json from being empty. + describe('while one tool has no record in managed-mcp.json', () => { + const cursorJson = (): string => path.join(projectRoot, '.cursor', 'mcp.json'); + const manifestFile = async (): Promise => { + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + return managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + }; + + it('lists a config holding a stale entry, and keeps it on the pulls after the one that rebuilds that tool\'s record', async () => { + const { readResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, projectConfig); + const manifest = await fse.readJson(await manifestFile()) as Record; + delete manifest['cursor:project']; + await fse.writeJson(await manifestFile(), manifest); + const sidecar = await fse.readJson(await sidecarFile()) as { files: Record }; + delete sidecar.files[cursorJson()]; + await fse.writeJson(await sidecarFile(), sidecar); + await writeMcpYaml(open); + vi.stubEnv('SECRET_TOKEN', ''); + + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await fse.readFile(cursorJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + expect((await readResolvedMcpFiles(projectConfig)).files[cursorJson()]?.unverified).toEqual(['with-secret']); + }); + + // The cost, as with no managed-mcp.json at all: a tool's first delivery cannot tell a member's own server from a stale one. + it('lists a config holding only a server of the member\'s own at that tool\'s first delivery, until it leaves', async () => { + await fse.outputJson(await manifestFile(), { 'claude:project': [] }); + await fse.writeJson(cursorJson(), { mcpServers: { mine: { type: 'http', url: 'https://mine.example/mcp' } } }); + await writeMcpYaml(open); + + await reconcileMcpForConfig(teamConfig, projectConfig); + await reconcileMcpForConfig(teamConfig, projectConfig); + expect(await excludeOf(projectRoot)).toMatch(/^\/\.cursor\/mcp\.json$/m); + const doc = await fse.readJson(cursorJson()) as { mcpServers: Record }; + delete doc.mcpServers.mine; + await fse.writeJson(cursorJson(), doc); + await reconcileMcpForConfig(teamConfig, projectConfig); + + expect(await excludeOf(projectRoot)).not.toMatch(/\.cursor\/mcp\.json/); + }); + }); + + // Cursor's file: CodeBuddy's built-in location is Claude's .mcp.json, which TOOL_PATHS moves CodeBuddy off. + describe('releases the line of a stale entry whose value is no longer set', () => { + const cursorJson = (): string => path.join(projectRoot, '.cursor', 'mcp.json'); + const cursorOnly = (): LocalConfig => ({ ...projectConfig, disabledAgents: ['claude', 'tclaude'] } as LocalConfig); + + it.each([ + ['it is deleted after a pull rebuilt the lost record', async () => { + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, cursorOnly()); + await loseManifest(); + await writeMcpYaml(open); + vi.stubEnv('SECRET_TOKEN', ''); + await reconcileMcpForConfig(teamConfig, cursorOnly()); + await fse.remove(await sidecarFile()); + }], + ['an older teamai, which kept none, wrote the config and rebuilt the lost record', async () => { + await writeMcpYaml(`${withSecret}${open.replace('servers:\n', '')}`); + await reconcileMcpForConfig(teamConfig, cursorOnly()); + await fse.remove(await sidecarFile()); + // Its rebuild records what it wrote, open, and notes nothing else. + const { getDataHome, managedMcpManifestPath } = await import('../types.js'); + const manifestFile = managedMcpManifestPath(getDataHome(projectConfig), projectRoot); + const manifest = await fse.readJson(manifestFile) as Record>; + await fse.writeJson(manifestFile, { 'cursor:project': manifest['cursor:project'].filter((record) => record.name === 'open') }); + await writeMcpYaml(open); + vi.stubEnv('SECRET_TOKEN', ''); + }], + ])('when %s', async (_label, arrange) => { + await arrange(); + + await reconcileMcpForConfig(teamConfig, cursorOnly()); + + // The documented limit: without the note, the stale entry looks like the member's own. + expect(await fse.readFile(cursorJson(), 'utf-8')).toContain('super-secret-value'); + expect(await excludeOf(projectRoot)).not.toContain('teamai'); + }); + }); + }); + + it('records a config an older teamai wrote a resolved value to on the first pull that finds it', async () => { + const { readResolvedMcpFiles, resolvedMcpFilesPath } = await import('../mcp-resolved-files.js'); + await writeMcpYaml(withSecret); + await reconcileMcpForConfig(teamConfig, projectConfig); + await fse.remove(resolvedMcpFilesPath(projectConfig) ?? ''); + + await reconcileMcpForConfig(teamConfig, { ...projectConfig, disabledAgents: ['cursor'] } as LocalConfig); + + expect((await readResolvedMcpFiles(projectConfig)).files[path.join(projectRoot, '.cursor', 'mcp.json')]).toEqual({ tools: ['cursor'] }); + }); + }); + it('skips tools that are not installed', async () => { await writeMcpYaml(` servers: @@ -1004,6 +3020,160 @@ servers: expect(changes.some((c) => c.server === 'volces-search' && c.action === 'added')).toBe(true); }); + // #879: a resolved ${VAR} may be a team secret, so the file holding it is the member's alone. + describe.skipIf(process.platform === 'win32')('file modes', () => { + let previousUmask: number; + const mode = async (file: string): Promise => (await fse.stat(file)).mode & 0o777; + const SECRET_SERVER = ` +servers: + - name: with-secret + transport: http + url: https://example.com/mcp + headers: + Authorization: Bearer \${SECRET_TOKEN} +`; + beforeEach(() => { + previousUmask = process.umask(0o022); + vi.stubEnv('SECRET_TOKEN', 'super-secret-value'); + }); + afterEach(() => { + process.umask(previousUmask); + vi.restoreAllMocks(); + }); + + it('writes an existing 0644 .mcp.json and ~/.claude.json 0600 once they hold a resolved value', async () => { + const projectRoot = path.join(tmpDir, 'proj-mode'); + await fse.ensureDir(path.join(projectRoot, '.claude', 'skills')); + const projectFile = path.join(projectRoot, '.mcp.json'); + const userFile = path.join(homeDir, '.claude.json'); + for (const file of [projectFile, userFile]) { + await fse.writeFile(file, '{}\n'); + await fse.chmod(file, 0o644); + } + await writeMcpYaml(SECRET_SERVER); + + await reconcileMcpForConfig(teamConfig, { ...localConfig, scope: 'project', projectRoot } as unknown as LocalConfig); + await reconcileMcpForConfig(teamConfig, localConfig); + + for (const file of [projectFile, userFile]) { + expect((await fse.readJson(file)).mcpServers['with-secret'].headers.Authorization).toBe('Bearer super-secret-value'); + expect(await mode(file)).toBe(0o600); + } + }); + + it('tightens a 0644 project config it protects without writing, as for a disabled tool', async () => { + const projectRoot = path.join(tmpDir, 'proj-mode-disabled'); + for (const d of ['.claude', '.cursor']) await fse.ensureDir(path.join(projectRoot, d, 'skills')); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + const project = { ...localConfig, scope: 'project', projectRoot } as unknown as LocalConfig; + await writeMcpYaml(SECRET_SERVER); + await reconcileMcpForConfig(teamConfig, project); + const cursorFile = path.join(projectRoot, '.cursor', 'mcp.json'); + expect(await fse.readFile(cursorFile, 'utf-8')).toContain('super-secret-value'); + await fse.chmod(cursorFile, 0o644); + + await reconcileMcpForConfig(teamConfig, { ...project, disabledAgents: ['cursor'] } as LocalConfig); + + expect(await fse.readFile(cursorFile, 'utf-8')).toContain('super-secret-value'); + expect(await mode(cursorFile)).toBe(0o600); + }); + + it('keeps the mode of an existing config whose servers hold no resolved value', async () => { + const userFile = path.join(homeDir, '.claude.json'); + await fse.writeFile(userFile, '{}\n'); + await fse.chmod(userFile, 0o644); + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + + await reconcileMcpForConfig(teamConfig, localConfig); + + expect((await fse.readJson(userFile)).mcpServers.open).toBeDefined(); + expect(await mode(userFile)).toBe(0o644); + }); + + it('tightens a 0644 config whose managed entry holds a resolved value, though nothing changed', async () => { + await fse.ensureDir(path.join(homeDir, '.codex', 'skills')); + await writeMcpYaml(SECRET_SERVER); + // Twice: the second pull pads the Codex block it appended last with a blank line. + await reconcileMcpForConfig(teamConfig, localConfig); + await reconcileMcpForConfig(teamConfig, localConfig); + const files = [path.join(homeDir, '.claude.json'), path.join(homeDir, '.codex', 'config.toml')]; + const contents: string[] = []; + for (const file of files) { + await fse.chmod(file, 0o644); + contents.push(await fse.readFile(file, 'utf-8')); + } + + await reconcileMcpForConfig(teamConfig, localConfig, { dryRun: true }); + for (const file of files) expect(await mode(file)).toBe(0o644); + + const { wrote, changes } = await reconcileMcpForConfig(teamConfig, localConfig); + + expect(changes).toEqual([]); + expect(wrote).toBe(false); + for (const [i, file] of files.entries()) { + expect(await fse.readFile(file, 'utf-8')).toBe(contents[i]); + expect(await mode(file)).toBe(0o600); + } + }); + + it('tightens a 0644 config whose kept entry holds an earlier resolved value', async () => { + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: SECRET_TOKEN\n'); + await writeMcpYaml(SECRET_SERVER); + await reconcileMcpForConfig(teamConfig, localConfig); + const userFile = path.join(homeDir, '.claude.json'); + await fse.chmod(userFile, 0o644); + vi.stubEnv('SECRET_TOKEN', undefined); + + await reconcileMcpForConfig(teamConfig, localConfig); + + expect((await fse.readJson(userFile)).mcpServers['with-secret']).toBeDefined(); + expect(await mode(userFile)).toBe(0o600); + }); + + it('keeps the mode of an unchanged config whose servers hold no resolved value', async () => { + await writeMcpYaml('servers:\n - name: open\n transport: http\n url: https://example.com/open\n'); + await reconcileMcpForConfig(teamConfig, localConfig); + const userFile = path.join(homeDir, '.claude.json'); + await fse.chmod(userFile, 0o644); + + await reconcileMcpForConfig(teamConfig, localConfig); + + expect(await mode(userFile)).toBe(0o644); + }); + + it('never lets the Codex config temp file be wider than 0600, and names it at random', async () => { + await fse.ensureDir(path.join(homeDir, '.codex', 'skills')); + await writeMcpYaml(`${SECRET_SERVER} tools: [codex]\n`); + const isCodexTemp = (file: string): boolean => path.basename(file).startsWith('config.toml.'); + const renamed: { file: string; mode: number }[] = []; + const rename = fs.promises.rename.bind(fs.promises); + vi.spyOn(fs.promises, 'rename').mockImplementation(async (from: fs.PathLike, to: fs.PathLike) => { + if (typeof from === 'string' && isCodexTemp(from)) renamed.push({ file: from, mode: await mode(from) }); + return rename(from, to); + }); + const created: number[] = []; + const writeFile = fs.promises.writeFile.bind(fs.promises); + vi.spyOn(fs.promises, 'writeFile').mockImplementation(async (...args: Parameters) => { + await writeFile(...args); + const [file] = args; + if (typeof file === 'string' && isCodexTemp(file)) created.push(await mode(file)); + }); + const configToml = path.join(homeDir, '.codex', 'config.toml'); + + await reconcileMcpForConfig(teamConfig, localConfig); + await writeCodexAtomic(configToml, 'model = "gpt-5"\n'); + + expect(created).toHaveLength(2); + expect(created.every((m) => (m & 0o077) === 0)).toBe(true); + expect(renamed.map((r) => path.basename(r.file))).toEqual([ + expect.stringMatching(/^config\.toml\.\d+\.[0-9a-f]{12}\.tmp$/), + expect.stringMatching(/^config\.toml\.\d+\.[0-9a-f]{12}\.tmp$/), + ]); + expect(renamed.every((r) => r.mode === 0o600)).toBe(true); + expect(await mode(configToml)).toBe(0o600); + }); + }); + // Verified against codex-cli 0.142.5: it speaks streamable HTTP. Secrets are // resolved to plaintext like every other tool — codex's env-var naming // (`bearer_token_env_var`) is not used, so the token is present regardless of @@ -1360,6 +3530,177 @@ servers: const oldFile = await fse.readJson(path.join(projectRoot, '.teamai', 'managed-mcp.json')); expect(oldFile['claude:project']).toBeUndefined(); }); + + // #875: the session-start pull inherits the agent's environment, which often + // lacks the member's shell export, so a declared secret is there for one + // pull and gone for the next. Removing the entry then would undo the pull + // that found it. + describe('a missing declared secret (#875)', () => { + const GITHUB = ` + - name: github + transport: http + url: https://api.example.com/mcp + headers: + Authorization: Bearer \${GITHUB_TOKEN} +`; + const DOCS = ` + - name: docs + transport: http + url: https://docs.example.com/mcp +`; + const claudeJson = () => path.join(homeDir, '.claude.json'); + const codexToml = () => path.join(homeDir, '.codex', 'config.toml'); + const manifestNames = async (key: string): Promise => { + const manifest = await fse.readJson(path.join(homeDir, '.teamai', 'managed-mcp.json')); + return (manifest[key] ?? []).map((r: { name: string }) => r.name).sort(); + }; + + beforeEach(async () => { + await fse.ensureDir(path.join(homeDir, '.codex', 'skills')); + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: GITHUB_TOKEN\n'); + }); + + it('keeps the entry an earlier pull wrote, in a JSON config and in Codex', async () => { + await writeMcpYaml(`servers:${GITHUB}`); + vi.stubEnv('GITHUB_TOKEN', 'first-token'); + await reconcileMcpForConfig(teamConfig, localConfig); + const jsonBefore = (await fse.readJson(claudeJson())).mcpServers.github; + const tomlBefore = await fse.readFile(codexToml(), 'utf-8'); + expect(tomlBefore).toContain('Bearer first-token'); + + vi.stubEnv('GITHUB_TOKEN', undefined); + const { changes } = await reconcileMcpForConfig(teamConfig, localConfig); + + expect(changes.filter((c) => c.action !== 'skipped')).toEqual([]); + expect((await fse.readJson(claudeJson())).mcpServers.github).toEqual(jsonBefore); + expect(await fse.readFile(codexToml(), 'utf-8')).toBe(tomlBefore); + }); + + it('keeps its ownership record when another server is written in the same pull', async () => { + await writeMcpYaml(`servers:${GITHUB}`); + vi.stubEnv('GITHUB_TOKEN', 'first-token'); + await reconcileMcpForConfig(teamConfig, localConfig); + + vi.stubEnv('GITHUB_TOKEN', undefined); + await writeMcpYaml(`servers:${GITHUB}${DOCS}`); + const { wrote } = await reconcileMcpForConfig(teamConfig, localConfig); + expect(wrote).toBe(true); + expect(await manifestNames('claude')).toEqual(['docs', 'github']); + expect(await manifestNames('codex')).toEqual(['docs', 'github']); + + // Still teamai's: a later pull that finds a new value updates it rather + // than treating it as a server the member added. + vi.stubEnv('GITHUB_TOKEN', 'second-token'); + const { changes } = await reconcileMcpForConfig(teamConfig, localConfig); + expect(changes.filter((c) => c.server === 'github').map((c) => `${c.tool}:${c.action}`).sort()) + .toEqual(['claude:updated', 'codex:updated', 'cursor:updated']); + expect((await fse.readJson(claudeJson())).mcpServers.github.headers.Authorization).toBe('Bearer second-token'); + expect(await fse.readFile(codexToml(), 'utf-8')).toContain('Bearer second-token'); + }); + + it('skips a server no pull has written yet', async () => { + vi.stubEnv('GITHUB_TOKEN', undefined); + await writeMcpYaml(`servers:${GITHUB}${DOCS}`); + + const { changes } = await reconcileMcpForConfig(teamConfig, localConfig); + + expect(changes.filter((c) => c.server === 'github').map((c) => `${c.tool}:${c.action}`).sort()) + .toEqual(['claude:skipped', 'codex:skipped', 'cursor:skipped']); + expect((await fse.readJson(claudeJson())).mcpServers.github).toBeUndefined(); + expect(await fse.readFile(codexToml(), 'utf-8')).not.toContain('[mcp_servers.github]'); + expect(await manifestNames('claude')).toEqual(['docs']); + }); + + it('removes a kept server once it leaves mcp.yaml', async () => { + await writeMcpYaml(`servers:${GITHUB}`); + vi.stubEnv('GITHUB_TOKEN', 'first-token'); + await reconcileMcpForConfig(teamConfig, localConfig); + vi.stubEnv('GITHUB_TOKEN', undefined); + await reconcileMcpForConfig(teamConfig, localConfig); + + await writeMcpYaml('servers: []\n'); + await reconcileMcpForConfig(teamConfig, localConfig); + + expect((await fse.readJson(claudeJson())).mcpServers.github).toBeUndefined(); + expect(await fse.readFile(codexToml(), 'utf-8')).not.toContain('[mcp_servers.github]'); + }); + + it('keeps every managed server as it is while the declarations cannot be read', async () => { + await writeMcpYaml(`servers:${GITHUB}${DOCS}`); + vi.stubEnv('GITHUB_TOKEN', 'first-token'); + await reconcileMcpForConfig(teamConfig, localConfig); + const jsonBefore = await fse.readFile(claudeJson(), 'utf-8'); + const tomlBefore = await fse.readFile(codexToml(), 'utf-8'); + + vi.stubEnv('GITHUB_TOKEN', undefined); + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets: [unclosed\n'); + await writeMcpYaml(`servers:${GITHUB}`); + const { changes, wrote, unresolved } = await reconcileMcpForConfig(teamConfig, localConfig); + + expect({ changes, wrote, unresolved }).toEqual({ changes: [], wrote: false, unresolved: true }); + expect(await fse.readFile(claudeJson(), 'utf-8')).toBe(jsonBefore); + expect(await fse.readFile(codexToml(), 'utf-8')).toBe(tomlBefore); + const { log } = await import('../utils/logger.js'); + expect(vi.mocked(log.warn).mock.calls.map(([m]) => String(m)).join('\n')).toContain('env/secrets.yaml'); + }); + + it('still removes a server whose missing variable is not declared as a secret', async () => { + await writeMcpYaml(` +servers: + - name: plain + transport: http + url: https://plain.example.com/mcp + headers: + X-Key: \${PLAIN_KEY} +`); + vi.stubEnv('PLAIN_KEY', 'k'); + await reconcileMcpForConfig(teamConfig, localConfig); + expect((await fse.readJson(claudeJson())).mcpServers.plain).toBeDefined(); + + vi.stubEnv('PLAIN_KEY', undefined); + await reconcileMcpForConfig(teamConfig, localConfig); + + expect((await fse.readJson(claudeJson())).mcpServers.plain).toBeUndefined(); + expect(await fse.readFile(codexToml(), 'utf-8')).not.toContain('[mcp_servers.plain]'); + }); + + it('still removes a server that also misses a variable not declared as a secret', async () => { + await writeMcpYaml(` +servers: + - name: both + transport: http + url: https://both.example.com/mcp + headers: + Authorization: Bearer \${GITHUB_TOKEN} + X-Key: \${PLAIN_KEY} + tools: [claude] +`); + vi.stubEnv('GITHUB_TOKEN', 't'); + vi.stubEnv('PLAIN_KEY', 'k'); + await reconcileMcpForConfig(teamConfig, localConfig); + + vi.stubEnv('GITHUB_TOKEN', undefined); + vi.stubEnv('PLAIN_KEY', undefined); + await reconcileMcpForConfig(teamConfig, localConfig); + + expect((await fse.readJson(claudeJson())).mcpServers.both).toBeUndefined(); + }); + + it('removeAll removes a kept server, while the declarations cannot be read too', async () => { + await writeMcpYaml(`servers:${GITHUB}`); + vi.stubEnv('GITHUB_TOKEN', 'first-token'); + await reconcileMcpForConfig(teamConfig, localConfig); + vi.stubEnv('GITHUB_TOKEN', undefined); + await reconcileMcpForConfig(teamConfig, localConfig); + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets: [unclosed\n'); + + const { changes } = await reconcileMcpForConfig(teamConfig, localConfig, { removeAll: true }); + + expect(changes.map((c) => `${c.tool}:${c.action}`).sort()).toEqual(['claude:removed', 'codex:removed', 'cursor:removed']); + expect((await fse.readJson(claudeJson())).mcpServers.github).toBeUndefined(); + expect(await fse.readFile(codexToml(), 'utf-8')).not.toContain('[mcp_servers.github]'); + }); + }); }); describe('MCP reconcile — OpenCode', () => { diff --git a/src/__tests__/mcp-resolved-files.test.ts b/src/__tests__/mcp-resolved-files.test.ts new file mode 100644 index 000000000..2845287bf --- /dev/null +++ b/src/__tests__/mcp-resolved-files.test.ts @@ -0,0 +1,276 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; + +vi.mock('../utils/logger.js', () => ({ + log: { debug: vi.fn(), error: vi.fn(), info: vi.fn(), success: vi.fn(), warn: vi.fn(), dim: vi.fn() }, +})); + +import { + readResolvedMcpFiles, + recordUnverifiedMcpServers, + resolvedMcpFilesPath, + settleResolvedMcpFiles, + trackResolvedMcpFiles, + untrackResolvedMcpFiles, +} from '../mcp-resolved-files.js'; +import { acquireLock, releaseLock } from '../update.js'; +import type { LocalConfig } from '../types.js'; + +/** The per-worktree record of the project MCP configs teamai wrote a resolved value to (#882). */ +describe('managed-mcp-files.json', () => { + let tmp: string; + let cfg: LocalConfig; + let sidecar: string; + const cursor = (): string => path.join(tmp, 'project', '.cursor', 'mcp.json'); + const custom = (): string => path.join(tmp, 'project', 'team', 'mcp.json'); + + beforeEach(async () => { + tmp = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-mcp-files-')); + cfg = { + repo: { localPath: path.join(tmp, 'team'), remote: 'r' }, + username: 'u', + scope: 'project', + projectRoot: path.join(tmp, 'project'), + dataHome: path.join(tmp, 'data'), + additionalRoles: [], + }; + sidecar = resolvedMcpFilesPath(cfg) ?? ''; + }); + + afterEach(async () => { + await fse.remove(tmp); + }); + + it('lives next to the worktree\'s managed-mcp.json', async () => { + const { managedMcpManifestPath } = await import('../types.js'); + expect(sidecar).toBe(path.join(path.dirname(managedMcpManifestPath(path.join(tmp, 'data'), path.join(tmp, 'project'))), 'managed-mcp-files.json')); + }); + + it.each([ + ['missing', null], + ['not JSON', '{ "version": 1, "files": '], + ['an array', '[]'], + ['without files', '{ "version": 1 }'], + ['a newer version', JSON.stringify({ version: 2, files: { '/x/mcp.json': { tools: ['claude'] } } })], + ])('reads as no files when it is %s', async (_label, content) => { + if (content !== null) await fse.outputFile(sidecar, content); + + expect(await readResolvedMcpFiles(cfg)).toEqual({ version: 1, files: {} }); + }); + + it('drops a relative path and an entry of the wrong shape, keeping the rest', async () => { + await fse.outputJson(sidecar, { + version: 1, + files: { + 'relative/mcp.json': { tools: ['cursor'] }, + [custom()]: { tools: 'cursor' }, + [path.join(tmp, 'project', '.mcp.json')]: { tools: ['claude'], unverified: ['jira', 3] }, + [cursor()]: { tools: ['cursor'], unverified: ['jira'] }, + }, + }); + + expect((await readResolvedMcpFiles(cfg)).files).toEqual({ [cursor()]: { tools: ['cursor'], unverified: ['jira'] } }); + }); + + it('records a file under the lock, 0600, and keeps fields it does not know', async () => { + await fse.outputJson(sidecar, { version: 1, note: 'kept', files: { [cursor()]: { tools: ['cursor'], since: 'kept' } } }); + + expect(await trackResolvedMcpFiles(cfg, [{ tool: 'claude', file: custom() }])).toBe('written'); + expect(await trackResolvedMcpFiles(cfg, [{ tool: 'claude', file: custom() }])).toBe('unchanged'); + + expect(await fse.readJson(sidecar)).toEqual({ + version: 1, + note: 'kept', + files: { [cursor()]: { tools: ['cursor'], since: 'kept' }, [custom()]: { tools: ['claude'] } }, + }); + expect((await fse.stat(sidecar)).mode & 0o777).toBe(0o600); + }); + + it('rewrites one that does not parse from empty', async () => { + await fse.outputFile(sidecar, '{ "version": 1, "files": '); + + expect(await trackResolvedMcpFiles(cfg, [{ tool: 'cursor', file: cursor() }])).toBe('written'); + + expect(await readResolvedMcpFiles(cfg)).toEqual({ version: 1, files: { [cursor()]: { tools: ['cursor'] } } }); + }); + + it('writes nothing while another command holds its lock', async () => { + await fse.ensureDir(path.dirname(sidecar)); + const lock = `${sidecar}.teamai-lock`; + expect(await acquireLock(lock)).toBe(true); + try { + expect(await trackResolvedMcpFiles(cfg, [{ tool: 'cursor', file: cursor() }])).toBe('locked'); + } finally { + await releaseLock(lock); + } + + expect(await fse.pathExists(sidecar)).toBe(false); + }); + + it('keeps every file five concurrent commands record', async () => { + const files = [0, 1, 2, 3, 4].map((i) => path.join(tmp, 'project', `tool-${i}`, 'mcp.json')); + + const results = await Promise.all(files.map((file) => trackResolvedMcpFiles(cfg, [{ tool: 'cursor', file }]))); + + expect(results).toEqual(['written', 'written', 'written', 'written', 'written']); + expect(Object.keys((await readResolvedMcpFiles(cfg)).files).sort()).toEqual([...files].sort()); + }); + + it('notes servers only for a file it already lists', async () => { + await trackResolvedMcpFiles(cfg, [{ tool: 'cursor', file: cursor() }]); + + await recordUnverifiedMcpServers(cfg, [{ file: cursor(), names: ['jira'] }, { file: custom(), names: ['mine'] }]); + + expect((await readResolvedMcpFiles(cfg)).files).toEqual({ [cursor()]: { tools: ['cursor'], unverified: ['jira'] } }); + }); + + it('takes back only the tool a record was added for, and the file with its last tool', async () => { + await trackResolvedMcpFiles(cfg, [{ tool: 'claude', file: custom() }, { tool: 'codebuddy', file: custom() }, { tool: 'cursor', file: cursor() }]); + + expect(await untrackResolvedMcpFiles(cfg, [{ tool: 'codebuddy', file: custom() }, { tool: 'cursor', file: cursor() }])).toBe('written'); + + expect((await readResolvedMcpFiles(cfg)).files).toEqual({ [custom()]: { tools: ['claude'] } }); + expect(await untrackResolvedMcpFiles(cfg, [{ tool: 'cursor', file: cursor() }])).toBe('unchanged'); + }); + + it('writes nothing to note when a file lists no servers', async () => { + await trackResolvedMcpFiles(cfg, [{ tool: 'cursor', file: cursor() }]); + + expect(await recordUnverifiedMcpServers(cfg, [{ file: cursor(), names: [] }])).toBe('unchanged'); + }); + + describe('settling it against what the files hold', () => { + beforeEach(async () => { + await fse.outputJson(sidecar, { + version: 1, + files: { + [cursor()]: { tools: ['cursor'], unverified: ['jira', 'mine', 'wiki'] }, + [custom()]: { tools: ['claude'] }, + }, + }); + }); + + it.each([ + ['is gone', { kind: 'missing' } as const], + ['holds no server', { kind: 'parsed', servers: [] } as const], + ])('forgets a file that %s', async (_label, state) => { + await settleResolvedMcpFiles(cfg, [{ file: custom(), tool: 'claude', state, holding: false, owned: [] }]); + + expect(Object.keys((await readResolvedMcpFiles(cfg)).files)).toEqual([cursor()]); + }); + + it('keeps a file that does not parse', async () => { + await settleResolvedMcpFiles(cfg, [{ file: custom(), tool: 'claude', state: { kind: 'unparsable' }, holding: false, owned: [] }]); + + expect(Object.keys((await readResolvedMcpFiles(cfg)).files)).toContain(custom()); + }); + + it('drops a noted server that left the file or that teamai owns again', async () => { + await settleResolvedMcpFiles(cfg, [ + { file: cursor(), tool: 'cursor', state: { kind: 'parsed', servers: ['mine', 'wiki'] }, holding: true, owned: ['wiki'] }, + ]); + + expect((await readResolvedMcpFiles(cfg)).files[cursor()]).toEqual({ tools: ['cursor'], unverified: ['mine'] }); + }); + + it('settles a file tools of different formats share on what all of them see, not each alone', async () => { + // Cursor sees no server under mcpServers and owns wiki there; OpenCode sees jira and wiki under mcp. + await settleResolvedMcpFiles(cfg, [ + { file: cursor(), tool: 'cursor', state: { kind: 'parsed', servers: ['wiki'] }, holding: false, owned: ['wiki'] }, + { file: cursor(), tool: 'opencode', state: { kind: 'parsed', servers: ['jira', 'wiki'] }, holding: true, owned: [] }, + ]); + expect((await readResolvedMcpFiles(cfg)).files[cursor()]).toEqual({ tools: ['cursor'], unverified: ['jira', 'wiki'] }); + + await settleResolvedMcpFiles(cfg, [ + { file: cursor(), tool: 'cursor', state: { kind: 'parsed', servers: [] }, holding: false, owned: [] }, + { file: cursor(), tool: 'opencode', state: { kind: 'parsed', servers: ['jira'] }, holding: true, owned: [] }, + ]); + expect((await readResolvedMcpFiles(cfg)).files[cursor()]).toEqual({ tools: ['cursor'], unverified: ['jira'] }); + }); + + it('lists a file holding a resolved value it did not know of', async () => { + const other = path.join(tmp, 'project', '.mcp.json'); + + await settleResolvedMcpFiles(cfg, [ + { file: other, tool: 'claude', state: { kind: 'parsed', servers: ['jira'] }, holding: true, owned: ['jira'] }, + ]); + + expect((await readResolvedMcpFiles(cfg)).files[other]).toEqual({ tools: ['claude'] }); + }); + + it('remembers that the files earlier teamai.yaml mappings reach were read, through later settles', async () => { + expect((await readResolvedMcpFiles(cfg)).earlierMappingsRead).toBeUndefined(); + + await settleResolvedMcpFiles(cfg, [], { earlierMappingsRead: true }); + await settleResolvedMcpFiles(cfg, [{ file: custom(), tool: 'claude', state: { kind: 'missing' }, holding: false, owned: [] }]); + + expect(await readResolvedMcpFiles(cfg)).toEqual({ version: 1, files: { [cursor()]: expect.anything() }, earlierMappingsRead: true }); + }); + + it('leaves a file it does not list alone when nothing holds a value there', async () => { + const other = path.join(tmp, 'project', '.mcp.json'); + + expect(await settleResolvedMcpFiles(cfg, [ + { file: other, tool: 'claude', state: { kind: 'parsed', servers: ['open'] }, holding: false, owned: ['open'] }, + ])).toBe('unchanged'); + }); + + it('records a file git tracks as tracked, and keeps it whatever it holds while git does', async () => { + const old = path.join(tmp, 'project', '.cursor', 'team-mcp.json'); + + await settleResolvedMcpFiles(cfg, [{ file: old, tool: 'cursor', state: { kind: 'parsed', servers: ['mine'] }, holding: false, owned: [], tracked: true }]); + await settleResolvedMcpFiles(cfg, [{ file: old, tool: 'cursor', state: { kind: 'missing' }, holding: false, owned: [], tracked: true }]); + + expect((await readResolvedMcpFiles(cfg)).files[old]).toEqual({ tools: ['cursor'], tracked: true }); + }); + + it('makes a tracked record an ordinary one once git no longer tracks the file', async () => { + await fse.outputJson(sidecar, { version: 1, files: { [custom()]: { tools: ['claude'], tracked: true } } }); + + await settleResolvedMcpFiles(cfg, [{ file: custom(), tool: 'claude', state: { kind: 'parsed', servers: ['jira'] }, holding: true, owned: [], tracked: false }]); + + expect((await readResolvedMcpFiles(cfg)).files[custom()]).toEqual({ tools: ['claude'] }); + }); + + it('takes a tool another now maps the file for off the record once it holds nothing of that tool\'s, and the file with its last one', async () => { + await fse.outputJson(sidecar, { version: 1, files: { [custom()]: { tools: ['claude', 'cursor'] }, [cursor()]: { tools: ['codebuddy'] } } }); + const state = { kind: 'parsed', servers: ['open'] } as const; + + await settleResolvedMcpFiles(cfg, [ + { file: custom(), tool: 'cursor', state, holding: false, owned: ['open'], remapped: true }, + { file: cursor(), tool: 'codebuddy', state, holding: false, owned: ['open'], remapped: true }, + ]); + expect(await settleResolvedMcpFiles(cfg, [ + { file: custom(), tool: 'claude', state, holding: true, owned: ['open'], remapped: true }, + ])).toBe('unchanged'); + + expect((await readResolvedMcpFiles(cfg)).files).toEqual({ [custom()]: { tools: ['claude'] } }); + }); + + it('adds a tool another now maps the file for to its record while it holds what teamai may have written for that tool', async () => { + await fse.outputJson(sidecar, { version: 1, files: { [custom()]: { tools: ['claude'] } } }); + const other = path.join(tmp, 'project', '.mcp.json'); + const state = { kind: 'parsed', servers: ['open', 'jira'] } as const; + + await settleResolvedMcpFiles(cfg, [ + { file: custom(), tool: 'cursor', state, holding: true, owned: ['open'], remapped: true }, + { file: other, tool: 'cursor', state, holding: true, owned: ['open'], remapped: true }, + ]); + + expect((await readResolvedMcpFiles(cfg)).files).toEqual({ [custom()]: { tools: ['claude', 'cursor'] }, [other]: { tools: ['cursor'] } }); + }); + + it('adds a tool git tracks the file for to its record, marked tracked, and forgets it only once git no longer tracks it', async () => { + await fse.outputJson(sidecar, { version: 1, files: { [custom()]: { tools: ['claude'] } } }); + + await settleResolvedMcpFiles(cfg, [{ file: custom(), tool: 'cursor', state: { kind: 'parsed', servers: ['jira'] }, holding: false, owned: [], tracked: true }]); + await settleResolvedMcpFiles(cfg, [{ file: custom(), tool: 'claude', state: { kind: 'missing' }, holding: false, owned: [] }]); + expect((await readResolvedMcpFiles(cfg)).files[custom()]).toEqual({ tools: ['claude', 'cursor'], tracked: true }); + + await settleResolvedMcpFiles(cfg, [{ file: custom(), tool: 'cursor', state: { kind: 'missing' }, holding: false, owned: [], tracked: false }]); + expect(Object.keys((await readResolvedMcpFiles(cfg)).files)).not.toContain(custom()); + }); + }); +}); diff --git a/src/__tests__/mcp-secrets.test.ts b/src/__tests__/mcp-secrets.test.ts new file mode 100644 index 000000000..b71736919 --- /dev/null +++ b/src/__tests__/mcp-secrets.test.ts @@ -0,0 +1,313 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; + +vi.mock('../utils/logger.js', () => ({ + log: { + debug: vi.fn(), error: vi.fn(), info: vi.fn(), success: vi.fn(), warn: vi.fn(), dim: vi.fn(), persist: vi.fn(), + }, +})); + +vi.mock('../config.js', async (importOriginal) => ({ + ...(await importOriginal()), + detectProjectConfig: vi.fn().mockResolvedValue(null), + requireInit: vi.fn(), +})); + +vi.mock('../utils/prompt.js', async (importOriginal) => ({ + ...(await importOriginal()), + readStdin: vi.fn(), +})); + +import { requireInit } from '../config.js'; +import { envSet, envUnset } from '../env-commands.js'; +import { readStdin } from '../utils/prompt.js'; +import { buildVarTable, reconcileMcpForConfig } from '../mcp-reconcile.js'; +import { resolvePlaceholders } from '../resources/mcp-format.js'; +import type { McpServerDef } from '../types.js'; +import { envShMarker } from '../env-sh-exports.js'; +import { getMachineSecretsPath, getTeamSecretsPath, writeSecretStore } from '../secret-store.js'; +import { log } from '../utils/logger.js'; +import { resetWarnOnce } from '../utils/warn-once.js'; +import type { LocalConfig, TeamaiConfig } from '../types.js'; + +/** + * `${VAR}` in mcp.yaml for a declared secret (#875): the member's value for + * this team, then their value for the machine, then their own environment + * (#879 Conflict 10), never the repo's + * env.yaml value for the same key. + */ +describe('MCP servers and declared secrets', () => { + let tmpDir: string; + let homeDir: string; + let repoPath: string; + let localConfig: LocalConfig; + const teamConfig: TeamaiConfig = { + team: 't', description: '', repo: 'r', provider: 'tgit', reviewers: [], + sharing: { skills: {}, rules: { enforced: [] }, docs: { localDir: '~/.teamai/docs' }, env: { injectShellProfile: false } }, + toolPaths: { claude: { skills: '.claude/skills', settings: '.claude/settings.json', mcp: '.claude.json', mcpProject: '.mcp.json' } }, + }; + + const write = (relativePath: string, content: string): Promise => + fse.outputFile(path.join(repoPath, ...relativePath.split('/')), content); + const githubAuthorization = async (): Promise => { + const file = path.join(homeDir, '.claude.json'); + if (!await fse.pathExists(file)) return undefined; + const config = await fse.readJson(file) as { mcpServers?: Record }> }; + return config.mcpServers?.github?.headers?.Authorization; + }; + + beforeEach(async () => { + resetWarnOnce(); + vi.mocked(log.warn).mockClear(); + tmpDir = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-mcp-secrets-')); + homeDir = path.join(tmpDir, 'home'); + repoPath = path.join(tmpDir, 'team-repo'); + await fse.ensureDir(path.join(homeDir, '.claude', 'skills')); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('USERPROFILE', homeDir); + vi.stubEnv('GITHUB_TOKEN', undefined); + localConfig = { repo: { localPath: repoPath, remote: 'r' }, username: 'u', scope: 'user', additionalRoles: [] }; + await write('mcp/mcp.yaml', [ + 'servers:', + ' - name: github', + ' transport: http', + ' url: https://api.example.com/mcp/', + ' headers:', + ' Authorization: Bearer ${GITHUB_TOKEN}', + ].join('\n')); + await write('env/secrets.yaml', 'secrets:\n - key: GITHUB_TOKEN\n'); + }); + afterEach(async () => { + vi.unstubAllEnvs(); + await fse.remove(tmpDir); + }); + + it('gives a server the team value over an exported one', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'team-token' } }); + vi.stubEnv('GITHUB_TOKEN', 'exported-token'); + + await reconcileMcpForConfig(teamConfig, localConfig); + + expect(await githubAuthorization()).toBe('Bearer team-token'); + }); + + it('gives a server the machine value when the team has none, over an exported one', async () => { + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'machine-token' } }); + vi.stubEnv('GITHUB_TOKEN', 'exported-token'); + + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await githubAuthorization()).toBe('Bearer machine-token'); + + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'team-token' } }); + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await githubAuthorization()).toBe('Bearer team-token'); + }); + + // What a member runs: set a value, pull, unset it with nothing exported, pull. + it('keeps the entry and still owns it after env unset, so a later value replaces it', async () => { + vi.mocked(requireInit).mockResolvedValue({ localConfig, teamConfig }); + vi.mocked(readStdin).mockResolvedValueOnce('first-token').mockResolvedValueOnce('second-token'); + + await envSet('GITHUB_TOKEN', { stdin: true }); + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await githubAuthorization()).toBe('Bearer first-token'); + + await envUnset('GITHUB_TOKEN', {}); + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await githubAuthorization()).toBe('Bearer first-token'); + + await envSet('GITHUB_TOKEN', { stdin: true }); + const { changes } = await reconcileMcpForConfig(teamConfig, localConfig); + expect(await githubAuthorization()).toBe('Bearer second-token'); + expect(changes.filter((change) => change.server === 'github').map((change) => change.action)).not.toContain('skipped'); + }); + + it("uses the member's own export when no team value is set", async () => { + vi.stubEnv('GITHUB_TOKEN', 'exported-token'); + + await reconcileMcpForConfig(teamConfig, localConfig); + + expect(await githubAuthorization()).toBe('Bearer exported-token'); + }); + + it("does not use a value another scope's env.sh exported", async () => { + await fse.outputFile(path.join(homeDir, '.teamai', 'projects', 'other-abc', 'env.sh'), "export GITHUB_TOKEN='other-team-token'\n"); + vi.stubEnv('GITHUB_TOKEN', 'other-team-token'); + + await reconcileMcpForConfig(teamConfig, localConfig); + + expect(await githubAuthorization()).toBeUndefined(); + }); + + it('does not use a value an env.sh no scan finds exported, and gives no server the marker that says so', async () => { + const marker = envShMarker(path.join(tmpDir, 'elsewhere', '.teamai'), [['GITHUB_TOKEN', 'project-a-token']]); + expect(marker).not.toBeNull(); + const [name, digests] = marker ?? ['', '']; + vi.stubEnv('GITHUB_TOKEN', 'project-a-token'); + vi.stubEnv(name, digests); + + await reconcileMcpForConfig(teamConfig, localConfig); + + expect(await githubAuthorization()).toBeUndefined(); + expect(Object.hasOwn(await buildVarTable(localConfig), name)).toBe(false); + }); + + it("ignores the env.yaml value of a key declared as a secret, and the shell's copy of it", async () => { + await write('env/env.yaml', 'variables:\n - key: GITHUB_TOKEN\n value: repo-token\n - key: API_URL\n value: u\n'); + vi.stubEnv('GITHUB_TOKEN', 'repo-token'); + + const vars = await buildVarTable(localConfig); + + expect(vars.GITHUB_TOKEN).toBeUndefined(); + expect(vars.API_URL).toBe('u'); + }); + + // #879: secrets and variable overrides share the team store; an entry is used only as the kind it was set as. + it('gives a server the env.yaml value of a former secret, never its stored value, and a secret never a variable override', async () => { + await write('env/env.yaml', 'variables:\n - key: API_URL\n value: team-url\n'); + await writeSecretStore(getTeamSecretsPath(localConfig), { + API_URL: { value: 'fixture-old-secret' }, + GITHUB_TOKEN: { value: 'fixture-override', kind: 'variable' }, + }); + vi.stubEnv('GITHUB_TOKEN', 'exported-token'); + + expect((await buildVarTable(localConfig)).API_URL).toBe('team-url'); + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await githubAuthorization()).toBe('Bearer exported-token'); + }); + + it('on Windows, fills ${token} with the value of TOKEN, the same environment variable', () => { + const def = { name: 'github', transport: 'http', url: 'https://api.example.com/mcp', headers: { Authorization: 'Bearer ${github_token}' } } as McpServerDef; + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + let resolved: ReturnType; + try { + resolved = resolvePlaceholders(def, { GITHUB_TOKEN: 'fixture-token' }); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(resolved.missing).toEqual([]); + expect(resolved.def.headers?.Authorization).toBe('Bearer fixture-token'); + expect(resolvePlaceholders(def, { GITHUB_TOKEN: 'fixture-token' }).missing).toEqual(['github_token']); + }); + + it('on Windows, never lets an inherited value under another case of a team variable\'s name in', async () => { + await write('env/env.yaml', 'variables:\n - key: API_URL\n value: team-url\n'); + vi.stubEnv('api_url', 'exported-url'); + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + let vars: Record; + try { + vars = await buildVarTable(localConfig); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(vars.API_URL).toBe('team-url'); + expect(Object.hasOwn(vars, 'api_url')).toBe(false); + }); + + // #875 (#879 S9): one order for a variable, member team value > env.yaml, + // with no environment override and no machine value. + it("resolves a variable from the member's value for this team, then env.yaml, never the environment", async () => { + await write('env/env.yaml', 'variables:\n - key: API_URL\n value: team-url\n'); + vi.stubEnv('API_URL', 'exported-url'); + vi.stubEnv('UNRELATED_URL', 'exported-unrelated'); + vi.stubEnv('MY_API_URL', undefined); + await writeSecretStore(getMachineSecretsPath(), { API_URL: { value: 'machine-url' } }); + + let vars = await buildVarTable(localConfig); + expect(vars.API_URL).toBe('team-url'); + expect(vars.UNRELATED_URL).toBe('exported-unrelated'); + + await writeSecretStore(getTeamSecretsPath(localConfig), { API_URL: { value: 'member-url', kind: 'variable' } }); + expect((await buildVarTable(localConfig)).API_URL).toBe('member-url'); + + await writeSecretStore(getTeamSecretsPath(localConfig), { API_URL: { env: 'MY_API_URL', kind: 'variable' } }); + expect((await buildVarTable(localConfig)).API_URL).toBe('team-url'); + vi.stubEnv('MY_API_URL', 'member-env-url'); + vars = await buildVarTable(localConfig); + expect(vars.API_URL).toBe('member-env-url'); + }); + + // `__proto__` is a valid env key: an ordinary object's inherited setter + // would swallow it, and a missing one would read as Object.prototype. + describe('a key named __proto__', () => { + const protoAuthorization = async (): Promise => { + const file = path.join(homeDir, '.claude.json'); + if (!await fse.pathExists(file)) return undefined; + const config = await fse.readJson(file) as { mcpServers?: Record }> }; + return config.mcpServers?.proto?.headers?.Authorization; + }; + beforeEach(async () => { + await write('mcp/mcp.yaml', [ + 'servers:', + ' - name: proto', + ' transport: http', + ' url: https://api.example.com/mcp/', + ' headers:', + ' Authorization: Bearer ${__proto__}', + ].join('\n')); + }); + + it('delivers a declared secret named __proto__, and leaves it unresolved when it has no value', async () => { + await write('env/secrets.yaml', 'secrets:\n - key: __proto__\n'); + + let vars = await buildVarTable(localConfig); + expect(Object.hasOwn(vars, '__proto__')).toBe(false); + expect(vars['__proto__']).toBeUndefined(); + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await protoAuthorization()).toBeUndefined(); + + await writeSecretStore(getTeamSecretsPath(localConfig), { ['__proto__']: { value: 'proto-secret' } }); + vars = await buildVarTable(localConfig); + expect(Object.hasOwn(vars, '__proto__')).toBe(true); + expect(vars['__proto__']).toBe('proto-secret'); + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await protoAuthorization()).toBe('Bearer proto-secret'); + }); + + it('delivers a variable named __proto__, and leaves it unresolved when nothing sets it', async () => { + let vars = await buildVarTable(localConfig); + expect(vars['__proto__']).toBeUndefined(); + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await protoAuthorization()).toBeUndefined(); + + await write('env/env.yaml', 'variables:\n - key: __proto__\n value: proto-team\n'); + vars = await buildVarTable(localConfig); + expect(vars['__proto__']).toBe('proto-team'); + + vi.mocked(requireInit).mockResolvedValue({ localConfig, teamConfig }); + vi.mocked(readStdin).mockResolvedValueOnce('proto-member'); + await envSet('__proto__', { stdin: true }); + expect((await buildVarTable(localConfig))['__proto__']).toBe('proto-member'); + await reconcileMcpForConfig(teamConfig, localConfig); + expect(await protoAuthorization()).toBe('Bearer proto-member'); + }); + }); + + it('keeps the variables the last pull wrote when the store cannot be read, without the value', async () => { + await write('env/env.yaml', 'variables:\n - key: API_URL\n value: team-url\n'); + await fse.outputFile(path.join(homeDir, '.teamai', 'env'), 'API_URL=member-url\n'); + await fse.outputFile(getTeamSecretsPath(localConfig), '{"API_URL": {"value": fixture_member_url}}'); + vi.stubEnv('API_URL', 'exported-url'); + + expect((await buildVarTable(localConfig)).API_URL).toBe('member-url'); + const warnings = vi.mocked(log.warn).mock.calls.map((call) => String(call[0])).join('\n'); + expect(warnings).toContain(`${getTeamSecretsPath(localConfig)} is not valid JSON`); + expect(warnings).not.toContain('fixture_member_url'); + }); + + it('warns without the value when the store cannot be read, and resolves the secret to nothing', async () => { + await fse.outputFile(getTeamSecretsPath(localConfig), '{"GITHUB_TOKEN": {"value": ghp_fixture_value}}'); + vi.stubEnv('GITHUB_TOKEN', 'exported-token'); + + const vars = await buildVarTable(localConfig); + + expect(vars.GITHUB_TOKEN).toBeUndefined(); + const warnings = vi.mocked(log.warn).mock.calls.map((call) => String(call[0])); + expect(warnings).toEqual([expect.stringContaining(`${getTeamSecretsPath(localConfig)} is not valid JSON.`)]); + expect(warnings.join('\n')).not.toContain('ghp_fixture_value'); + }); +}); diff --git a/src/__tests__/models-cmd.test.ts b/src/__tests__/models-cmd.test.ts index d48a06f51..24618d0a1 100644 --- a/src/__tests__/models-cmd.test.ts +++ b/src/__tests__/models-cmd.test.ts @@ -1,6 +1,7 @@ import fse from 'fs-extra'; import os from 'node:os'; import path from 'node:path'; +import { Readable } from 'node:stream'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { modelsAdd, modelsConfigure, modelsList, modelsRemove, modelsRestore, modelsSwitch } from '../models-cmd.js'; import { getLocalValuesPath, loadLocalProfiles, loadModelInputs } from '../models/profile.js'; @@ -128,6 +129,27 @@ describe('models commands', () => { expect((await loadLocalProfiles()).profiles[0].name).toBe('Renamed'); }); + it('stores a key piped with --api-key-stdin and refuses a terminal or empty input', async () => { + const original = Object.getOwnPropertyDescriptor(process, 'stdin'); + const pipeStdin = (chunks: string[], isTTY?: true) => Object.defineProperty(process, 'stdin', { + value: Object.assign(Readable.from(chunks), { isTTY }), configurable: true, + }); + try { + await addMine(); + pipeStdin(['sk-pi', 'ped\r\n']); + await modelsConfigure('local:mine', { apiKeyStdin: true }); + expect(Object.values(await loadModelInputs(getLocalValuesPath())).map((entry) => entry.API_KEY)) + .toEqual([{ value: 'sk-piped' }]); + + pipeStdin([], true); + await expect(modelsConfigure('local:mine', { apiKeyStdin: true })).rejects.toThrow('--api-key-stdin expects piped stdin'); + pipeStdin(['\n']); + await expect(modelsConfigure('local:mine', { apiKeyStdin: true })).rejects.toThrow('No API key was provided on stdin'); + } finally { + if (original) Object.defineProperty(process, 'stdin', original); + } + }); + it('switches every compatible installed agent by default and lists where a profile is active', async () => { await fse.outputJson(path.join(home, '.claude', 'settings.json'), {}); await fse.outputJson(path.join(home, '.codebuddy', 'models.json'), { models: [] }); diff --git a/src/__tests__/pull-env-advisories.test.ts b/src/__tests__/pull-env-advisories.test.ts new file mode 100644 index 000000000..867bfa96a --- /dev/null +++ b/src/__tests__/pull-env-advisories.test.ts @@ -0,0 +1,312 @@ +/** + * #875 (#879 S6): an interactive pull names a declared secret with no value, + * the server that needs it and the command that fixes it; the silent + * session-start pull prints nothing. Harness from pull-env-shape-warning.test.ts. + */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; + +vi.mock('../config.js', async (importOriginal) => ({ + ...(await importOriginal()), + detectProjectConfig: vi.fn().mockResolvedValue(null), + loadLocalConfigForScope: vi.fn(), + loadStateForScope: vi.fn().mockResolvedValue({ lastPull: null, lastPullRev: null }), + loadTeamConfig: vi.fn(), + requireInit: vi.fn(), + saveStateForScope: vi.fn(), +})); + +vi.mock('../utils/git.js', () => ({ + getHeadRev: vi.fn().mockResolvedValue('abc1234'), + pullRepo: vi.fn().mockResolvedValue('already up to date'), +})); + +vi.mock('../utils/logger.js', () => ({ + log: { + debug: vi.fn(), error: vi.fn(), info: vi.fn(), success: vi.fn(), warn: vi.fn(), dim: vi.fn(), persist: vi.fn(), + }, + spinner: vi.fn(() => ({ + fail: vi.fn().mockReturnThis(), info: vi.fn().mockReturnThis(), + start: vi.fn().mockReturnThis(), stop: vi.fn().mockReturnThis(), + succeed: vi.fn().mockReturnThis(), warn: vi.fn().mockReturnThis(), + })), +})); + +vi.mock('../roles.js', () => ({ + loadRolesManifest: vi.fn().mockResolvedValue({ + version: 1, + roles: [{ + id: 'dev', + name: 'Dev', + description: '', + resources: { knowledge: ['common'], skills: ['common'], learnings: ['common'], agents: [] }, + }], + defaults: { shareTarget: 'primary-role' }, + }), + resolveRoleResourceNamespaces: vi.fn(() => ({ + knowledge: ['common'], skills: ['common'], learnings: ['common'], agents: [], + })), +})); + +// Isolation: pull() takes a real ~/.teamai/.sync-lock. Parallel vitest workers +// sharing that path race and skip/error, so these tests mock the lock. +vi.mock('../update.js', () => ({ + acquireLock: vi.fn().mockResolvedValue(true), + releaseLock: vi.fn().mockResolvedValue(undefined), +})); + +// Counts the value store reads, for the one-resolution-per-pull test. +vi.mock('../secret-store.js', async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, readSecretStore: vi.fn(actual.readSecretStore) }; +}); + +// The end-of-pull checks are exercised in pull-post-checks.test.ts; keep them +// out of the way here so a warning under test is the only thing on the wire. +vi.mock('../doctor.js', async (importOriginal) => ({ + ...await importOriginal(), + resolveDoctorContext: vi.fn(), + buildChecks: vi.fn(), +})); + +import { detectProjectConfig, loadLocalConfigForScope, loadStateForScope, loadTeamConfig } from '../config.js'; +import { acquireLock } from '../update.js'; +import { buildChecks, resolveDoctorContext, type DoctorContext } from '../doctor.js'; +import { log } from '../utils/logger.js'; +import { pull } from '../pull.js'; +import type { LocalConfig, TeamaiConfig } from '../types.js'; +import { resetWarnOnce } from '../utils/warn-once.js'; +import { getMachineSecretsPath, getTeamSecretsPath, readSecretStore, writeSecretStore } from '../secret-store.js'; +import { secretsEntryReader } from '../resources/secrets.js'; + +const GITHUB_LINE = 'github: GITHUB_TOKEN is not set. Run `teamai env set GITHUB_TOKEN` (https://github.com/settings/tokens).'; +const KEPT_LINE = 'github: the entry an earlier pull wrote stays in claude and may hold an old GITHUB_TOKEN until a pull finds its value.'; + +describe('pull advisories for team secrets', () => { + let tempDir: string; + let homeDir: string; + let repoPath: string; + let scopeConfig: LocalConfig; + + const write = (relativePath: string, content: string): Promise => + fse.outputFile(path.join(repoPath, ...relativePath.split('/')), content); + const warned = (): string[] => vi.mocked(log.warn).mock.calls.map(([message]) => String(message)); + /** Every printed line; debug.log still records the skip reason, as before. */ + const printed = (): string[] => (['warn', 'info', 'dim', 'success', 'error'] as const) + .flatMap((level) => vi.mocked(log[level]).mock.calls.map(([message]) => String(message))); + + beforeEach(async () => { + resetWarnOnce(); + tempDir = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-pull-env-advisories-')); + homeDir = path.join(tempDir, 'home'); + repoPath = path.join(tempDir, 'team-repo'); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('USERPROFILE', homeDir); + vi.stubEnv('GITHUB_TOKEN', undefined); + vi.stubEnv('GITLAB_TOKEN', undefined); + vi.stubEnv('GITLAB_HOST', undefined); + vi.stubEnv('MY_GITLAB_HOST', undefined); + + await write('skills/common/kept-skill/SKILL.md', '---\nname: kept-skill\ndescription: kept\n---\n'); + await write('manifest/roles.yaml', 'version: 1\n'); + await write('mcp/mcp.yaml', [ + 'servers:', + ' - name: github', + ' transport: http', + ' url: https://api.example.com/mcp/', + ' headers:', + ' Authorization: Bearer ${GITHUB_TOKEN}', + ].join('\n')); + await write('env/secrets.yaml', 'secrets:\n - key: GITHUB_TOKEN\n url: https://github.com/settings/tokens\n'); + await fse.ensureDir(path.join(homeDir, '.claude', 'skills')); + + const localConfig: LocalConfig = { + repo: { localPath: repoPath, remote: 'owner/repo' }, + username: 'tester', + scope: 'user', + primaryRole: 'dev', + additionalRoles: [], + }; + scopeConfig = localConfig; + const teamConfig: TeamaiConfig = { + team: 'test', + description: '', + repo: 'owner/repo', + provider: 'github', + reviewers: [], + sharing: { + skills: {}, rules: { enforced: [] }, docs: { localDir: '' }, env: { injectShellProfile: false }, + }, + toolPaths: { claude: { skills: '.claude/skills', mcp: '.claude.json' } }, + }; + + vi.mocked(detectProjectConfig).mockResolvedValue(null); + vi.mocked(loadLocalConfigForScope).mockResolvedValue(localConfig); + vi.mocked(loadTeamConfig).mockResolvedValue(teamConfig); + vi.mocked(loadStateForScope).mockResolvedValue({ lastPull: null, lastPullRev: null } as never); + const ctx: DoctorContext = { + localConfig, + teamConfig, + toolPaths: teamConfig.toolPaths, + hookToolPaths: teamConfig.toolPaths, + baseDir: homeDir, + }; + vi.mocked(resolveDoctorContext).mockResolvedValue(ctx); + vi.mocked(buildChecks).mockResolvedValue([]); + vi.mocked(acquireLock).mockResolvedValue(true); + }); + + afterEach(async () => { + vi.unstubAllEnvs(); + vi.clearAllMocks(); + await fse.remove(tempDir); + }); + + it('names the server, the key, the command and the url', async () => { + await pull({ force: true }); + + expect(warned()).toContain(GITHUB_LINE); + }); + + it('names a secret no MCP server uses, with no mcp.yaml', async () => { + await fse.remove(path.join(repoPath, 'mcp')); + await write('env/secrets.yaml', 'secrets:\n - key: GITLAB_TOKEN\n'); + + await pull({ force: true }); + + expect(warned()).toContain('GITLAB_TOKEN is not set. Run `teamai env set GITLAB_TOKEN`.'); + }); + + // One resolution per scope serves env.sh, the MCP reconcile and the advisories. + it('reads env/secrets.yaml and each value store once', async () => { + await write('env/env.yaml', 'variables:\n - key: API_URL\n value: u\n'); + const secretsRead = vi.spyOn(secretsEntryReader, 'read'); + vi.mocked(readSecretStore).mockClear(); + + await pull({ force: true }); + + expect(warned()).toContain(GITHUB_LINE); + expect(secretsRead.mock.calls.map(([, relativePath]) => relativePath)).toEqual(['env/secrets.yaml']); + expect(vi.mocked(readSecretStore).mock.calls.map(([file]) => file).sort()) + .toEqual([getMachineSecretsPath(), getTeamSecretsPath(scopeConfig)].sort()); + secretsRead.mockRestore(); + }); + + it('prints nothing about it on a silent pull', async () => { + await pull({ force: true, silent: true }); + + expect(printed().some((message) => message.includes('GITHUB_TOKEN'))).toBe(false); + }); + + it('says a kept entry may hold an old value, and that pull did write none of it', async () => { + vi.stubEnv('GITHUB_TOKEN', 'shell-token'); + await pull({ force: true }); + const claudeJson = await fse.readFile(path.join(homeDir, '.claude.json'), 'utf8'); + expect(claudeJson).toContain('Bearer shell-token'); + + vi.stubEnv('GITHUB_TOKEN', undefined); + vi.mocked(log.warn).mockClear(); + resetWarnOnce(); + await pull({ force: true }); + + expect(warned()).toEqual(expect.arrayContaining([GITHUB_LINE, KEPT_LINE])); + expect(await fse.readFile(path.join(homeDir, '.claude.json'), 'utf8')).toBe(claudeJson); + expect(printed().some((message) => message.includes('shell-token'))).toBe(false); + }); + + it('warns about a key declared as a secret and also set in env.yaml', async () => { + await write('env/env.yaml', 'variables:\n - key: GITHUB_TOKEN\n value: repo-token\n'); + + await pull({ force: true }); + + expect(warned()).toContain( + 'GITHUB_TOKEN is a team secret and is also set in env/env.yaml, whose value is ignored. ' + + 'Remove it from env/env.yaml and run `teamai push`.', + ); + expect(printed().some((message) => message.includes('repo-token'))).toBe(false); + }); + + it('says nothing once the secret has a value', async () => { + vi.stubEnv('GITHUB_TOKEN', 'shell-token'); + + await pull({ force: true }); + + expect(warned().some((message) => message.includes('GITHUB_TOKEN'))).toBe(false); + }); + + // #875 (#879 S9): the environment no longer overrides a plain variable; pull + // says so, and env.sh exports the member's value for this team. + describe('a plain variable the environment no longer overrides', () => { + const IGNORED_LINE = 'GITLAB_HOST in your environment differs from the value in env/env.yaml, which this team uses. ' + + 'To use yours for this team, run `teamai env set GITLAB_HOST`.'; + const envSh = (): Promise => fse.readFile(path.join(homeDir, '.teamai', 'env.sh'), 'utf8'); + + beforeEach(async () => { + await write('env/env.yaml', 'variables:\n - key: GITLAB_HOST\n value: gitlab.team.example\n - key: API_URL\n value: https://team.example\n'); + }); + + it('tells the member to run env set when it ignores a differing export, and never prints either value', async () => { + vi.stubEnv('GITLAB_HOST', 'gitlab.dave.example'); + + await pull({ force: true }); + + expect(warned()).toContain(IGNORED_LINE); + expect(warned().filter((message) => message.includes('--from-env'))).toEqual([]); + expect(printed().some((message) => message.includes('gitlab.dave.example') || message.includes('gitlab.team.example'))).toBe(false); + expect(await envSh()).toContain("export GITLAB_HOST='gitlab.team.example'"); + }); + + it('says nothing on a silent pull', async () => { + vi.stubEnv('GITLAB_HOST', 'gitlab.dave.example'); + + await pull({ force: true, silent: true }); + + expect(printed().some((message) => message.includes('GITLAB_HOST'))).toBe(false); + }); + + it("says nothing for an export that is another scope's env.sh value, or equals the team's", async () => { + await fse.outputFile(path.join(homeDir, '.teamai', 'projects', 'other-abc', 'env.sh'), "export GITLAB_HOST='gitlab.other.example'\n"); + vi.stubEnv('GITLAB_HOST', 'gitlab.other.example'); + vi.stubEnv('API_URL', 'https://team.example'); + + await pull({ force: true }); + + expect(printed().some((message) => message.includes('GITLAB_HOST') || message.includes('API_URL'))).toBe(false); + }); + + // #879: a secret's value stays one after the key stops being a secret; an entry without a kind counts as one. + it('exports the env.yaml value, never a value stored while the key was a secret, in env.sh and the env backup', async () => { + await writeSecretStore(getTeamSecretsPath(scopeConfig), { + GITLAB_HOST: { value: 'fixture-old-secret' }, + API_URL: { value: 'fixture-old-secret-2', kind: 'secret' }, + }); + + await pull({ force: true }); + + const exported = await envSh(); + expect(exported).toContain("export GITLAB_HOST='gitlab.team.example'"); + expect(exported).toContain("export API_URL='https://team.example'"); + const backup = await fse.readFile(path.join(homeDir, '.teamai', 'env'), 'utf8'); + expect(`${exported}${backup}${printed().join('\n')}`).not.toContain('fixture-old-secret'); + }); + + it("exports the member's literal value in env.sh, leaves out a --from-env one, and says nothing then", async () => { + await writeSecretStore(getTeamSecretsPath(scopeConfig), { + GITLAB_HOST: { value: 'gitlab.dave.example', kind: 'variable' }, + API_URL: { env: 'MY_API_URL', kind: 'variable' }, + }); + vi.stubEnv('GITLAB_HOST', 'gitlab.other.example'); + vi.stubEnv('MY_API_URL', 'https://mine.example'); + + await pull({ force: true }); + + const exported = await envSh(); + expect(exported).toContain("export GITLAB_HOST='gitlab.dave.example'"); + expect(exported).not.toContain('API_URL'); + expect(await fse.readFile(path.join(homeDir, '.teamai', 'env'), 'utf8')).toBe('GITLAB_HOST=gitlab.dave.example\n'); + expect(printed().some((message) => message.includes('GITLAB_HOST') || message.includes('API_URL'))).toBe(false); + }); + }); +}); diff --git a/src/__tests__/pull-env-shape-warning.test.ts b/src/__tests__/pull-env-shape-warning.test.ts index 042fabca2..f21132f6f 100644 --- a/src/__tests__/pull-env-shape-warning.test.ts +++ b/src/__tests__/pull-env-shape-warning.test.ts @@ -180,6 +180,50 @@ describe('env.yaml shape warning on a real pull', () => { expect(log.warn).not.toHaveBeenCalledWith(expect.stringContaining(SHAPE_WARNING)); }); + // #875 (#879 Conflict 14): a failed declaration is never "no secrets", so env.sh stays as it is. + it('warns about a secrets file that does not parse, in secret wording, and leaves env.sh and the backup as they are', async () => { + await fse.outputFile(path.join(repoPath, 'env', 'env.yaml'), 'variables:\n - key: FOO\n value: bar\n'); + await pull({ force: true }); + const envSh = await fse.readFile(path.join(homeDir, '.teamai', 'env.sh'), 'utf8'); + const backup = await fse.readFile(path.join(homeDir, '.teamai', 'env'), 'utf8'); + + await fse.outputFile(path.join(repoPath, 'env', 'env.yaml'), 'variables:\n - key: FOO\n value: changed\n'); + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secret:\n - key: GITHUB_TOKEN\n'); + await pull({ force: true }); + + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining( + 'env/secrets.yaml declares no secrets: it has no top-level `secrets:` key, only `secret`. ' + + 'Team secrets were not resolved this run; env variables and MCP servers stay as they are.', + )); + expect(await fse.readFile(path.join(homeDir, '.teamai', 'env.sh'), 'utf8')).toBe(envSh); + expect(await fse.readFile(path.join(homeDir, '.teamai', 'env'), 'utf8')).toBe(backup); + }); + + // #875 (#879 Conflict 13): a key declared as a secret and set in env.yaml resolves as the secret. + it.each([ + ['a full pull', false], + ['the unchanged-rev fast path', true], + ])('leaves the env.yaml value of a key declared as a secret out of env.sh and the backup, on %s', async (_name, fastPath) => { + await fse.outputFile( + path.join(repoPath, 'env', 'env.yaml'), + 'variables:\n - key: FOO\n value: bar\n - key: GITHUB_TOKEN\n value: repo-token\n', + ); + await pull({}); + expect(await fse.readFile(path.join(homeDir, '.teamai', 'env.sh'), 'utf8')).toContain('repo-token'); + + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: GITHUB_TOKEN\n'); + vi.mocked(log.success).mockClear(); + await pull(fastPath ? {} : { force: true }); + + if (fastPath) expect(log.success).toHaveBeenCalledWith(expect.stringContaining('Already synced')); + const envSh = await fse.readFile(path.join(homeDir, '.teamai', 'env.sh'), 'utf8'); + const backup = await fse.readFile(path.join(homeDir, '.teamai', 'env'), 'utf8'); + expect(envSh).toContain("export FOO='bar'"); + expect(backup).toContain('FOO=bar'); + expect(envSh).not.toContain('repo-token'); + expect(backup).not.toContain('repo-token'); + }); + it('warns from the unchanged-rev fast path too', async () => { // The machine pulled once while the CLI still accepted a bad shape, so it // stored the rev. The repo has not moved since — every later pull takes the diff --git a/src/__tests__/push-env.test.ts b/src/__tests__/push-env.test.ts index 2dead29b1..e6d113dcb 100644 --- a/src/__tests__/push-env.test.ts +++ b/src/__tests__/push-env.test.ts @@ -159,6 +159,23 @@ describe('push publishes the env files env add leaves in a standalone clone (#88 expect(pushed).toContain('value: changed'); }); + // #879: `env add --secret` leaves env/secrets.yaml for push the same way. + it('pushes the secrets.yaml that env add --secret creates', async () => { + const { envAdd } = await import('../env-commands.js'); + const { push } = await import('../push.js'); + await envAdd('GITHUB_TOKEN', undefined, { secret: true }); + expect(await simpleGit(teamRepo).raw(['status', '--porcelain', '--untracked-files=all'])).toContain('env/secrets.yaml'); + + await push({ all: true }); + + expect(process.exitCode).toBe(previousExitCode); + const [branch] = await pushBranches(remote); + expect(branch).toBeDefined(); + const pushed = await simpleGit(remote).show([`${branch}:env/secrets.yaml`]); + expect(pushed).toContain('key: GITHUB_TOKEN'); + expect(pushed).not.toContain('value'); + }); + it('keeps the env.yaml edit when the refresh fails after the reset', async () => { const { envAdd } = await import('../env-commands.js'); const { push } = await import('../push.js'); diff --git a/src/__tests__/secret-values.test.ts b/src/__tests__/secret-values.test.ts new file mode 100644 index 000000000..bd4e78541 --- /dev/null +++ b/src/__tests__/secret-values.test.ts @@ -0,0 +1,562 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; + +vi.mock('../utils/logger.js', () => ({ + log: { + debug: vi.fn(), error: vi.fn(), info: vi.fn(), success: vi.fn(), warn: vi.fn(), dim: vi.fn(), persist: vi.fn(), + }, +})); + +import YAML from 'yaml'; +import { EnvHandler, parseEnvFile, type EnvVariable } from '../resources/env.js'; +import { resolveTeamEnv, secretState, type SecretValue, type StoreResolution, type TeamEnv } from '../env-resolution.js'; +import { getMachineSecretsPath, getTeamSecretsPath, readSecretStore, writeSecretStore } from '../secret-store.js'; +import type { LocalConfig, TeamaiConfig } from '../types.js'; + +/** + * A member's value for a declared secret (#875): stored per team repo under + * ~/.teamai/secrets/, or once for the machine, resolved team value > machine + * value > the member's own environment. + */ +describe('team secret values', () => { + let tmpDir: string; + let home: string; + let localConfig: LocalConfig; + + const teamConfig: TeamaiConfig = { + team: 'acme', description: '', repo: 'https://example.com/acme/team.git', provider: 'git', reviewers: [], + sharing: { skills: {}, rules: { enforced: [] }, docs: { localDir: '' }, env: { injectShellProfile: false } }, + toolPaths: {}, + }; + const variable = (key: string, value: string): EnvVariable => ({ key, value }); + const keys = (...names: string[]): readonly string[] => names; + const values = (resolution: StoreResolution): Record => + resolution.kind === 'resolved' ? Object.fromEntries([...resolution.values].map(([k, v]) => [k, `${v.source}:${v.value}`])) : {}; + /** This scope's env with this env, as the team repo declares `declared` and sets `variables`. */ + const resolveTeamEnvWith = async ( + declared: readonly string[], + variables: readonly EnvVariable[], + env: NodeJS.ProcessEnv = {}, + ): Promise => { + const repoPath = localConfig.repo.localPath; + await fse.outputFile(path.join(repoPath, 'env', 'secrets.yaml'), YAML.stringify({ secrets: declared.map((key) => ({ key })) })); + await fse.outputFile(path.join(repoPath, 'env', 'env.yaml'), YAML.stringify({ variables })); + return resolveTeamEnv(localConfig, undefined, env); + }; + /** The secrets `declared` resolve to with this env. */ + const resolveSecretValues = async ( + declared: readonly string[], + variables: readonly EnvVariable[], + env: NodeJS.ProcessEnv, + ): Promise> => (await resolveTeamEnvWith(declared, variables, env)).secrets; + + const variableSources = (teamEnv: TeamEnv): Record => teamEnv.variableValues.kind === 'resolved' + ? Object.fromEntries([...teamEnv.variableValues.values].map(([k, v]) => [k, `${v.source}:${v.value}`])) + : {}; + + beforeEach(async () => { + tmpDir = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-secret-values-')); + home = path.join(tmpDir, 'home'); + vi.stubEnv('HOME', home); + vi.stubEnv('USERPROFILE', home); + const repoPath = path.join(tmpDir, 'team-repo'); + await fse.outputFile(path.join(repoPath, 'teamai.yaml'), 'team: acme\n'); + localConfig = { repo: { localPath: repoPath, remote: 'https://example.com/acme/team.git' }, username: 't', scope: 'user', additionalRoles: [] }; + }); + afterEach(async () => { + vi.unstubAllEnvs(); + await fse.remove(tmpDir); + }); + + describe('store', () => { + it('keeps values per team repo under ~/.teamai/secrets/teams, readable by the member only', async () => { + const file = getTeamSecretsPath(localConfig); + expect(path.dirname(file)).toBe(path.join(home, '.teamai', 'secrets', 'teams')); + expect(path.basename(file)).toMatch(/^[0-9a-f]{64}\.json$/); + + await writeSecretStore(file, { GITHUB_TOKEN: { value: 'fixture-token' }, GITLAB_TOKEN: { env: 'WORK_GITLAB_TOKEN' } }); + + expect(await readSecretStore(file)).toEqual({ + ok: true, + values: { GITHUB_TOKEN: { value: 'fixture-token' }, GITLAB_TOKEN: { env: 'WORK_GITLAB_TOKEN' } }, + }); + if (process.platform !== 'win32') expect((await fse.stat(file)).mode & 0o777).toBe(0o600); + }); + + it("keeps the values when the team is renamed in teamai.yaml, and only for this team repo", async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'team-token' } }); + await fse.outputFile(path.join(localConfig.repo.localPath, 'teamai.yaml'), 'team: Acme Engineering\n'); + + expect(await readSecretStore(getTeamSecretsPath(localConfig))).toEqual({ ok: true, values: { GITHUB_TOKEN: { value: 'team-token' } } }); + + const otherRepo: LocalConfig = { ...localConfig, repo: { ...localConfig.repo, remote: 'https://example.com/other/team.git' } }; + expect(getTeamSecretsPath(otherRepo)).not.toBe(getTeamSecretsPath(localConfig)); + expect(await readSecretStore(getTeamSecretsPath(otherRepo))).toEqual({ ok: true, values: {} }); + }); + + it("keys the values by the configured team repo URL, not by teamai.yaml's repo:", async () => { + const otherPath = path.join(tmpDir, 'copied-repo'); + const claim = 'team: acme\nrepo: https://example.com/acme/team.git\n'; + await fse.outputFile(path.join(localConfig.repo.localPath, 'teamai.yaml'), claim); + await fse.outputFile(path.join(otherPath, 'teamai.yaml'), claim); + const copied: LocalConfig = { ...localConfig, repo: { localPath: otherPath, remote: 'https://example.com/mallory/team.git' } }; + + expect(getTeamSecretsPath(copied)).not.toBe(getTeamSecretsPath(localConfig)); + }); + + it('keys the values by the repo URL when the remote is only an alias (fork), so two teams behind one alias stay apart', () => { + const behindFork = (url: string): LocalConfig => ({ ...localConfig, repo: { ...localConfig.repo, remote: 'fork', url } }); + + expect(getTeamSecretsPath(behindFork('https://example.com/acme/team.git'))) + .not.toBe(getTeamSecretsPath(behindFork('https://example.com/other/team.git'))); + expect(getTeamSecretsPath(behindFork('https://example.com/acme/team.git'))) + .toBe(getTeamSecretsPath({ ...localConfig, repo: { ...localConfig.repo, remote: 'origin', url: 'https://example.com/acme/team.git' } })); + }); + + describe('naming the file by the team repo URL', () => { + const fileFor = (remote: string): string => getTeamSecretsPath({ ...localConfig, repo: { ...localConfig.repo, remote } }); + + it('gives the credentialed, default-port and trailing-slash forms of one https URL the same file', () => { + for (const remote of [ + 'https://user:fixture-pass@EXAMPLE.com/acme/team/', + 'https://example.com:443/acme/team.git', + ]) expect(fileFor(remote)).toBe(getTeamSecretsPath(localConfig)); + }); + + it('gives the http and https URLs of a repo different files, each with its default port written or not', () => { + expect(fileFor('http://example.com/acme/team.git')).not.toBe(getTeamSecretsPath(localConfig)); + expect(fileFor('http://example.com:80/acme/team')).toBe(fileFor('http://example.com/acme/team.git')); + expect(fileFor('http://example.com:443/acme/team.git')).not.toBe(getTeamSecretsPath(localConfig)); + }); + + it('gives the scp form and the ssh URL of one repo the same file, with the default port written or not', () => { + for (const remote of [ + 'ssh://git@example.com/~/acme/team', + 'ssh://git@EXAMPLE.com:22/~/acme/team.git/', + 'git+ssh://git@example.com/~/acme/team.git', + 'ssh+git://git@example.com:22/~/acme/team.git', + 'git@example.com:~/acme/team.git', + ]) { + expect(fileFor(remote)).toBe(fileFor('git@example.com:acme/team.git')); + } + expect(fileFor('ssh://git@example.com:22/acme/team.git')).toBe(fileFor('git@example.com:/acme/team.git')); + expect(fileFor('ssh://example.com/~/acme/team')).toBe(fileFor('example.com:acme/team')); + }); + + it('gives an scp path in the ssh user\'s home and the ssh URL of that path from the root different files', () => { + expect(fileFor('ssh://git@example.com/acme/team.git')).not.toBe(fileFor('git@example.com:acme/team.git')); + expect(fileFor('git@example.com:/acme/team.git')).not.toBe(fileFor('git@example.com:acme/team.git')); + }); + + it('gives two ssh users on one host different files, in the scp form and the ssh URL alike', () => { + expect(fileFor('alice@example.com:team.git')).not.toBe(fileFor('bob@example.com:team.git')); + expect(fileFor('ssh://alice@example.com/team')).not.toBe(fileFor('ssh://bob@example.com/team')); + expect(fileFor('ssh://alice@example.com:22/~/team.git')).toBe(fileFor('alice@example.com:team.git')); + expect(fileFor('ssh://example.com/team')).not.toBe(fileFor('alice@example.com:team.git')); + }); + + it('gives repos on one host with different ports different files', () => { + expect(fileFor('ssh://git@example.com:2222/acme/team.git')).not.toBe(fileFor('ssh://git@example.com:2223/acme/team.git')); + expect(fileFor('ssh://git@example.com:2222/acme/team.git')).not.toBe(fileFor('git@example.com:acme/team.git')); + expect(fileFor('https://example.com:8443/acme/team.git')).not.toBe(getTeamSecretsPath(localConfig)); + }); + + it('gives file:// repos that differ only by a .git suffix different files: they are two directories', () => { + expect(fileFor('file:///srv/team')).not.toBe(fileFor('file:///srv/team.git')); + expect(fileFor('file:///srv/team/')).toBe(fileFor('file:///srv/team')); + }); + + it('gives URLs that differ only in the query or the fragment different files', () => { + expect(fileFor('https://example.com/acme/team?tenant=a')).not.toBe(fileFor('https://example.com/acme/team?tenant=b')); + expect(fileFor('https://example.com/acme/team?tenant=a')).not.toBe(getTeamSecretsPath(localConfig)); + expect(fileFor('https://example.com/acme/team#a')).not.toBe(fileFor('https://example.com/acme/team#b')); + expect(fileFor('ssh://git@example.com/acme/team?tenant=a')).not.toBe(fileFor('ssh://git@example.com/acme/team?tenant=b')); + }); + + it('keeps the query while dropping the credentials, the default port and a trailing .git or slash before it', () => { + for (const remote of [ + 'https://user:fixture-pass@EXAMPLE.com/acme/team.git?tenant=a', + 'https://example.com:443/acme/team/?tenant=a', + ]) expect(fileFor(remote)).toBe(fileFor('https://example.com/acme/team?tenant=a')); + }); + + it('gives the ssh and https URLs of a repo different files', () => { + expect(fileFor('git@example.com:acme/team.git')).not.toBe(getTeamSecretsPath(localConfig)); + }); + + it('keeps the case of the path', () => { + expect(fileFor('https://example.com/Acme/Team.git')).not.toBe(getTeamSecretsPath(localConfig)); + }); + }); + + it('keeps an entry named __proto__ as an own key, through a write and a read', async () => { + const file = getTeamSecretsPath(localConfig); + await writeSecretStore(file, { ['__proto__']: { value: 'proto-value' }, API_URL: { value: 'u' } }); + + expect(JSON.parse(await fse.readFile(file, 'utf8'))).toEqual({ ['__proto__']: { value: 'proto-value' }, API_URL: { value: 'u' } }); + const read = await readSecretStore(file); + expect(read.ok).toBe(true); + if (!read.ok) return; + expect(Object.keys(read.values)).toEqual(['__proto__', 'API_URL']); + expect(Object.hasOwn(read.values, '__proto__')).toBe(true); + }); + + it('keeps whether an entry is a secret or a variable override, and rejects any other kind', async () => { + const file = getTeamSecretsPath(localConfig); + await writeSecretStore(file, { GITHUB_TOKEN: { value: 't', kind: 'secret' }, API_URL: { env: 'MY_API_URL', kind: 'variable' } }); + + expect(await readSecretStore(file)).toEqual({ + ok: true, values: { GITHUB_TOKEN: { value: 't', kind: 'secret' }, API_URL: { env: 'MY_API_URL', kind: 'variable' } }, + }); + await fse.outputJson(file, { GITHUB_TOKEN: { value: 't', kind: 'token' } }); + expect((await readSecretStore(file)).ok).toBe(false); + }); + + it('does not touch the env backup file ~/.teamai/env', async () => { + await fse.outputFile(path.join(home, '.teamai', 'env'), 'API_URL=u\n'); + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'fixture-token' } }); + expect(await fse.readFile(path.join(home, '.teamai', 'env'), 'utf8')).toBe('API_URL=u\n'); + }); + + it('keeps the machine values beside the team files, in ~/.teamai/secrets/machine.json', () => { + expect(getMachineSecretsPath()).toBe(path.join(home, '.teamai', 'secrets', 'machine.json')); + }); + + it('reads a missing file as no values', async () => { + expect(await readSecretStore(getTeamSecretsPath(localConfig))).toEqual({ ok: true, values: {} }); + }); + + it('reports a hand-corrupted file by path only, never with the value or the parser message', async () => { + const file = getTeamSecretsPath(localConfig); + await fse.outputFile(file, '{\n "GITHUB_TOKEN": { "value": ghp_fixture_value }\n}\n'); + + const read = await readSecretStore(file); + + expect(read.ok).toBe(false); + if (read.ok) return; + expect(read.reason).toBe(`${file} is not valid JSON. Fix the file, or delete it and set the values again with \`teamai env set\`.`); + }); + + it.skipIf(process.platform === 'win32' || process.getuid?.() === 0)('says what to check when the file cannot be read', async () => { + const file = getTeamSecretsPath(localConfig); + await writeSecretStore(file, { GITHUB_TOKEN: { value: 'fixture-token' } }); + await fse.chmod(file, 0o000); + try { + const read = await readSecretStore(file); + + expect(read).toEqual({ + ok: false, + reason: `Cannot read your secret values at ${file} (EACCES). Check that the file is yours and readable (\`ls -l ${file}\`), ` + + 'or delete it and set the values again with `teamai env set`.', + }); + } finally { + await fse.chmod(file, 0o600); + } + }); + + it('rejects an entry that is not exactly one of a value or a variable reference', async () => { + const file = getTeamSecretsPath(localConfig); + for (const entry of ['{"value": "ghp_fixture_value", "env": "X"}', '{}', '"ghp_fixture_value"']) { + await fse.outputFile(file, `{"OK": {"env": "X"}, "GITHUB_TOKEN": ${entry}}`); + const read = await readSecretStore(file); + expect(read.ok).toBe(false); + if (read.ok) continue; + expect(read.reason).toContain(`${file} has an invalid entry (entry 2)`); + expect(read.reason).not.toContain('ghp_fixture_value'); + } + }); + }); + + describe('resolution', () => { + it('takes the team value over the environment, and the environment when no team value is set', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'team-token' } }); + const resolved = await resolveSecretValues(keys('GITHUB_TOKEN', 'GITLAB_TOKEN', 'ACME_TOKEN'), [], { + GITHUB_TOKEN: 'exported-token', GITLAB_TOKEN: 'exported-gitlab', ACME_TOKEN: '', + }); + + expect(values(resolved)).toEqual({ GITHUB_TOKEN: 'team:team-token', GITLAB_TOKEN: 'environment:exported-gitlab' }); + expect(secretState(resolved, 'GITHUB_TOKEN')).toBe('team'); + expect(secretState(resolved, 'GITLAB_TOKEN')).toBe('environment'); + expect(secretState(resolved, 'ACME_TOKEN')).toBe('missing'); + }); + + it('reads a --from-env reference when the value is used, and does not fall back to the environment when it is unset', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN' } }); + const secret = keys('GITHUB_TOKEN'); + + expect(values(await resolveSecretValues(secret, [], { WORK_GITHUB_TOKEN: 'work-1', GITHUB_TOKEN: 'personal' }))) + .toEqual({ GITHUB_TOKEN: 'team:work-1' }); + expect(values(await resolveSecretValues(secret, [], { WORK_GITHUB_TOKEN: 'work-2' }))) + .toEqual({ GITHUB_TOKEN: 'team:work-2' }); + expect(values(await resolveSecretValues(secret, [], { GITHUB_TOKEN: 'personal' }))).toEqual({}); + }); + + it('takes the team value over the machine value, and the machine value over the environment', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'team-token' } }); + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'machine-github' }, GITLAB_TOKEN: { value: 'machine-gitlab' } }); + const resolved = await resolveSecretValues(keys('GITHUB_TOKEN', 'GITLAB_TOKEN', 'SENTRY_TOKEN'), [], { + GITHUB_TOKEN: 'exported-github', GITLAB_TOKEN: 'exported-gitlab', SENTRY_TOKEN: 'exported-sentry', + }); + + expect(values(resolved)).toEqual({ + GITHUB_TOKEN: 'team:team-token', GITLAB_TOKEN: 'global:machine-gitlab', SENTRY_TOKEN: 'environment:exported-sentry', + }); + expect(secretState(resolved, 'GITLAB_TOKEN')).toBe('global'); + }); + + it('lets an entry decide even when its --from-env variable is unset: a team entry over the machine, a machine entry over the environment', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { env: 'WORK_GITHUB_TOKEN' } }); + await writeSecretStore(getMachineSecretsPath(), { GITHUB_TOKEN: { value: 'personal' }, GITLAB_TOKEN: { env: 'PERSONAL_GITLAB_TOKEN' } }); + + expect(values(await resolveSecretValues(keys('GITHUB_TOKEN', 'GITLAB_TOKEN'), [], { GITLAB_TOKEN: 'exported' }))) + .toEqual({}); + }); + + // #879: secrets and variable overrides share the team store; each entry says which it is. + it('never resolves a variable from a value stored while the key was a secret, an entry without a kind counting as one', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { + API_URL: { value: 'fixture-old-secret' }, GITLAB_HOST: { value: 'fixture-old-secret-2', kind: 'secret' }, + }); + const teamEnv = await resolveTeamEnvWith(keys(), [variable('API_URL', 'team-url'), variable('GITLAB_HOST', 'gitlab.team')]); + + expect(variableSources(teamEnv)).toEqual({ API_URL: 'env.yaml:team-url', GITLAB_HOST: 'env.yaml:gitlab.team' }); + expect([...teamEnv.staleEntries]).toEqual([['API_URL', 'secret'], ['GITLAB_HOST', 'secret']]); + }); + + it('never resolves a secret from a variable override of the same key', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'fixture-override', kind: 'variable' } }); + await writeSecretStore(getMachineSecretsPath(), { GITLAB_TOKEN: { value: 'machine-gitlab', kind: 'secret' } }); + const teamEnv = await resolveTeamEnvWith(keys('GITHUB_TOKEN', 'GITLAB_TOKEN'), [], { GITHUB_TOKEN: 'exported-token' }); + + expect(values(teamEnv.secrets)).toEqual({ GITHUB_TOKEN: 'environment:exported-token', GITLAB_TOKEN: 'global:machine-gitlab' }); + expect([...teamEnv.staleEntries]).toEqual([['GITHUB_TOKEN', 'variable']]); + }); + + it('on Windows, treats an env.yaml variable as the secret declared under the same name in another case', async () => { + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + let teamEnv: TeamEnv; + try { + teamEnv = await resolveTeamEnvWith(keys('GITHUB_TOKEN'), [variable('github_token', 'fixture-repo-token')]); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(variableSources(teamEnv)).toEqual({}); + + teamEnv = await resolveTeamEnvWith(keys('GITHUB_TOKEN'), [variable('github_token', 'repo-value')]); + expect(variableSources(teamEnv)).toEqual({ github_token: 'env.yaml:repo-value' }); + }); + + it("on Windows, never takes an env.yaml value under another case of a secret's name for the member's own", async () => { + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + let teamEnv: TeamEnv; + try { + teamEnv = await resolveTeamEnvWith(keys('GITHUB_TOKEN'), [variable('github_token', 'fixture-repo-token')], { GITHUB_TOKEN: 'fixture-repo-token' }); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(values(teamEnv.secrets)).toEqual({}); + }); + + it('on Windows, resolves a secret from a value stored under another case of its name', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { github_token: { value: 'fixture-team-token', kind: 'secret' } }); + const original = process.platform; + Object.defineProperty(process, 'platform', { value: 'win32', configurable: true }); + let teamEnv: TeamEnv; + try { + teamEnv = await resolveTeamEnvWith(keys('GITHUB_TOKEN'), []); + } finally { + Object.defineProperty(process, 'platform', { value: original, configurable: true }); + } + expect(values(teamEnv.secrets)).toEqual({ GITHUB_TOKEN: 'team:fixture-team-token' }); + }); + + it("resolves a variable from the member's override and a secret from its value, each by its kind", async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { + API_URL: { value: 'member-url', kind: 'variable' }, GITHUB_TOKEN: { value: 'team-token', kind: 'secret' }, + }); + const teamEnv = await resolveTeamEnvWith(keys('GITHUB_TOKEN'), [variable('API_URL', 'team-url')]); + + expect(variableSources(teamEnv)).toEqual({ API_URL: 'team:member-url' }); + expect(values(teamEnv.secrets)).toEqual({ GITHUB_TOKEN: 'team:team-token' }); + expect(teamEnv.staleEntries.size).toBe(0); + }); + + it('leaves every secret without a value when the machine store cannot be read', async () => { + await writeSecretStore(getTeamSecretsPath(localConfig), { GITHUB_TOKEN: { value: 'team-token' } }); + await fse.outputFile(getMachineSecretsPath(), '{ "GITLAB_TOKEN": { "value": ghp_fixture_value } }'); + const resolved = await resolveSecretValues(keys('GITHUB_TOKEN', 'GITLAB_TOKEN'), [], { GITLAB_TOKEN: 'exported' }); + + expect(resolved.kind).toBe('store-unreadable'); + if (resolved.kind !== 'store-unreadable') return; + expect(resolved.reason).toContain(`${getMachineSecretsPath()} is not valid JSON`); + expect(resolved.reason).not.toContain('ghp_fixture_value'); + }); + + it('leaves every secret without a value when the store cannot be read, and says so rather than missing', async () => { + await fse.outputFile(getTeamSecretsPath(localConfig), '{ "GITHUB_TOKEN": { "value": ghp_fixture_value } }'); + const resolved = await resolveSecretValues(keys('GITHUB_TOKEN'), [], { GITHUB_TOKEN: 'exported' }); + + expect(resolved.kind).toBe('store-unreadable'); + expect(secretState(resolved, 'GITHUB_TOKEN')).toBe('unreadable'); + expect(JSON.stringify(resolved)).not.toContain('ghp_fixture_value'); + }); + }); + + // #879 Conflict 10: the environment in the order is the member's own. + describe("the member's environment", () => { + const resolve = async (env: NodeJS.ProcessEnv, variables: EnvVariable[] = []): Promise> => + values(await resolveSecretValues(keys('GITHUB_TOKEN'), variables, env)); + + it.each([ + ['counts a value the member exported by hand', null, [], 'hand-export', { GITHUB_TOKEN: 'environment:hand-export' }], + ["leaves out a value another scope's env.sh exports", 'projects/other-slug/env.sh', [], 'other-team-token', {}], + ["leaves out a value the user scope's env.sh exports", 'env.sh', [], 'user-scope-token', {}], + ["leaves out this scope's env.yaml value for a key now declared as a secret", null, [variable('GITHUB_TOKEN', 'repo-token')], 'repo-token', {}], + ] as const)('%s', async (_name, envSh, variables, exported, expected) => { + if (envSh) await fse.outputFile(path.join(home, '.teamai', envSh), `export GITHUB_TOKEN='${exported}'\n`); + expect(await resolve({ GITHUB_TOKEN: exported }, [...variables])).toEqual(expected); + }); + + it("leaves out this scope's previous env.sh value after a pull rewrote it", async () => { + const envSh = path.join(home, '.teamai', 'env.sh'); + await fse.outputFile(envSh, "export GITHUB_TOKEN='old-repo-token'\n"); + + await new EnvHandler().writeResolvedEnv([], teamConfig, localConfig); + + expect(await fse.readFile(envSh, 'utf8')).not.toContain('old-repo-token'); + expect(await resolve({ GITHUB_TOKEN: 'old-repo-token' })).toEqual({}); + expect(await resolve({ GITHUB_TOKEN: 'hand-export' })).toEqual({ GITHUB_TOKEN: 'environment:hand-export' }); + }); + + // A shell opened before a pull keeps what env.sh exported then, through + // every later command, not only the one that rewrote it. + it('leaves out a value an earlier rewrite of env.sh exported, after a later rewrite dropped it', async () => { + const write = (value?: string): Promise => + new EnvHandler().writeResolvedEnv(value ? [{ key: 'GITHUB_TOKEN', value }] : [], teamConfig, localConfig); + await write('repo-token'); + await write(); + + expect(await resolve({ GITHUB_TOKEN: 'repo-token' })).toEqual({}); + expect(await resolve({ GITHUB_TOKEN: 'hand-export' })).toEqual({ GITHUB_TOKEN: 'environment:hand-export' }); + }); + + // No scan finds every env.sh a shell may have loaded: a non-git project + // keeps its own under `/.teamai/`. The marker each one exports says so. + describe('an env.sh at a path no scan reaches', () => { + const MARKER_LINE = /^export (TEAMAI_ENV_SH_[0-9a-f]{64})='([^']*)'$/m; + /** The environment of a shell that sourced a project's env.sh in `/elsewhere/.teamai`. */ + const sourcedProjectEnvSh = async (variables: EnvVariable[]): Promise<{ env: NodeJS.ProcessEnv; content: string }> => { + const projectRoot = path.join(tmpDir, 'elsewhere'); + const project: LocalConfig = { ...localConfig, scope: 'project', projectRoot }; + await new EnvHandler().writeResolvedEnv(variables, teamConfig, project); + const content = await fse.readFile(path.join(projectRoot, '.teamai', 'env.sh'), 'utf8'); + const env = Object.fromEntries([...content.matchAll(/^export (\w+)='([^']*)'$/gm)].map((m) => [m[1], m[2]])); + return { env, content }; + }; + + it('leaves out a value that env.sh exported', async () => { + const { env } = await sourcedProjectEnvSh([variable('GITHUB_TOKEN', 'project-a-token'), variable('API_URL', 'https://a')]); + + expect(env.GITHUB_TOKEN).toBe('project-a-token'); + expect(await resolve(env)).toEqual({}); + }); + + it('counts a different value the member exported by hand for the same key', async () => { + const { env } = await sourcedProjectEnvSh([variable('GITHUB_TOKEN', 'project-a-token')]); + + expect(await resolve({ ...env, GITHUB_TOKEN: 'hand-export' })).toEqual({ GITHUB_TOKEN: 'environment:hand-export' }); + }); + + it('exports one marker of hashes, never a value, that env.sh does not read back as a variable', async () => { + const { content } = await sourcedProjectEnvSh([variable('GITHUB_TOKEN', 'project-a-token'), variable('API_URL', 'https://a')]); + const marker = content.match(MARKER_LINE); + + expect(marker?.[2]).toMatch(/^[0-9a-f]{12} [0-9a-f]{12}$/); + expect(marker?.[0]).not.toContain('project-a-token'); + expect(marker?.[0]).not.toContain('https://a'); + expect([...parseEnvFile(content).keys()]).toEqual(['GITHUB_TOKEN', 'API_URL']); + }); + + // The shell re-sources the rewritten env.sh: the old value stays + // exported, and the new marker replaces the old one. + it('keeps marking a value that env.sh exported before a rewrite dropped it, in a shell that sources it again', async () => { + const first = await sourcedProjectEnvSh([variable('GITHUB_TOKEN', 'project-a-token'), variable('API_URL', 'https://a')]); + const second = await sourcedProjectEnvSh([variable('API_URL', 'https://a')]); + const env = { ...first.env, ...second.env }; + + expect(second.content).not.toContain('project-a-token'); + expect(env.GITHUB_TOKEN).toBe('project-a-token'); + expect(await resolve(env)).toEqual({}); + expect(await resolve({ ...env, GITHUB_TOKEN: 'hand-export' })).toEqual({ GITHUB_TOKEN: 'environment:hand-export' }); + }); + }); + + // Windows compares environment names case-insensitively, so `github_token` + // another scope exported is the member's GITHUB_TOKEN there. + describe('a key exported in another case', () => { + const original = process.platform; + const onPlatform = (platform: NodeJS.Platform): void => { + Object.defineProperty(process, 'platform', { value: platform, configurable: true }); + }; + afterEach(() => onPlatform(original)); + + const exportedInLowerCase = async (): Promise => { + await fse.outputFile(path.join(home, '.teamai', 'projects', 'other-slug', 'env.sh'), "export github_token='other-team-token'\n"); + }; + const recordedInLowerCase = async (): Promise => { + const write = (exports: EnvVariable[]): Promise => new EnvHandler().writeResolvedEnv(exports, teamConfig, localConfig); + await write([variable('github_token', 'repo-token')]); + await write([]); + }; + /** A shell that sourced a project's env.sh no scan finds, exporting github_token. */ + const markedInLowerCase = async (): Promise => { + const projectRoot = path.join(tmpDir, 'elsewhere'); + await new EnvHandler().writeResolvedEnv([variable('github_token', 'project-token')], teamConfig, { ...localConfig, scope: 'project', projectRoot }); + const content = await fse.readFile(path.join(projectRoot, '.teamai', 'env.sh'), 'utf8'); + const marker = /^export (TEAMAI_ENV_SH_[0-9a-f]{64})='([^']*)'$/m.exec(content); + expect(marker).not.toBeNull(); + return marker?.[1] ? { [marker[1]]: marker[2] } : {}; + }; + + it('on Windows, leaves out a value another env.sh exports, has recorded or has marked in any case', async () => { + onPlatform('win32'); + await exportedInLowerCase(); + await recordedInLowerCase(); + const marker = await markedInLowerCase(); + + expect(await resolve({ GITHUB_TOKEN: 'other-team-token' })).toEqual({}); + expect(await resolve({ GITHUB_TOKEN: 'repo-token' })).toEqual({}); + expect(await resolve({ ...marker, GITHUB_TOKEN: 'project-token' })).toEqual({}); + expect(await resolve({ ...marker, GITHUB_TOKEN: 'hand-export' })).toEqual({ GITHUB_TOKEN: 'environment:hand-export' }); + }); + + it('elsewhere, counts a value exported, recorded or marked only under another case as the member\'s', async () => { + onPlatform('linux'); + await exportedInLowerCase(); + await recordedInLowerCase(); + const marker = await markedInLowerCase(); + + expect(await resolve({ GITHUB_TOKEN: 'other-team-token' })).toEqual({ GITHUB_TOKEN: 'environment:other-team-token' }); + expect(await resolve({ GITHUB_TOKEN: 'repo-token' })).toEqual({ GITHUB_TOKEN: 'environment:repo-token' }); + expect(await resolve({ ...marker, GITHUB_TOKEN: 'project-token' })).toEqual({ GITHUB_TOKEN: 'environment:project-token' }); + }); + }); + + it('records what env.sh exported as hashes beside it, readable by the member only, and forgets the oldest', async () => { + const write = (value: string): Promise => + new EnvHandler().writeResolvedEnv([{ key: 'GITHUB_TOKEN', value }], teamConfig, localConfig); + for (let i = 1; i <= 21; i++) await write(`repo-token-${i}`); + await new EnvHandler().writeResolvedEnv([], teamConfig, localConfig); + + const record = path.join(home, '.teamai', 'env.sh.exports.json'); + expect(await fse.readFile(record, 'utf8')).not.toContain('repo-token'); + if (process.platform !== 'win32') expect((await fse.stat(record)).mode & 0o777).toBe(0o600); + expect(await resolve({ GITHUB_TOKEN: 'repo-token-1' })).toEqual({ GITHUB_TOKEN: 'environment:repo-token-1' }); + expect(await resolve({ GITHUB_TOKEN: 'repo-token-2' })).toEqual({}); + }); + }); +}); diff --git a/src/__tests__/secrets-declarations.test.ts b/src/__tests__/secrets-declarations.test.ts new file mode 100644 index 000000000..0f7335d9b --- /dev/null +++ b/src/__tests__/secrets-declarations.test.ts @@ -0,0 +1,129 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import fse from 'fs-extra'; +import os from 'node:os'; +import path from 'node:path'; + +vi.mock('../utils/logger.js', () => ({ + log: { + debug: vi.fn(), error: vi.fn(), info: vi.fn(), success: vi.fn(), warn: vi.fn(), dim: vi.fn(), persist: vi.fn(), + }, +})); + +import { describeEntryFailure } from '../namespaced-entries.js'; +import { resolveSecretDeclarations } from '../resources/secrets.js'; +import type { LocalConfig } from '../types.js'; + +/** + * `env/secrets.yaml` and `env//secrets.yaml` (#875): declared keys, no + * values, resolved like env.yaml, with absent, valid and failed kept apart. + */ +describe('team secret declarations', () => { + let repoPath: string; + + const config = (projects?: string[]): LocalConfig => ({ + repo: { localPath: repoPath, remote: 'owner/repo' }, username: 't', scope: 'user', additionalRoles: [], + ...(projects ? { projects } : {}), + }); + const write = (relativePath: string, content: string): Promise => + fse.outputFile(path.join(repoPath, ...relativePath.split('/')), content); + + beforeEach(async () => { + repoPath = await fse.mkdtemp(path.join(os.tmpdir(), 'teamai-secrets-')); + }); + afterEach(async () => { + await fse.remove(repoPath); + }); + + it('resolves the root file and the namespaces resources.env activates, a namespace entry replacing the root one', async () => { + await write('manifest/projects.yaml', [ + 'version: 1', + 'projects:', + ' - { id: checkout, resources: { env: [checkout], mcp: [billing] } }', + ' - { id: billing, resources: { env: [billing] } }', + '', + ].join('\n')); + await write('env/secrets.yaml', [ + 'secrets:', + ' - { key: GITHUB_TOKEN, description: root token, url: https://github.com/settings/tokens }', + ' - { key: NPM_TOKEN }', + '', + ].join('\n')); + await write('env/checkout/secrets.yaml', 'secrets:\n - { key: GITHUB_TOKEN, description: checkout token }\n'); + await write('env/billing/secrets.yaml', 'secrets:\n - { key: BILLING_TOKEN }\n'); + + const declarations = await resolveSecretDeclarations(config(['checkout'])); + + expect(declarations.kind).toBe('resolved'); + if (declarations.kind !== 'resolved') return; + expect(declarations.entries.map(({ name, entry, source, replaces }) => ({ name, entry, source, replaces }))).toEqual([ + { name: 'GITHUB_TOKEN', entry: { key: 'GITHUB_TOKEN', description: 'checkout token' }, source: 'env/checkout/secrets.yaml', replaces: 'env/secrets.yaml' }, + { name: 'NPM_TOKEN', entry: { key: 'NPM_TOKEN' }, source: 'env/secrets.yaml', replaces: null }, + ]); + }); + + it('uses the active namespaces it is given, as pull passes env\'s', async () => { + await write('env/secrets.yaml', 'secrets:\n - { key: A }\n'); + await write('env/billing/secrets.yaml', 'secrets:\n - { key: B }\n'); + + const declarations = await resolveSecretDeclarations(config(), { active: ['billing'] }); + + expect(declarations.kind === 'resolved' && declarations.entries.map((entry) => entry.name)).toEqual(['A', 'B']); + }); + + it('is absent when no file it reads exists, and valid when a file declares none', async () => { + await write('env/env.yaml', 'variables:\n - { key: PLAIN, value: x }\n'); + expect(await resolveSecretDeclarations(config())).toEqual({ kind: 'absent' }); + + // A file in a namespace that is not active here is not read. + await write('env/billing/secrets.yaml', 'secrets:\n - { key: B }\n'); + expect(await resolveSecretDeclarations(config())).toEqual({ kind: 'absent' }); + + for (const content of ['', 'secrets: []\n']) { + await write('env/secrets.yaml', content); + const declarations = await resolveSecretDeclarations(config()); + expect(declarations.kind === 'resolved' && declarations.entries).toEqual([]); + } + }); + + it.each([ + ['not YAML', 'secrets: [\n', 'env/secrets.yaml is not valid YAML'], + ['no secrets: key', 'secret:\n - { key: A }\n', 'env/secrets.yaml declares no secrets: it has no top-level `secrets:` key, only `secret`'], + ['an entry without a key', 'secrets:\n - { description: x }\n', 'env/secrets.yaml does not match the secrets.yaml schema: secrets.0.key: Required'], + ['a key that is no variable name', 'secrets:\n - { key: MY-TOKEN }\n', 'secrets.0.key: must be a shell variable name'], + ['secrets: not a list', 'secrets: GITHUB_TOKEN\n', 'env/secrets.yaml does not match the secrets.yaml schema: secrets: Expected array'], + ])('fails, in secret wording, on %s', async (_label, content, reason) => { + await write('env/secrets.yaml', content); + + const declarations = await resolveSecretDeclarations(config()); + + expect(declarations.kind).toBe('failed'); + if (declarations.kind !== 'failed') return; + const message = describeEntryFailure(declarations.failure); + expect(message).toContain(reason); + expect(message).toContain('Team secrets were not resolved this run; env variables and MCP servers stay as they are. Fix the file in the team repo and push.'); + }); + + it('fails when a namespace file repeats a key, naming it a secret', async () => { + await write('env/secrets.yaml', 'secrets:\n - { key: A }\n - { key: A }\n'); + + const declarations = await resolveSecretDeclarations(config(), { active: [] }); + + expect(declarations.kind === 'failed' && describeEntryFailure(declarations.failure)).toBe( + 'env/secrets.yaml defines secret "A" more than once. Team secrets were not resolved this run; env variables and MCP servers stay as they are. ' + + 'Keep one of them in the team repo and push.', + ); + }); + + it('does not declare a secret written with a value, and says why', async () => { + await write('env/secrets.yaml', 'secrets:\n - { key: A, value: committed }\n - { key: B }\n'); + + const declarations = await resolveSecretDeclarations(config()); + + expect(declarations.kind).toBe('resolved'); + if (declarations.kind !== 'resolved') return; + expect(declarations.entries.map((entry) => entry.name)).toEqual(['B']); + expect(declarations.notices.map((notice) => notice.message)).toEqual([ + 'env/secrets.yaml: secret "A" has unknown key `value:`, so this entry is not delivered. Correct the key or remove it.', + ]); + }); +}); diff --git a/src/__tests__/self-mode-env-agents.test.ts b/src/__tests__/self-mode-env-agents.test.ts index a0dbe80fe..f9e735ffa 100644 --- a/src/__tests__/self-mode-env-agents.test.ts +++ b/src/__tests__/self-mode-env-agents.test.ts @@ -105,6 +105,18 @@ describe('single-repo mode: env + agents direct .teamai scan', () => { expect(await fse.pathExists(path.join(worktreeTeamai, 'env', 'env.yaml'))).toBe(false); }); + it('env: detects a changed secrets file and pushes it to the same path (#875)', async () => { + await fse.writeFile(path.join(worktreeTeamai, 'env', 'env.yaml'), 'variables: []\n'); + await fse.writeFile(path.join(bizRoot, '.teamai', 'env', 'env.yaml'), 'variables: []\n'); + await fse.writeFile(path.join(bizRoot, '.teamai', 'env', 'secrets.yaml'), 'secrets:\n - key: GITHUB_TOKEN\n'); + const items = await new EnvHandler().scanLocalForPush(teamConfig, localConfig); + expect(items.map((item) => item.relativePath)).toEqual(['env/secrets.yaml']); + const [item] = items; + if (!item) throw new Error('expected one item'); + await new EnvHandler().pushItem(item, teamConfig, localConfig); + expect(await fse.readFile(path.join(worktreeTeamai, 'env', 'secrets.yaml'), 'utf8')).toContain('GITHUB_TOKEN'); + }); + // Regression: a teammate who clones a self-mode repo has .teamai/env/ as a // committed DIRECTORY (holding env.yaml). pullItem must NOT try to write its // KEY=value backup at /env (that path is the dir → EISDIR). It must diff --git a/src/__tests__/skill-commands-exist.test.ts b/src/__tests__/skill-commands-exist.test.ts index 62a2d2da9..0a215fd21 100644 --- a/src/__tests__/skill-commands-exist.test.ts +++ b/src/__tests__/skill-commands-exist.test.ts @@ -105,7 +105,9 @@ function validate(program: Command, invocations: Invocation[]): string[] { } const flags = knownFlags(command, program); - for (const token of rest) { + // After `--` the words are another command's (`teamai env exec -- gh …`). + const end = rest.indexOf('--'); + for (const token of end === -1 ? rest : rest.slice(0, end)) { if (!token.startsWith('-') || token === '-') continue; const flag = token.split('=')[0]; // Placeholders and prose inside an example are not flags to resolve. diff --git a/src/__tests__/status-list.test.ts b/src/__tests__/status-list.test.ts index 4f20cfcdc..860d0dc93 100644 --- a/src/__tests__/status-list.test.ts +++ b/src/__tests__/status-list.test.ts @@ -148,6 +148,61 @@ describe('teamai list / status resource coverage', () => { expect(out).toContain('SECRET_TOKEN=super-secret-value'); }); + // #875: a declared secret shows where its value comes from, never the value. + it('list env shows each declared secret with its state and never its value, --reveal included', async () => { + await fse.writeFile( + path.join(repoPath, 'env', 'secrets.yaml'), + 'secrets:\n - key: GITHUB_TOKEN\n description: GitHub token\n - key: GITLAB_TOKEN\n', + ); + vi.stubEnv('GITHUB_TOKEN', 'fixture-github-value'); + vi.stubEnv('GITLAB_TOKEN', ''); + + await list('env', { source: 'repo', reveal: true, verbose: true }); + const out = lines.join('\n'); + expect(out).toContain('SECRET_TOKEN=super-secret-value'); + expect(out).toContain('GITHUB_TOKEN environment (root)'); + expect(out).toContain(' GitHub token'); + expect(out).toContain('GITLAB_TOKEN missing (root)'); + expect(out).not.toContain('fixture-github-value'); + }); + + // #875 (#879 Conflict 13): a key declared twice is listed only as a secret. + it('list env --reveal leaves out the env.yaml value of a key declared as a secret, and shows a team value as team', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secrets:\n - key: SECRET_TOKEN\n'); + const { getTeamSecretsPath, writeSecretStore } = await import('../secret-store.js'); + const { localConfig } = await mockAutoDetectInit() as { localConfig: LocalConfig }; + await writeSecretStore(getTeamSecretsPath(localConfig), { SECRET_TOKEN: { value: 'fixture-team-value' } }); + + await list('env', { source: 'repo', reveal: true }); + const out = lines.join('\n'); + expect(out).not.toContain('super-secret-value'); + expect(out).not.toContain('fixture-team-value'); + expect(out).toContain('SECRET_TOKEN team (root)'); + }); + + // #875: list env is the listing env list prints, so it shows a member's override too. + it('list env shows the member\'s value of an overridden variable, as team', async () => { + const { getTeamSecretsPath, writeSecretStore } = await import('../secret-store.js'); + const { localConfig } = await mockAutoDetectInit() as { localConfig: LocalConfig }; + await writeSecretStore(getTeamSecretsPath(localConfig), { SECRET_TOKEN: { value: 'fixture-member-value', kind: 'variable' } }); + + await list('env', { source: 'repo', reveal: true }); + const out = lines.join('\n'); + expect(out).toContain('SECRET_TOKEN=fixture-member-value team (root)'); + expect(out).not.toContain('super-secret-value'); + }); + + it('list env still lists the variables when the secrets file is broken, without their values, and names it', async () => { + await fse.writeFile(path.join(repoPath, 'env', 'secrets.yaml'), 'secret:\n - key: GITHUB_TOKEN\n'); + + await list('env', { source: 'repo', reveal: true }); + const out = lines.join('\n'); + expect(out).toContain('SECRET_TOKEN (root)'); + expect(out).not.toContain('super-secret-value'); + expect(out).toContain('env/secrets.yaml declares no secrets'); + expect(out).toContain('Team secrets were not resolved this run'); + }); + it('list rejects unknown types', async () => { await list('widgets', { source: 'repo' }); expect(log.error).toHaveBeenCalledWith(expect.stringContaining('Unknown resource type')); diff --git a/src/__tests__/uninstall.test.ts b/src/__tests__/uninstall.test.ts index cc1e90890..589f5a639 100644 --- a/src/__tests__/uninstall.test.ts +++ b/src/__tests__/uninstall.test.ts @@ -3,6 +3,7 @@ import path from 'node:path'; import os from 'node:os'; import fse from 'fs-extra'; import { fileURLToPath } from 'node:url'; +import { execFileSync } from 'node:child_process'; import { shipped, shippedSkillDigestsMock } from './helpers/shipped-skills.js'; const PACKAGE_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..'); @@ -48,7 +49,7 @@ vi.mock('../utils/logger.js', () => ({ import { uninstall } from '../uninstall.js'; import { EnvHandler } from '../resources/env.js'; -import { TeamaiConfigSchema } from '../types.js'; +import { TeamaiConfigSchema, getDataHome, managedMcpManifestKey, managedMcpManifestPath } from '../types.js'; import { ModelProfileSchema, resolveProfile } from '../models/profile.js'; import { switchModelProfile } from '../models/switch.js'; import type { TeamaiConfig, LocalConfig } from '../types.js'; @@ -755,6 +756,508 @@ describe('uninstall', () => { expect(after.mcpServers['my-own']).toEqual({ command: 'my-server' }); }); + it('project-scope uninstall removes only the teamai block from .git/info/exclude (#882)', async () => { + const { homeDir, repoPath } = await setupFixture(tmpDir); + const projectRoot = path.join(tmpDir, 'business-repo'); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('SHELL', '/bin/zsh'); + await fse.outputFile(path.join(projectRoot, '.claude', 'skills', 'team-skill', 'SKILL.md'), '# Team Skill'); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + const excludeFile = path.join(projectRoot, '.git', 'info', 'exclude'); + await fse.writeFile(excludeFile, [ + '# my own', + 'scratch/', + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/.mcp.json', + '# [teamai:mcp-exclude:end]', + '*.local', + '', + ].join('\n')); + + const localConfig = makeLocalConfig(homeDir, repoPath, { + scope: 'project', + projectRoot, + repo: { localPath: repoPath, remote: '', kind: 'self', businessRepoRoot: projectRoot }, + }); + mockAutoDetectInit.mockResolvedValue({ localConfig, teamConfig: makeTeamConfig() }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe('# my own\nscratch/\n*.local\n'); + }); + + it('project-scope uninstall removes the .git/info/exclude block when it is all teamai left (#882)', async () => { + const homeDir = path.join(tmpDir, 'home'); + const repoPath = path.join(tmpDir, 'team-repo'); + const projectRoot = path.join(tmpDir, 'business-repo'); + await fse.ensureDir(homeDir); + await fse.ensureDir(repoPath); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('SHELL', '/bin/zsh'); + await fse.ensureDir(projectRoot); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + const excludeFile = path.join(projectRoot, '.git', 'info', 'exclude'); + await fse.writeFile(excludeFile, [ + 'scratch/', + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/.mcp.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n')); + + const localConfig = makeLocalConfig(homeDir, repoPath, { + scope: 'project', + projectRoot, + repo: { localPath: repoPath, remote: '', kind: 'self', businessRepoRoot: projectRoot }, + }); + mockAutoDetectInit.mockResolvedValue({ localConfig, teamConfig: makeTeamConfig() }); + const { log } = await import('../utils/logger.js'); + vi.mocked(log.info).mockClear(); + + await uninstall({ force: true }); + + expect(log.info).not.toHaveBeenCalledWith('Nothing to uninstall'); + expect(await fse.readFile(excludeFile, 'utf8')).toBe('scratch/\n'); + }); + + it('project-scope uninstall keeps the .git/info/exclude block while a config still holds teamai servers (#882)', async () => { + const { homeDir, repoPath } = await setupFixture(tmpDir); + const projectRoot = path.join(tmpDir, 'business-repo'); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('SHELL', '/bin/zsh'); + await fse.ensureDir(path.join(projectRoot, '.claude', 'skills')); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + // Hand-edited into invalid JSON: uninstall cannot take the resolved token out. + await fse.writeFile(path.join(projectRoot, '.mcp.json'), '{ "mcpServers": { "jira": { "headers": { "Authorization": "Bearer t0ken" } } },\n'); + const excludeFile = path.join(projectRoot, '.git', 'info', 'exclude'); + const block = [ + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/.mcp.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n'); + await fse.writeFile(excludeFile, block); + + const localConfig = makeLocalConfig(homeDir, repoPath, { + scope: 'project', + projectRoot, + repo: { localPath: repoPath, remote: '', kind: 'self', businessRepoRoot: projectRoot }, + }); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'jira', hash: 'h' }], + }); + const teamConfig = makeTeamConfig({ + toolPaths: { claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: '.mcp.json' } }, + }); + mockAutoDetectInit.mockResolvedValue({ localConfig, teamConfig }); + const { log } = await import('../utils/logger.js'); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining(`Kept \`/.mcp.json\` in ${await fse.realpath(excludeFile)}`)); + }); + + describe('the block protects a config holding a resolved value (#882)', () => { + const block = [ + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/.mcp.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n'); + const jira = { type: 'http', url: 'https://jira.example/mcp', headers: { Authorization: 'Bearer t0ken' } }; + + async function setup(): Promise<{ homeDir: string; repoPath: string; projectRoot: string; excludeFile: string; localConfig: LocalConfig }> { + const { homeDir, repoPath } = await setupFixture(tmpDir); + const projectRoot = path.join(tmpDir, 'business-repo'); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('SHELL', '/bin/zsh'); + await fse.outputFile(path.join(repoPath, 'mcp', 'mcp.yaml'), [ + 'servers:', + ' - name: jira', + ' transport: http', + ' url: https://jira.example/mcp', + ' headers:', + ' Authorization: "Bearer ${JIRA_TOKEN}"', + '', + ].join('\n')); + await fse.ensureDir(path.join(projectRoot, '.claude', 'skills')); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + const excludeFile = path.join(projectRoot, '.git', 'info', 'exclude'); + await fse.writeFile(excludeFile, block); + const localConfig = makeLocalConfig(homeDir, repoPath, { scope: 'project', projectRoot }); + const teamConfig = makeTeamConfig({ + toolPaths: { claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: '.mcp.json' } }, + }); + mockAutoDetectInit.mockResolvedValue({ localConfig, teamConfig }); + return { homeDir, repoPath, projectRoot, excludeFile, localConfig }; + } + + it('keeps the block when managed-mcp.json is gone and the token is still in .mcp.json', async () => { + const { projectRoot, excludeFile } = await setup(); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: { jira } }); + const { log } = await import('../utils/logger.js'); + + await uninstall({ force: true }); + + expect(await fse.readJson(path.join(projectRoot, '.mcp.json'))).toEqual({ mcpServers: { jira } }); + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining(`Kept \`/.mcp.json\` in ${await fse.realpath(excludeFile)}`)); + }); + + it('keeps the block when a server dropped from mcp.yaml left its token behind with no manifest', async () => { + const { repoPath, projectRoot, excludeFile } = await setup(); + await fse.outputFile(path.join(repoPath, 'mcp', 'mcp.yaml'), 'servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + vi.stubEnv('JIRA_TOKEN', 't0ken-still-set-9f2'); + const stale = { ...jira, headers: { Authorization: 'Bearer t0ken-still-set-9f2' } }; + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: { jira: stale } }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + }); + + it('keeps the entry, naming the file, when the manifest is lost, the server left mcp.yaml and its value is not set', async () => { + const { repoPath, projectRoot, excludeFile } = await setup(); + await fse.outputFile(path.join(repoPath, 'mcp', 'mcp.yaml'), 'servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: { jira } }); + const { log } = await import('../utils/logger.js'); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + const warning = vi.mocked(log.warn).mock.calls.map(([message]) => String(message)).find((m) => m.includes('/.mcp.json')); + expect(warning).toContain(path.join(await fse.realpath(projectRoot), '.mcp.json')); + expect(warning).toContain(await fse.realpath(excludeFile)); + }); + + it.each([ + ['empty', ''], + ['truncated', '{ "claude:project": [ { "name": "ji'], + ])('keeps the entry when managed-mcp.json is %s, the server left mcp.yaml and its value is not set', async (_label, content) => { + const { repoPath, projectRoot, excludeFile, localConfig } = await setup(); + await fse.outputFile(path.join(repoPath, 'mcp', 'mcp.yaml'), 'servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: { jira } }); + await fse.outputFile(managedMcpManifestPath(getDataHome(localConfig), projectRoot), content); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + }); + + it('removes the entry when the file holds no server, with no manifest', async () => { + const { projectRoot, excludeFile } = await setup(); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: {} }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(''); + }); + + it('removes only the entry whose file is gone', async () => { + const { projectRoot, excludeFile } = await setup(); + await fse.writeFile(excludeFile, [ + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/.mcp.json', + '/other.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n')); + await fse.writeJson(path.join(projectRoot, 'other.json'), { mcpServers: { jira } }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe([ + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/other.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n')); + }); + + // CodeBuddy stays at its built-in .mcp.json: moved or dropped, any server there Claude's records don't own holds it. + const withCodeBuddy = (localConfig: LocalConfig): void => { + mockAutoDetectInit.mockResolvedValue({ + localConfig, + teamConfig: makeTeamConfig({ + toolPaths: { + claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: '.mcp.json' }, + codebuddy: { skills: '.codebuddy/skills', mcp: '.codebuddy/mcp.json', mcpProject: '.mcp.json' }, + }, + }), + }); + }; + + it('names the variable whose value keeps the block', async () => { + const { projectRoot, excludeFile, localConfig } = await setup(); + withCodeBuddy(localConfig); + vi.stubEnv('TEAM_BASE_URL', 'https://base.example'); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: { mine: { url: 'https://base.example/mcp' } } }); + const { log } = await import('../utils/logger.js'); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining('$TEAM_BASE_URL')); + }); + + it('keeps the block while a tool uninstall no longer reaches still holds teamai\'s server', async () => { + const { repoPath, projectRoot, excludeFile, localConfig } = await setup(); + // The server left mcp.yaml, its variable is not set here, and Claude is no longer detected. + await fse.outputFile(path.join(repoPath, 'mcp', 'mcp.yaml'), 'servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + await fse.remove(path.join(projectRoot, '.claude')); + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: { jira } }); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'jira', hash: 'h' }], + }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + }); + + it('keeps the block, naming the server, while the config holds one that was there when a pull rebuilt the lost record', async () => { + const { repoPath, projectRoot, excludeFile, localConfig } = await setup(); + await fse.outputFile(path.join(repoPath, 'mcp', 'mcp.yaml'), 'servers:\n - name: docs\n transport: http\n url: https://docs.example/mcp\n'); + const file = path.join(projectRoot, '.mcp.json'); + await fse.writeJson(file, { mcpServers: { jira, docs: { type: 'http', url: 'https://docs.example/mcp' } } }); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'docs', hash: 'h' }], + }); + const { trackResolvedMcpFiles, recordUnverifiedMcpServers } = await import('../mcp-resolved-files.js'); + await trackResolvedMcpFiles(localConfig, [{ tool: 'claude', file }]); + expect(await recordUnverifiedMcpServers(localConfig, [{ file, names: ['jira'] }])).toBe('written'); + const { log } = await import('../utils/logger.js'); + + await uninstall({ force: true }); + + expect(await fse.readJson(file)).toEqual({ mcpServers: { jira } }); + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + const warning = vi.mocked(log.warn).mock.calls.map(([message]) => String(message)).find((m) => m.includes('/.mcp.json')); + expect(warning).toContain('jira'); + }); + + describe('for a config a pull wrote under a mcpProject the team has since changed', () => { + const oldBlock = block.replace('/.mcp.json', '/.cursor/team-mcp.json'); + + async function setupRecorded(content: unknown): Promise<{ excludeFile: string; old: string }> { + const { projectRoot, excludeFile, localConfig } = await setup(); + const old = path.join(projectRoot, '.cursor', 'team-mcp.json'); + await fse.outputJson(old, content); + const { trackResolvedMcpFiles } = await import('../mcp-resolved-files.js'); + expect(await trackResolvedMcpFiles(localConfig, [{ tool: 'cursor', file: old }])).toBe('written'); + await fse.writeFile(excludeFile, oldBlock); + return { excludeFile, old }; + } + + it('keeps the block while it holds a server, naming it', async () => { + const { excludeFile, old } = await setupRecorded({ mcpServers: { jira } }); + const { log } = await import('../utils/logger.js'); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(oldBlock); + const warning = vi.mocked(log.warn).mock.calls.map(([message]) => String(message)).find((m) => m.includes('team-mcp.json')); + expect(warning).toContain(path.join(await fse.realpath(path.dirname(old)), 'team-mcp.json')); + expect(warning).toContain('earlier toolPaths mapping'); + }); + + it('removes the block once it holds no server', async () => { + const { excludeFile } = await setupRecorded({ mcpServers: {} }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(''); + }); + }); + + it('removes the block once uninstall has taken teamai\'s servers out of .mcp.json', async () => { + const { homeDir, projectRoot, excludeFile, localConfig } = await setup(); + // A path and the login name are in the environment and in ordinary configs: neither holds the block. + vi.stubEnv('USER', 'longusername1'); + const mine = { command: path.join(homeDir, 'bin', 'mine'), env: { OWNER: 'longusername1' } }; + await fse.writeJson(path.join(projectRoot, '.mcp.json'), { mcpServers: { jira, mine } }); + withCodeBuddy(localConfig); + await fse.outputJson(managedMcpManifestPath(getDataHome(localConfig), projectRoot), { + [managedMcpManifestKey('claude', true)]: [{ name: 'jira', hash: 'h' }], + [managedMcpManifestKey('codebuddy', true)]: [], + }); + + await uninstall({ force: true }); + + expect(await fse.readJson(path.join(projectRoot, '.mcp.json'))).toEqual({ mcpServers: { mine } }); + expect(await fse.readFile(excludeFile, 'utf8')).toBe(''); + }); + // The appliers replace the file itself but follow its directories (#886). + describe('for a config under a symlinked directory, judged where the write lands', () => { + const landedBlock = block.replace('/.mcp.json', '/config/mcp.json'); + + async function setupLinked(servers: Record): Promise<{ excludeFile: string }> { + const { projectRoot, excludeFile, localConfig } = await setup(); + await fse.outputJson(path.join(projectRoot, 'config', 'mcp.json'), { mcpServers: servers }); + await fse.symlink('config', path.join(projectRoot, 'cfg'), 'dir'); + await fse.writeFile(excludeFile, landedBlock); + mockAutoDetectInit.mockResolvedValue({ + localConfig, + teamConfig: makeTeamConfig({ toolPaths: { claude: { skills: '.claude/skills', mcp: '.claude.json', mcpProject: 'cfg/mcp.json' } } }), + }); + return { excludeFile }; + } + + it('keeps the landing path\'s line while the file there holds the token', async () => { + const { excludeFile } = await setupLinked({ jira }); + const { log } = await import('../utils/logger.js'); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(landedBlock); + expect(log.warn).toHaveBeenCalledWith(expect.stringContaining(`Kept \`/config/mcp.json\` in ${await fse.realpath(excludeFile)}`)); + }); + + it('removes it once the file there holds no server', async () => { + const { excludeFile } = await setupLinked({}); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(''); + }); + }); + }); + + it('project-scope uninstall keeps a nested repository\'s block while its linked worktree holds a token (#882)', async () => { + const homeDir = path.join(tmpDir, 'home'); + const repoPath = path.join(tmpDir, 'team-repo'); + const projectRoot = path.join(tmpDir, 'business-repo'); + await fse.ensureDir(homeDir); + await fse.ensureDir(repoPath); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('SHELL', '/bin/zsh'); + const cursorDir = path.join(projectRoot, '.cursor'); + await fse.ensureDir(path.join(cursorDir, 'skills')); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + execFileSync('git', ['init', '-q'], { cwd: cursorDir }); + execFileSync('git', ['-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '--allow-empty', '-m', 'init'], { cwd: cursorDir }); + const linked = path.join(tmpDir, 'cursor-linked'); + execFileSync('git', ['worktree', 'add', '-q', linked], { cwd: cursorDir }); + await fse.writeJson(path.join(linked, 'mcp.json'), { mcpServers: { jira: { headers: { Authorization: 'Bearer t0ken' } } } }); + const excludeFile = path.join(cursorDir, '.git', 'info', 'exclude'); + const block = [ + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/mcp.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n'); + await fse.writeFile(excludeFile, block); + + const localConfig = makeLocalConfig(homeDir, repoPath, { + scope: 'project', + projectRoot, + repo: { localPath: repoPath, remote: '', kind: 'self', businessRepoRoot: projectRoot }, + }); + const teamConfig = makeTeamConfig({ + toolPaths: { cursor: { skills: '.cursor/skills', mcp: '.cursor/mcp.json', mcpProject: '.cursor/mcp.json' } }, + }); + mockAutoDetectInit.mockResolvedValue({ localConfig, teamConfig }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe(block); + }); + + describe('a nested repository\'s linked worktree (#882)', () => { + async function setupNestedLinked(content: unknown): Promise<{ excludeFile: string; linked: string; cursorDir: string }> { + const homeDir = path.join(tmpDir, 'home'); + const repoPath = path.join(tmpDir, 'team-repo'); + const projectRoot = path.join(tmpDir, 'business-repo'); + await fse.ensureDir(homeDir); + await fse.ensureDir(repoPath); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('SHELL', '/bin/zsh'); + const cursorDir = path.join(projectRoot, '.cursor'); + await fse.ensureDir(path.join(cursorDir, 'skills')); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + execFileSync('git', ['init', '-q'], { cwd: cursorDir }); + execFileSync('git', ['-c', 'user.name=t', '-c', 'user.email=t@t', 'commit', '-q', '--allow-empty', '-m', 'init'], { cwd: cursorDir }); + const linked = path.join(tmpDir, 'cursor-linked'); + execFileSync('git', ['worktree', 'add', '-q', linked], { cwd: cursorDir }); + await fse.writeJson(path.join(linked, 'mcp.json'), content); + const excludeFile = path.join(cursorDir, '.git', 'info', 'exclude'); + await fse.writeFile(excludeFile, [ + 'scratch/', + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/mcp.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n')); + const localConfig = makeLocalConfig(homeDir, repoPath, { + scope: 'project', + projectRoot, + repo: { localPath: repoPath, remote: '', kind: 'self', businessRepoRoot: projectRoot }, + }); + const teamConfig = makeTeamConfig({ + toolPaths: { cursor: { skills: '.cursor/skills', mcp: '.cursor/mcp.json', mcpProject: '.cursor/mcp.json' } }, + }); + mockAutoDetectInit.mockResolvedValue({ localConfig, teamConfig }); + return { excludeFile, linked, cursorDir }; + } + + it('removes the block when the config there holds no server', async () => { + const { excludeFile } = await setupNestedLinked({ mcpServers: {} }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe('scratch/\n'); + }); + + it('keeps the block while the config there holds a server, saying teamai cannot judge it', async () => { + const { excludeFile, linked, cursorDir } = await setupNestedLinked({ mcpServers: { mine: { url: 'https://mine.example/mcp' } } }); + const { log } = await import('../utils/logger.js'); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toContain('/mcp.json'); + const warning = vi.mocked(log.warn).mock.calls.map(([message]) => String(message)).find((m) => m.includes('/mcp.json')); + expect(warning).toContain(path.join(await fse.realpath(linked), 'mcp.json')); + expect(warning).toContain(`in a linked worktree of the repository at ${await fse.realpath(cursorDir)}`); + }); + }); + + it('project-scope uninstall removes the block from a nested repository holding an MCP config (#882)', async () => { + const homeDir = path.join(tmpDir, 'home'); + const repoPath = path.join(tmpDir, 'team-repo'); + const projectRoot = path.join(tmpDir, 'business-repo'); + await fse.ensureDir(homeDir); + await fse.ensureDir(repoPath); + vi.stubEnv('HOME', homeDir); + vi.stubEnv('SHELL', '/bin/zsh'); + const cursorDir = path.join(projectRoot, '.cursor'); + await fse.ensureDir(path.join(cursorDir, 'skills')); + execFileSync('git', ['init', '-q'], { cwd: projectRoot }); + execFileSync('git', ['init', '-q'], { cwd: cursorDir }); + const excludeFile = path.join(cursorDir, '.git', 'info', 'exclude'); + await fse.writeFile(excludeFile, [ + 'scratch/', + '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values', + '/mcp.json', + '# [teamai:mcp-exclude:end]', + '', + ].join('\n')); + + const localConfig = makeLocalConfig(homeDir, repoPath, { + scope: 'project', + projectRoot, + repo: { localPath: repoPath, remote: '', kind: 'self', businessRepoRoot: projectRoot }, + }); + const teamConfig = makeTeamConfig({ + toolPaths: { cursor: { skills: '.cursor/skills', mcp: '.cursor/mcp.json', mcpProject: '.cursor/mcp.json' } }, + }); + mockAutoDetectInit.mockResolvedValue({ localConfig, teamConfig }); + + await uninstall({ force: true }); + + expect(await fse.readFile(excludeFile, 'utf8')).toBe('scratch/\n'); + }); + it('移除 OpenClaw 系 agent 的 HOOK.md 目录（无 settings 路径）', async () => { const { homeDir, repoPath, teamaiHome } = await setupFixture(tmpDir); vi.stubEnv('HOME', homeDir); diff --git a/src/config.ts b/src/config.ts index bd9ead2e1..74913d948 100644 --- a/src/config.ts +++ b/src/config.ts @@ -371,15 +371,23 @@ export async function resolveDataHomeForScope(scope: Scope, projectRoot?: string * longer exists (a hook payload naming a deleted worktree) holds no project * config; git refuses to open it, so it is not asked. Hooks go through * resolveHookConfig (dashboard-collector.ts), which gives such a payload the - * scope its session last recorded (#810). + * scope its session last recorded (#810). `onUnreadable` is told which file + * could not be read, for a caller that names it. */ -export async function resolveConfigForDir(dir?: string): Promise { +export async function resolveConfigForDir( + dir?: string, + onUnreadable?: UnreadableConfigSink, + options: LoadOptions = {}, +): Promise { const target = dir ?? process.cwd(); - if (!(await pathExists(target))) return loadLocalConfig(); + if (!(await pathExists(target))) return loadLocalConfig(options); let unreadable = false; - const project = await detectProjectConfig(target, () => { unreadable = true; }); + const project = await detectProjectConfig(target, (configPath, error) => { + unreadable = true; + onUnreadable?.(configPath, error); + }, options); if (unreadable) return null; - return project ?? loadLocalConfig(); + return project ?? loadLocalConfig(options); } /** diff --git a/src/doctor-delivery.ts b/src/doctor-delivery.ts index 844e04233..2a330c207 100644 --- a/src/doctor-delivery.ts +++ b/src/doctor-delivery.ts @@ -2,12 +2,14 @@ import path from 'node:path'; import fs from 'node:fs'; import { isDeepStrictEqual } from 'node:util'; import { expandHome, listFilesRecursive, pathExists, readFileSafe } from './utils/fs.js'; -import { getDataHome, getMcpSharing, isAgentExcluded } from './types.js'; -import type { DeliveryTarget, LocalConfig, ResourceItem, TeamaiConfig } from './types.js'; -import type { EntryResolution, EntryType } from './namespaced-entries.js'; +import { getDataHome, getMcpSharing, isAgentExcluded, managedMcpManifestKey } from './types.js'; +import type { DeliveryTarget, LocalConfig, ManagedMcpManifest, ResourceItem, TeamaiConfig } from './types.js'; +import type { EntryLayout, EntryResolution } from './namespaced-entries.js'; import { splitFrontmatter } from './utils/frontmatter.js'; import type { ResourceHandler } from './resources/base.js'; import type { Check, DoctorContext } from './doctor.js'; +import type { DesiredMcpContext } from './mcp-reconcile.js'; +import type { ResolvedMcpFile } from './mcp-resolved-files.js'; import { findEnvBlockFor, envBlockSourcesPath, @@ -463,6 +465,7 @@ export async function buildMcpDeliveryChecks(ctx: DoctorContext): Promise !excludedByUser.has(change.server)) + .filter((change) => !excludedByUser.has(change.server) && !kept.has(change.server)) .map((change) => `${change.server} (${change.reason ?? 'skipped'})`); const problems: string[] = []; + // Its fix is the exclusion's own, not another pull (#882). + let withheld: string | undefined; const installed = await installedMcpEntries(target); if (installed === null) { problems.push(`${target.file} could not be parsed, so no server was injected`); @@ -510,28 +517,131 @@ export async function buildMcpDeliveryChecks(ctx: DoctorContext): Promise 0) problems.push(`not injected: ${nameList(absent)}`); - if (foreign.length > 0) problems.push(`not the team's definition: ${nameList(foreign)}`); + // Pull writes a resolved value only into a file git leaves out of a + // commit (#882), and otherwise leaves the whole file as it was. + const exclusion = carriesResolvedValue(target, teamDefs, [...absent, ...foreign]) + ? await ensureExcludedFromGit(target.file, { dryRun: true }) + : undefined; + if (exclusion?.kind === 'failed') { + withheld = `In ${target.file}, withheld: ${nameList([...absent, ...foreign])}, as git would commit the file: ${exclusion.reason}. ${exclusion.fix}`; + } else { + if (absent.length > 0) problems.push(`not injected: ${nameList(absent)}`); + if (foreign.length > 0) problems.push(`not the team's definition: ${nameList(foreign)}`); + } } if (blocked.length > 0) problems.push(`skipped: ${nameList(blocked)}`); - if (problems.length === 0 && desired.size === 0) continue; + if (problems.length === 0 && !withheld && desired.size === 0) continue; + const delivery = problems.length === 0 ? [] : [`In ${target.file}, ${problems.join('; ')}. A server needing a variable reads it from ` + + '`env/env.yaml` or an active `env//env.yaml`, whose top-level key is `variables:` — a plain `KEY: value` mapping ' + + 'parses as no variables at all. Then run `teamai pull --force`: a pull leaves an entry ' + + 'teamai does not own untouched, so a server of your own under a team name only gives ' + + 'way to `--force`.']; checks.push({ name: `MCP servers delivered to ${target.tool}`, source: 'local', - check: async () => problems.length === 0, - fix: `In ${target.file}, ${problems.join('; ')}. A server needing a variable reads it from ` - + '`env/env.yaml` or an active `env//env.yaml`, whose top-level key is `variables:` — a plain `KEY: value` mapping ' - + 'parses as no variables at all. Then run `teamai pull --force`: a pull leaves an entry ' - + 'teamai does not own untouched, so a server of your own under a team name only gives ' - + 'way to `--force`.', + check: async () => problems.length === 0 && !withheld, + fix: [...withheld ? [withheld] : [], ...delivery].join(' '), }); } return checks; } +/** + * A project MCP config holding a resolved `${VAR}` that git would commit + * (#882). Pull lists such a file in `.git/info/exclude`; this is the standing + * check for a file that is tracked already, or a repo whose exclude could not + * be written. Read-only: `git check-ignore` changes nothing. + */ +export async function buildMcpGitExcludeCheck(ctx: DoctorContext): Promise { + const { localConfig, teamConfig } = ctx; + const { projectRoot } = localConfig; + if (!teamConfig || localConfig.scope !== 'project' || !projectRoot || localConfig.repo.kind === 'http') return []; + + const { + resolveMcpTargets, resolvedValueEvidence, buildVarTable, buildDesiredMcpContext, recordedMcpTargets, recordedMcpFileEvidence, + earlierMappedMcpTargets, earlierMappedMcpFileEvidence, unrecordedMcpTool, unmappedMcpDefaults, unrecordedUnmappedMcpDefaults, unclaimedMcpServers, + } = await import('./mcp-reconcile.js'); + const { readResolvedMcpFiles } = await import('./mcp-resolved-files.js'); + const { gitPathOf, gitTracking, gitTracks } = await import('./mcp-git-exclude.js'); + const { sameServerKey } = await import('./resources/mcp-format.js'); + const { mcpEntryReader, teamMcpToDef } = await import('./resources/mcp.js'); + const { resolveEntriesFor } = await import('./namespaced-entries.js'); + const { loadProjectMcpManifest } = await import('./utils/mcp-manifest.js'); + + // Unreadable team servers still leave teamai's entries on disk: judged by the manifest, as pull does. + const resolution = await resolveEntriesFor(mcpEntryReader, localConfig); + const teamDefs = resolution.kind === 'failed' ? null : resolution.entries.map((entry) => teamMcpToDef(entry.entry)); + let manifest: ManagedMcpManifest | undefined; + let vars: Record | undefined; + let ledger: Record | undefined; + let desiredContext: Promise | undefined; + const desired = (): Promise => desiredContext ??= buildDesiredMcpContext(teamConfig, localConfig); + + const holding = new Set(); + const tracked: string[] = []; + const hold = async (file: string): Promise => { + holding.add(file); + const tracking = await gitTracking(file); + if (tracking.kind === 'would-commit') tracked.push((await gitPathOf(file)).label); + else if (tracking.kind === 'unknown') tracked.push(`${(await gitPathOf(file)).label} (git failed: ${tracking.error})`); + }; + // Every tool's file, delivery on or off, the same files and evidence pull protects. Two tools may share one. + const mapped = await resolveMcpTargets(teamConfig, localConfig, { includeUndetected: true }); + // A built-in location no mapping reaches today (its tool moved or dropped): its tool's records describe another file. + const unmapped = await unmappedMcpDefaults(mapped); + const targets = mapped.filter((target) => !unmapped.has(target)); + for (const target of targets) { + if (holding.has(target.file) || !await pathExists(target.file)) continue; + manifest ??= (await loadProjectMcpManifest(getDataHome(localConfig), projectRoot, { dryRun: true })).manifest; + vars ??= await buildVarTable(localConfig); + ledger ??= (await readResolvedMcpFiles(localConfig)).files; + const owned = manifest[managedMcpManifestKey(target.tool, true)] ?? []; + // No managed-mcp.json at all, no record for this installed tool the team maps, or a record a pull wrote + // without one whose note hasn't landed: any server no record claims may be teamai's, as pull judges it. + const claimed = targets.filter((t) => t.file === target.file && sameServerKey(t.format, target.format)) + .flatMap((t) => manifest?.[managedMcpManifestKey(t.tool, true)] ?? []).map((record) => record.name); + const unrecorded = unrecordedMcpTool(target, targets, ledger[target.file]?.tools) && manifest[managedMcpManifestKey(target.tool, true)] === undefined; + if (((Object.keys(manifest).length === 0 || unrecorded || owned.some((record) => record.unnoted)) + && (await unclaimedMcpServers(target, claimed)).length > 0) + || await resolvedValueEvidence(target, teamDefs, { owned, unverified: ledger[target.file]?.unverified }, vars, desired)) await hold(target.file); + } + // And a file a pull wrote under a mapping the team has since changed, but one recorded as tracked while git + // tracks it: no line protects it. In a file another tool now maps, that tool's records tell its own servers. + for (const [file, { targets: group, mappedBy, tracked }] of await recordedMcpTargets(localConfig, targets)) { + if (holding.has(file) || (tracked && (await gitTracks(file)).kind === 'tracked')) continue; + manifest ??= (await loadProjectMcpManifest(getDataHome(localConfig), projectRoot, { dryRun: true })).manifest; + const owned = mappedBy.length === 0 ? undefined + : mappedBy.flatMap((tool) => manifest?.[managedMcpManifestKey(tool, true)] ?? []).map((record) => record.name); + if (await recordedMcpFileEvidence(group, owned)) await hold(file); + } + // And, until a pull on this version reads them, those an older teamai wrote under a mapping an earlier + // teamai.yaml made. Read-only: the record of that read is pull's. Unreadable history skips them. + // A built-in location no mapping reaches today, which no record covers, is judged as one of them. + const earlier = (await readResolvedMcpFiles(localConfig)).earlierMappingsRead ? [] + : await earlierMappedMcpTargets(localConfig, mapped).catch(() => null) ?? []; + for (const { tracked, mappedBy, ...target } of [...earlier, ...await unrecordedUnmappedMcpDefaults(localConfig, unmapped, targets)]) { + if (tracked || holding.has(target.file)) continue; + vars ??= await buildVarTable(localConfig); + manifest ??= (await loadProjectMcpManifest(getDataHome(localConfig), projectRoot, { dryRun: true })).manifest; + const owned = mappedBy.length === 0 ? undefined + : mappedBy.flatMap((tool) => manifest?.[managedMcpManifestKey(tool, true)] ?? []).map((record) => record.name); + if (await earlierMappedMcpFileEvidence(target, teamDefs, vars, desired, owned)) await hold(target.file); + } + if (holding.size === 0) return []; + + return [{ + name: 'Project MCP configs with resolved values are kept out of git', + source: 'local', + check: async () => tracked.length === 0, + fix: `${tracked.join(', ')} may hold MCP variables resolved to plaintext, and git would commit them or cannot say. ` + + 'Fix any git error shown, then run `teamai pull` to list them in .git/info/exclude. If git already tracks one, run ' + + '`git rm --cached ` and rotate the values it held.', + }]; +} + /** * Env, hook and MCP entries carrying a key to fix: the per-entry `roles:` / * `projects:` keys that namespace files replace (#707), or a key the entry's @@ -556,26 +666,42 @@ export async function buildEntryScopeKeyCheck(ctx: DoctorContext): Promise { const { describeEntryFailure } = await import('./namespaced-entries.js'); - const names: Partial> = { - hooks: 'Team hooks can be resolved', - models: 'Team model profiles can be resolved', - }; const checks: Check[] = []; - for (const { type, resolution } of await resolveEntryTypes(ctx.localConfig)) { - const name = names[type]; - if (name === undefined || resolution.kind !== 'failed') continue; + for (const { checkName: name, resolution } of await resolveEntryTypes(ctx.localConfig)) { + if (name === null || resolution.kind !== 'failed') continue; checks.push({ name, source: 'local', check: async () => false, fix: describeEntryFailure(resolution.failure) }); } return checks; } +/** + * The member's values for this team and machine can be read (#875). While one + * can't, every secret has no value and MCP keeps what the last pull wrote, + * which the MCP check can't see. Only for a scope whose secrets or variables + * read those files. + */ +export function buildSecretValuesCheck(ctx: DoctorContext): Check[] { + const { teamEnv } = ctx; + if (!teamEnv) return []; + const reads = (teamEnv.declarations.kind === 'resolved' && teamEnv.declarations.entries.length > 0) + || (teamEnv.variables.kind === 'resolved' && teamEnv.variables.entries.length > 0); + if (!reads) return []; + const unreadable = [teamEnv.secrets, teamEnv.variableValues].find((values) => values.kind === 'store-unreadable'); + return [{ + name: 'Your team secret values can be read', + source: 'local', + check: async () => unreadable === undefined, + fix: unreadable?.kind === 'store-unreadable' ? unreadable.reason : undefined, + }]; +} + /** * Info lines for `doctor`: which namespace entry replaces which root entry, * and in legacy mode each name the root file repeats. They answer "why do I @@ -583,21 +709,42 @@ export async function buildEntryResolutionChecks(ctx: DoctorContext): Promise { const { describeEntryNotes } = await import('./namespaced-entries.js'); - return (await resolveEntryTypes(ctx.localConfig)).flatMap(({ type, resolution }) => describeEntryNotes(type, resolution)); + return (await resolveEntryTypes(ctx.localConfig)).flatMap(({ layout, resolution }) => describeEntryNotes(layout, resolution)); } -async function resolveEntryTypes(localConfig: LocalConfig): Promise<{ type: EntryType; resolution: EntryResolution }[]> { +/** + * Every namespaced entry file set, each with the layout its messages use and + * the doctor check that fails when it does not resolve (null for env and MCP, + * whose delivery checks report it). + */ +async function resolveEntryTypes( + localConfig: LocalConfig, +): Promise<{ layout: EntryLayout; resolution: EntryResolution; checkName: string | null }[]> { if (localConfig.repo.kind === 'http') return []; - const { resolveEntriesFor } = await import('./namespaced-entries.js'); + const { entryLayout, resolveEntriesFor } = await import('./namespaced-entries.js'); const { envEntryReader } = await import('./resources/env.js'); + const { SECRETS_LAYOUT, secretsEntryReader } = await import('./resources/secrets.js'); const { hooksEntryReader } = await import('./resources/hooks.js'); const { mcpEntryReader } = await import('./resources/mcp.js'); const { modelsEntryReader } = await import('./models/profile.js'); return [ - { type: 'env', resolution: await resolveEntriesFor(envEntryReader, localConfig) }, - { type: 'hooks', resolution: await resolveEntriesFor(hooksEntryReader, localConfig) }, - { type: 'mcp', resolution: await resolveEntriesFor(mcpEntryReader, localConfig) }, - { type: 'models', resolution: await resolveEntriesFor(modelsEntryReader, localConfig) }, + { layout: entryLayout('env'), resolution: await resolveEntriesFor(envEntryReader, localConfig), checkName: null }, + { + layout: SECRETS_LAYOUT, + resolution: await resolveEntriesFor(secretsEntryReader, localConfig), + checkName: 'Team secrets can be resolved', + }, + { + layout: entryLayout('hooks'), + resolution: await resolveEntriesFor(hooksEntryReader, localConfig), + checkName: 'Team hooks can be resolved', + }, + { layout: entryLayout('mcp'), resolution: await resolveEntriesFor(mcpEntryReader, localConfig), checkName: null }, + { + layout: entryLayout('models'), + resolution: await resolveEntriesFor(modelsEntryReader, localConfig), + checkName: 'Team model profiles can be resolved', + }, ]; } @@ -648,17 +795,26 @@ async function envDeliveryProblems( const none = { problems: [], staleProfiles: [] }; if (teamConfig?.sharing?.env?.injectShellProfile === false) return none; - const { EnvHandler, envEntryReader } = await import('./resources/env.js'); + const { EnvHandler } = await import('./resources/env.js'); const envHandler = new EnvHandler(); // The variables this member and directory receive: the same resolution pull // writes env.sh from, not a second copy of it. A file that cannot be used, or // a name defined twice, is reported here as pull reports it (#662), and a // deliberate `variables: []` is not. - const { resolveEntriesFor, describeEntryFailure } = await import('./namespaced-entries.js'); - const resolution = await resolveEntriesFor(envEntryReader, localConfig); + const { describeEntryFailure } = await import('./namespaced-entries.js'); + const { envShVariables, resolveTeamEnv } = await import('./env-resolution.js'); + const teamEnv = ctx.teamEnv ?? await resolveTeamEnv(localConfig); + const { variables: resolution, declarations: secrets, variableValues: values } = teamEnv; if (resolution.kind === 'failed') return { problems: [describeEntryFailure(resolution.failure)], staleProfiles: [] }; - const declared = resolution.entries.map((entry) => entry.entry); + // A key the team also declares as a secret is not delivered (#875); declarations + // that cannot be read keep env.sh as it is, as a broken env file does. + if (secrets.kind === 'failed') return { problems: [describeEntryFailure(secrets.failure)], staleProfiles: [] }; + // A variable the member set for this team is owed their value, and one set + // with `--from-env` is not owed at all (#875); a values file that cannot be + // read keeps env.sh as it is, as pull does. + if (values.kind === 'store-unreadable') return { problems: [values.reason], staleProfiles: [] }; + const declared = envShVariables(resolution.entries, values.values); const deliverable = new Set(declared.map((variable) => variable.key)); const problems: string[] = []; @@ -690,7 +846,7 @@ async function envDeliveryProblems( if (undelivered.length > 0) problems.push(`${envShPath} is missing ${nameList(undelivered)}`); if (stale.length > 0) { problems.push( - `${envShPath} has a stale value for ${nameList(stale)}: env.yaml declares a different one`, + `${envShPath} has a stale value for ${nameList(stale)}: env.yaml or your value for this team is a different one`, ); } // env.sh holds only what pull wrote, so a key the resolved set lacks is diff --git a/src/doctor.ts b/src/doctor.ts index 11f42887c..524d5bded 100644 --- a/src/doctor.ts +++ b/src/doctor.ts @@ -26,8 +26,10 @@ import { buildAgentsDeliveryChecks, buildNamespaceNotes, buildMcpDeliveryChecks, + buildMcpGitExcludeCheck, buildEnvDeliveryCheck, buildEntryResolutionChecks, + buildSecretValuesCheck, buildEntryScopeKeyCheck, entryNamespaceNotes, buildDocsCheck, @@ -42,6 +44,8 @@ import { */ export type CheckSource = 'local' | 'provider'; import { hasPiHooks } from './pi-hooks.js'; +import { describeEnvAdvisory, envAdvisories } from './env-advisories.js'; +import { resolveTeamEnv, type TeamEnv } from './env-resolution.js'; export interface Check { name: string; @@ -89,6 +93,8 @@ export interface DoctorContext { hookToolPaths: TeamaiConfig['toolPaths']; /** Where hooks are actually injected — see `resolveHookScope` (#264). */ baseDir: string; + /** This scope's env, resolved once for every check that reads it (env-resolution.ts); none in HTTP mode. */ + teamEnv?: TeamEnv; } export interface DoctorOptions extends GlobalOptions { @@ -111,7 +117,10 @@ export interface DoctorReport { checks: CheckResult[]; /** Present only when the team repo declares packages. Human text, not checks. */ packages?: { ok: boolean; lines: string[] }; - /** Advisories that are not checks: namespace overrides, the Codex trust-gate reminder. */ + /** + * Advisories that are not checks: namespace overrides, a team secret with no + * value (#875), the Codex trust-gate reminder. + */ notes?: string[]; } @@ -331,8 +340,9 @@ export async function resolveDoctorContext(): Promise { ) : {}; const baseDir = hookScope.baseDir; + const teamEnv = localConfig.repo.kind === 'http' ? undefined : await resolveTeamEnv(localConfig); - return { localConfig, teamConfig, toolPaths, hookToolPaths, baseDir }; + return { localConfig, teamConfig, toolPaths, hookToolPaths, baseDir, teamEnv }; } /** @@ -458,9 +468,11 @@ export async function buildChecks(ctx: DoctorContext, stage: CheckStage = 'docto ...(stage === 'doctor' ? await buildRulesDeliveryChecks(ctx) : []), ...(stage === 'doctor' ? await buildAgentsDeliveryChecks(ctx) : []), ...await buildMcpDeliveryChecks(ctx), + ...await buildMcpGitExcludeCheck(ctx), ...await buildDocsCheck(ctx), ...await buildEnvDeliveryCheck(ctx), ...await buildEntryResolutionChecks(ctx), + ...buildSecretValuesCheck(ctx), ...await buildEntryScopeKeyCheck(ctx), ); @@ -555,6 +567,7 @@ export async function doctor(options: DoctorOptions): Promise { const notes = [ ...await buildNamespaceNotes(ctx), ...await entryNamespaceNotes(ctx), + ...(await envAdvisories(localConfig, ctx.teamConfig, ctx.teamEnv)).map(describeEnvAdvisory), ...(codexNote ? [codexNote] : []), ]; diff --git a/src/env-advisories.ts b/src/env-advisories.ts new file mode 100644 index 000000000..4a127326d --- /dev/null +++ b/src/env-advisories.ts @@ -0,0 +1,150 @@ +/** + * What a member should know about this scope's env and team secrets (#875): + * one result that `pull`, `doctor`, `mcp list` and `env list` print from, so + * each says the same thing. None is a failure; `doctor` reports them as notes. + */ +import { keptMcpEntries } from './mcp-reconcile.js'; +import { resolveTeamEnv, secretState, type TeamEnv, type UnsetReference } from './env-resolution.js'; +import { referencedVars } from './resources/mcp-format.js'; +import { envName } from './resources/env-key.js'; +import { mcpEntryReader, teamMcpToDef } from './resources/mcp.js'; +import { declaredSecretKeys } from './resources/secrets.js'; +import { resolveEntriesFor } from './namespaced-entries.js'; +import type { LocalConfig, TeamaiConfig } from './types.js'; +import { log } from './utils/logger.js'; + +export type EnvAdvisory = + /** A declared secret with no value; `servers` are the team MCP servers that use it. */ + | { + readonly kind: 'missing-secret'; + readonly key: string; + readonly url?: string; + readonly servers: readonly string[]; + /** The member's entry for it reads this variable, which is unset. */ + readonly reference?: UnsetReference; + } + /** An entry an earlier pull wrote, kept while its secret is missing, so it may hold an old value. */ + | { readonly kind: 'kept-entry'; readonly server: string; readonly tools: readonly string[]; readonly keys: readonly string[] } + /** A key declared as a secret and also set as a variable in `source`, whose value is ignored. */ + | { readonly kind: 'secret-also-variable'; readonly key: string; readonly source: string } + /** A variable the member exports with another value than `source`'s, which this team uses (#875). */ + | { readonly kind: 'ignored-export'; readonly key: string; readonly source: string }; + +/** + * The advisories for this scope, from its declarations, so a secret no MCP + * server uses is reported too. `teamConfig` null leaves out the kept entries, + * which need the team's tool paths. Declarations or a store that cannot be + * read give none: the command reading them reports that failure itself. + * `teamEnv` is for a caller that already resolved it. + */ +export async function envAdvisories( + localConfig: LocalConfig, + teamConfig: TeamaiConfig | null, + teamEnv?: TeamEnv, +): Promise { + if (localConfig.repo.kind === 'http') return []; + const resolved = teamEnv ?? await resolveTeamEnv(localConfig); + const { declarations } = resolved; + if (declarations.kind === 'failed') return []; + const variables = resolved.variables.kind === 'resolved' ? resolved.variables.entries : []; + const ignored = ignoredExports(resolved); + if (declarations.kind === 'absent' || declarations.entries.length === 0) return ignored; + const secretKeys = declaredSecretKeys(declarations); + const mcp = await resolveEntriesFor(mcpEntryReader, localConfig); + const excluded = new Set(localConfig.excludedSkills ?? []); + const servers = (mcp.kind === 'resolved' ? mcp.entries : []) + .map((entry) => teamMcpToDef(entry.entry)) + .filter((server) => !excluded.has(server.name)); + const usedBy = (key: string): string[] => + servers.filter((server) => referencedVars(server).some((name) => envName(name) === envName(key))).map((server) => server.name); + + const advisories: EnvAdvisory[] = []; + for (const secret of declarations.entries) { + const state = secretState(resolved.secrets, secret.name); + switch (state) { + case 'missing': + advisories.push({ + kind: 'missing-secret', key: secret.name, url: secret.entry.url, servers: usedBy(secret.name), + reference: resolved.unsetReferences.get(secret.name), + }); + break; + // Nobody knows while the store can't be read; the command reports that itself. + case 'unreadable': + case 'team': + case 'global': + case 'environment': + break; + default: { + const unhandled: never = state; + return unhandled; + } + } + } + if (teamConfig) { + for (const [server, tools] of await keptMcpEntries(teamConfig, localConfig, resolved)) { + const def = servers.find((candidate) => candidate.name === server); + const keys = def ? referencedVars(def).filter((key) => secretKeys.has(key)) : []; + advisories.push({ kind: 'kept-entry', server, tools, keys }); + } + } + for (const variable of variables) { + if (secretKeys.has(variable.name)) advisories.push({ kind: 'secret-also-variable', key: variable.name, source: variable.source }); + } + return [...advisories, ...ignored]; +} + +/** Warn about each declared secret with no value, with the command that sets it (#875). */ +export async function reportMissingSecrets(localConfig: LocalConfig, teamEnv?: TeamEnv): Promise { + for (const advisory of await envAdvisories(localConfig, null, teamEnv)) { + if (advisory.kind === 'missing-secret') log.warn(describeEnvAdvisory(advisory)); + } +} + +/** + * The variables whose export the MCP servers and `env exec` no longer use: the + * member's own value (see member-env.ts), differing from the team's, for a key + * they set no value for with `teamai env set`. A store that cannot be read + * gives none. + */ +function ignoredExports(teamEnv: TeamEnv): EnvAdvisory[] { + const { variableValues: values, member } = teamEnv; + if (values.kind === 'store-unreadable' || values.values.size === 0) return []; + const variables = teamEnv.variables.kind === 'resolved' ? teamEnv.variables.entries : []; + return variables.flatMap((variable): EnvAdvisory[] => { + const resolved = values.values.get(variable.name); + if (!resolved || resolved.source !== 'env.yaml' || resolved.fromEnv) return []; + const exported = member(variable.name); + return exported !== undefined && exported !== resolved.value + ? [{ kind: 'ignored-export', key: variable.name, source: variable.source }] + : []; + }); +} + +/** The line a command prints for `advisory`. It never carries a value. */ +export function describeEnvAdvisory(advisory: EnvAdvisory): string { + switch (advisory.kind) { + case 'missing-secret': { + const servers = advisory.servers.length > 0 ? `${advisory.servers.join(', ')}: ` : ''; + const { reference } = advisory; + if (reference) { + return `${servers}${advisory.key} reads ${reference.variable}, which is not set. Set ${reference.variable}, ` + + `or run \`teamai env set ${advisory.key}${reference.global ? ' --global' : ''}\` to replace the reference.`; + } + const url = advisory.url ? ` (${advisory.url})` : ''; + return `${servers}${advisory.key} is not set. Run \`teamai env set ${advisory.key}\`${url}.`; + } + case 'kept-entry': + return `${advisory.server}: the entry an earlier pull wrote stays in ${advisory.tools.join(', ')} ` + + `and may hold an old ${advisory.keys.join(', ') || 'value'} until a pull finds its value.`; + case 'ignored-export': + return `${advisory.key} in your environment differs from the value in ${advisory.source}, which this team uses. ` + + `To use yours for this team, run \`teamai env set ${advisory.key}\`.`; + case 'secret-also-variable': + return `${advisory.key} is a team secret and is also set in ${advisory.source}, whose value is ignored. ` + + `Remove it from ${advisory.source} and run \`teamai push\`.`; + default: { + const unhandled: never = advisory; + return unhandled; + } + } +} diff --git a/src/env-commands.ts b/src/env-commands.ts index 65281e7fe..b120d1c41 100644 --- a/src/env-commands.ts +++ b/src/env-commands.ts @@ -1,94 +1,342 @@ -import { requireInit, detectProjectConfig } from './config.js'; +import { requireInit, detectProjectConfig, describeUnreadableConfig, NotInitializedError } from './config.js'; import { pullRepo } from './utils/git.js'; import { pathExists } from './utils/fs.js'; import { log, spinner } from './utils/logger.js'; -import { EnvHandler, maskEnvValue, ENV_KEY_RE, envEntryReader, unknownEnvVariableKeys, type EnvYaml } from './resources/env.js'; -import { describeEntryFailure, describeOrigin, entryFileAbsolutePath, entryFilePath, entryNamespaceFromFlags, moveTo, reportUndeliveredEntryNotices, resolveEntriesFor, TargetFiles } from './namespaced-entries.js'; +import { EnvHandler, envEntryReader, unknownEnvVariableKeys, type EnvYaml } from './resources/env.js'; +import { ENV_KEY_RE, envName, envValue, sameEnvName } from './resources/env-key.js'; +import { + SECRETS_LAYOUT, declaredSecretKeys, readSecretsForEdit, resolveSecretDeclarations, unknownSecretDeclarationKeys, + writeSecretsFile, +} from './resources/secrets.js'; +import { getMachineSecretsPath, getTeamSecretsPath, readSecretStore, updateSecretStore, type StoredSecret } from './secret-store.js'; +import { askSecret, isInteractive, readStdin } from './utils/prompt.js'; +import { reportMissingSecrets } from './env-advisories.js'; +import { envListing } from './env-listing.js'; +import { resolveTeamEnv } from './env-resolution.js'; +import { + describeEntryFailure, entryFileAbsolutePath, entryFilePath, entryLayout, entryNamespaceFromFlags, moveTo, reportUndeliveredEntryNotices, resolveEntriesFor, TargetFiles, + type EntryLayout, type EntryType, +} from './namespaced-entries.js'; import type { GlobalOptions, LocalConfig } from './types.js'; import { isSelfMode } from './types.js'; const envHandler = new EnvHandler(); /** - * List the team env variables this directory receives: env/env.yaml plus the - * active env//env.yaml files, each with the namespace it comes from. - * - * By default, values are masked. Pass `reveal: true` to show plaintext. + * List the team env variables this directory receives and the secrets it + * declares (env-listing.ts). By default, variable values are masked. Pass + * `reveal: true` to show plaintext. A file that cannot be used fails its own + * list only, and the command exits non-zero. */ export async function envList(options: GlobalOptions & { reveal?: boolean }): Promise { - const projectConfig = await detectProjectConfig(); - const localConfig = projectConfig ?? (await requireInit()).localConfig; - - const resolution = await resolveEntriesFor(envEntryReader, localConfig); - if (resolution.kind === 'failed') { - reportUndeliveredEntryNotices(resolution); - log.error(describeEntryFailure(resolution.failure)); - process.exitCode = 1; - return; - } + // Read-only, so `dryRun: true` unconditionally, as in `status` and `list` (#850). + const localConfig = await requireScope(true); + if (!localConfig) return; + const teamEnv = await resolveTeamEnv(localConfig); // An entry an unknown or removed key takes out of the delivered set never // appears in the list below, so say why it is missing (#822). - reportUndeliveredEntryNotices(resolution); - const variables = resolution.entries; - if (variables.length === 0) { - log.info('No env variables defined'); + reportUndeliveredEntryNotices(teamEnv.variables); + const listing = envListing(teamEnv, options); + for (const problem of listing.problems) fail(problem); + if (listing.lines.length === 0) { + if (listing.problems.length === 0) log.info('No env variables defined'); return; } + if (listing.revealed) process.stderr.write('[warn] Env values will be shown in plaintext\n'); + console.log(''); + for (const line of listing.lines) { + if (line.detail) log.dim(line.text); + else console.log(line.text); + } + if (listing.hasSecrets) await reportMissingSecrets(localConfig, teamEnv); +} - if (options.reveal) { - process.stderr.write('[warn] Env values will be shown in plaintext\n'); +/** + * Keep this member's value for a secret the scope declares, for this team + * repo, on this machine (#875); with `global`, for every team on the machine. + * Without `global`, also for an env variable the scope receives, which then + * replaces the team's value for this team (a machine value is for secrets only). + * The value comes from a hidden prompt, from piped stdin, or is a reference to + * another variable read each time it is used; never from an argument, so it + * stays out of shell history. + */ +export async function envSet( + typed: string, + options: GlobalOptions & { stdin?: boolean; fromEnv?: string; global?: boolean }, +): Promise { + // Stored under the name the scope declares, which on Windows may differ in case from the one typed. + let key = typed; + if (!ENV_KEY_RE.test(key)) return fail(invalidKeyMessage(key)); + if (options.stdin && options.fromEnv !== undefined) return fail('Pass either --stdin or --from-env, not both. Nothing was changed.'); + if (options.fromEnv !== undefined && !ENV_KEY_RE.test(options.fromEnv)) { + return fail(invalidKeyMessage(options.fromEnv, '--from-env variable name')); } - console.log(''); - console.log(`Team env variables (${variables.length}):`); - console.log(''); - for (const v of variables) { - const displayValue = options.reveal ? v.entry.value : maskEnvValue(v.entry.value); - console.log(` ${v.name}=${displayValue} (${describeOrigin(v)})`); - if (v.entry.description && options.verbose) { - log.dim(` ${v.entry.description}`); + const scope = await scopeHere(options.global, options.dryRun); + if (scope.kind === 'reported') return; + const localConfig = scope.kind === 'scope' ? scope.localConfig : null; + let isVariable = false; + if (localConfig) { + const declarations = await resolveSecretDeclarations(localConfig); + if (declarations.kind === 'failed') { + log.error(describeEntryFailure(declarations.failure)); + return fail(`Cannot tell whether ${key} is a secret this team declares. Nothing was changed.`); + } + const declared = declaredSecretKeys(declarations); + key = sameEnvName(declared, key) ?? key; + if (!declared.has(key)) { + if (options.global) { + const list = declared.size > 0 ? ` It declares: ${[...declared].sort().join(', ')}.` : ' It declares none.'; + return fail( + `${key} is not a secret this directory's team declares, so it was not set.${list} ` + + 'If the team declared it recently, run `teamai pull` first.', + ); + } + // #875: without --global, a member may also override a variable the scope receives, for this team. + const env = await resolveEntriesFor(envEntryReader, localConfig); + if (env.kind === 'failed') { + log.error(describeEntryFailure(env.failure)); + return fail(`Cannot tell whether ${key} is an env variable this team sets. Nothing was changed.`); + } + const variables = new Set(env.entries.map((variable) => variable.name).filter((name) => !declared.has(name))); + key = sameEnvName(variables, key) ?? key; + if (!variables.has(key)) { + const named = (keys: ReadonlySet): string => (keys.size > 0 ? [...keys].sort().join(', ') : 'none'); + return fail( + `${key} is neither a secret nor an env variable this directory's team declares, so it was not set. ` + + `Its secrets: ${named(declared)}. Its variables: ${named(variables)}. ` + + 'If the team added it recently, run `teamai pull` first.', + ); + } + isVariable = true; } } - console.log(''); + + const { file, target } = valuesFile(localConfig, options.global); + // Read before asking for the value, so an unusable store fails first; the write re-reads it under the lock. + const store = await readSecretStore(file); + if (!store.ok) return fail(`${store.reason} Nothing was changed.`); + if (options.dryRun) { + log.info(`[dry-run] Would set ${key} ${target} in ${file}`); + return; + } + + const input = await secretInput(key, options); + if (!input.ok) return fail(`${input.message} Nothing was changed.`); + const { entry } = input; + + // The kind the scope declares the key as now, so the value is never used as the other one (#879). + const kind = isVariable ? 'variable' : 'secret'; + // On Windows an entry under another case of the key is this key's: it is replaced too. + const update = await updateSecretStore(file, (values) => ({ ...withoutKey(values, key), [key]: { ...entry, kind } })); + if (update.kind === 'failed') return fail(`${update.reason} Nothing was changed.`); + if ('env' in entry) { + log.success(`${key} now reads ${entry.env} from your environment ${target} (${file}).`); + if (!envValue(process.env, entry.env)) log.warn(`${entry.env} is not set in this shell; ${key} has no value until it is.`); + } else { + log.success(`Set ${key} ${target} (${file}).`); + } + if (localConfig) { + log.info(isVariable ? 'Run `teamai pull` to update MCP servers and env.sh.' : 'Run `teamai pull` to update MCP servers.'); + } else { + log.info(`No teamai scope here, so no team declares ${key} yet. The value applies to every team on this machine that declares it.`); + } +} + +/** Remove this member's value for a secret or variable, for this team repo or, with `global`, for the machine. */ +export async function envUnset(key: string, options: GlobalOptions & { global?: boolean }): Promise { + if (!ENV_KEY_RE.test(key)) return fail(invalidKeyMessage(key)); + const scope = await scopeHere(options.global, options.dryRun); + if (scope.kind === 'reported') return; + const localConfig = scope.kind === 'scope' ? scope.localConfig : null; + + const { file, value } = valuesFile(localConfig, options.global); + const hasNoValue = (): void => { log.info(`${key} has no ${value}. Nothing was changed.`); }; + if (options.dryRun) { + const store = await readSecretStore(file); + if (!store.ok) return fail(`${store.reason} Nothing was changed.`); + const stored = sameEnvName(Object.keys(store.values), key); + if (stored === undefined) return hasNoValue(); + log.info(`[dry-run] Would remove ${stored}'s ${value} from ${file}`); + return; + } + // On Windows the stored name may differ in case from the one typed: it is the same variable. + const update = await updateSecretStore(file, (values) => (sameEnvName(Object.keys(values), key) === undefined ? null : withoutKey(values, key))); + switch (update.kind) { + case 'failed': + return fail(`${update.reason} Nothing was changed.`); + case 'unchanged': + return hasNoValue(); + case 'written': + break; + default: { + const unhandled: never = update; + return unhandled; + } + } + log.success(`Removed ${key}'s ${value} (${file}).`); + if (!localConfig) return; + log.info(`Run \`teamai pull\` to ${await unsetApplies(localConfig, key, options.global)}.`); +} + +/** + * This directory's scope. Outside any scope `env set --global` still has + * somewhere to write, so there `global` gives `none`; otherwise "not + * initialized" is reported (exit 1) and gives `reported`. A project config + * that cannot be read is reported too: detection would answer with the user + * scope, whose team may not be this project's (the rule `pull` follows, #784). + * A `dryRun` lookup writes nothing (#866). + */ +async function scopeHere( + global: boolean | undefined, + dryRun: boolean | undefined, +): Promise<{ kind: 'scope'; localConfig: LocalConfig } | { kind: 'none' } | { kind: 'reported' }> { + const unreadable: string[] = []; + const projectConfig = await detectProjectConfig( + process.cwd(), + (configPath, error) => { unreadable.push(`${configPath}: ${error}`); }, + { dryRun }, + ); + const [problem] = unreadable; + if (problem !== undefined) { + fail(`Cannot tell which team this directory belongs to: ${describeUnreadableConfig(problem)}`); + return { kind: 'reported' }; + } + if (projectConfig) return { kind: 'scope', localConfig: projectConfig }; + try { + return { kind: 'scope', localConfig: (await requireInit({ dryRun })).localConfig }; + } catch (e) { + if (!(e instanceof NotInitializedError)) throw e; + if (global) return { kind: 'none' }; + fail(e.message); + return { kind: 'reported' }; + } +} + +/** This directory's scope, or null once "not initialized" is reported. */ +async function requireScope(dryRun: boolean | undefined): Promise { + const scope = await scopeHere(false, dryRun); + return scope.kind === 'scope' ? scope.localConfig : null; +} + +/** + * The store `env set` / `env unset` write, and how their messages name it and + * a value in it. Without a scope, only the machine's: `global` there, as the + * flag and `env list` call it. + */ +function valuesFile( + localConfig: LocalConfig | null, + global: boolean | undefined, +): { file: string; target: string; value: string } { + return localConfig && !global + ? { file: getTeamSecretsPath(localConfig), target: 'for this team', value: 'value for this team' } + : { + file: getMachineSecretsPath(), + target: 'as your global value (every team on this machine)', + value: 'global value (every team on this machine)', + }; } /** - * Add or update an env variable locally. + * What the pull after `env unset` updates: env.sh exports a member's value for + * a variable (#875), not for a secret. Declarations that fail can't say which + * the key is. + */ +async function unsetApplies(localConfig: LocalConfig, key: string, global: boolean | undefined): Promise { + if (global) return 'update MCP servers'; + const declared = declaredSecretKeys(await resolveSecretDeclarations(localConfig)); + if (!declared) return 'apply it'; + return declared.has(key) ? 'update MCP servers' : 'update MCP servers and env.sh'; +} + +/** The entry `env set` stores: a `--from-env` reference, piped stdin, or the hidden prompt. */ +async function secretInput( + key: string, + options: { stdin?: boolean; fromEnv?: string }, +): Promise<{ ok: true; entry: StoredSecret } | { ok: false; message: string }> { + if (options.fromEnv !== undefined) return { ok: true, entry: { env: options.fromEnv } }; + if (options.stdin) { + if (process.stdin.isTTY) return { ok: false, message: '--stdin expects piped stdin; run without it to be prompted.' }; + process.stdin.setEncoding('utf8'); + const value = await readStdin(); + return value ? { ok: true, entry: { value } } : { ok: false, message: 'No value was provided on stdin.' }; + } + let value: string; + try { + value = await askSecret(`Value for ${key}: `); + } catch (e) { + if (isInteractive()) throw e; + return { ok: false, message: `Cannot prompt for ${key} without a terminal. Pipe the value with --stdin, or pass --from-env .` }; + } + return value ? { ok: true, entry: { value } } : { ok: false, message: 'No value was entered.' }; +} + +function invalidKeyMessage(key: string, what = 'env variable name'): string { + return `Invalid ${what} "${key}": use letters, digits and underscores, starting with a letter or underscore.`; +} + +function fail(message: string): void { + log.error(message); + process.exitCode = 1; +} + +/** + * Add or update an env variable locally, or with `secret` declare a secret: + * the key, what it is for and where to get a value, never a value. * Changes are deferred — run `teamai push` to sync to team repo. */ export async function envAdd( key: string, - value: string, - options: GlobalOptions & { description?: string; role?: string; project?: string }, + value: string | undefined, + options: GlobalOptions & { description?: string; role?: string; project?: string; secret?: boolean; url?: string }, ): Promise { // env.sh is generated as `export =...` and sourced by every member, so a // key that is not a shell identifier either breaks that line or runs as code. // `generateEnvFile` drops such keys, which would make this command report // success for a variable that never reaches anyone's shell — reject it here, // where the user still sees what they typed. - if (!ENV_KEY_RE.test(key)) { - log.error( - `Invalid env variable name "${key}": use letters, digits and underscores, starting with a letter or underscore.`, + if (!ENV_KEY_RE.test(key)) return fail(invalidKeyMessage(key)); + // Every member supplies a secret's value on their own machine; the value + // passed here is neither stored nor printed. + if (options.secret && value !== undefined) { + return fail( + `A secret has no value in the team repo, so --secret takes none. Nothing was changed. ` + + `Run \`teamai env add ${key} --secret\` without the value.`, + ); + } + if (!options.secret && options.url !== undefined) { + return fail('--url says where a member gets a secret\'s value, so it needs --secret. Nothing was changed.'); + } + if (!options.secret && value === undefined) { + return fail( + `No value for "${key}". Run \`teamai env add ${key} \`, ` + + `or \`teamai env add ${key} --secret\` to declare a secret each member sets.`, ); - return; } - const projectConfig = await detectProjectConfig(); - const localConfig = projectConfig ?? (await requireInit()).localConfig; + const localConfig = await requireScope(options.dryRun); + if (!localConfig) return; const repoPath = localConfig.repo.localPath; if (!await refreshTeamRepo(localConfig, options.project)) return; + if (options.secret) { + await declareSecret(repoPath, key, options); + return; + } + // Refused above: a variable needs a value. + if (value === undefined) return; const target = await envFileFromFlags(repoPath, options); if (!target) return; - const { envYamlPath, relativePath, where } = target; + const { filePath: envYamlPath, relativePath, where } = target; // The target env.yaml, or a new one when it does not exist. const envConfig = await readEnvFileForEdit(envYamlPath); if (!envConfig) return; // Check if key already exists - const existingIdx = envConfig.variables.findIndex(v => v.key === key); + // On Windows a key typed in another case is this variable: it is updated, not added a second time. + const existingIdx = envConfig.variables.findIndex(v => sameEntryKey(v.key, key)); const isUpdate = existingIdx !== -1; if (isUpdate) { @@ -116,7 +364,7 @@ export async function envAdd( // The remediation has to name the namespace file, as pull's notice does: // dropping a root-scoped key where it sits would deliver the secret to // everyone — the outcome the per-entry key was scoping against. - const targets = new TargetFiles(repoPath, 'env'); + const targets = new TargetFiles(repoPath, entryLayout('env')); const files: string[] = []; for (const key of removed) { files.push(...await targets.forIds(key as 'roles' | 'projects', updated[key as 'roles' | 'projects'] ?? [])); @@ -149,32 +397,94 @@ export async function envAdd( } /** - * Remove an env variable locally. + * Declare a secret in env/secrets.yaml or env//secrets.yaml, or update + * the description and url of one already declared there. + */ +async function declareSecret( + repoPath: string, + key: string, + options: GlobalOptions & { description?: string; role?: string; project?: string; url?: string }, +): Promise { + const target = await envFileFromFlags(repoPath, options, SECRETS_LAYOUT); + if (!target) return; + const secrets = await readSecretsFileForEdit(target); + if (!secrets) return; + + const index = secrets.findIndex((secret) => sameEntryKey(secret.key, key)); + const isUpdate = index !== -1; + // On Windows a key typed in another case is the declared one: it keeps its declared name. + if (isUpdate && typeof secrets[index]?.key === 'string') key = secrets[index].key; + const declaration = { + ...(isUpdate ? secrets[index] : {}), + key, + ...(options.description !== undefined ? { description: options.description } : {}), + ...(options.url !== undefined ? { url: options.url } : {}), + }; + if (isUpdate) secrets[index] = declaration; + else secrets.push(declaration); + // A key declared twice fails every read of the file, so the update leaves one. + const duplicates = dropDuplicateSecrets(secrets, key); + // The update keeps an unknown key, so the secret stays undeclared. + const unknown = unknownSecretDeclarationKeys(declaration); + if (unknown.length > 0) { + const one = unknown.length === 1; + log.warn( + `${target.relativePath}: secret "${key}" has unknown ${one ? 'key' : 'keys'} ` + + `${unknown.map((k) => `\`${k}:\``).join(', ')}, so it is not declared. ` + + `Correct the ${one ? 'key' : 'keys'} or remove ${one ? 'it' : 'them'} in ${target.relativePath}.`, + ); + } + + const removedToo = duplicates === 0 ? '' : `, and ${options.dryRun ? 'remove' : 'removed'} ${declarationCount(duplicates)} of it`; + if (options.dryRun) { + log.info(`[dry-run] Would ${isUpdate ? 'update' : 'declare'} secret${target.where}: ${key}${removedToo}`); + return; + } + await writeSecretsFile(target.filePath, secrets); + log.success(`${isUpdate ? 'Updated' : 'Declared'} secret${target.where}: ${key}${removedToo}`); + log.info('Run `teamai push` to sync to team repo.'); +} + +/** + * Remove an env variable locally. A key that env.yaml does not set is removed + * from the secrets file next to it; `secret` removes from the secrets file + * only, for a key both files carry. * Changes are deferred — run `teamai push` to sync to team repo. */ -export async function envRemove(key: string, options: GlobalOptions & { role?: string; project?: string }): Promise { - const projectConfig = await detectProjectConfig(); - const localConfig = projectConfig ?? (await requireInit()).localConfig; +export async function envRemove( + key: string, + options: GlobalOptions & { role?: string; project?: string; secret?: boolean }, +): Promise { + const localConfig = await requireScope(options.dryRun); + if (!localConfig) return; const repoPath = localConfig.repo.localPath; if (!await refreshTeamRepo(localConfig, options.project)) return; - const target = await envFileFromFlags(repoPath, options); + const target = await envFileFromFlags(repoPath, options, options.secret ? SECRETS_LAYOUT : 'env'); if (!target) return; - const { envYamlPath, relativePath, where } = target; + if (options.secret) { + if (await removeSecret(key, target, options) !== 'absent') return; + return fail( + `Secret "${key}" is not declared in ${target.relativePath}. Nothing was changed. For a namespace's file, pass ` + + '--role or --project ; `teamai env list` shows where each secret this directory receives comes from.', + ); + } + const { filePath: envYamlPath, relativePath, where } = target; + const secretsFile = entryFileIn(repoPath, SECRETS_LAYOUT, target.namespace); if (!await pathExists(envYamlPath)) { - log.error(`No env variables defined (${relativePath} not found)`); - return; + if (await removeSecret(key, secretsFile, options) === 'removed') return; + return fail(`No env variables defined (${relativePath} not found)`); } const envConfig = await readEnvFileForEdit(envYamlPath); if (!envConfig) return; - const idx = envConfig.variables.findIndex(v => v.key === key); + const idx = envConfig.variables.findIndex(v => sameEntryKey(v.key, key)); if (idx === -1) { - log.error(`Env variable "${key}" not found${where}`); - return; + if (await removeSecret(key, secretsFile, options) === 'removed') return; + return fail(`Env variable "${key}" not found${where}`); } if (options.dryRun) { @@ -189,6 +499,62 @@ export async function envRemove(key: string, options: GlobalOptions & { role?: s log.info('Run `teamai push` to sync to team repo.'); } +/** + * Remove a declared secret from `file`: `removed` (or would be, on dry-run), + * `absent` when the file does not declare it, `reported` when the file does + * not parse and the reason was printed. + */ +async function removeSecret( + key: string, + file: EntryFileTarget, + options: GlobalOptions, +): Promise<'removed' | 'absent' | 'reported'> { + const secrets = await readSecretsFileForEdit(file); + if (!secrets) return 'reported'; + const index = secrets.findIndex((secret) => sameEntryKey(secret.key, key)); + if (index === -1) return 'absent'; + + // Every declaration of it: one left behind still declares the key. + const duplicates = dropDuplicateSecrets(secrets, key, 0) - 1; + const removedToo = duplicates === 0 ? '' : `, and ${declarationCount(duplicates)} of it`; + if (options.dryRun) { + log.info(`[dry-run] Would remove secret${file.where}: ${key}${removedToo}`); + return 'removed'; + } + await writeSecretsFile(file.filePath, secrets); + log.success(`Removed secret${file.where}: ${key}${removedToo}`); + log.info('Run `teamai push` to sync to team repo.'); + return 'removed'; +} + +/** `values` without `key`, in every case of it on Windows, where they are one environment variable. */ +function withoutKey(values: Readonly>, key: string): Record { + const rest = { ...values }; + for (const other of Object.keys(rest)) if (envName(other) === envName(key)) delete rest[other]; + return rest; +} + +/** Whether an entry's `key` is `key`: the same environment variable, so in any case on Windows. */ +function sameEntryKey(declared: unknown, key: string): boolean { + return typeof declared === 'string' && envName(declared) === envName(key); +} + +/** Remove the declarations of `key` after the first `keep` of them from `secrets`; answers how many. */ +function dropDuplicateSecrets(secrets: Record[], key: string, keep = 1): number { + let seen = 0; + let removed = 0; + for (let i = 0; i < secrets.length; i++) { + if (!sameEntryKey(secrets[i]?.key, key) || ++seen <= keep) continue; + secrets.splice(i--, 1); + removed++; + } + return removed; +} + +function declarationCount(duplicates: number): string { + return `${duplicates} duplicate declaration${duplicates === 1 ? '' : 's'}`; +} + /** * Pull the team repo before an edit. A failure only warns, except with * `--project`: that resolves through manifest/projects.yaml, and a stale copy @@ -208,11 +574,10 @@ async function refreshTeamRepo(localConfig: LocalConfig, project: string | undef return true; } pullSpin.fail(`Pull failed: ${(e as Error).message}`); - log.error( + fail( `The team repo could not be refreshed (${(e as Error).message}), so the env namespace of project "${project}" ` + 'may be out of date. Nothing was changed. Fix the pull (run `teamai pull` to see why) and retry, or pass --role .', ); - process.exitCode = 1; return false; } } @@ -224,30 +589,54 @@ async function refreshTeamRepo(localConfig: LocalConfig, project: string | undef async function readEnvFileForEdit(envYamlPath: string): Promise { const read = await envHandler.readEnvYaml(envYamlPath); if (read.ok) return { variables: read.variables }; - log.error(`${read.reason}. Nothing was changed. Fix the file in the team repo, then retry.`); - process.exitCode = 1; + fail(`${read.reason}. Nothing was changed. Fix the file in the team repo, then retry.`); return null; } +/** A file `env add` / `env remove` edit, and how messages name it. */ +interface EntryFileTarget { + readonly namespace: string | null; + readonly filePath: string; + readonly relativePath: string; + readonly where: string; +} + /** - * The env file `--role ` / `--project ` name, or env/env.yaml without - * either. Reports the reason and returns null when the flags name none. + * The secrets file to edit, or null when it does not parse, reported as for + * env.yaml. + */ +async function readSecretsFileForEdit(file: EntryFileTarget): Promise[] | null> { + const read = await readSecretsForEdit(file.filePath, file.relativePath); + if (read.ok) return read.secrets; + fail(`${read.reason}. Nothing was changed. Fix the file in the team repo, then retry.`); + return null; +} + +/** + * The env file (env.yaml, or secrets.yaml for `SECRETS_LAYOUT`) that + * `--role ` / `--project ` name, or the root one without either. + * Reports the reason and returns null when the flags name none. */ async function envFileFromFlags( repoPath: string, flags: { role?: string; project?: string }, -): Promise<{ envYamlPath: string; relativePath: string; where: string } | null> { - const target = await entryNamespaceFromFlags(repoPath, 'env', flags); + layout: EntryType | EntryLayout = 'env', +): Promise { + const target = await entryNamespaceFromFlags(repoPath, layout, flags); if (!target.ok) { - log.error(target.message); - process.exitCode = 1; + fail(target.message); return null; } - const relativePath = entryFilePath('env', target.namespace); + return entryFileIn(repoPath, layout, target.namespace); +} + +function entryFileIn(repoPath: string, layout: EntryType | EntryLayout, namespace: string | null): EntryFileTarget { + const relativePath = entryFilePath(layout, namespace); return { - envYamlPath: entryFileAbsolutePath(repoPath, 'env', target.namespace), + namespace, + filePath: entryFileAbsolutePath(repoPath, layout, namespace), relativePath, // Messages name the file only for a namespace; the root is the default. - where: target.namespace === null ? '' : ` in ${relativePath}`, + where: namespace === null ? '' : ` in ${relativePath}`, }; } diff --git a/src/env-exec.ts b/src/env-exec.ts new file mode 100644 index 000000000..b3b5e4dd5 --- /dev/null +++ b/src/env-exec.ts @@ -0,0 +1,272 @@ +/** + * `teamai env exec -- ` (#875): run a CLI such as `gh` or `glab` with + * this directory's team env. The command inherits teamai's environment, + * overlaid with the scope's env.yaml variables and its team secrets in the + * resolution order (resources/secrets.ts). The scope is the one that governs + * the directory (resolveConfigForDir), so every worktree of a project resolves + * to that project. + * + * Everything teamai prints goes to stderr, so the command's stdout can be + * piped. No value is written anywhere: the child gets it in its environment + * only. + */ +import { execFile } from 'node:child_process'; +import os from 'node:os'; +import path from 'node:path'; +import { promisify } from 'node:util'; +import crossSpawn from 'cross-spawn'; +import { resolveConfigForDir } from './config.js'; +import { reportMissingSecrets } from './env-advisories.js'; +import { resolveTeamEnv } from './env-resolution.js'; +import { memberEnvironmentWithoutScope, type MemberEnvironment } from './member-env.js'; +import { describeEntryFailure } from './namespaced-entries.js'; +import { envTable } from './resources/env-key.js'; +import { declaredSecretKeys } from './resources/secrets.js'; +import { getDataHome, type GlobalOptions, type LocalConfig } from './types.js'; +import { log, setStderrOnly } from './utils/logger.js'; + +/** How the command ended. */ +export type ExecOutcome = + | { readonly kind: 'exited'; readonly code: number } + | { readonly kind: 'signaled'; readonly signal: NodeJS.Signals }; + +/** Signals sent to teamai alone, which the command gets only if teamai passes them on. */ +const FORWARDED_SIGNALS: readonly NodeJS.Signals[] = ['SIGTERM', 'SIGHUP']; +/** + * A terminal sends Ctrl-C and Ctrl-\ to its whole foreground process group, so + * when teamai is in that group the command has them already; passing them on + * would send a second, which tools such as terraform take as "force quit". + * teamai then ignores them and waits. Anywhere else (a background job, no + * terminal) one is sent to teamai alone, and is passed on. + */ +const TERMINAL_SIGNALS: readonly NodeJS.Signals[] = ['SIGINT', 'SIGQUIT']; + +/** + * Whether `ps -o pgid=,tpgid=` says the process is in its terminal's + * foreground process group. No controlling terminal prints a tpgid of 0 + * (macOS) or -1 (Linux). + */ +export function inTerminalForeground(ps: string): boolean { + const [pgid, tpgid] = ps.trim().split(/\s+/).map(Number); + return Number.isInteger(tpgid) && tpgid > 0 && pgid === tpgid; +} + +/** + * Whether a terminal delivers Ctrl-C to the command as well as to teamai. + * Windows delivers it to every process on the console. Elsewhere, when `ps` + * cannot say, teamai takes a SIGINT to be its alone: passing on one the + * command already had is a second Ctrl-C, keeping it leaves the command + * running. + */ +async function terminalReachesCommand(): Promise { + if (process.platform === 'win32') return true; + try { + const { stdout } = await promisify(execFile)('ps', ['-o', 'pgid=,tpgid=', '-p', String(process.pid)]); + return inTerminalForeground(stdout); + } catch { + return false; + } +} + +/** + * Run the command in `words`, what was typed after `exec`: teamai's own + * options, then `--`, then the command. Without `--`, a flag of the command + * (`gh pr list --dry-run`) would be read as teamai's, so it is refused. + */ +export async function envExec(words: readonly string[], options: GlobalOptions, cwd = process.cwd()): Promise { + // Before the scope lookup, which can print (a role migration, for one). + setStderrOnly(true); + const separator = words.indexOf('--'); + if (separator === -1 || !words.slice(0, separator).every((word) => word.startsWith('-'))) { + log.error('Put -- before the command: teamai env exec -- '); + return { kind: 'exited', code: 2 }; + } + const [file, ...args] = words.slice(separator + 1); + if (!file) { + log.error('No command to run. Usage: teamai env exec -- [args...]'); + return { kind: 'exited', code: 2 }; + } + const env = await commandEnvironment(cwd, options.dryRun); + if (options.dryRun) { + log.info(`[dry-run] Would run ${file} with this directory's team env`); + return { kind: 'exited', code: 0 }; + } + return run(file, args, env, cwd, await terminalReachesCommand()); +} + +/** + * Signals that end the command but not Node: Node ignores SIGPIPE, and SIGUSR1 + * starts its inspector. Re-raising one would leave teamai running. + */ +const SURVIVED_SIGNALS: ReadonlySet = new Set(['SIGPIPE', 'SIGUSR1']); + +/** + * Exit the way the command did: its exit code, or the signal that ended it. + * The shell's 128 + signal number is set first, for a signal teamai survives. + */ +export function exitLike(outcome: ExecOutcome): void { + switch (outcome.kind) { + case 'exited': + process.exitCode = outcome.code; + return; + case 'signaled': + process.exitCode = 128 + os.constants.signals[outcome.signal]; + if (!SURVIVED_SIGNALS.has(outcome.signal)) process.kill(process.pid, outcome.signal); + return; + default: { + const unhandled: never = outcome; + return unhandled; + } + } +} + +/** The environment the command runs with, reporting on stderr whatever it leaves out. */ +async function commandEnvironment(cwd: string, dryRun: boolean | undefined): Promise { + const unreadable: { configPath: string; error: string }[] = []; + const localConfig = await resolveConfigForDir(cwd, (configPath, error) => { unreadable.push({ configPath, error }); }, { dryRun }); + if (unreadable.length > 0) { + // The project's config is unknown, so as while its declarations fail, a value a teamai env.sh exported is removed. + const env = inheritedEnvironment(); + const removed = withoutTeamExports(env, await memberEnvironmentWithoutScope(unreadable.map(({ configPath }) => path.dirname(configPath)))); + log.warn(`${unreadable.map(({ configPath, error }) => `${configPath} could not be read: ${error}.`).join(' ')} No team env ` + + `variables or secrets were applied; the command runs with the inherited environment${exportedClause(removed)}. Fix the ` + + 'file, or run `teamai init` again in this project.'); + return env; + } + // No scope delivers team values here, so as above, a value a teamai env.sh exported is removed. + if (!localConfig) { + const env = inheritedEnvironment(); + const removed = withoutTeamExports(env, await memberEnvironmentWithoutScope([])); + log.warn('No teamai config applies to this directory, so no team env variables or secrets were applied; the command ' + + `runs with the inherited environment${exportedClause(removed)}.`); + return env; + } + if (localConfig.repo.kind === 'http') { + const env = inheritedEnvironment(); + const removed = withoutTeamExports(env, await memberEnvironmentWithoutScope([getDataHome(localConfig)])); + log.warn('An HTTP team repo delivers no env variables or secrets here, so the command runs with the inherited ' + + `environment${exportedClause(removed)}.`); + return env; + } + return overlayTeamEnv(localConfig); +} + +/** A copy of the inherited environment that keeps `__proto__` an own key when the overlay sets it. */ +function inheritedEnvironment(): NodeJS.ProcessEnv { + return envTable(Object.entries(process.env)); +} + +/** + * The inherited environment with this scope's variables and secrets. A key the + * scope declares as a secret gets its resolved value or is removed, so the + * command never sees a value `teamai env list` doesn't show for this scope: + * another team's export, or the member's own export when this team's value + * names another variable. A key that is also an env.yaml variable resolves as + * a secret. A variable takes the member's value for this team, else the + * team's, as in MCP; the inherited value never overrides it (#875). While the + * declarations fail, nothing is overlaid, and every inherited value that is + * not the member's own (member-env.ts) is removed; so is it while env.yaml + * fails or the values file cannot be read. + */ +async function overlayTeamEnv(localConfig: LocalConfig): Promise { + const env = inheritedEnvironment(); + const teamEnv = await resolveTeamEnv(localConfig); + const { variables, declarations, variableValues, secrets } = teamEnv; + if (declarations.kind === 'failed') { + // Any env.yaml key may be a secret the file declares, so no team value is applied (#879 Conflict 14), + // and one a teamai env.sh exported is a team value, not the member's: it is removed. + const failures = [variables, declarations].flatMap((entries) => entries.kind === 'failed' ? [describeEntryFailure(entries.failure)] : []); + const without = exportedClause(withoutTeamExports(env, teamEnv.member)); + log.warn(`${failures.join(' ')} The command runs with the inherited environment, without team env variables or secrets${without}.`); + return env; + } + // Without this team's variables, one a teamai env.sh exported may be another team's: it is removed. + if (variables.kind === 'failed') { + const without = exportedClause(withoutTeamExports(env, teamEnv.member)); + log.warn(`${describeEntryFailure(variables.failure)} The command runs without the team's env variables${without}.`); + } + if (variableValues.kind === 'store-unreadable') { + const without = exportedClause(withoutTeamExports(env, teamEnv.member)); + log.warn(`${variableValues.reason} The command runs without the team's env variables${without}.`); + } else { + for (const [key, variable] of variableValues.values) setKey(env, key, variable.value); + } + const secretKeys = declaredSecretKeys(declarations); + if (!secretKeys || secretKeys.size === 0) return env; + + if (secrets.kind === 'store-unreadable') { + log.warn(`${secrets.reason} The command runs without team secrets.`); + } + for (const key of secretKeys) { + const secret = secrets.kind === 'resolved' ? secrets.values.get(key) : undefined; + if (secret) setKey(env, key, secret.value); + else removeKey(env, key); + } + if (secrets.kind === 'resolved') await reportMissingSecrets(localConfig, teamEnv); + return env; +} + +/** Remove `key` from `env`, in every case on Windows, where environment names are case-insensitive. */ +function removeKey(env: NodeJS.ProcessEnv, key: string): void { + if (process.platform !== 'win32') { + delete env[key]; + return; + } + const name = key.toUpperCase(); + for (const other of Object.keys(env)) if (other.toUpperCase() === name) delete env[other]; +} + +/** Set `key` in `env`, replacing it in any case on Windows rather than adding a competing name. */ +function setKey(env: NodeJS.ProcessEnv, key: string, value: string): void { + removeKey(env, key); + env[key] = value; +} + +/** Remove from `env` every value that is not the member's own (member-env.ts), and return the keys removed. */ +function withoutTeamExports(env: NodeJS.ProcessEnv, member: MemberEnvironment): string[] { + const removed = Object.keys(env).filter((key) => env[key] !== '' && member(key) === undefined); + for (const key of removed) delete env[key]; + return removed; +} + +/** The warning's clause naming the keys `withoutTeamExports` removed, never their values. */ +function exportedClause(removed: readonly string[]): string { + return removed.length > 0 ? `, and without ${removed.join(', ')}, whose values a teamai env.sh exported` : ''; +} + +/** + * Run the command with the terminal's stdio. A signal sent to teamai alone is + * passed on, one from the terminal is not, and either way teamai waits for + * the command to end rather than exit first. + */ +function run( + file: string, + args: readonly string[], + env: NodeJS.ProcessEnv, + cwd: string, + terminalReachesCommand: boolean, +): Promise { + const forwarded = terminalReachesCommand ? FORWARDED_SIGNALS : [...FORWARDED_SIGNALS, ...TERMINAL_SIGNALS]; + const ignored = terminalReachesCommand ? TERMINAL_SIGNALS : []; + return new Promise((resolve) => { + // cross-spawn: on Windows, npm installs CLIs as .cmd shims spawn can't start. + const child = crossSpawn(file, [...args], { cwd, env, stdio: 'inherit' }); + const forward = (signal: NodeJS.Signals): void => { child.kill(signal); }; + const ignore = (): void => {}; + for (const signal of forwarded) process.on(signal, forward); + for (const signal of ignored) process.on(signal, ignore); + let settled = false; + const settle = (outcome: ExecOutcome): void => { + if (settled) return; + settled = true; + for (const signal of forwarded) process.off(signal, forward); + for (const signal of ignored) process.off(signal, ignore); + resolve(outcome); + }; + child.on('error', (e) => { + log.error(`Could not run ${file}: ${e.message}`); + settle({ kind: 'exited', code: 127 }); + }); + child.on('exit', (code, signal) => settle(signal ? { kind: 'signaled', signal } : { kind: 'exited', code: code ?? 1 })); + }); +} diff --git a/src/env-listing.ts b/src/env-listing.ts new file mode 100644 index 000000000..7cdec2cae --- /dev/null +++ b/src/env-listing.ts @@ -0,0 +1,92 @@ +/** + * One scope's env as `teamai env list` and `teamai list env` show it (#875): + * the env.yaml variables it receives, each with the value it resolves to and + * where that comes from (`team` for the member's value, `env.yaml`), then the + * secrets it declares, each with where its value comes from, never the value. + * A key declared as a secret is listed only as one: its env.yaml value is not + * delivered. + * + * A value nobody can tell is not shown: while the declarations fail, any + * variable may be a secret whose repo value is ignored, and while the member's + * values can't be read, a variable's value is `unreadable`, as a secret's is. + */ +import { secretState, type TeamEnv } from './env-resolution.js'; +import { describeEntryFailure, describeOrigin } from './namespaced-entries.js'; +import { maskEnvValue } from './resources/env.js'; +import { declaredSecretKeys } from './resources/secrets.js'; + +export interface EnvListingLine { + readonly text: string; + /** A description or url, shown with `verbose`. */ + readonly detail: boolean; +} + +export interface EnvListing { + /** Why part of the listing can't be resolved, one message each. */ + readonly problems: readonly string[]; + /** Empty when the scope has no variables and declares no secrets. */ + readonly lines: readonly EnvListingLine[]; + /** A line shows a value in plaintext (`reveal`). */ + readonly revealed: boolean; + /** The listing has a secrets section. */ + readonly hasSecrets: boolean; +} + +export function envListing(teamEnv: TeamEnv, options: { reveal?: boolean; verbose?: boolean }): EnvListing { + const { variables, declarations, variableValues, secrets, staleEntries } = teamEnv; + const problems = new Set(); + if (variables.kind === 'failed') problems.add(describeEntryFailure(variables.failure)); + if (declarations.kind === 'failed') problems.add(describeEntryFailure(declarations.failure)); + if (variableValues.kind === 'store-unreadable') problems.add(variableValues.reason); + if (secrets.kind === 'store-unreadable') problems.add(secrets.reason); + + const declared = declarations.kind === 'resolved' ? declarations.entries : []; + const secretKeys = declaredSecretKeys(declarations) ?? new Set(); + const received = (variables.kind === 'resolved' ? variables.entries : []).filter((variable) => !secretKeys.has(variable.name)); + const lines: EnvListingLine[] = []; + const line = (text: string): void => { lines.push({ text, detail: false }); }; + const detail = (text: string | undefined): void => { if (text && options.verbose) lines.push({ text: ` ${text}`, detail: true }); }; + let revealed = false; + // A value set as the other kind is not used (secret-store.ts); `env set` again stores it as this one. + const stale = (key: string, now: 'secret' | 'env variable'): void => { + const kind = staleEntries.get(key); + if (!kind) return; + line( + ` Your value for this team was set while ${key} was ${kind === 'secret' ? 'a secret' : 'an env variable'}, so it is not used. ` + + `Run \`teamai env unset ${key}\` to remove it, then \`teamai env set ${key}\` to set one for the ${now}.`, + ); + }; + + if (received.length > 0) { + line(`Team env variables (${received.length}):`); + line(''); + for (const variable of received) { + const origin = `(${describeOrigin(variable)})`; + if (declarations.kind === 'failed') { + line(` ${variable.name} ${origin}`); + } else if (variableValues.kind === 'store-unreadable') { + line(` ${variable.name} unreadable ${origin}`); + } else { + const resolved = variableValues.values.get(variable.name); + const value = resolved?.value ?? variable.entry.value; + revealed ||= options.reveal === true; + line(` ${variable.name}=${options.reveal ? value : maskEnvValue(value)} ${resolved?.source ?? 'env.yaml'} ${origin}`); + stale(variable.name, 'env variable'); + } + detail(variable.entry.description); + } + line(''); + } + if (declared.length > 0) { + line(`Team secrets (${declared.length}):`); + line(''); + for (const secret of declared) { + line(` ${secret.name} ${secretState(secrets, secret.name)} (${describeOrigin(secret)})`); + stale(secret.name, 'secret'); + detail(secret.entry.description); + detail(secret.entry.url); + } + line(''); + } + return { problems: [...problems], lines, revealed, hasSecrets: declared.length > 0 }; +} diff --git a/src/env-resolution.ts b/src/env-resolution.ts new file mode 100644 index 000000000..b4db687a4 --- /dev/null +++ b/src/env-resolution.ts @@ -0,0 +1,209 @@ +/** + * This scope's env for the member (#875): the env.yaml variables it receives + * and the secrets it declares, each with the member's value in the resolution + * order (docs/designs/team-secrets.md#resolution). + * + * `resolveTeamEnv` reads env.yaml, secrets.yaml, both value stores and the + * env.sh exports once. A command passes the one result to everything that + * lists, delivers or advises from it, so it reads each file once and every + * part of its output gives the same answer. + */ +import { memberEnvironment, type MemberEnvironment } from './member-env.js'; +import { resolveEntries, resolveEntriesFor, type EntryResolution, type ResolvedEntry } from './namespaced-entries.js'; +import { envEntryReader, type EnvVariable } from './resources/env.js'; +import { sameEnvName } from './resources/env-key.js'; +import { declaredSecretKeys, resolveSecretDeclarations, type KnownNamespaces, type SecretDeclarations } from './resources/secrets.js'; +import { + getMachineSecretsPath, getTeamSecretsPath, readSecretStore, storedEntryKind, storedSecretValue, type SecretStore, + type SecretStoreRead, type StoredEntryKind, type StoredSecret, +} from './secret-store.js'; +import type { LocalConfig } from './types.js'; + +export interface SecretValue { + readonly source: 'team' | 'global' | 'environment'; + readonly value: string; +} + +/** Where a declared secret's value comes from; `unreadable` when a value store can't be read, so nobody knows. */ +export type SecretState = SecretValue['source'] | 'missing' | 'unreadable'; + +/** + * The value of an env.yaml variable this scope receives: the member's value + * for this team (`teamai env set KEY`), then the team's. The environment + * doesn't override either, so a value exported for one team doesn't reach + * another team's servers, and `--global` doesn't apply: it is for secrets only. + * `fromEnv` says the member's entry reads another variable, so `env.sh` leaves + * the key out rather than hold a copy of that variable's value. While that + * variable is unset the team's value is used: unlike a secret's next source, + * it is the value every other member of the team gets. + */ +export interface VariableValue { + readonly source: 'team' | 'env.yaml'; + readonly value: string; + readonly fromEnv: boolean; +} + +/** Each key's value, or why the member's values can't be read; `reason` carries no value. */ +export type StoreResolution = + | { readonly kind: 'resolved'; readonly values: ReadonlyMap } + | { readonly kind: 'store-unreadable'; readonly reason: string }; + +export interface TeamEnv { + /** The env.yaml variables this scope receives, a declared secret's included. */ + readonly variables: EntryResolution; + readonly declarations: SecretDeclarations; + /** + * Each declared secret's value: the member's value for this team, then for + * the machine, then their own environment. A key without one is absent. The + * first entry found decides even when its `--from-env` variable is unset: + * falling back to the next source would send another account's token to + * this team. A store that can't be read leaves every secret without a value, + * for the same reason. None when the declarations failed. + */ + readonly secrets: StoreResolution; + /** + * The secrets without a value whose deciding entry reads a variable that is + * unset (`--from-env`): the variable, and whether it is the machine's entry. + */ + readonly unsetReferences: ReadonlyMap; + /** The value of each variable that isn't a declared secret (each one when the declarations failed). */ + readonly variableValues: StoreResolution; + /** + * The keys whose entry for this team is of the other kind, so it is not + * applied: a secret's value for a key now a variable, or a variable override + * for a key now a secret. The key maps to the entry's kind. + */ + readonly staleEntries: ReadonlyMap; + /** The member's own environment (member-env.ts). */ + readonly member: MemberEnvironment; +} + +export interface UnsetReference { + readonly variable: string; + readonly global: boolean; +} + +const NO_VALUES: SecretStoreRead = { ok: true, values: {} }; + +/** + * Resolve this scope's env, in env's active namespaces: `namespaces` when the + * caller has them, else resolved from `resources.env`. A store is read only + * when a key needs it. + */ +export async function resolveTeamEnv( + localConfig: LocalConfig, + namespaces?: KnownNamespaces, + env: NodeJS.ProcessEnv = process.env, +): Promise { + const variables = namespaces + ? await resolveEntries(envEntryReader, localConfig, namespaces.active) + : await resolveEntriesFor(envEntryReader, localConfig); + const declarations = await resolveSecretDeclarations(localConfig, namespaces); + const secretKeys = declaredSecretKeys(declarations) ?? new Set(); + const received = variables.kind === 'resolved' ? variables.entries : []; + const plain = received.filter((variable) => !secretKeys.has(variable.name)); + const envYaml = new Map(received.map((variable) => [variable.name, variable.entry.value])); + const member = await memberEnvironment(localConfig, { secretKeys, envYaml }, env); + const team = secretKeys.size > 0 || plain.length > 0 ? await readSecretStore(getTeamSecretsPath(localConfig)) : NO_VALUES; + const machine = secretKeys.size > 0 ? await readSecretStore(getMachineSecretsPath()) : NO_VALUES; + const secrets = secretValues(secretKeys, team, machine, member, env); + return { + variables, + declarations, + secrets: secrets.values, + unsetReferences: secrets.unsetReferences, + variableValues: variableValues(plain, team, env), + staleEntries: staleEntries(secretKeys, plain, team), + member, + }; +} + +/** The entry for `key` when it is of this kind: a secret never resolves from a variable override, nor the reverse. */ +function storeEntry(store: SecretStore, key: string, kind: StoredEntryKind): StoredSecret | undefined { + // On Windows a value stored as `token` is `TOKEN`'s: the same environment variable. + const stored = sameEnvName(Object.keys(store), key); + const entry = stored === undefined ? undefined : store[stored]; + return entry && storedEntryKind(entry) === kind ? entry : undefined; +} + +function staleEntries( + secretKeys: ReadonlySet, + variables: readonly ResolvedEntry[], + team: SecretStoreRead, +): ReadonlyMap { + const stale = new Map(); + if (!team.ok) return stale; + const check = (key: string, kind: StoredEntryKind): void => { + const stored = sameEnvName(Object.keys(team.values), key); + const entry = stored === undefined ? undefined : team.values[stored]; + if (entry && storedEntryKind(entry) !== kind) stale.set(key, storedEntryKind(entry)); + }; + for (const key of secretKeys) check(key, 'secret'); + for (const variable of variables) check(variable.name, 'variable'); + return stale; +} + +function secretValues( + keys: ReadonlySet, + team: SecretStoreRead, + machine: SecretStoreRead, + member: MemberEnvironment, + env: NodeJS.ProcessEnv, +): { values: StoreResolution; unsetReferences: ReadonlyMap } { + const values = new Map(); + const unsetReferences = new Map(); + const result = { values: { kind: 'resolved', values }, unsetReferences } as const; + if (keys.size === 0) return result; + if (!team.ok) return { values: { kind: 'store-unreadable', reason: team.reason }, unsetReferences }; + if (!machine.ok) return { values: { kind: 'store-unreadable', reason: machine.reason }, unsetReferences }; + for (const key of keys) { + const teamEntry = storeEntry(team.values, key, 'secret'); + const machineEntry = storeEntry(machine.values, key, 'secret'); + const entry = teamEntry ?? machineEntry; + const [source, value]: [SecretValue['source'], string | undefined] = teamEntry ? ['team', storedSecretValue(teamEntry, env)] + : machineEntry ? ['global', storedSecretValue(machineEntry, env)] + : ['environment', member(key)]; + if (value !== undefined) values.set(key, { source, value }); + else if (entry && 'env' in entry) unsetReferences.set(key, { variable: entry.env, global: teamEntry === undefined }); + } + return result; +} + +function variableValues( + variables: readonly ResolvedEntry[], + team: SecretStoreRead, + env: NodeJS.ProcessEnv, +): StoreResolution { + const values = new Map(); + if (variables.length === 0) return { kind: 'resolved', values }; + if (!team.ok) return { kind: 'store-unreadable', reason: team.reason }; + for (const variable of variables) { + const entry = storeEntry(team.values, variable.name, 'variable'); + const member = entry ? storedSecretValue(entry, env) : undefined; + const fromEnv = entry !== undefined && 'env' in entry; + values.set(variable.name, member !== undefined + ? { source: 'team', value: member, fromEnv } + : { source: 'env.yaml', value: variable.entry.value, fromEnv }); + } + return { kind: 'resolved', values }; +} + +export function secretState(secrets: StoreResolution, key: string): SecretState { + return secrets.kind === 'resolved' ? secrets.values.get(key)?.source ?? 'missing' : 'unreadable'; +} + +/** The variables `env.sh` exports, with their resolved values: every one in `values` but a `--from-env` override. */ +export function envShVariables( + variables: readonly ResolvedEntry[], + values: ReadonlyMap, +): EnvVariable[] { + return variables.flatMap((variable) => { + const resolved = values.get(variable.name); + return resolved && !resolved.fromEnv ? [{ ...variable.entry, value: resolved.value }] : []; + }); +} + +/** What a pull and MCP do while the member's values can't be read: keep what the last pull wrote. */ +export function variablesKeptWarning(reason: string): string { + return `${reason} Team env variables keep the values the last pull wrote until it is fixed.`; +} diff --git a/src/env-sh-exports.ts b/src/env-sh-exports.ts new file mode 100644 index 000000000..63f9d6d95 --- /dev/null +++ b/src/env-sh-exports.ts @@ -0,0 +1,116 @@ +/** + * What each scope's env.sh has exported (#879 Conflict 10), kept beside it in + * `env.sh.exports.json`. A shell opened before a pull rewrote env.sh still + * carries the values it exported then, in every command it runs afterwards, + * and those are the team's values, not the member's: without a record, the + * next command would read an old team token as the member's own. + * + * Each entry is a SHA-256 of `KEY=VALUE`, never the value, so the record adds + * no copy of a team value or token to the machine. It keeps the latest + * `KEPT_PER_KEY` per key: a shell older than that many changes of one key is + * not expected. + */ +import crypto from 'node:crypto'; +import path from 'node:path'; +import { z } from 'zod'; +import { envName } from './resources/env-key.js'; +import { readFileSafe, writeJsonAtomic } from './utils/fs.js'; +import { log } from './utils/logger.js'; + +const RECORD_FILE = 'env.sh.exports.json'; +const KEPT_PER_KEY = 20; + +const RecordSchema = z.record(z.string(), z.array(z.string())); + +/** Per key, the digests of the values an env.sh exported. */ +export type EnvShExports = ReadonlyMap>; + +/** Of the key as the platform compares it (`envName`), so on Windows `token` and `TOKEN` hash alike. */ +export function exportDigest(key: string, value: string): string { + return crypto.createHash('sha256').update(`${envName(key)}=${value}`).digest('hex'); +} + +/** + * The variable each env.sh exports beside its own, naming what it exported: + * `TEAMAI_ENV_SH_= ...`, one digest prefix per + * export and per value the record keeps, never a value. The record above is found by scanning known paths, + * and a non-git project keeps its env.sh at `/.teamai/`, which no scan + * reaches; a shell that sourced it carries the marker instead. Named after + * the data home, so a shell that sourced the user's env.sh and a project's + * keeps both. + */ +const MARKER_RE = /^TEAMAI_ENV_SH_[0-9a-f]{64}$/i; +const MARKED_DIGEST_LENGTH = 12; + +export function isEnvShMarker(key: string): boolean { + return MARKER_RE.test(key); +} + +function markedDigest(key: string, value: string): string { + return exportDigest(key, value).slice(0, MARKED_DIGEST_LENGTH); +} + +/** + * The marker an env.sh in `dataHome` exports for `exports` and what it + * `recorded` exporting before, as [name, value]; null when it lists nothing. + * A shell that sources the rewritten env.sh keeps a value an earlier one + * exported, while the new marker replaces the old, so it lists both. + */ +export function envShMarker( + dataHome: string, + exports: Iterable, + recorded: EnvShExports = new Map(), +): [string, string] | null { + const digests = new Set([...exports].map(([key, value]) => markedDigest(key, value))); + for (const kept of recorded.values()) for (const digest of kept) digests.add(digest.slice(0, MARKED_DIGEST_LENGTH)); + if (digests.size === 0) return null; + const name = `TEAMAI_ENV_SH_${crypto.createHash('sha256').update(path.resolve(dataHome)).digest('hex')}`; + return [name, [...digests].join(' ')]; +} + +/** Whether a teamai env.sh this environment sourced exported `key=value`, by its markers. */ +export function markedAsExported(env: NodeJS.ProcessEnv): (key: string, value: string) => boolean { + const marked = new Set(); + for (const [name, digests] of Object.entries(env)) { + if (digests !== undefined && isEnvShMarker(name)) for (const digest of digests.split(' ')) marked.add(digest); + } + return (key, value) => marked.has(markedDigest(key, value)); +} + +function recordPath(envShPath: string): string { + return path.join(path.dirname(envShPath), RECORD_FILE); +} + +async function readRecord(envShPath: string): Promise> { + const file = recordPath(envShPath); + const content = await readFileSafe(file); + if (content === null) return new Map(); + let raw: unknown; + try { + raw = JSON.parse(content); + } catch { + raw = null; + } + const parsed = RecordSchema.safeParse(raw); + if (parsed.success) return new Map(Object.entries(parsed.data)); + // The next env.sh write replaces it; until then an old export may count as the member's. + log.debug(`${file} is not a record of env.sh exports; ignoring it until the next pull rewrites it.`); + return new Map(); +} + +/** What the env.sh at `envShPath` has exported, as digests. */ +export async function readEnvShExports(envShPath: string): Promise { + return new Map([...await readRecord(envShPath)].map(([key, digests]) => [key, new Set(digests)])); +} + +/** Add `exports` to the record beside `envShPath`, readable by this user only; answers the record. */ +export async function recordEnvShExports(envShPath: string, exports: Iterable): Promise { + const record = await readRecord(envShPath); + for (const [key, value] of exports) { + const digest = exportDigest(key, value); + const kept = (record.get(key) ?? []).filter((entry) => entry !== digest); + record.set(key, [...kept, digest].slice(-KEPT_PER_KEY)); + } + await writeJsonAtomic(recordPath(envShPath), Object.fromEntries(record), { mode: 0o600 }); + return new Map([...record].map(([key, digests]) => [key, new Set(digests)])); +} diff --git a/src/hook-handlers.ts b/src/hook-handlers.ts index e8f46e5f5..2415a9978 100644 --- a/src/hook-handlers.ts +++ b/src/hook-handlers.ts @@ -699,6 +699,33 @@ const packageHintHandler: HookHandler = { }, }; +/** + * SessionStart: tell the agent which secrets the scope declares and to run the + * CLIs that need them through `teamai env exec` (#875). Nothing when the scope + * declares none, or when its secrets files don't parse (doctor and pull say so). + */ +const secretsHintHandler: HookHandler = { + name: 'secrets-hint', + async execute(_stdin, _tool, config) { + if (!config) return null; + const { resolveSecretDeclarations } = await import('./resources/secrets.js'); + const declarations = await resolveSecretDeclarations(config); + if (declarations.kind !== 'resolved' || declarations.entries.length === 0) return null; + const keys = declarations.entries.map(({ name, entry }) => { + const description = entry.description?.replace(/\s+/g, ' ').trim(); + return description ? `${name} (${description})` : name; + }); + return JSON.stringify({ + hookSpecificOutput: { + hookEventName: 'SessionStart', + additionalContext: `Team secrets in this scope: ${keys.join(', ')}. ` + + 'Run the CLIs that need them through `teamai env exec -- ` so they get this team\'s values. ' + + 'Never ask for, read or print a secret value; if one is missing, ask the member to run `teamai env set KEY` in their own terminal.', + }, + }); + }, +}; + /** HTTP local-agent report/sync + workspace binding prompts. */ const localAgentHandler: HookHandler = { name: 'local-agent-sync', @@ -812,6 +839,7 @@ export function buildHandlerRegistry(): HandlerRegistration[] { { event: 'session-start', matcher: '*', handler: dashboardReportHandler, timeoutMs: FOREGROUND_HOOK_TIMEOUT_MS, requiresConfig: true }, { event: 'session-start', matcher: '*', handler: mrHintHandler, timeoutMs: FOREGROUND_HOOK_TIMEOUT_MS, gitOnly: true, requiresConfig: true }, { event: 'session-start', matcher: '*', handler: packageHintHandler, timeoutMs: FOREGROUND_HOOK_TIMEOUT_MS, requiresConfig: true }, + { event: 'session-start', matcher: '*', handler: secretsHintHandler, timeoutMs: FOREGROUND_HOOK_TIMEOUT_MS, requiresConfig: true }, { event: 'session-start', matcher: '*', handler: localAgentHandler, timeoutMs: FOREGROUND_HOOK_TIMEOUT_MS }, { event: 'session-start', matcher: '*', handler: webhookHandler, timeoutMs: FOREGROUND_HOOK_TIMEOUT_MS, background: true, requiresConfig: true }, diff --git a/src/index.ts b/src/index.ts index d8bb3f7fd..bedf39d5b 100644 --- a/src/index.ts +++ b/src/index.ts @@ -669,10 +669,12 @@ envCmd }); envCmd - .command('add ') - .description('Add or update a team environment variable') - .option('-d, --description ', 'Description for the variable') - .option('--role ', 'Write to env//env.yaml instead of env/env.yaml') + .command('add [value]') + .description('Add or update a team environment variable, or declare a secret with --secret') + .option('-d, --description ', 'Description for the variable or secret') + .option('--secret', 'Declare a secret in env/secrets.yaml: no value, each member sets their own') + .option('--url ', 'Where a member gets a value for the secret (with --secret)') + .option('--role ', 'Write to env// instead of env/ (env.yaml, or secrets.yaml with --secret)') .option('--project ', "Write to the project's env namespace (resources.env in manifest/projects.yaml)") .action(async (key, value, cmdOpts) => { const globalOpts = program.opts() as GlobalOptions; @@ -682,8 +684,9 @@ envCmd envCmd .command('remove ') - .description('Remove a team environment variable') - .option('--role ', 'Remove from env//env.yaml instead of env/env.yaml') + .description('Remove a team environment variable or declared secret') + .option('--secret', 'Remove the declared secret only (env/secrets.yaml), for a key env.yaml also sets') + .option('--role ', 'Remove from env// instead of env/') .option('--project ', "Remove from the project's env namespace (resources.env in manifest/projects.yaml)") .action(async (key, cmdOpts) => { const globalOpts = program.opts() as GlobalOptions; @@ -691,6 +694,39 @@ envCmd await envRemove(key, { ...globalOpts, ...cmdOpts }); }); +envCmd + .command('set ') + .description("Set your value for a secret the team declares, or an env variable it sets, for this directory's team, on this machine (prompts without echo)") + .option('--stdin', 'Read the value from piped stdin') + .option('--from-env ', 'Read the value from this environment variable each time it is used; no copy is stored') + .option('--global', 'Set a secret for every team on this machine; a value set for a team still wins') + .action(async (key, cmdOpts) => { + const globalOpts = program.opts() as GlobalOptions; + const { envSet } = await import('./env-commands.js'); + await envSet(key, { ...globalOpts, ...cmdOpts }); + }); + +envCmd + .command('unset ') + .description("Remove your value for a secret or env variable, for this directory's team, from this machine") + .option('--global', 'Remove the value set for every team on this machine instead') + .action(async (key, cmdOpts) => { + const globalOpts = program.opts() as GlobalOptions; + const { envUnset } = await import('./env-commands.js'); + await envUnset(key, { ...globalOpts, ...cmdOpts }); + }); + +envCmd + .command('exec ') + .description("Run a command with this directory's team env variables and secrets (put -- before the command)") + .action(async () => { + const globalOpts = program.opts() as GlobalOptions; + const { envExec, exitLike } = await import('./env-exec.js'); + // What was typed after `exec`, `--` included: Commander drops it. + const argv = process.argv.slice(2); + exitLike(await envExec(argv.slice(argv.indexOf('exec', argv.indexOf('env')) + 1), globalOpts)); + }); + // ─── Hooks commands ───────────────────────────────────── const hooksCmd = program diff --git a/src/init.ts b/src/init.ts index 11c12f046..ffc046c4e 100644 --- a/src/init.ts +++ b/src/init.ts @@ -775,6 +775,8 @@ export function buildSelfModeGitignore(): string { // and teammates get them on clone. env.yaml holds plaintext key/value pairs, so // only put non-secret config there; keep real secrets out of the repo. 'env.sh', + // What env.sh has exported, as hashes (#879 Conflict 10). + 'env.sh.exports.json', // env.local is the machine-local KEY=value backup pull writes for ${VAR} // resolution (self mode uses this name to avoid colliding with the env/ dir). 'env.local', @@ -821,6 +823,7 @@ export function buildProjectScopeGitignore(): string { '.update-lock', 'env', 'env.sh', + 'env.sh.exports.json', 'sessions/', 'dashboard/', 'usage.jsonl', diff --git a/src/mcp-cmd.ts b/src/mcp-cmd.ts index c2e6ed2ed..75aa2ae09 100644 --- a/src/mcp-cmd.ts +++ b/src/mcp-cmd.ts @@ -4,11 +4,18 @@ import { mcpEntryReader, teamMcpToDef } from './resources/mcp.js'; import { describeEntryFailure, describeOrigin, reportUndeliveredEntryNotices, resolveEntriesFor } from './namespaced-entries.js'; import { reconcileMcpForConfig, + releaseCleanMcpGitExcludes, resolveMcpTargets, - buildVarTable, + buildDesiredMcpContext, + desiredMcpForTarget, + mcpTargetExcluded, type McpChange, + type McpTarget, } from './mcp-reconcile.js'; -import { referencedVars } from './resources/mcp-format.js'; +import { placeholderValue, referencedVars } from './resources/mcp-format.js'; +import { reportMissingSecrets } from './env-advisories.js'; +import { resolveTeamEnv } from './env-resolution.js'; +import { carriesResolvedValue, ensureExcludedFromGit } from './mcp-git-exclude.js'; import { log } from './utils/logger.js'; import type { GlobalOptions } from './types.js'; import { managedMcpManifestPath, managedMcpManifestKey, getDataHome } from './types.js'; @@ -38,13 +45,26 @@ export async function mcpList(_options: GlobalOptions): Promise { reportUndeliveredEntryNotices(resolution); const servers = resolution.entries; + // HTTP mode has no repo tree to declare secrets in. + const teamEnv = localConfig.repo.kind === 'http' ? undefined : await resolveTeamEnv(localConfig); + // A failed declaration is not "no secrets" (#879 Conflict 14): nothing says + // which of a server's variables are secrets, so none is called set. + const declarationsFailed = teamEnv?.declarations.kind === 'failed'; + if (teamEnv?.declarations.kind === 'failed') { + log.error(describeEntryFailure(teamEnv.declarations.failure)); + process.exitCode = 1; + } + if (servers.length === 0) { log.info('No team MCP servers reach this directory (mcp/mcp.yaml and active mcp//mcp.yaml files are absent or empty)'); + await reportMissingSecrets(localConfig, teamEnv); return; } const targets = await resolveMcpTargets(teamConfig, localConfig); - const vars = await buildVarTable(localConfig); + // The team env already resolved above: resolving it again repeats its warnings. + const desiredContext = await buildDesiredMcpContext(teamConfig, localConfig, { teamEnv }); + const { vars } = desiredContext; // Project scope reads THIS worktree's own per-worktree manifest; user the global file. const manifest = (await readJson( managedMcpManifestPath( @@ -67,15 +87,24 @@ export async function mcpList(_options: GlobalOptions): Promise { const needed = referencedVars(s); if (needed.length > 0) { - const missing = needed.filter((v) => !vars[v]); - const state = missing.length === 0 ? 'all set' : `MISSING: ${missing.join(', ')}`; + const missing = needed.filter((v) => !placeholderValue(vars, v)); + const state = declarationsFailed ? 'not resolved' : missing.length === 0 ? 'all set' : `MISSING: ${missing.join(', ')}`; console.log(` secrets: ${needed.join(', ')} (${state})`); } - const installedIn = targets - .filter((t) => (manifest[managedMcpManifestKey(t.tool, t.projectScope)] ?? []).some((r) => r.name === s.name)) - .map((t) => t.tool); + const installed = (t: McpTarget): boolean => + (manifest[managedMcpManifestKey(t.tool, t.projectScope)] ?? []).some((r) => r.name === s.name); + const installedIn = targets.filter(installed).map((t) => t.tool); console.log(` installed: ${installedIn.length > 0 ? installedIn.join(', ') : '(none)'}`); + // Pull writes a resolved value only into a file git leaves out of a commit + // (#882); an entry an earlier pull wrote there stays as it was. Only where + // delivery would write it: its tools, transport, policy and requirements. + for (const t of targets) { + if (mcpTargetExcluded(localConfig, t)) continue; + if (!carriesResolvedValue(t, [s], desiredMcpForTarget(t, [s], desiredContext).desired.keys())) continue; + const exclusion = await ensureExcludedFromGit(t.file, { dryRun: true }); + if (exclusion.kind === 'failed') console.log(` withheld: ${t.tool} — ${exclusion.reason}. ${exclusion.fix}`); + } console.log(''); } @@ -85,6 +114,7 @@ export async function mcpList(_options: GlobalOptions): Promise { } else { for (const t of targets) console.log(` ${t.tool.padEnd(16)} ${displayPath(t.file)}`); } + await reportMissingSecrets(localConfig, teamEnv); } function reportChanges(changes: McpChange[]): void { @@ -121,6 +151,8 @@ export async function mcpInject( export async function mcpRemove(_options: GlobalOptions): Promise { const { localConfig, teamConfig } = await autoDetectInit(); const { changes, wrote } = await reconcileMcpForConfig(teamConfig, localConfig, { removeAll: true }); + // Nothing of teamai's is left for .git/info/exclude to protect (#882). + await releaseCleanMcpGitExcludes(teamConfig, localConfig); console.log('MCP remove:'); reportChanges(changes); diff --git a/src/mcp-git-exclude.ts b/src/mcp-git-exclude.ts new file mode 100644 index 000000000..fb94efaf6 --- /dev/null +++ b/src/mcp-git-exclude.ts @@ -0,0 +1,388 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import fse from 'fs-extra'; +import type { McpServerDef } from './types.js'; +import type { McpTarget } from './mcp-reconcile.js'; +import { referencedVars, supportsEnvExpansion } from './resources/mcp-format.js'; +import { execCommand } from './utils/exec.js'; +import { pathExists, readFileSafe, writeFileAtomic } from './utils/fs.js'; +import { listWorktrees } from './utils/git.js'; +import { log } from './utils/logger.js'; + +// ─── Project MCP configs and git ───────────────────────────── +// +// A project-scope MCP config that holds a resolved `${VAR}` sits in the +// business repo's working tree with the value in plaintext, and one +// `git add -A` commits it (#882). teamai lists such a file in the clone's own +// `.git/info/exclude`, inside a block it owns: local to the clone, nothing +// committed, and the team's `.gitignore` never touched. + +export const MCP_EXCLUDE_START = '# [teamai:mcp-exclude:start] project MCP configs holding resolved ${VAR} values'; +export const MCP_EXCLUDE_END = '# [teamai:mcp-exclude:end]'; + +/** + * Whether `target`'s file carries a value teamai resolved from a `${VAR}`: a + * project-scope file holding one of `names` whose definition references a + * variable the tool does not expand itself. + */ +export function carriesResolvedValue( + target: McpTarget, + teamDefs: McpServerDef[], + names: Iterable, +): boolean { + if (!target.projectScope) return false; + const present = new Set(names); + return teamDefs.some((def) => present.has(def.name) + && referencedVars(def).length > 0 + && !supportsEnvExpansion(target.format, target.projectScope, def)); +} + +/** + * The variable whose value, resolved by teamai into `target`, `raw` (a project + * file's text) holds, or null: one `teamDefs` references that the tool does not + * expand itself, with a value in `vars` of 8+ characters (shorter ones turn up + * anywhere). Needs no ownership manifest. + */ +export function resolvedVariableIn( + target: McpTarget, + teamDefs: McpServerDef[], + vars: Record, + raw: string, +): string | null { + if (!target.projectScope) return null; + for (const def of teamDefs) { + if (supportsEnvExpansion(target.format, target.projectScope, def)) continue; + const found = referencedVars(def).find((name) => { + const value = vars[name]; + return value !== undefined && value.length >= 8 && raw.includes(value); + }); + if (found) return found; + } + return null; +} + +/** + * The `info/exclude` git reads for `dir`'s checkout (worktrees and submodules + * included), the checkout's root, and `dir`'s path from it. + */ +async function gitExcludeFile(dir: string): Promise<{ excludeFile: string; root: string; prefix: string } | null> { + const result = await execCommand('git', ['rev-parse', '--show-toplevel', '--show-prefix', '--git-path', 'info/exclude'], { cwd: dir, timeoutMs: 10_000 }) + .catch(() => null); + if (!result || result.code !== 0) return null; + const [root = '', prefix = '', gitPath = ''] = result.stdout.split(/\r?\n/); + if (!root || !gitPath) return null; + // Real path, so one repository reached through a symlink (macOS /var) is one file. + const base = await fse.realpath(dir).catch(() => dir); + return { excludeFile: path.resolve(base, gitPath), root, prefix }; +} + +/** The closest directory above `file` that exists. */ +export async function existingAncestor(file: string): Promise { + let dir = path.dirname(path.resolve(file)); + while (!await pathExists(dir) && path.dirname(dir) !== dir) dir = path.dirname(dir); + return dir; +} + +/** + * Where a write to `file` lands: the real path of its closest existing + * directory, the rest appended. The appliers replace the file itself (tmp + + * rename) but follow its directories, so every check of whether git would + * commit the file judges this path (#886), and reads keep `file`. + */ +export async function realFilePath(file: string): Promise { + const dir = await existingAncestor(file); + const real = await fs.promises.realpath(dir).catch(() => dir); + return path.join(real, path.relative(dir, file)); +} + +/** + * Whether git would put a file in a commit. `unknown` is a repository git could + * not answer for (unsafe ownership, a bad config): never read it as safe. + */ +export type GitTracking = + | { kind: 'ignored' } + | { kind: 'would-commit' } + | { kind: 'outside-repo' } + | { kind: 'unknown'; error: string }; + + +/** + * `file` as a message names it, and the path to give git for it: the one a + * write lands in, named with `file`, when a directory inside its checkout is a + * symlink (#886), where git refuses `file` ("beyond a symbolic link"). A + * symlink above the checkout (macOS /var) changes no path git uses. + */ +export async function gitPathOf(file: string): Promise<{ label: string; path: string }> { + const landed = await realFilePath(file); + if (landed === file) return { label: file, path: file }; + const location = await gitExcludeFile(await existingAncestor(landed)); + const inCheckout = location ? path.relative(location.root, landed) : ''; + if (inCheckout && !inCheckout.startsWith('..') && file.endsWith(`${path.sep}${inCheckout}`)) return { label: file, path: file }; + return { label: `${landed} (where ${file} is written)`, path: landed }; +} + +/** + * Whether git would put `file` in a commit: tracked, or untracked without an + * ignore rule. Judged where a write to it lands. Read-only. + */ +export async function gitTracking(file: string): Promise { + file = await realFilePath(file); + const dir = await existingAncestor(file); + const result = await execCommand('git', ['check-ignore', '-q', '--', path.relative(dir, file)], { cwd: dir, timeoutMs: 10_000 }) + .catch((e: unknown) => ({ code: -1, stdout: '', stderr: e instanceof Error ? e.message : String(e) })); + if (result.code === 0) return { kind: 'ignored' }; + if (result.code === 1) return { kind: 'would-commit' }; + // Anything else is no repository at all, or git failing inside one. + for (let d = path.resolve(dir); ; d = path.dirname(d)) { + if (await pathExists(path.join(d, '.git'))) return { kind: 'unknown', error: result.stderr.trim() || `git exited with ${result.code}` }; + if (path.dirname(d) === d) return { kind: 'outside-repo' }; + } +} + +/** + * Whether git tracks `file` (#879): the next `git commit -a` commits a change to + * it, and no exclude rule stops that. Read-only. `unknown` is git failing to + * answer: never read it as untracked. + */ +export async function gitTracks(file: string): Promise<{ kind: 'tracked' } | { kind: 'untracked' } | { kind: 'unknown'; error: string }> { + file = await realFilePath(file); + // The file, or even its directory, may be gone from disk and still be in the index. + const dir = await existingAncestor(file); + const result = await execCommand('git', ['--literal-pathspecs', 'ls-files', '--error-unmatch', '--', path.relative(dir, file)], { cwd: dir, timeoutMs: 10_000 }) + .catch((e: unknown) => ({ code: -1, stdout: '', stderr: e instanceof Error ? e.message : String(e) })); + if (result.code === 0) return { kind: 'tracked' }; + if (result.code === 1) return { kind: 'untracked' }; + return { kind: 'unknown', error: result.stderr.trim() || `git exited with ${result.code}` }; +} + +/** + * teamai's block and what surrounds it; null without both markers, so a damaged + * block never takes the member's lines with it. The last start marker opens it: + * one that lost its end marker is left behind, not paired with the next block's end. + */ +function splitBlock(content: string): { before: string; patterns: string[]; after: string } | null { + const start = content.lastIndexOf(MCP_EXCLUDE_START); + const endAt = start === -1 ? -1 : content.indexOf(MCP_EXCLUDE_END, start); + if (endAt === -1) return null; + const patterns = content.slice(start + MCP_EXCLUDE_START.length, endAt) + .split(/\r?\n/).map((l) => l.trim()).filter((l) => l && !l.startsWith('#')); + const after = content.slice(endAt + MCP_EXCLUDE_END.length).replace(/^\r?\n/, ''); + return { before: content.slice(0, start), patterns, after }; +} + +/** + * Whether `file` is kept out of git, or why teamai could not keep it out and + * what the member does about it. `added`: this call listed it. `pending`: a dry + * run found nothing in the way of listing it. + */ +export type GitExclusion = + | { kind: 'excluded'; added: boolean } + | { kind: 'pending' } + | { kind: 'failed'; reason: string; fix: string }; + +/** + * Add `file` to its repository's `.git/info/exclude` unless git ignores it + * already, and whether git now leaves it out of a commit. Idempotent; a path + * already ignored, or outside any repository, adds nothing. One git tracks + * fails before anything else is checked, and so does one git cannot say it + * does not track: an exclude rule does not apply to a tracked file, and a git + * error is never read as safe. `file` need not exist yet: pull calls this + * before writing a resolved value into it. `dryRun` writes nothing and reports + * what would stop the write. + */ +export async function ensureExcludedFromGit(file: string, options: { dryRun?: boolean } = {}): Promise { + const tracking = await gitTracking(file); + if (tracking.kind === 'ignored' || tracking.kind === 'outside-repo') return { kind: 'excluded', added: false }; + const repair = 'Fix the repository, or add the file to its .git/info/exclude yourself, then run `teamai pull` again.'; + const tracked = async (): Promise => { + const named = await gitPathOf(file); + return { + kind: 'failed', + reason: `git already tracks ${named.label}`, + fix: `Run \`git rm --cached ${named.path}\` (rotate any value a commit of it holds), then \`teamai pull\` again.`, + }; + }; + const inIndex = await gitTracks(file); + if (inIndex.kind === 'tracked') return tracked(); + if (inIndex.kind === 'unknown') return { kind: 'failed', reason: inIndex.error, fix: repair }; + // Where the write lands. It and its directory need not exist yet: git is asked from the nearest one that does. + const landed = await realFilePath(file); + const dir = await existingAncestor(landed); + const location = await gitExcludeFile(dir); + if (!location) { + return { + kind: 'failed', + reason: tracking.kind === 'unknown' ? tracking.error : 'git could not locate .git/info/exclude', + fix: repair, + }; + } + const { excludeFile } = location; + // Anchored at the working tree root, glob characters escaped. + const rel = path.relative(dir, landed).split(path.sep).join('/'); + const pattern = `/${location.prefix}${rel}`.replace(/[\\*?[\]!#]/g, '\\$&'); + const retry = `Make it writable, or add \`${pattern}\` to it yourself, then run \`teamai pull\` again.`; + // A read-only exclude file is the member's choice; the atomic write would replace it all the same. + for (const writable of [path.dirname(excludeFile), ...(await pathExists(excludeFile) ? [excludeFile] : [])]) { + const denied = await fse.access(writable, fse.constants.W_OK).then(() => false, () => true); + if (denied) return { kind: 'failed', reason: `${writable} is not writable`, fix: retry }; + } + const add = (content: string): string | null => { + const block = splitBlock(content); + if (block?.patterns.includes(pattern)) return null; + const head = block ? block.before : content; + const patterns = [...(block?.patterns ?? []), pattern]; + const body = [MCP_EXCLUDE_START, ...patterns, MCP_EXCLUDE_END].join('\n'); + const sep = head === '' || head.endsWith('\n') ? '' : '\n'; + return `${head}${sep}${body}\n${block?.after ?? ''}`; + }; + let result: ExcludeUpdate; + try { + if (options.dryRun) { + if (add((await readFileSafe(excludeFile)) ?? '') !== null) { + // A negated rule in a .gitignore outranks .git/info/exclude: the line would change nothing. + const rule = await reincludingRule(landed); + return rule && path.basename(rule.source) === '.gitignore' ? reincluded(await gitPathOf(file), rule) : { kind: 'pending' }; + } + result = 'unchanged'; + } else { + result = await updateFileLocked(excludeFile, add); + } + } catch (e) { + return { kind: 'failed', reason: `adding it to ${excludeFile} failed: ${e instanceof Error ? e.message : String(e)}`, fix: retry }; + } + if (result === 'locked') { + return { + kind: 'failed', + reason: `another teamai command held ${excludeFile} past the wait`, + fix: 'Run `teamai pull` again.', + }; + } + if (result === 'written') log.debug(`Added ${pattern} to ${excludeFile}`); + if ((await gitTracking(file)).kind !== 'would-commit') return { kind: 'excluded', added: result === 'written' }; + // Untracked, as checked above: a rule git reads after teamai's line, or before it, re-includes the file. + return reincluded(await gitPathOf(file), await reincludingRule(landed)); +} + +/** The failure for a file a rule of the member's re-includes, naming `rule` when git could. */ +function reincluded(named: { label: string }, rule: { source: string; line: string; pattern: string } | null): GitExclusion { + return rule + ? { + kind: 'failed', + reason: `a rule in your git ignore files re-includes ${named.label}: \`${rule.pattern}\` (${rule.source}:${rule.line})`, + fix: `Remove \`${rule.pattern}\` from ${rule.source}, then run \`teamai pull\` again.`, + } + : { + kind: 'failed', + reason: `a rule in your git ignore files re-includes ${named.label}`, + fix: 'Remove the rule in .gitignore, .git/info/exclude or core.excludesFile that re-includes it (`git check-ignore -v` names it), then run `teamai pull` again.', + }; +} + +/** The negated rule `git check-ignore -v` says decides `file`, or null when it names none. */ +async function reincludingRule(file: string): Promise<{ source: string; line: string; pattern: string } | null> { + const dir = await existingAncestor(file); + const result = await execCommand('git', ['check-ignore', '-v', '--', path.relative(dir, file)], { cwd: dir, timeoutMs: 10_000 }) + .catch(() => null); + // ::, the source as git names it from `dir`. + const match = result?.code === 0 ? /^(.*):(\d+):(!.*)\t/.exec(result.stdout) : null; + return match ? { source: path.resolve(dir, match[1]), line: match[2], pattern: match[3] } : null; +} + +/** + * `ensureExcludedFromGit` for a file already on disk that may hold a resolved + * value, warning when it fails rather than failing the sync that wrote the file. + */ +export async function excludeFromGit(file: string): Promise { + if (!await pathExists(file)) return; + const exclusion = await ensureExcludedFromGit(file); + if (exclusion.kind === 'failed') { + log.warn( + `${file} may hold a resolved MCP variable, and teamai could not keep it out of git: ${exclusion.reason}. ` + + `${exclusion.fix} Do not commit the file meanwhile.`, + ); + } +} + +/** How `updateFileLocked` left the file: `locked` wrote nothing, another command held it past the wait. */ +export type ExcludeUpdate = 'written' | 'unchanged' | 'locked'; + +/** + * Rewrite `file` with `edit` (null: leave it as it is), holding a lock + * across the read and an atomic write: the worktrees of a repository share + * `.git/info/exclude`, so two commands adding different paths must not drop each other's. + * A lock still held after the wait writes nothing: an unlocked write could drop + * the holder's pattern, leaving that path unprotected. `mode` forces the file's + * mode; without it the file keeps its own. + */ +export async function updateFileLocked( + file: string, + edit: (content: string) => string | null, + options: { mode?: number } = {}, +): Promise { + const { acquireLock, releaseLock } = await import('./update.js'); + const lockPath = `${file}.teamai-lock`; + let held = false; + for (let attempt = 0; attempt < 25 && !held; attempt++) { + held = await acquireLock(lockPath); + if (!held) await new Promise((resolve) => setTimeout(resolve, 100)); + } + if (!held) return 'locked'; + try { + const next = edit((await readFileSafe(file)) ?? ''); + if (next === null) return 'unchanged'; + await writeFileAtomic(file, next, options); + return 'written'; + } finally { + await releaseLock(lockPath); + } +} + +/** + * The `.git/info/exclude` files holding teamai's block, one per repository + * among those `dirs` are in (a config inside a nested repository or submodule + * is excluded from that repository, not from the project root's), each with + * its patterns and the absolute paths each protects in the checkouts `dirs` reach. + */ +export async function findMcpGitExcludes(dirs: Iterable): Promise>> { + const roots = new Map>(); + for (const dir of new Set(dirs)) { + const location = await gitExcludeFile(dir); + if (!location) continue; + const seen = roots.get(location.excludeFile) ?? new Set(); + roots.set(location.excludeFile, seen.add(location.root)); + } + const found = new Map>(); + for (const [excludeFile, checkouts] of roots) { + const content = await readFileSafe(excludeFile); + const block = content === null ? null : splitBlock(content); + if (!block) continue; + // Every checkout sharing the file, including a nested repository's linked worktrees elsewhere. + const [anyCheckout] = checkouts; + if (anyCheckout) for (const worktree of await listWorktrees(anyCheckout)) checkouts.add(worktree); + found.set(excludeFile, block.patterns.map((pattern) => { + const rel = mcpExcludePatternPath(pattern); + return { pattern, files: [...checkouts].map((root) => path.join(root, rel)) }; + })); + } + return found; +} + +/** The path from its checkout's root one of teamai's patterns stands for: `/`, glob characters escaped (see ensureExcludedFromGit). */ +export function mcpExcludePatternPath(pattern: string): string { + return pattern.replace(/^\//, '').replace(/\\(.)/g, '$1'); +} + +/** + * Remove `patterns` from teamai's block in `excludeFile` (one `findMcpGitExcludes` + * returned), and the block with its last pattern. + */ +export async function removeMcpGitExclude(excludeFile: string, patterns: string[]): Promise { + return updateFileLocked(excludeFile, (content) => { + const block = splitBlock(content); + if (!block) return null; + const kept = block.patterns.filter((p) => !patterns.includes(p)); + if (kept.length === block.patterns.length) return null; + const body = kept.length > 0 ? `${[MCP_EXCLUDE_START, ...kept, MCP_EXCLUDE_END].join('\n')}\n` : ''; + return block.before + body + block.after; + }); +} diff --git a/src/mcp-reconcile.ts b/src/mcp-reconcile.ts index 5507906ec..7c12c7b71 100644 --- a/src/mcp-reconcile.ts +++ b/src/mcp-reconcile.ts @@ -1,6 +1,6 @@ import crypto from 'node:crypto'; +import fs from 'node:fs'; import path from 'node:path'; -import fse from 'fs-extra'; import type { LocalConfig, TeamaiConfig, @@ -17,7 +17,9 @@ import { managedMcpManifestKey, resolveToolBaseDir, scopedToolPaths, + TeamaiConfigSchema, } from './types.js'; +import YAML from 'yaml'; import { detectMcpFormat, supportsTransport, @@ -28,10 +30,14 @@ import { referencedVars, entryHash, MCP_SERVER_KEY, + sameServerKey, type McpFormat, } from './resources/mcp-format.js'; import { mcpEntryReader, teamMcpToDef } from './resources/mcp.js'; -import { envEntryReader } from './resources/env.js'; +import { envName, envTable } from './resources/env-key.js'; +import { declaredSecretKeys, type SecretDeclarations } from './resources/secrets.js'; +import { resolveTeamEnv, variablesKeptWarning, type TeamEnv } from './env-resolution.js'; +import { isEnvShMarker } from './env-sh-exports.js'; import { isToolInstalledForConfig } from './resources/base.js'; import { reportEntryResolution, resolveEntriesFor } from './namespaced-entries.js'; import { @@ -42,8 +48,30 @@ import { expandHome, } from './utils/fs.js'; import { log } from './utils/logger.js'; +import { warnOnce } from './utils/warn-once.js'; import { loadProjectMcpManifest } from './utils/mcp-manifest.js'; import { isOnPath, SAFE_BIN_RE, type LookPathOptions } from './utils/lookpath.js'; +import { + carriesResolvedValue, + ensureExcludedFromGit, + excludeFromGit, + findMcpGitExcludes, + gitTracks, + mcpExcludePatternPath, + realFilePath, + removeMcpGitExclude, + resolvedVariableIn, + type GitExclusion, +} from './mcp-git-exclude.js'; +import { createGit, getFileContentAtRev, listWorktrees } from './utils/git.js'; +import { + readResolvedMcpFiles, + recordUnverifiedMcpServers, + settleResolvedMcpFiles, + trackResolvedMcpFiles, + untrackResolvedMcpFiles, + type McpFileObservation, +} from './mcp-resolved-files.js'; // ─── Reconcile engine ──────────────────────────────────────── // @@ -73,6 +101,8 @@ export interface McpReconcileOptions { * without mutating the host platform. */ lookPath?: LookPathOptions; + /** This scope's env, when the caller already resolved it (env-resolution.ts). */ + teamEnv?: TeamEnv; } export interface McpChange { @@ -87,8 +117,9 @@ export interface McpReconcileResult { /** True when any file was actually written. */ wrote: boolean; /** - * Set when the team's servers could not be resolved (a file that does not - * parse, a name twice): nothing was changed, and the reason was reported. + * Set when the team's servers, or the secrets they may need, could not be + * resolved (a file that does not parse, a name twice): nothing was changed, + * and the reason was reported. */ unresolved?: true; } @@ -105,32 +136,53 @@ async function readManifest(manifestPath: string): Promise { /** * Build the ${VAR} lookup table: the team env variables this member receives * (root plus active namespace files, the same set pull writes env.sh from), - * then process env on top. + * each with the member's value for this team when they set one, then process + * env for every other key; it no longer overrides a team variable (#875). + * A declared secret (#875) resolves from the + * member's value for this team, then their value for the machine, then their + * own environment (not a value a teamai env.sh exported); its env.yaml value, + * if the team also sets one, is ignored. * * The installed KEY=value backup is read instead only when that set cannot be - * resolved (pull then keeps env.sh as it is, so MCP sees what the shell sees) - * or the team has no repo tree to resolve it from (HTTP mode). - */ -export async function buildVarTable(localConfig: LocalConfig): Promise> { - const table: Record = {}; - const env = localConfig.repo.kind === 'http' - ? null - : await resolveEntriesFor(envEntryReader, localConfig); - if (env?.kind === 'resolved') { - for (const variable of env.entries) table[variable.name] = variable.entry.value; + * resolved, or the secret declarations or the member's values cannot (pull + * then keeps env.sh as it is, so MCP sees what the shell sees), or the team has no repo tree to + * resolve it from (HTTP mode, which declares no secrets). + * + * `teamEnv` is for a caller that already resolved it, so one command reads + * each file once. HTTP mode ignores it. + */ +export async function buildVarTable(localConfig: LocalConfig, teamEnv?: TeamEnv): Promise> { + const table = envTable(); + const resolved = localConfig.repo.kind === 'http' ? null : teamEnv ?? await resolveTeamEnv(localConfig); + const secretKeys = resolved ? declaredSecretKeys(resolved.declarations) : new Set(); + const isSecret = (key: string): boolean => secretKeys?.has(key) ?? false; + const variables = resolved?.variables.kind === 'resolved' && secretKeys ? resolved.variableValues : null; + if (variables?.kind === 'resolved') { + for (const [key, variable] of variables.values) table[key] = variable.value; } else { - Object.assign(table, await readEnvBackup(localConfig)); + if (variables) warnOnce(variablesKeptWarning(variables.reason)); + for (const [key, value] of Object.entries(await readEnvBackup(localConfig))) if (!isSecret(key)) table[key] = value; } - // process.env wins: it lets a user override a team-provided value locally. + // The environment fills only what the team sets nothing for (#875): a + // member overrides a team variable with `teamai env set`, for that team. + // An env.sh marker says what a shell sourced, and is no server's value. + // On Windows `api_url` is the team's `API_URL`: names compare as the platform does. + const teamSet = new Set(Object.keys(table).map(envName)); for (const [k, v] of Object.entries(process.env)) { - if (v !== undefined) table[k] = v; + if (v !== undefined && !isSecret(k) && !isEnvShMarker(k) && !teamSet.has(envName(k))) table[k] = v; } + if (!resolved || !secretKeys || secretKeys.size === 0) return table; + if (resolved.secrets.kind === 'store-unreadable') { + warnOnce(`${resolved.secrets.reason} Team secrets have no value until it is fixed.`); + return table; + } + for (const [key, secret] of resolved.secrets.values) table[key] = secret.value; return table; } /** The KEY=value file the env channel last wrote. */ async function readEnvBackup(localConfig: LocalConfig): Promise> { - const table: Record = {}; + const table = envTable(); // Must use the same path the env channel wrote (getEnvBackupPath) — self mode // uses env.local, not env (which is a committed directory there). const envFile = getEnvBackupPath(localConfig); @@ -196,12 +248,19 @@ function requirementsMet(def: McpServerDef, lookPath?: LookPathOptions): string // ─── Tool targeting ────────────────────────────────────────── -interface McpTarget { +export interface McpTarget { tool: string; format: McpFormat; /** Absolute path of the config file to edit. */ file: string; projectScope: boolean; + /** + * Added by `includeUndetected`: the built-in location of a tool the team maps + * elsewhere or not at all. No mapping of today's reaches it for this tool. + */ + builtinFallback?: true; + /** Added by `includeUndetected`: a tool not installed on this machine, so no pull of this checkout delivers to it. */ + undetected?: true; } /** @@ -215,13 +274,26 @@ interface McpTarget { export async function resolveMcpTargets( teamConfig: TeamaiConfig, localConfig: LocalConfig, + /** + * Also the tools not detected here, and in project scope the built-in + * location of a tool the team dropped or moved: a file an earlier pull + * wrote outlives its tool and its mapping. + */ + options: { includeUndetected?: boolean } = {}, ): Promise { const projectScope = localConfig.scope === 'project'; const targets: McpTarget[] = []; // Skills/settings/agents probe paths must reflect the active scope: OpenCode's // user-scope resources live under ~/.config/opencode, not ~/.opencode. - for (const [tool, paths] of Object.entries(scopedToolPaths(teamConfig, localConfig))) { + const toolPaths = scopedToolPaths(teamConfig, localConfig); + const entries: Array<[string, (typeof toolPaths)[string], boolean?]> = Object.entries(toolPaths); + if (options.includeUndetected && projectScope) { + for (const [tool, paths] of Object.entries(TeamaiConfigSchema.shape.toolPaths.parse(undefined))) { + if (paths.mcpProject && toolPaths[tool]?.mcpProject !== paths.mcpProject) entries.push([tool, paths, true]); + } + } + for (const [tool, paths, builtinFallback] of entries) { const format = detectMcpFormat(tool); if (!format) continue; @@ -238,16 +310,73 @@ export async function resolveMcpTargets( const probe = paths.skills ?? paths.settings ?? paths.agents; if (!probe) continue; - if (!await isToolInstalledForConfig(tool, probe, localConfig, file)) { + const installed = await isToolInstalledForConfig(tool, probe, localConfig, file); + if (!options.includeUndetected && !installed) { log.debug(`Skipping MCP sync for ${tool}: tool not installed`); continue; } - targets.push({ tool, format, file, projectScope }); + targets.push({ + tool, format, file, projectScope, + ...builtinFallback ? { builtinFallback: true as const } : {}, + ...installed ? {} : { undetected: true as const }, + }); } return targets; } +/** + * Whether a missing record of `target`'s tool makes its file's unclaimed servers suspect (#882): a tool the + * team maps there, installed, or not installed while no installed tool maps that file or while + * managed-mcp-files.json lists it as having written a resolved value there (`writers`). + */ +export function unrecordedMcpTool(target: McpTarget, targets: McpTarget[], writers: readonly string[] = []): boolean { + if (target.builtinFallback) return false; + return !target.undetected || writers.includes(target.tool) + || !targets.some((other) => other.file === target.file && !other.undetected); +} + +/** + * The built-in fallbacks among `targets` their own tool's current mapping does + * not reach (#882): the team moved or dropped the tool, so its manifest + * records describe another file, or none, while an earlier pull may have + * written this one. In one another tool maps today (CodeBuddy's `.mcp.json`, + * which Claude maps), that tool's records tell its own servers. + */ +export async function unmappedMcpDefaults(targets: McpTarget[]): Promise> { + const unmapped = new Set(); + for (const target of targets) { + if (!target.builtinFallback) continue; + const own = await Promise.all(targets.filter((t) => t.tool === target.tool && !t.builtinFallback).map((t) => realFilePath(t.file))); + if (!own.includes(await realFilePath(target.file))) unmapped.add(target); + } + return unmapped; +} + +/** + * The files of `unmapped` (`unmappedMcpDefaults`) that exist and `cfg`'s + * worktree has not recorded for their tool, as `earlierMappedMcpTargets` + * returns its files: judged as one an earlier mapping reached. `known`: the + * other targets. + */ +export async function unrecordedUnmappedMcpDefaults( + cfg: LocalConfig, + unmapped: Iterable, + known: McpTarget[], +): Promise> { + const reach = await Promise.all(known.map(async ({ tool, file }) => ({ tool, real: await realFilePath(file) }))); + const recorded = await Promise.all(Object.entries((await readResolvedMcpFiles(cfg)).files) + .flatMap(([file, { tools }]) => tools.map(async (tool) => ({ tool, real: await realFilePath(file) })))); + const found: Array = []; + for (const target of unmapped) { + const real = await realFilePath(target.file); + if (recorded.some((r) => r.tool === target.tool && r.real === real) || !await pathExists(target.file)) continue; + const mappedBy = [...new Set(reach.filter((r) => r.real === real && r.tool !== target.tool).map((r) => r.tool))]; + found.push({ ...target, tracked: (await gitTracks(target.file)).kind === 'tracked', mappedBy }); + } + return found; +} + // ─── JSON target I/O ───────────────────────────────────────── export interface JsonDoc { @@ -284,18 +413,22 @@ export async function readJsonDoc( } } -/** Write a parsed JSON MCP config while preserving its original container shape. */ +/** + * Write a parsed JSON MCP config while preserving its original container + * shape, and its mode unless `options.mode` forces one. + */ export async function writeJsonDoc( file: string, serverKey: string, doc: JsonDoc, + options?: { mode?: number }, ): Promise { if (doc.bare) { - await writeJsonAtomic(file, doc.servers); + await writeJsonAtomic(file, doc.servers, options); return; } doc.data[serverKey] = doc.servers; - await writeJsonAtomic(file, doc.data); + await writeJsonAtomic(file, doc.data, options); } // ─── Codex TOML target I/O ─────────────────────────────────── @@ -362,6 +495,8 @@ export interface DesiredMcpEntry { hash: string; /** Codex alone stores a TOML block rather than a JSON value. */ block?: string; + /** The entry holds a `${VAR}` value teamai resolved, which may be a team secret. */ + resolvedValue: boolean; } /** Everything the per-server filters need, resolved once per run. */ @@ -369,6 +504,8 @@ export interface DesiredMcpContext { sharing: ReturnType; excluded: Set; vars: Record; + /** Which `${VAR}` names are declared secrets, whose missing value keeps an entry (#875). */ + secrets: SecretDeclarations; lookPath?: McpReconcileOptions['lookPath']; } @@ -377,10 +514,13 @@ export async function buildDesiredMcpContext( localConfig: LocalConfig, options: McpReconcileOptions = {}, ): Promise { + // HTTP mode has no repo tree to declare secrets in. + const teamEnv = localConfig.repo.kind === 'http' ? undefined : options.teamEnv ?? await resolveTeamEnv(localConfig); return { sharing: getMcpSharing(teamConfig), excluded: new Set(localConfig.excludedSkills ?? []), - vars: await buildVarTable(localConfig), + vars: await buildVarTable(localConfig, teamEnv), + secrets: teamEnv?.declarations ?? { kind: 'absent' }, lookPath: options.lookPath, }; } @@ -393,14 +533,24 @@ export async function buildDesiredMcpContext( * the filters (#624). A second copy of them is how an MCP server ends up * skipped for `unresolved variable(s)` during one pull and reported as * correctly delivered forever after. + * + * `kept` names the skipped servers whose only missing variables are declared + * secrets (#875): the session-start pull inherits the agent's environment, so + * a secret that lives in the member's shell is there for one pull and gone for + * the next, and an entry an earlier pull wrote stays as it is. With + * declarations that failed, every skipped server is kept. */ export function desiredMcpForTarget( target: McpTarget, teamDefs: McpServerDef[], ctx: DesiredMcpContext, -): { desired: Map; skipped: McpChange[] } { +): { desired: Map; skipped: McpChange[]; kept: Set } { const desired = new Map(); const skipped: McpChange[] = []; + const kept = new Set(); + // Declarations that failed can't say which variables are secrets, so every + // missing one may be: pull keeps every installed entry then. + const declared = declaredSecretKeys(ctx.secrets); for (const raw of teamDefs) { if (raw.tools && !raw.tools.includes(target.tool)) continue; @@ -430,9 +580,8 @@ export function desiredMcpForTarget( // Pass ${VAR} through where the tool expands it itself, so the secret // never lands on disk; otherwise resolve and require every var to exist. - // A resolved value is written verbatim into the target file, including - // project-scope files that get committed — the team has opted into that - // by declaring the server with a ${VAR} a tool cannot expand itself. + // A resolved value is written verbatim into the target file; a project + // file gets one only once it is kept out of git (#882, reconcileTargets). const passthrough = supportsEnvExpansion(target.format, target.projectScope, raw); let def = raw; if (!passthrough) { @@ -444,6 +593,7 @@ export function desiredMcpForTarget( action: 'skipped', reason: `unresolved variable(s): ${missing.join(', ')}`, }); + if (missing.every((key) => declared?.has(key) ?? true)) kept.add(raw.name); continue; } def = resolved; @@ -451,16 +601,17 @@ export function desiredMcpForTarget( log.debug(`${raw.name}: passing ${referencedVars(raw).join(', ')} through to ${target.tool}`); } + const resolvedValue = !passthrough && referencedVars(raw).length > 0; if (target.format === 'codex') { const block = renderCodexBlock(def); - desired.set(raw.name, { entry: block, hash: entryHash(block), block }); + desired.set(raw.name, { entry: block, hash: entryHash(block), block, resolvedValue }); } else { const entry = renderJsonEntry(target.format, def); - desired.set(raw.name, { entry, hash: entryHash(entry) }); + desired.set(raw.name, { entry, hash: entryHash(entry), resolvedValue }); } } - return { desired, skipped }; + return { desired, skipped, kept }; } /** @@ -489,6 +640,508 @@ export async function installedMcpEntries(target: McpTarget): Promise { + const dataHome = getDataHome(localConfig); + if (localConfig.scope === 'project' && localConfig.projectRoot) { + return loadProjectMcpManifest(dataHome, localConfig.projectRoot, { dryRun }); + } + const manifestPath = managedMcpManifestPath(dataHome); + return { manifestPath, manifest: await readManifest(manifestPath) }; +} + +/** + * The team servers whose entry an earlier pull wrote and a pull now keeps, + * because a declared secret has no value (#875), with the tools holding one. + * Read-only: for the note that such an entry may hold an old value. + */ +export async function keptMcpEntries( + teamConfig: TeamaiConfig, + localConfig: LocalConfig, + teamEnv?: TeamEnv, +): Promise> { + const kept = new Map(); + if (localConfig.repo.kind === 'http') return kept; + const resolution = await resolveEntriesFor(mcpEntryReader, localConfig); + if (resolution.kind === 'failed' || resolution.entries.length === 0) return kept; + const teamDefs = resolution.entries.map((entry) => teamMcpToDef(entry.entry)); + const targets = await resolveMcpTargets(teamConfig, localConfig); + if (targets.length === 0) return kept; + const ctx = await buildDesiredMcpContext(teamConfig, localConfig, { teamEnv }); + if (ctx.secrets.kind !== 'resolved') return kept; + const { manifest } = await loadMcpManifest(localConfig, true); + + for (const target of targets) { + if (mcpTargetExcluded(localConfig, target)) continue; + const owned = new Set((manifest[managedMcpManifestKey(target.tool, target.projectScope)] ?? []).map((r) => r.name)); + const installed = await installedMcpEntries(target); + for (const name of desiredMcpForTarget(target, teamDefs, ctx).kept) { + if (!owned.has(name) || !installed?.has(name)) continue; + kept.set(name, [...kept.get(name) ?? [], target.tool]); + } + } + return kept; +} + +/** + * Why `target`'s file may hold a value teamai resolved (#882), or null when it + * is missing or proven not to. Judged by what is on disk and in the manifest + * (`owned`: the records it holds for the file's tool), never by delivery: an + * owned entry whose definition cannot be read, or has left the team's servers, + * is unproven, and so is one still as a pull wrote it with a resolved value, + * whatever its definition says now. So is a server `unverified` names + * (managed-mcp-files.json): one in the file when teamai rebuilt its lost + * record. A file that does not parse is judged by the ledger alone. `ctx` is + * asked for only by a record an older teamai wrote. + */ +export async function resolvedValueEvidence( + target: McpTarget, + teamDefs: McpServerDef[] | null, + ledger: { owned: ManagedMcpRecord[]; unverified?: string[] }, + vars: Record, + ctx: () => Promise, +): Promise { + const raw = await readFileSafe(target.file); + if (raw === null) return null; + const installed = await installedMcpEntries(target); + const records = installed ? ledger.owned.filter((record) => installed.has(record.name)) : ledger.owned; + const present = records.map((record) => record.name); + const unverified = (ledger.unverified ?? []).find((name) => !installed || installed.has(name)); + if (unverified) return `${unverified}, which was in the file when teamai rebuilt its lost record, so teamai cannot tell whether a pull wrote it`; + if (!teamDefs) return present.length > 0 ? `teamai's ${present.join(', ')}, and the team's MCP servers cannot be read` : null; + const dropped = present.find((name) => !teamDefs.some((def) => def.name === name)); + if (dropped) return `teamai's ${dropped}, which has left the team's MCP servers`; + const needing = present.find((name) => carriesResolvedValue(target, teamDefs, [name])); + if (needing) return `teamai's ${needing}, which needs a resolved \${VAR}`; + // An entry as a pull wrote it holds what that pull resolved, whatever its definition says now. + let desired: Map | undefined; + for (const record of installed ? records : []) { + if (entryHash(installed?.get(record.name)) !== record.hash) continue; + if (record.resolved === true) return `teamai's ${record.name}, as a pull wrote it with a resolved \${VAR}`; + if (record.resolved !== undefined) continue; + // An older teamai did not note it: stale, unless today's definition writes the same entry. + desired ??= desiredMcpForTarget(target, teamDefs, await ctx()).desired; + if (desired.get(record.name)?.hash !== record.hash) { + return `teamai's ${record.name}, which an earlier pull wrote and its current definition no longer produces`; + } + } + const variable = resolvedVariableIn(target, teamDefs, vars, raw); + return variable ? `the value of $${variable}` : null; +} + +/** + * The servers in `target`'s file that none of `claimed` names, in a file git + * does not track (#882): judged while the worktree has no managed-mcp.json + * (`claimed`: the records a pull wrote there since, if any), when any of them + * may be one teamai wrote. None for a file git tracks: no line protects it. + */ +export async function unclaimedMcpServers(target: McpTarget, claimed: readonly string[]): Promise { + const unclaimed = [...(await installedMcpEntries(target))?.keys() ?? []].filter((name) => !claimed.includes(name)); + return unclaimed.length === 0 || (await gitTracks(target.file)).kind === 'tracked' ? [] : unclaimed; +} + +/** `load`, run once, on the first call. */ +function once(load: () => Promise): () => Promise { + let value: Promise | undefined; + return () => value ??= load(); +} + +/** A file `recordedMcpTargets` returns. */ +export interface RecordedMcpFile { + /** A target per tool it was recorded for whose mapping in `known` no longer reaches it. */ + targets: McpTarget[]; + /** The tools whose target in `known` reaches it: their manifest records tell their own servers there. */ + mappedBy: string[]; + /** Recorded as one git tracked (managed-mcp-files.json): no line protects it while git does. */ + tracked: boolean; +} + +/** + * The files `cfg`'s worktree recorded writing a resolved value to (#882) for + * a tool no target in `known` reaches them for: the team has since changed or + * removed the toolPaths mapping they were written under. A file another + * tool's target reaches is among them while a tool it was recorded for is not + * one of those. + */ +export async function recordedMcpTargets(cfg: LocalConfig, known: McpTarget[]): Promise> { + const reach = await Promise.all(known.map(async (target) => ({ tool: target.tool, real: await realFilePath(target.file) }))); + const recorded = new Map(); + for (const [file, entry] of Object.entries((await readResolvedMcpFiles(cfg)).files)) { + const real = await realFilePath(file); + const mappedBy = [...new Set(reach.filter((r) => r.real === real).map((r) => r.tool))]; + const targets = entry.tools.filter((tool) => !mappedBy.includes(tool)).flatMap((tool): McpTarget[] => { + const format = detectMcpFormat(tool); + return format ? [{ tool, format, file, projectScope: true }] : []; + }); + if (targets.length > 0) recorded.set(file, { targets, mappedBy, tracked: entry.tracked === true }); + } + return recorded; +} + +// Built-in mcpProject defaults an older teamai wrote to and no longer maps: +// no teamai.yaml revision names them. +const EARLIER_BUILTIN_MCP_PROJECT = { + codebuddy: { mcpProject: '.codebuddy/mcp.json' }, // before 57636a27 +}; + +/** + * The files earlier revisions of the team's teamai.yaml mapped a tool's + * project MCP config to (`toolPaths..mcpProject`) that exist under the + * project root, and that neither the tool's own target in `known` nor a file + * `cfg`'s worktree recorded for the tool is (#882), each saying whether git + * tracks it (no exclude line applies to one it does) and which other tools' + * targets in `known` reach it: a teamai from before managed-mcp-files.json may have + * written a resolved value there, under a mapping the team changed before + * this member's first pull on a teamai that records one, plus those under a + * built-in default teamai has since changed. Read from the team + * repo's history of teamai.yaml, as far as the clone has it (a shallow clone + * has less). Null when git cannot read it: not a repository, no commits, a + * git error. + */ +export async function earlierMappedMcpTargets( + cfg: LocalConfig, + known: McpTarget[], +): Promise | null> { + const { projectRoot } = cfg; + if (!projectRoot) return []; + const repoPath = cfg.repo.localPath; + let revisions: string[]; + try { + revisions = (await createGit(repoPath).raw(['log', '--format=%H', 'HEAD', '--', 'teamai.yaml'])).split('\n').filter(Boolean); + } catch (e) { + log.debug(`Could not read the history of teamai.yaml in ${repoPath}: ${e instanceof Error ? e.message : String(e)}. The next pull tries again.`); + return null; + } + const root = await realFilePath(projectRoot); + // Each path, by real path, with the tools today's targets or the record reach it for. + const mapped = await Promise.all(known.map(async ({ tool, file }) => ({ tool, real: await realFilePath(file) }))); + const recorded = await Promise.all(Object.entries((await readResolvedMcpFiles(cfg)).files) + .flatMap(([file, { tools }]) => tools.map(async (tool) => ({ tool, real: await realFilePath(file) })))); + const reached = (tool: string, real: string): boolean => [...mapped, ...recorded].some((r) => r.tool === tool && r.real === real); + const found = new Map(); + for (const revision of [null, ...revisions]) { + let toolPaths: unknown = EARLIER_BUILTIN_MCP_PROJECT; + if (revision !== null) { + try { + toolPaths = (YAML.parse((await getFileContentAtRev(repoPath, revision, './teamai.yaml'))?.toString() ?? '') as { toolPaths?: unknown } | null)?.toolPaths; + } catch { + continue; + } + } + if (typeof toolPaths !== 'object' || toolPaths === null) continue; + for (const [tool, paths] of Object.entries(toolPaths)) { + const rel: unknown = typeof paths === 'object' && paths !== null ? (paths as { mcpProject?: unknown }).mcpProject : undefined; + const format = detectMcpFormat(tool); + if (typeof rel !== 'string' || !format) continue; + const file = path.resolve(resolveToolBaseDir(tool, cfg), rel); + const key = `${tool}\0${file}`; + if (found.has(key)) continue; + const real = await realFilePath(file); + const inside = path.relative(root, real); + if (inside === '' || inside === '..' || inside.startsWith(`..${path.sep}`) || path.isAbsolute(inside)) continue; + if (reached(tool, real) || !await pathExists(file)) continue; + const mappedBy = [...new Set(mapped.filter((r) => r.real === real).map((r) => r.tool))]; + found.set(key, { tool, format, file, projectScope: true, tracked: (await gitTracks(file)).kind === 'tracked', mappedBy }); + } + } + return [...found.values()]; +} + +/** What one file, read in the format of each of `targets` (all for that file), holds. */ +async function mcpFileState(targets: McpTarget[]): Promise { + const servers = new Set(); + for (const target of targets) { + if (!await pathExists(target.file)) return { kind: 'missing' }; + const installed = await installedMcpEntries(target); + if (!installed) return { kind: 'unparsable' }; + for (const name of installed.keys()) servers.add(name); + } + return { kind: 'parsed', servers: [...servers] }; +} + +/** + * Why a file `recordedMcpTargets` returned may still hold a value teamai + * resolved, or null once it is gone or holds no server: with no tool's + * definitions to judge its entries by, any server it holds may be teamai's. + * `owned`, for a file other tools' targets now reach: the servers their + * manifest records say they wrote there, which their own rules judge. Any + * other server may be what teamai wrote for `targets`' tools. + */ +export async function recordedMcpFileEvidence(targets: McpTarget[], owned?: readonly string[]): Promise { + const state = await mcpFileState(targets); + if (state.kind === 'unparsable') return 'it does not parse'; + if (state.kind !== 'parsed') return null; + if (!owned) { + return state.servers.length > 0 + ? 'teamai may have written a resolved value to it under an earlier toolPaths mapping, and it still holds MCP servers' + : null; + } + const other = state.servers.find((name) => !owned.includes(name)); + return other === undefined ? null + : `teamai may have written a resolved value to it for ${targets.map((t) => t.tool).join(', ')} under an earlier toolPaths mapping, ` + + `and it holds ${other}, which no tool that maps it now owns`; +} + +/** + * Why a file `earlierMappedMcpTargets` returned may hold a value an older + * teamai resolved, or null: judged as a recorded file is, since the + * manifest's records for its tool describe the file today's mapping reaches, + * not this one, plus the value scan. `owned`: for one other tools' targets + * reach today, the servers their manifest records say they wrote there. + */ +export async function earlierMappedMcpFileEvidence( + target: McpTarget, + teamDefs: McpServerDef[] | null, + vars: Record, + ctx: () => Promise, + owned?: readonly string[], +): Promise { + return await recordedMcpFileEvidence([target], owned) ?? await resolvedValueEvidence(target, teamDefs, { owned: [] }, vars, ctx); +} + +/** + * What each of this worktree's project MCP configs holds, for + * `settleResolvedMcpFiles`: each of `targets`' files, judged by `holds`, and + * each file `recordedMcpTargets` returns, by `recordedMcpFileEvidence`, but + * for one recorded as tracked that git still tracks: no line protects it. + */ +async function observeMcpConfigs( + localConfig: LocalConfig, + targets: McpTarget[], + manifest: ManagedMcpManifest, + holds: (target: McpTarget, owned: ManagedMcpRecord[]) => Promise, +): Promise { + const observations: McpFileObservation[] = []; + for (const target of targets) { + const owned = manifest[managedMcpManifestKey(target.tool, true)] ?? []; + const state = await mcpFileState([target]); + observations.push({ file: target.file, tool: target.tool, state, holding: await holds(target, owned), owned: owned.map((r) => r.name) }); + } + for (const [file, { targets: group, mappedBy, tracked }] of await recordedMcpTargets(localConfig, targets)) { + const state = await mcpFileState(group); + const stillTracked = tracked && (await gitTracks(file)).kind === 'tracked'; + const owned = mappedBy.length === 0 ? undefined + : mappedBy.flatMap((tool) => manifest[managedMcpManifestKey(tool, true)] ?? []).map((record) => record.name); + const holding = !stillTracked && await recordedMcpFileEvidence(group, owned) !== null; + for (const { tool } of group) { + observations.push({ + file, tool, state, holding, owned: owned ?? [], + ...tracked ? { tracked: stillTracked } : {}, + ...owned && !stillTracked ? { remapped: true as const } : {}, + }); + } + } + return observations; +} + +/** `settleResolvedMcpFiles`, which only ever brings the record closer to the disk: a failure waits for the next pull. */ +async function settleRecordedMcpConfigs( + localConfig: LocalConfig, + observations: McpFileObservation[], + options?: { earlierMappingsRead?: boolean }, +): Promise { + const result = await settleResolvedMcpFiles(localConfig, observations, options).catch((e: unknown) => e instanceof Error ? e.message : String(e)); + if (result !== 'written' && result !== 'unchanged') { + log.debug(`Did not update managed-mcp-files.json: ${result === 'locked' ? 'another teamai command held it past the wait' : result}. The next pull tries again.`); + } +} + +/** + * `localConfig` and, in project scope, one config per other linked worktree: + * each worktree has its own MCP configs and managed-mcp manifest. + */ +export async function projectWorktreeConfigs(localConfig: LocalConfig): Promise { + const configs: LocalConfig[] = [localConfig]; + if (localConfig.scope === 'project' && localConfig.projectRoot) { + const { resolveProjectDataHome } = await import('./config.js'); + for (const wt of await listWorktrees(localConfig.projectRoot)) { + if (wt === localConfig.projectRoot) continue; + configs.push({ ...localConfig, projectRoot: wt, dataHome: await resolveProjectDataHome(wt) }); + } + } + return configs; +} + +/** A project worktree's managed-mcp.json: `{}` when it is gone, empty or does not parse. */ +async function readProjectMcpManifest(cfg: LocalConfig, projectRoot: string): Promise { + return (await loadProjectMcpManifest(getDataHome(cfg), projectRoot, { dryRun: true })).manifest; +} + +/** + * The files of `groups` (the checkouts of one exclude line) not proven free of + * a value teamai resolved (#882), each with why. A missing file is clean; so is + * one a tool reads that parses and holds no server at all, and one in a nested + * repository's linked worktree, read as the file of its line this project maps + * is, that parses and holds none, and one a worktree recorded writing a + * resolved value to under a toolPaths mapping since changed (managed-mcp-files.json) + * that parses and holds none, as is a tool's built-in location no mapping reaches today. One a tool reads + * holding servers is clean only when its worktree's manifest + * records what teamai wrote to that tool's file (an empty list once teamai took + * its last server out), and the file holds none of the team's servers that need + * a resolved `${VAR}` there, none of teamai's own entries the manifest records + * and cleanup left (their definition may have left mcp.yaml), and none of the + * values of the variables set in this environment. Anything else (no tool reads + * it, it does not parse, the team's servers cannot be read, the manifest is + * lost, empty, does not parse, has no record for the tool (for a file + * managed-mcp-files.json does not list, for any tool mapping it today), or a record rebuilt + * without noting the file's other servers in managed-mcp-files.json) is not: a server + * teamai wrote, since dropped from mcp.yaml, with a value no longer set, looks + * like the member's own. + * `before` is `localConfig`'s manifest as it stood before a reconcile rewrote it. + * With `otherWorktrees: 'empty'` another worktree's file is clean only when it + * holds no server at all: today's definitions and values cannot judge an entry + * that worktree's last pull wrote (a `${VAR}` since made a literal), only a + * pull there can. + */ +export async function mcpConfigsNotProvenClean( + teamConfig: TeamaiConfig, + localConfig: LocalConfig, + groups: Array<{ pattern: string; files: string[] }>, + options: { before?: ManagedMcpManifest; otherWorktrees?: 'judged' | 'empty' } = {}, +): Promise> { + const { before, otherWorktrees = 'judged' } = options; + const resolution = await resolveEntriesFor(mcpEntryReader, localConfig); + const teamDefs = resolution.kind === 'failed' ? null : resolution.entries.map((entry) => teamMcpToDef(entry.entry)); + // Keyed by real path: the protected paths come from git, which resolves symlinks (macOS /var). + const targets = new Map; mapsToday: Set; proven: Set; writers: Set; + /** Every tool's target on this file: tools of different formats read different keys of it. */ + all: McpTarget[]; + }>(); + const realRoot = (root: string | undefined): Promise => + root ? fs.promises.realpath(root).catch(() => root) : Promise.resolve(undefined); + const ownRoot = await realRoot(localConfig.projectRoot); + const recordedBy = new Map(); + // A built-in location no mapping reaches today, in each worktree: judged as a file an earlier mapping reached. + const unmappedBy = new Map(); + for (const cfg of await projectWorktreeConfigs(localConfig)) { + const manifest = cfg === localConfig && before ? before + : cfg.projectRoot ? await readProjectMcpManifest(cfg, cfg.projectRoot) + : {}; + // This checkout listed again under its real path is not another worktree. + const foreign = cfg !== localConfig && await realRoot(cfg.projectRoot) !== ownRoot; + const cfgTargets: McpTarget[] = []; + recordedBy.set(cfg, cfgTargets); + const { files: ledger } = await readResolvedMcpFiles(cfg); + const unmapped = [...await unmappedMcpDefaults(await resolveMcpTargets(teamConfig, cfg, { includeUndetected: true }))]; + unmappedBy.set(cfg, unmapped); + for (const target of await resolveMcpTargets(teamConfig, cfg, { includeUndetected: true })) { + const key = await realFilePath(target.file); + cfgTargets.push(target); + // Judged below, as a file an earlier mapping reached. + if (unmapped.some((t) => t.tool === target.tool && t.file === target.file)) continue; + const records = manifest[managedMcpManifestKey(target.tool, true)]; + const owned = Array.isArray(records) ? records : []; + // A rebuilt record whose file's other servers could not be noted says nothing of them yet. + const recorded = Array.isArray(records) && !records.some((record) => record.unnoted); + // One file reached twice (two tools share it, or a checkout through a symlink) merges what each says. + // It counts as recorded only while every tool managed-mcp-files.json says wrote a resolved value + // there still has its record: another tool's intact one proves nothing of that tool's entries. + // (A writer that no longer maps the file is judged by the remapped rule below.) With no such list + // (a file no pull on this version recorded), every tool whose mapping reaches it today needs one. + const seen = targets.get(key); + const mappers = new Set([...seen?.mappers ?? [], target.tool]); + const mapsToday = new Set([...seen?.mapsToday ?? [], ...target.builtinFallback ? [] : [target.tool]]); + const proven = new Set([...seen?.proven ?? [], ...recorded ? [target.tool] : []]); + const writers = new Set([...seen?.writers ?? [], ...ledger[target.file]?.tools ?? []]); + targets.set(key, { + target, + owned: [...seen?.owned ?? [], ...owned], + unverified: [...seen?.unverified ?? [], ...ledger[target.file]?.unverified ?? []], + recorded: proven.size > 0 && (writers.size > 0 + ? [...writers].every((tool) => proven.has(tool) || !mappers.has(tool)) + : [...mapsToday].every((tool) => proven.has(tool))), + mappers, + mapsToday, + proven, + writers, + all: [...seen?.all ?? [], target], + foreign: foreign || seen?.foreign === true, + }); + } + } + // Files a pull wrote under a mapping since changed, in any worktree: nothing but the file itself can judge them, + // and in one another tool now maps, nothing but that tool's records. + const recorded = new Map(); + const remapped = new Map(); + for (const [cfg, cfgTargets] of recordedBy) { + const groups = [...(await recordedMcpTargets(cfg, cfgTargets)).values()].map(({ targets: group }) => group); + for (const group of [...groups, ...[...unmappedBy.get(cfg) ?? []].map((target) => [target])]) { + const key = await realFilePath(group[0].file); + const map = targets.has(key) ? remapped : recorded; + const known = map.get(key) ?? []; + map.set(key, [...known, ...group.filter((t) => !known.some((k) => k.tool === t.tool && k.file === t.file))]); + } + } + // Short values, paths and the login name turn up in ordinary configs, so they prove nothing. + const identity = new Set(['USER', 'LOGNAME', 'USERNAME']); + const vars = await buildVarTable(localConfig); + const ctx = once(() => buildDesiredMcpContext(teamConfig, localConfig)); + const values = Object.entries(vars) + .filter(([name, value]) => value.length >= 8 && !identity.has(name) && !/^([/~]|[A-Za-z]:[\\/])/.test(value)); + const held = new Map(); + for (const { pattern, files } of groups) { + // A file no worktree of this project maps, in a checkout of the same repository as + // one it does: a nested repository's linked worktree, read as that one is. + const siblingFile = files.find((file) => targets.has(file)); + const sibling = siblingFile === undefined ? undefined : targets.get(siblingFile); + const nested = siblingFile && path.join(siblingFile, ...mcpExcludePatternPath(pattern).split('/').map(() => '..')); + for (const file of files) { + if (!await pathExists(file)) continue; + const earlier = recorded.get(file); + if (earlier) { + const why = await recordedMcpFileEvidence(earlier); + if (why) held.set(file, why); + continue; + } + const moved = remapped.get(file); + const movedWhy = moved && await recordedMcpFileEvidence(moved, targets.get(file)?.owned.map((record) => record.name) ?? []); + if (movedWhy) { + held.set(file, movedWhy); + continue; + } + const mappedHere = targets.get(file); + const knownHere = mappedHere + ?? (sibling && nested ? { target: { ...sibling.target, file }, owned: [], unverified: [], recorded: false, foreign: true, nested } : undefined); + const raw = (await readFileSafe(file)) ?? ''; + // Judged in the format of every tool that maps it: one tool's key may hold what another's doesn't. + const judge = async (known: NonNullable, target: McpTarget): Promise => { + const installed = await installedMcpEntries(target); + const named = installed && teamDefs + ? [...installed.keys()].find((name) => carriesResolvedValue(target, teamDefs, [name])) + : undefined; + return !installed ? 'it does not parse' + : installed.size === 0 ? undefined + : 'nested' in known ? `it holds MCP servers in a linked worktree of the repository at ${known.nested}, which teamai cannot judge` + : known.foreign && otherWorktrees === 'empty' ? 'it holds MCP servers in another worktree, which only a pull there can judge' + : !teamDefs ? 'the team\'s MCP servers cannot be read' + : named ? `it holds the team's ${named}, which needs a resolved \${VAR}` + : await resolvedValueEvidence(target, teamDefs, known, vars, ctx).then((e) => e && `it holds ${e}`) + ?? values.filter(([, value]) => raw.includes(value)).map(([name]) => `it holds the value of $${name}`)[0] + ?? (known.recorded ? undefined : 'it holds MCP servers, and managed-mcp.json, teamai\'s record of which it wrote there, is gone, does not parse, has no entry for it or was rebuilt without noting its other servers'); + }; + let why = knownHere ? undefined : 'no tool teamai knows reads it'; + const formats = mappedHere ? mappedHere.all.filter((t, i, all) => all.findIndex((o) => o.format === t.format) === i) + : knownHere ? [knownHere.target] : []; + for (const target of formats) { + why = knownHere && await judge(knownHere, target); + if (why) break; + } + if (why) held.set(file, why); + } + } + return held; +} + // ─── Main entry ────────────────────────────────────────────── export function mcpTargetExcluded(localConfig: LocalConfig, target: McpTarget): boolean { @@ -506,6 +1159,264 @@ export async function reconcileMcpForConfig( teamConfig: TeamaiConfig, localConfig: LocalConfig, options: McpReconcileOptions = {}, +): Promise { + // Each project config's exclusion from git, established before a resolved value is written into it. + const exclusions = new Map(); + // The project configs this run wrote: a line it added for one stays, whatever fails after. + const written = new Set(); + // The (file, tool) pairs managed-mcp-files.json first recorded this run, before their write, until that + // tool's records hold a resolved value there: another tool's write to the same file proves nothing of it. + const recorded: McpTarget[] = []; + const protect = !options.removeAll && !options.dryRun; + // Read before the reconcile records what it writes: a manifest it recreates says nothing of what came before. + const before = protect && localConfig.projectRoot ? await readProjectMcpManifest(localConfig, localConfig.projectRoot) : undefined; + try { + return await reconcileTargets(teamConfig, localConfig, options, exclusions, written, recorded); + } finally { + // A record this run added for a tool that then wrote no value goes, as its exclude line does. The settle + // below records the file again if it holds a resolved value all the same (an earlier pull wrote it). + await forgetUnwrittenMcpConfigs(localConfig, recorded); + // Also after a failed write: what earlier pulls wrote is on disk either way. + if (protect) await protectResolvedMcpConfigs(teamConfig, localConfig, exclusions, written, before); + } +} + +/** + * List each project MCP config holding a value teamai resolved in + * `.git/info/exclude` (#882), and take out the line of one proven clean. It + * covers what is on disk, whether or not this run delivered to it: the file of + * a disabled or undetected tool, or one written before the team turned + * delivery off, still holds what a pull wrote. + */ +async function protectResolvedMcpConfigs( + teamConfig: TeamaiConfig, + localConfig: LocalConfig, + exclusions: Map, + written: Set, + before: ManagedMcpManifest | undefined, +): Promise { + const { projectRoot } = localConfig; + if (localConfig.scope !== 'project' || !projectRoot || localConfig.repo.kind === 'http') return; + try { + await protectProjectMcpConfigs(teamConfig, localConfig, projectRoot, exclusions, written, before); + } catch (e) { + log.warn( + `Could not check this project's MCP configs for resolved values to keep out of git: ${e instanceof Error ? e.message : String(e)}. ` + + 'Run `teamai doctor` to see whether git would commit one.', + ); + } +} + +async function protectProjectMcpConfigs( + teamConfig: TeamaiConfig, + localConfig: LocalConfig, + projectRoot: string, + exclusions: Map, + written: Set, + before: ManagedMcpManifest | undefined, +): Promise { + const resolution = await resolveEntriesFor(mcpEntryReader, localConfig); + const teamDefs = resolution.kind === 'failed' ? null : resolution.entries.map((entry) => teamMcpToDef(entry.entry)); + const { manifestPath, manifest } = await loadProjectMcpManifest(getDataHome(localConfig), projectRoot, { dryRun: true }); + const vars = await buildVarTable(localConfig); + const ctx = once(() => buildDesiredMcpContext(teamConfig, localConfig)); + const mapped = await resolveMcpTargets(teamConfig, localConfig, { includeUndetected: true }); + const unmapped = await unmappedMcpDefaults(mapped); + // Tried before its write this run, and reported there. + const targets = mapped.filter((target) => !unmapped.has(target) && exclusions.get(target.file)?.kind !== 'failed'); + const { files: ledger, earlierMappingsRead } = await readResolvedMcpFiles(localConfig); + // No managed-mcp.json when this pull began, or a record of a tool mapping the file still marked unnoted: + // a server no record claims may be one teamai wrote. Noted after the settle, as a rebuild of a lost record + // notes the servers it did not write. + const lost = Object.keys(before ?? manifest).length === 0; + // So, too, a tool the team maps there whose record alone is missing (lost, or never written): an installed + // one, or one uninstalled since that left the file behind, when no installed tool maps that file (CodeBuddy + // never installed beside Claude's .mcp.json would otherwise hold every member's own servers there). + const unnoted = (file: string): boolean => lost || targets.some((t) => t.file === file + && ((unrecordedMcpTool(t, targets, ledger[t.file]?.tools) && (before ?? manifest)[managedMcpManifestKey(t.tool, true)] === undefined) + || [before, manifest].some((m) => m?.[managedMcpManifestKey(t.tool, true)]?.some((record) => record.unnoted)))); + const unclaimed = new Map(); + const holds = async (target: McpTarget, owned: ManagedMcpRecord[]): Promise => { + // One file two tools map under one key: what either's record claims. A tool that reads another key of the + // file (OpenCode's `mcp` beside `mcpServers`) proves nothing of this one's. + const claimed = targets.filter((t) => t.file === target.file && sameServerKey(t.format, target.format)) + .flatMap((t) => manifest[managedMcpManifestKey(t.tool, true)] ?? []).map((record) => record.name); + const names = unnoted(target.file) ? await unclaimedMcpServers(target, claimed) : []; + // Tools of different formats sharing the file each find their own: every one is noted. + if (names.length > 0) unclaimed.set(target.file, [...new Set([...unclaimed.get(target.file) ?? [], ...names])]); + return names.length > 0 + || await resolvedValueEvidence(target, teamDefs, { owned, unverified: ledger[target.file]?.unverified }, vars, ctx) !== null; + }; + const observations = await observeMcpConfigs(localConfig, targets, manifest, holds); + // Once per worktree, what a teamai that kept no record of paths wrote under a mapping the team has since changed. + const earlier = earlierMappingsRead ? [] : await earlierMappedMcpTargets(localConfig, mapped).catch((e: unknown) => { + log.debug(`Did not read the MCP configs earlier toolPaths mappings reach: ${e instanceof Error ? e.message : String(e)}`); + return null; + }); + // And on every pull, a built-in location no mapping reaches today that no record of this version covers yet. + const fallbacks = await unrecordedUnmappedMcpDefaults(localConfig, unmapped, mapped.filter((target) => !unmapped.has(target))); + // Held through the release, which reads only what was recorded before this run. + const found: string[] = []; + for (const { tracked, mappedBy, ...target } of [...earlier ?? [], ...fallbacks]) { + const state = await mcpFileState([target]); + // No line protects a file git tracks: recorded as tracked, whatever it holds, and judged once git no longer tracks it. + if (tracked) { + observations.push({ file: target.file, tool: target.tool, state, holding: false, owned: [], tracked }); + continue; + } + // In a file other tools map today, their records tell their own servers. + const owned = mappedBy.length === 0 ? undefined + : mappedBy.flatMap((tool) => manifest[managedMcpManifestKey(tool, true)] ?? []).map((record) => record.name); + const holding = await earlierMappedMcpFileEvidence(target, teamDefs, vars, ctx, owned) !== null; + if (holding) found.push(target.file); + observations.push({ file: target.file, tool: target.tool, state, holding, owned: owned ?? [], ...owned ? { remapped: true as const } : {} }); + } + const holding = new Set(observations.filter((o) => o.holding).map((o) => o.file)); + const unproven = new Set(observations.filter((o) => !o.holding).map((o) => o.file)); + // Also a file listed before its write: a concurrent uninstall may have taken its line out since. + // And readable by this user only (#879), written this run or not: a disabled or moved tool's too. + for (const file of holding) { + await excludeFromGit(file); + await tightenMode(file).catch((e: unknown) => log.debug(`Could not make ${file} 0600: ${e instanceof Error ? e.message : String(e)}`)); + } + // A line this run added for a file it then did not write restores the file's state before the run. + // One it wrote holds the value even when no scan finds it (shorter than eight characters). + const addedNow = [...unproven].filter((file) => { + const exclusion = exclusions.get(file); + return !holding.has(file) && !written.has(file) && exclusion?.kind === 'excluded' && exclusion.added; + }); + await releaseMcpGitExcludes(teamConfig, localConfig, projectRoot, addedNow, before, found); + // After the release, which reads the files recorded before this run; also lists one an older teamai wrote. + await settleRecordedMcpConfigs(localConfig, observations, { earlierMappingsRead: !earlierMappingsRead && earlier !== null }); + const noted = await noteUnclaimedMcpServers(localConfig, unclaimed); + // A file that parses with no server left unclaimed has nothing to note. + const parses = (target: McpTarget): boolean => + observations.some((o) => o.file === target.file && o.tool === target.tool && o.state.kind !== 'unparsable'); + await markMcpRecordsNoted(manifestPath, manifest, targets.filter((target) => unnoted(target.file) + && (unclaimed.has(target.file) ? noted.has(target.file) : parses(target)))); +} + +/** + * Note the servers no record claimed in each config a pull that found no + * managed-mcp.json listed for them (#882): once its manifest is back, a stale + * entry teamai wrote looks like the member's own. After the settle, which + * records the file. Returns the files whose servers are noted now. + */ +async function noteUnclaimedMcpServers(localConfig: LocalConfig, unclaimed: Map): Promise> { + if (unclaimed.size === 0) return new Set(); + const found = [...unclaimed].map(([file, names]) => ({ file, names })); + const result = await recordUnverifiedMcpServers(localConfig, found).catch((e: unknown) => e instanceof Error ? e.message : String(e)); + // Read back: a file the settle did not record takes no note. + const { files } = await readResolvedMcpFiles(localConfig); + const noted = new Set(found.filter(({ file, names }) => names.every((name) => files[file]?.unverified?.includes(name))).map((f) => f.file)); + const missed = found.filter((f) => !noted.has(f.file)).map((f) => f.file); + if (missed.length > 0) { + const why = result === 'locked' ? 'another teamai command held managed-mcp-files.json past the wait' + : result === 'written' || result === 'unchanged' ? 'managed-mcp-files.json has no record of the file' : result; + log.debug( + `Did not note the MCP servers teamai found in ${missed.join(', ')} that no managed-mcp.json record claims: ${why}. ` + + 'They keep their .git/info/exclude lines while they hold MCP servers; the next pull tries again.', + ); + } + return noted; +} + +/** + * Take the unnoted mark off the records of `targets`' tools, whose files' + * other servers are noted (#882). A failed write keeps it: the file keeps its + * line while it holds a server, and the next pull notes them again. + */ +async function markMcpRecordsNoted(manifestPath: string, manifest: ManagedMcpManifest, targets: McpTarget[]): Promise { + let changed = false; + for (const { tool } of targets) { + for (const record of manifest[managedMcpManifestKey(tool, true)] ?? []) { + changed ||= record.unnoted === true; + delete record.unnoted; + } + } + if (!changed) return; + await writeJsonAtomic(manifestPath, manifest).catch((e: unknown) => { + log.debug(`Did not update ${manifestPath}: ${e instanceof Error ? e.message : String(e)}. The next pull notes its MCP configs' other servers again.`); + }); +} + +/** + * Take out of teamai's block in `.git/info/exclude` the line of each project + * MCP config proven free of a value teamai resolved (#882), in every worktree + * sharing it: `teamai mcp remove` leaves nothing of teamai's to protect. A + * config not proven clean keeps its line. + */ +export async function releaseCleanMcpGitExcludes(teamConfig: TeamaiConfig, localConfig: LocalConfig): Promise { + const { projectRoot } = localConfig; + if (localConfig.scope !== 'project' || !projectRoot || localConfig.repo.kind === 'http') return; + try { + await releaseMcpGitExcludes(teamConfig, localConfig, projectRoot, []); + const { manifest } = await loadProjectMcpManifest(getDataHome(localConfig), projectRoot, { dryRun: true }); + const mapped = await resolveMcpTargets(teamConfig, localConfig, { includeUndetected: true }); + const unmapped = await unmappedMcpDefaults(mapped); + const targets = mapped.filter((target) => !unmapped.has(target)); + await settleRecordedMcpConfigs(localConfig, await observeMcpConfigs(localConfig, targets, manifest, async () => false)); + } catch (e) { + log.warn( + `Could not check whether this project's MCP configs still need their .git/info/exclude lines: ${e instanceof Error ? e.message : String(e)}. ` + + 'The lines stay; `teamai uninstall` removes them.', + ); + } +} + +/** + * Remove each line of teamai's block whose files are all proven clean or in + * `addedNow`: files this run listed and holds no evidence for, whose line it + * takes back out even when they cannot be proven clean (one that does not parse). + * A line of a file in `kept` stays: this run found it holding by a record it + * has not written yet. + */ +async function releaseMcpGitExcludes( + teamConfig: TeamaiConfig, + localConfig: LocalConfig, + projectRoot: string, + addedNow: string[], + before?: ManagedMcpManifest, + kept: string[] = [], +): Promise { + const dirs = [projectRoot]; + for (const target of await resolveMcpTargets(teamConfig, localConfig, { includeUndetected: true })) dirs.push(path.dirname(target.file)); + for (const file of Object.keys((await readResolvedMcpFiles(localConfig)).files)) dirs.push(path.dirname(file)); + const excludes = await findMcpGitExcludes(dirs); + if (excludes.size === 0) return; + // Keyed as findMcpGitExcludes keys them: by real path (macOS /var). + const exempt = new Set(await Promise.all(addedNow.map(realFilePath))); + const keep = new Set(await Promise.all(kept.map(realFilePath))); + const held = await mcpConfigsNotProvenClean( + teamConfig, + localConfig, + [...excludes.values()].flat(), + { before, otherWorktrees: 'empty' }, + ); + for (const [excludeFile, entries] of excludes) { + const cleanEntries = entries.filter((entry) => entry.files.every((file) => (!held.has(file) || exempt.has(file)) && !keep.has(file))); + const clean = cleanEntries.map((entry) => entry.pattern); + if (clean.length === 0) continue; + const result = await removeMcpGitExclude(excludeFile, clean); + if (result === 'written') { + // A line this run added and took back out is no change the member saw. + const rolledBack = cleanEntries.filter((entry) => entry.files.some((file) => exempt.has(file))).map((entry) => entry.pattern); + const released = clean.filter((pattern) => !rolledBack.includes(pattern)); + if (released.length > 0) log.info(`Removed ${released.join(', ')} from ${excludeFile}: no MCP config there holds a value teamai resolved.`); + if (rolledBack.length > 0) log.debug(`Took ${rolledBack.join(', ')} back out of ${excludeFile}: this run wrote no resolved value there.`); + } + // Left as it is: the next pull tries again. + if (result === 'locked') log.debug(`Kept ${clean.join(', ')} in ${excludeFile}: another teamai command held it past the wait.`); + } +} + +async function reconcileTargets( + teamConfig: TeamaiConfig, + localConfig: LocalConfig, + options: McpReconcileOptions, + exclusions: Map, + written: Set, + recorded: McpTarget[], ): Promise { const changes: McpChange[] = []; let wrote = false; @@ -539,26 +1450,29 @@ export async function reconcileMcpForConfig( const targets = await resolveMcpTargets(teamConfig, localConfig); if (targets.length === 0) return { changes, wrote }; - const dataHome = getDataHome(localConfig); - const projectScope = localConfig.scope === 'project'; - // Project scope uses a PER-WORKTREE manifest under the partition (migrating this - // worktree's records out of any legacy shared file on first read); user scope - // keeps the single global file. Either way this reconcile owns exactly one file. - let manifestPath: string; - let manifest: ManagedMcpManifest; - if (projectScope && localConfig.projectRoot) { - ({ manifestPath, manifest } = await loadProjectMcpManifest(dataHome, localConfig.projectRoot, { dryRun: options.dryRun })); - } else { - manifestPath = managedMcpManifestPath(dataHome); - manifest = await readManifest(manifestPath); - } + const { manifestPath, manifest } = await loadMcpManifest(localConfig, options.dryRun); // An empty desired set still has to run: it is how servers dropped from // mcp.yaml get cleaned out of the tools we previously injected them into. const nothingOwned = Object.values(manifest).every((r) => r.length === 0); if (teamDefs.length === 0 && nothingOwned) return { changes, wrote }; + // The files an earlier pull recorded, and each record this run rebuilds after it was lost (#882). + const ledger = localConfig.scope === 'project' && !options.dryRun ? (await readResolvedMcpFiles(localConfig)).files : {}; + const listed = new Set(Object.keys(ledger)); + const rebuilt: Array<{ target: McpTarget; records: ManagedMcpRecord[] }> = []; + // The tools with no record in managed-mcp.json when this pull began (#882): theirs are marked below. + const unrecorded = new Set(localConfig.scope === 'project' && !options.dryRun + ? targets.filter((t) => manifest[managedMcpManifestKey(t.tool, true)] === undefined).map((t) => t.tool) : []); const desiredContext = await buildDesiredMcpContext(teamConfig, localConfig, options); + // A failed declaration is not "no secrets": read as none, every server whose + // secret the member left in their shell would be removed. Keep what is + // installed rather than guess which variables are secrets. `removeAll` + // (mcp remove, uninstall) still removes everything. + if (!removeAll && desiredContext.secrets.kind === 'failed') { + reportEntryResolution(desiredContext.secrets); + return { changes, wrote, unresolved: true }; + } for (const target of targets) { // Same enabledAgents / disabledAgents gate as the other resource syncs. The @@ -571,48 +1485,167 @@ export async function reconcileMcpForConfig( const nextRecords: ManagedMcpRecord[] = []; // Which of this team's servers apply to this tool, and in what rendered form. - const { desired, skipped } = desiredMcpForTarget(target, teamDefs, desiredContext); + const { desired, skipped, kept } = desiredMcpForTarget(target, teamDefs, desiredContext); changes.push(...skipped); - - if (target.format === 'codex') { - wrote = await applyCodex(target, desired, ownedNames, nextRecords, changes, options) || wrote; - } else { - wrote = await applyJson(target, desired, owned, ownedNames, nextRecords, changes, options) || wrote; + // Their old records, so a manifest this run writes still claims them. + const keep = new Map(owned.filter((r) => kept.has(r.name)).map((r) => [r.name, r])); + + // A resolved value lands only in a file git leaves out of a commit (#882). + // Otherwise the file stays as it was, its manifest entry with it. + if (carriesResolvedValue(target, teamDefs, desired.keys())) { + const exclusion = exclusions.get(target.file) ?? await ensureExcludedFromGit(target.file, { dryRun: options.dryRun }); + exclusions.set(target.file, exclusion); + if (exclusion.kind === 'failed') { + const reason = `${target.file} is not kept out of git: ${exclusion.reason}`; + for (const server of desired.keys()) changes.push({ tool: target.tool, server, action: 'skipped', reason }); + log.warn( + `Did not write ${target.tool}'s MCP servers to ${target.file}: it would hold resolved values, and teamai could not ` + + `keep it out of git first: ${exclusion.reason}. The file is left as it was. ${exclusion.fix}`, + ); + continue; + } + // Recorded before the write, so a later change to toolPaths still finds the file. + if (!options.dryRun) { + await recordResolvedMcpFile(localConfig, target); + if (!ledger[target.file]?.tools.includes(target.tool)) recorded.push(target); + } } - if (nextRecords.length > 0) manifest[manifestKey] = nextRecords; + const wroteTarget = target.format === 'codex' + ? await applyCodex(target, desired, keep, ownedNames, nextRecords, changes, options) + : await applyJson(target, desired, keep, owned, ownedNames, nextRecords, changes, options); + if (wroteTarget) written.add(target.file); + wrote = wroteTarget || wrote; + // Not read: its record stays as it was, or absent. An empty one would say teamai owns nothing there (#882). + if (wroteTarget === null) continue; + + // The unnoted mark stays until a note of what else is in the file lands. + const marked = manifest[manifestKey]?.some((record) => record.unnoted) ?? false; + // Whether each entry holds a resolved value: once its definition stops + // needing one, what this pull wrote still does (#882). + if (target.projectScope) { + for (const record of nextRecords) { + record.resolved ??= carriesResolvedValue(target, teamDefs, [record.name]); + if (marked) record.unnoted = true; + else delete record.unnoted; + } + const at = recorded.indexOf(target); + if (at >= 0 && nextRecords.some((record) => record.resolved === true)) recorded.splice(at, 1); + } + // Rebuilt this run, or by one that could not note what else was in the file. + const unnoted = manifest[manifestKey] === undefined || manifest[manifestKey].some((record) => record.unnoted); + if (listed.has(target.file) && unnoted && nextRecords.length > 0) rebuilt.push({ target, records: nextRecords }); + // An emptied project record stays: it says teamai owns nothing left in that + // file, which a lost record cannot, and so lets its exclude line go (#882). + // Not while the file's other servers are unnoted: it would say the same. + if (nextRecords.length > 0 || (target.projectScope && manifest[manifestKey] !== undefined && !marked)) manifest[manifestKey] = nextRecords; else delete manifest[manifestKey]; } - if (!options.dryRun && wrote) { + // A record of a tool that had none when this pull began, of a file holding a server no record claims, is + // unnoted until protectProjectMcpConfigs notes that server, after its settle records the file. + for (const target of targets.filter((t) => unrecorded.has(t.tool))) { + const records = manifest[managedMcpManifestKey(target.tool, true)] ?? []; + const claimed = targets.filter((t) => t.file === target.file) + .flatMap((t) => manifest[managedMcpManifestKey(t.tool, true)] ?? []).map((record) => record.name); + if (records.length > 0 && (await unclaimedMcpServers(target, claimed)).length > 0) { + for (const record of records) record.unnoted = true; + } + } + if (!options.dryRun && (wrote || rebuilt.length > 0)) { + // Before the manifest: once it is written, only a record marked unnoted says it was rebuilt. + const failed = await noteUnverifiedMcpServers(localConfig, rebuilt); + for (const { records } of rebuilt) { + for (const record of records) { + if (failed.includes(records)) record.unnoted = true; + else delete record.unnoted; + } + } await writeJsonAtomic(manifestPath, manifest); } return { changes, wrote }; } +/** + * Note, for each file whose lost record this run rebuilt, the servers in it + * the new record does not claim: a stale entry teamai wrote looks like the + * member's own once its value is no longer set (#882). Returns the records of + * each file it could not note them for: the manifest write marks them + * unnoted, so the file keeps its line and the next pull tries again, and + * still owns what this one wrote. + */ +async function noteUnverifiedMcpServers( + localConfig: LocalConfig, + rebuilt: Array<{ target: McpTarget; records: ManagedMcpRecord[] }>, +): Promise { + const found: Array<{ file: string; names: string[]; records: ManagedMcpRecord[] }> = []; + for (const { target, records } of rebuilt) { + const installed = await installedMcpEntries(target); + const names = [...installed?.keys() ?? []].filter((name) => !records.some((record) => record.name === name)); + if (names.length > 0) found.push({ file: target.file, names, records }); + } + if (found.length === 0) return []; + const result = await recordUnverifiedMcpServers(localConfig, found).catch((e: unknown) => e instanceof Error ? e.message : String(e)); + if (result === 'written' || result === 'unchanged') return []; + log.debug( + `Did not note the MCP servers teamai found in ${found.map((f) => f.file).join(', ')} while rebuilding its lost record of them: ` + + `${result === 'locked' ? 'another teamai command held managed-mcp-files.json past the wait' : result}. ` + + 'They keep their .git/info/exclude lines while they hold MCP servers; the next pull tries again.', + ); + return found.map((f) => f.records); +} + +/** + * `trackResolvedMcpFiles` for a file about to get a resolved value. A failure + * does not stop the write: the exclusion protects the file, and the next pull + * records it. + */ +async function recordResolvedMcpFile(localConfig: LocalConfig, target: McpTarget): Promise { + const result = await trackResolvedMcpFiles(localConfig, [target]).catch((e: unknown) => e instanceof Error ? e.message : String(e)); + if (result !== 'written' && result !== 'unchanged') { + log.debug(`Did not record ${target.file} in managed-mcp-files.json: ${result === 'locked' ? 'another teamai command held it past the wait' : result}. The next pull records it.`); + } +} + +/** + * `untrackResolvedMcpFiles`. A failure leaves the record, and the file its + * line while it holds a server once no mapping reaches it. + */ +async function forgetUnwrittenMcpConfigs(localConfig: LocalConfig, targets: McpTarget[]): Promise { + if (targets.length === 0) return; + const result = await untrackResolvedMcpFiles(localConfig, targets).catch((e: unknown) => e instanceof Error ? e.message : String(e)); + if (result !== 'written' && result !== 'unchanged') { + log.debug(`Did not take ${targets.map((t) => t.file).join(', ')} back out of managed-mcp-files.json: ${result === 'locked' ? 'another teamai command held it past the wait' : result}.`); + } +} + // ─── Appliers ──────────────────────────────────────────────── +/** Whether it wrote `target`'s file; null when the file does not parse, and so was not read. */ async function applyJson( target: McpTarget, - desired: Map, + desired: Map, + keep: Map, owned: ManagedMcpRecord[], ownedNames: Set, nextRecords: ManagedMcpRecord[], changes: McpChange[], options: McpReconcileOptions, -): Promise { +): Promise { const serverKey = MCP_SERVER_KEY[target.format as Exclude]; const allowBare = target.format === 'copilot' && target.projectScope; const doc = await readJsonDoc(target.file, serverKey, allowBare); if (!doc) { log.warn(`Could not parse ${target.file} — skipping MCP injection for ${target.tool}`); - return false; + return null; } const ownedHash = new Map(owned.map((r) => [r.name, r.hash])); let dirty = false; + // A kept entry holds the value an earlier pull resolved (desiredMcpForTarget). + let holdsResolvedValue = false; - for (const [name, { entry, hash }] of desired) { + for (const [name, { entry, hash, resolvedValue }] of desired) { const existing = doc.servers[name]; if (existing !== undefined && !ownedNames.has(name) && !options.force) { changes.push({ @@ -624,6 +1657,7 @@ async function applyJson( continue; } nextRecords.push({ name, hash }); + holdsResolvedValue ||= resolvedValue; if (existing !== undefined && ownedHash.get(name) === hash) continue; doc.servers[name] = entry; dirty = true; @@ -632,6 +1666,12 @@ async function applyJson( for (const name of ownedNames) { if (desired.has(name)) continue; + const kept = keep.get(name); + if (kept && doc.servers[name] !== undefined) { + nextRecords.push(kept); + holdsResolvedValue = true; + continue; + } if (doc.servers[name] !== undefined) { delete doc.servers[name]; dirty = true; @@ -639,19 +1679,25 @@ async function applyJson( changes.push({ tool: target.tool, server: name, action: 'removed' }); } - if (!dirty || options.dryRun) return false; + if (options.dryRun) return false; + if (!dirty) { + if (holdsResolvedValue) await tightenMode(target.file); + return false; + } // Key-level surgery: every unrelated top-level key is carried over untouched. // Some tools (OpenCode) key the server map under `mcp`, not `mcpServers`; // writing the wrong key would strip the servers and, worse, leave a phantom // empty `mcpServers` in a file the tool never reads under that name. - await writeJsonDoc(target.file, serverKey, doc); + // A file that holds a resolved value is the member's alone, an existing one tightened. + await writeJsonDoc(target.file, serverKey, doc, holdsResolvedValue ? { mode: 0o600 } : undefined); return true; } async function applyCodex( target: McpTarget, - desired: Map, + desired: Map, + keep: Map, ownedNames: Set, nextRecords: ManagedMcpRecord[], changes: McpChange[], @@ -660,8 +1706,9 @@ async function applyCodex( let source = (await readFileSafe(target.file)) ?? ''; const present = new Set(codexServerNames(source)); let dirty = false; + let holdsResolvedValue = false; - for (const [name, { hash, block }] of desired) { + for (const [name, { hash, block, resolvedValue }] of desired) { if (present.has(name) && !ownedNames.has(name) && !options.force) { changes.push({ tool: target.tool, @@ -672,6 +1719,7 @@ async function applyCodex( continue; } nextRecords.push({ name, hash }); + holdsResolvedValue ||= resolvedValue; const next = spliceCodexBlock(source, name, block!); if (next === source) continue; source = next; @@ -681,6 +1729,12 @@ async function applyCodex( for (const name of ownedNames) { if (desired.has(name)) continue; + const kept = keep.get(name); + if (kept && present.has(name)) { + nextRecords.push(kept); + holdsResolvedValue = true; + continue; + } const next = spliceCodexBlock(source, name, null); if (next !== source) { source = next; @@ -689,21 +1743,41 @@ async function applyCodex( changes.push({ tool: target.tool, server: name, action: 'removed' }); } - if (!dirty || options.dryRun) return false; + if (options.dryRun) return false; + if (!dirty) { + if (holdsResolvedValue) await tightenMode(target.file); + return false; + } - await fse.ensureDir(path.dirname(target.file)); - const tmp = `${target.file}.${process.pid}.tmp`; - await fse.writeFile(tmp, source, 'utf-8'); - await fse.chmod(tmp, 0o600); - await fse.rename(tmp, target.file); + await writeCodexAtomic(target.file, source); return true; } +/** + * Make an unchanged config readable by this user only, without rewriting it: + * an entry a CLI before #879 wrote holds its resolved value in a file that may + * still be 0644. + */ +async function tightenMode(file: string): Promise { + const { mode } = await fs.promises.stat(file); + if ((mode & 0o077) !== 0) await fs.promises.chmod(file, 0o600); +} + +/** + * Write a Codex config.toml atomically, readable by this user only: it may + * hold resolved values. A symlink at `file` is replaced, as `writeJsonAtomic` + * does for the JSON configs: git protection judges `file`, so a value must + * never land in the file it links to (#882). + */ export async function writeCodexAtomic(file: string, content: string): Promise { - await fse.ensureDir(path.dirname(file)); - const suffix = crypto.randomBytes(6).toString('hex'); - const tmp = `${file}.${process.pid}.${suffix}.tmp`; - await fse.writeFile(tmp, content, 'utf-8'); - await fse.chmod(tmp, 0o600); - await fse.rename(tmp, file); + await fs.promises.mkdir(path.dirname(file), { recursive: true }); + const tmp = `${file}.${process.pid}.${crypto.randomBytes(6).toString('hex')}.tmp`; + try { + await fs.promises.writeFile(tmp, content, { encoding: 'utf-8', mode: 0o600, flag: 'wx' }); + await fs.promises.chmod(tmp, 0o600); + await fs.promises.rename(tmp, file); + } catch (error) { + await fs.promises.rm(tmp, { force: true }); + throw error; + } } diff --git a/src/mcp-resolved-files.ts b/src/mcp-resolved-files.ts new file mode 100644 index 000000000..c34311880 --- /dev/null +++ b/src/mcp-resolved-files.ts @@ -0,0 +1,235 @@ +import path from 'node:path'; +import { z } from 'zod'; +import { getDataHome, managedMcpManifestPath, type LocalConfig } from './types.js'; +import { readFileSafe } from './utils/fs.js'; +import { updateFileLocked, type ExcludeUpdate } from './mcp-git-exclude.js'; + +// ─── Project MCP configs teamai wrote a resolved value to ──── +// +// managed-mcp.json records server names per tool, not paths, so a file an +// earlier pull wrote under a toolPaths mapping the team has since changed is +// no longer anyone's target, and a record rebuilt after it was lost cannot +// tell teamai's stale entries from the member's own (#882). This file, next +// to the worktree's managed-mcp.json, remembers both: each project MCP config +// a pull wrote a resolved value to, by absolute path, with the tools it wrote +// it for, and the servers it found there when it rebuilt a lost record. It +// also says whether a pull has read the files earlier revisions of the team's +// teamai.yaml mapped, which a teamai from before this file wrote to without +// recording them, and remembers one of those git tracked, which no line can +// protect until the member stops git tracking it. Nothing depends on it to +// keep a line: missing or unreadable, it reads as empty and the rules without +// it apply. + +export interface ResolvedMcpFile { + /** The tools whose MCP format the file was written in. */ + tools: string[]; + /** Servers in the file when teamai rebuilt its lost record: teamai may have written them. */ + unverified?: string[]; + /** Git tracked it when a pull found it under an earlier teamai.yaml mapping: judged once git no longer does. */ + tracked?: true; +} + +export interface ResolvedMcpFiles { + version: 1; + /** Keyed by the file's absolute path. */ + files: Record; + /** A pull has read the project MCP configs earlier revisions of teamai.yaml mapped. */ + earlierMappingsRead?: true; +} + +/** What a command found in a project MCP config, for `settleResolvedMcpFiles`. */ +export interface McpFileObservation { + file: string; + tool: string; + state: { kind: 'missing' } | { kind: 'unparsable' } | { kind: 'parsed'; servers: readonly string[] }; + /** It may hold a value teamai resolved (resolvedValueEvidence). */ + holding: boolean; + /** The server names managed-mcp.json records for it now. */ + owned: string[]; + /** Whether git tracks it, for a file recorded (or to record) as one it tracked: kept, whatever it holds, while git does. */ + tracked?: boolean; + /** `tool` does not map the file today, another tool does: `holding` says whether it holds what teamai may have written for `tool`. */ + remapped?: true; +} + +// Fields a later teamai adds are carried through a rewrite. +const FileSchema = z.object({ tools: z.array(z.string()), unverified: z.array(z.string()).optional() }).passthrough(); +const SidecarSchema = z.object({ version: z.literal(1), files: z.record(z.unknown()) }).passthrough(); + +type Sidecar = z.infer & { files: Record> }; + +/** `/workspaces//managed-mcp-files.json`, or null outside project scope. */ +export function resolvedMcpFilesPath(cfg: LocalConfig): string | null { + if (cfg.scope !== 'project' || !cfg.projectRoot) return null; + return path.join(path.dirname(managedMcpManifestPath(getDataHome(cfg), cfg.projectRoot)), 'managed-mcp-files.json'); +} + +/** Missing, not JSON, of another shape or version: no files. An entry of the wrong shape, or under a relative path, is left out. */ +function parse(content: string): Sidecar { + let data: unknown; + try { + data = JSON.parse(content); + } catch { + data = null; + } + const parsed = SidecarSchema.safeParse(data); + if (!parsed.success) return { version: 1, files: {} }; + const files: Sidecar['files'] = {}; + for (const [file, value] of Object.entries(parsed.data.files)) { + const entry = FileSchema.safeParse(value); + if (entry.success && path.isAbsolute(file)) files[file] = entry.data; + } + return { ...parsed.data, files }; +} + +/** The files this worktree's pulls wrote a resolved value to. Never throws. */ +export async function readResolvedMcpFiles(cfg: LocalConfig): Promise { + const file = resolvedMcpFilesPath(cfg); + const content = file === null ? null : await readFileSafe(file).catch(() => null); + if (content === null) return { version: 1, files: {} }; + const sidecar = parse(content); + return { version: 1, files: sidecar.files, ...sidecar.earlierMappingsRead === true ? { earlierMappingsRead: true } : {} }; +} + +/** + * Apply `edit` to the record under its lock (re-read, atomic write, 0600). + * `edit` returns false to leave it as it is. One that does not parse is + * rewritten from empty. + */ +export function updateResolvedMcpFiles(cfg: LocalConfig, edit: (files: Record) => boolean): Promise { + return updateSidecar(cfg, (sidecar) => edit(sidecar.files)); +} + +async function updateSidecar(cfg: LocalConfig, edit: (sidecar: Sidecar) => boolean): Promise { + const file = resolvedMcpFilesPath(cfg); + if (file === null) return 'unchanged'; + return updateFileLocked(file, (content) => { + const sidecar = parse(content); + return edit(sidecar) ? `${JSON.stringify(sidecar, null, 2)}\n` : null; + }, { mode: 0o600 }); +} + +/** Record each file as written with a resolved value, for its tool. */ +export function trackResolvedMcpFiles(cfg: LocalConfig, targets: Array<{ tool: string; file: string }>): Promise { + return updateResolvedMcpFiles(cfg, (files) => { + let changed = false; + for (const { tool, file } of targets) { + const entry = files[file]; + if (entry?.tools.includes(tool)) continue; + files[file] = entry ? { ...entry, tools: [...entry.tools, tool] } : { tools: [tool] }; + changed = true; + } + return changed; + }); +} + +/** Take back what `trackResolvedMcpFiles` recorded for a file it was not written to after all. */ +export function untrackResolvedMcpFiles(cfg: LocalConfig, targets: Array<{ tool: string; file: string }>): Promise { + return updateResolvedMcpFiles(cfg, (files) => { + let changed = false; + for (const { tool, file } of targets) { + const entry = files[file]; + if (!entry?.tools.includes(tool)) continue; + const tools = entry.tools.filter((t) => t !== tool); + if (tools.length > 0) files[file] = { ...entry, tools }; + else delete files[file]; + changed = true; + } + return changed; + }); +} + +/** + * Note `names`, servers found in a file whose lost record teamai rebuilt, as + * possibly teamai's: only for a file already recorded as holding a resolved value. + */ +export function recordUnverifiedMcpServers(cfg: LocalConfig, found: Array<{ file: string; names: string[] }>): Promise { + return updateResolvedMcpFiles(cfg, (files) => { + let changed = false; + for (const { file, names } of found) { + const entry = files[file]; + const added = names.filter((name) => !entry?.unverified?.includes(name)); + if (!entry || added.length === 0) continue; + entry.unverified = [...entry.unverified ?? [], ...added]; + changed = true; + } + return changed; + }); +} + +/** + * Bring the record up to date with what the files hold: forget a file that is + * gone or holds no server, record one holding a resolved value it did not + * list (written by an older teamai), keep a tool on the record of a file + * another tool now maps while the file holds what teamai may have written for + * it (adding it for one an older teamai wrote), and take it off after, and + * drop a noted server that left its file or that teamai owns again. A file + * that does not parse stays as it is, and so does one recorded as tracked + * until an observation says git no longer tracks it: a checkout brings back + * what git holds. A tool found in a file git tracks is added, marked tracked. + * `earlierMappingsRead`: the observations cover the files earlier revisions + * of teamai.yaml mapped, which later pulls need not read again. + */ +export function settleResolvedMcpFiles( + cfg: LocalConfig, + observations: McpFileObservation[], + options: { earlierMappingsRead?: boolean } = {}, +): Promise { + return updateSidecar(cfg, (sidecar) => { + const { files } = sidecar; + let changed = options.earlierMappingsRead === true && sidecar.earlierMappingsRead !== true; + if (changed) sidecar.earlierMappingsRead = true; + // Tools of different formats read different keys of one file: it is empty only when every one of them + // finds it so, and a noted server stays while any of them finds it and does not own it. + const ofFile = (file: string): McpFileObservation[] => observations.filter((o) => o.file === file); + const empty = (file: string): boolean => ofFile(file).every(({ state: s }) => s.kind === 'missing' || (s.kind === 'parsed' && s.servers.length === 0)); + const unparsable = (file: string): boolean => ofFile(file).some(({ state: s }) => s.kind === 'unparsable'); + const stillNoted = (file: string, name: string): boolean => + ofFile(file).some(({ state: s, owned: o }) => s.kind === 'parsed' && s.servers.includes(name) && !o.includes(name)); + for (const { file, tool, holding, tracked, remapped } of observations) { + const entry = files[file]; + if (tracked === true) { + if (entry?.tools.includes(tool)) continue; + files[file] = entry ? { ...entry, tools: [...entry.tools, tool], tracked: true } : { tools: [tool], tracked: true }; + changed = true; + continue; + } + if (empty(file)) { + const forget = entry !== undefined && (entry.tracked !== true || tracked === false); + if (forget) delete files[file]; + changed ||= forget; + continue; + } + if (entry?.tracked === true && tracked === false) { + delete entry.tracked; + changed = true; + } + if (remapped && holding) { + if (entry?.tools.includes(tool)) continue; + files[file] = entry ? { ...entry, tools: [...entry.tools, tool] } : { tools: [tool] }; + changed = true; + continue; + } + if (remapped) { + if (!entry?.tools.includes(tool)) continue; + const tools = entry.tools.filter((t) => t !== tool); + if (tools.length > 0 || entry.unverified) files[file] = { ...entry, tools }; + else delete files[file]; + changed = true; + continue; + } + if (!entry) { + if (holding) files[file] = { tools: [tool] }; + changed ||= holding; + continue; + } + if (unparsable(file) || !entry.unverified) continue; + const unverified = entry.unverified.filter((name) => stillNoted(file, name)); + if (unverified.length === entry.unverified.length) continue; + if (unverified.length > 0) entry.unverified = unverified; + else delete entry.unverified; + changed = true; + } + return changed; + }); +} diff --git a/src/member-env.ts b/src/member-env.ts new file mode 100644 index 000000000..930a29488 --- /dev/null +++ b/src/member-env.ts @@ -0,0 +1,94 @@ +/** + * The member's own environment (#879 Conflict 10). + * + * The profile loads whichever teamai `env.sh` a scope wrote, so the process + * environment also carries values teamai exported: another team's, this + * scope's from before a team edit, or this scope's repo value for a key the + * team now declares as a secret. Those are the team's values, not the + * member's, and a secret must not fall back to them. + * + * Each env.sh exports a marker of what it exported, which reaches a shell that + * sourced one no scan finds (`/.teamai/env.sh` of a non-git project), and + * keeps a record of what it has exported (env-sh-exports.ts), so a shell + * opened before any scope's pull keeps being discounted after it. + * + * Not covered: a value a shell got from an env.sh no scan finds, written by a + * CLI without the marker; a value exported before the last 20 changes of its + * key; and one an env.sh written by a CLI without the record dropped. + */ +import fs from 'node:fs'; +import path from 'node:path'; +import { exportDigest, markedAsExported, readEnvShExports, type EnvShExports } from './env-sh-exports.js'; +import { parseEnvFile } from './resources/env.js'; +import { envName, envValue } from './resources/env-key.js'; +import { getDataHome, getTeamaiHomeDir, type LocalConfig } from './types.js'; +import { readFileSafe } from './utils/fs.js'; + +/** A key's value in the member's own environment, or undefined. */ +export type MemberEnvironment = (key: string) => string | undefined; + +/** Every teamai env.sh on this machine that a shell may have loaded, with the one in each of `dataHomes`. */ +async function teamaiEnvShPaths(dataHomes: readonly string[]): Promise { + const home = getTeamaiHomeDir(); + const projects = path.join(home, 'projects'); + let partitions: string[] = []; + try { + partitions = (await fs.promises.readdir(projects)).map((name) => path.join(projects, name, 'env.sh')); + } catch { + // No project partitions on this machine. + } + return [...new Set([path.join(home, 'env.sh'), ...dataHomes.map((dataHome) => path.join(dataHome, 'env.sh')), ...partitions])]; +} + +/** + * For key K, `env[K]` is the member's unless it is empty, a marker in `env` + * says a teamai env.sh exported it for K, it equals what a teamai env.sh + * exports for K or has exported for K since it recorded its exports + * (env-sh-exports.ts), or K is a declared secret and it equals this scope's + * env.yaml value for K. + */ +export async function memberEnvironment( + localConfig: LocalConfig, + scope: { secretKeys: ReadonlySet; envYaml: ReadonlyMap }, + env: NodeJS.ProcessEnv = process.env, +): Promise { + return memberEnvironmentAt([getDataHome(localConfig)], scope, env); +} + +/** + * The member's own environment where the config that governs the directory + * cannot be read: no scope declares anything, and `dataHomes` are the + * directories of the configs that could not be read, whose env.sh a shell may + * have loaded. + */ +export function memberEnvironmentWithoutScope(dataHomes: readonly string[], env: NodeJS.ProcessEnv = process.env): Promise { + return memberEnvironmentAt(dataHomes, { secretKeys: new Set(), envYaml: new Map() }, env); +} + +async function memberEnvironmentAt( + dataHomes: readonly string[], + scope: { secretKeys: ReadonlySet; envYaml: ReadonlyMap }, + env: NodeJS.ProcessEnv, +): Promise { + const exported: ReadonlyMap[] = []; + const recorded: EnvShExports[] = []; + for (const envSh of await teamaiEnvShPaths(dataHomes)) { + const content = await readFileSafe(envSh); + if (content !== null) exported.push(parseEnvFile(content)); + recorded.push(await readEnvShExports(envSh)); + } + const marked = markedAsExported(env); + return (key) => { + const value = envValue(env, key); + if (value === undefined || value === '') return undefined; + if (marked(key, value)) return undefined; + // On Windows another scope's `token` is this key's `TOKEN`: compare names as the platform does. + const name = envName(key); + const sameName = ([other]: readonly [string, unknown]): boolean => envName(other) === name; + if (exported.some((exports) => [...exports].some((entry) => sameName(entry) && entry[1] === value))) return undefined; + const digest = exportDigest(key, value); + if (recorded.some((exports) => [...exports].some((entry) => sameName(entry) && entry[1].has(digest)))) return undefined; + if (scope.secretKeys.has(key) && [...scope.envYaml].some((entry) => sameName(entry) && entry[1] === value)) return undefined; + return value; + }; +} diff --git a/src/models-cmd.ts b/src/models-cmd.ts index d2358504b..65103ffbc 100644 --- a/src/models-cmd.ts +++ b/src/models-cmd.ts @@ -3,7 +3,7 @@ import { autoDetectInit } from './config.js'; import { describeEntryFailure, describeOrigin, reportEntryResolution, resolveEntriesFor } from './namespaced-entries.js'; import { pathExists } from './utils/fs.js'; import { log } from './utils/logger.js'; -import { askConfirmation, askQuestion, askSecret, isInteractive } from './utils/prompt.js'; +import { askConfirmation, askQuestion, askSecret, isInteractive, readStdin } from './utils/prompt.js'; import type { LocalConfig } from './types.js'; import { API_KEY_PLACEHOLDER, @@ -217,9 +217,7 @@ function parseProtocols(value: string | undefined): ModelProtocol[] { async function readSecretStdin(): Promise { if (process.stdin.isTTY) throw new Error('--api-key-stdin expects piped stdin'); - let value = ''; - for await (const chunk of process.stdin) value += String(chunk); - value = value.replace(/[\r\n]+$/, ''); + const value = await readStdin(); if (!value) throw new Error('No API key was provided on stdin'); return value; } diff --git a/src/namespaced-entries.ts b/src/namespaced-entries.ts index cd1347a2a..3bb38632b 100644 --- a/src/namespaced-entries.ts +++ b/src/namespaced-entries.ts @@ -4,6 +4,9 @@ * /.yaml root, shared * //.yaml read only where is active in resources. * + * A reader may declare its own directory, file and activation key instead + * (`EntryLayout`), for a second file under a type's directory. + * * Each file is a list of named entries. An active namespace entry replaces the * root entry of the same name, whole; the rule itself is `namespace-resolver`. * This module adds what is particular to list files: reading them, the failure @@ -44,14 +47,49 @@ const INSTALLED: Record = { models: 'agent model settings', }; +/** + * Where a reader's files are, which namespaces are active for it, and how its + * messages name what it reads. A reader that declares none has its type's. + */ +export interface EntryLayout { + /** `/` at the root, `//` in a namespace. */ + readonly dir: string; + readonly file: string; + /** The `resources.` that lists the active namespaces. */ + readonly activation: EntryType; + /** What messages call the whole set, as `env` or `secrets`. */ + readonly label: string; + /** What one entry is called, for messages. */ + readonly noun: string; + /** What a failure leaves unchanged, as a sentence, for messages. */ + readonly kept: string; +} + +/** A type's own layout: `/.yaml`, active through `resources.`. */ +export function entryLayout(type: EntryType): EntryLayout { + return { + dir: type, + file: ENTRY_FILE[type], + activation: type, + label: type, + noun: ENTRY_NOUN[type], + kept: `${type} was not applied this run, so your ${INSTALLED[type]} are unchanged.`, + }; +} + +function asLayout(where: EntryType | EntryLayout): EntryLayout { + return typeof where === 'string' ? entryLayout(where) : where; +} + /** Repo-relative (`/`-separated) path of a type's file in the root (`null`) or a namespace. */ -export function entryFilePath(type: EntryType, namespace: string | null): string { - return namespace === null ? `${type}/${ENTRY_FILE[type]}` : `${type}/${namespace}/${ENTRY_FILE[type]}`; +export function entryFilePath(where: EntryType | EntryLayout, namespace: string | null): string { + const { dir, file } = asLayout(where); + return namespace === null ? `${dir}/${file}` : `${dir}/${namespace}/${file}`; } /** `entryFilePath` under a checkout. */ -export function entryFileAbsolutePath(repoPath: string, type: EntryType, namespace: string | null): string { - return path.join(repoPath, ...entryFilePath(type, namespace).split('/')); +export function entryFileAbsolutePath(repoPath: string, where: EntryType | EntryLayout, namespace: string | null): string { + return path.join(repoPath, ...entryFilePath(where, namespace).split('/')); } /** One of a type's files that exists in a checkout. */ @@ -66,12 +104,12 @@ export interface EntryFile { * Every file of `type` in a checkout, active here or not: the root file, then * each `//` file in name order. Absent files are left out. */ -export async function listEntryFiles(repoPath: string, type: EntryType): Promise { - const namespaces = (await listDirs(path.join(repoPath, type))).sort(); +export async function listEntryFiles(repoPath: string, where: EntryType | EntryLayout): Promise { + const namespaces = (await listDirs(path.join(repoPath, asLayout(where).dir))).sort(); const files: EntryFile[] = []; for (const namespace of [null, ...namespaces]) { - const absolutePath = entryFileAbsolutePath(repoPath, type, namespace); - if (await pathExists(absolutePath)) files.push({ namespace, relativePath: entryFilePath(type, namespace), absolutePath }); + const absolutePath = entryFileAbsolutePath(repoPath, where, namespace); + if (await pathExists(absolutePath)) files.push({ namespace, relativePath: entryFilePath(where, namespace), absolutePath }); } return files; } @@ -161,6 +199,8 @@ export async function readEntryFileText( export interface EntryReader { readonly type: EntryType; + /** Defaults to `entryLayout(type)`. */ + readonly layout?: EntryLayout; /** null when the file does not exist. */ read(absolutePath: string, relativePath: string): Promise | null>; nameOf(entry: E): string; @@ -181,7 +221,7 @@ export interface ResolvedEntry { } /** Why a type was not applied this run. */ -export type EntryFailure = +export type EntryFailure = ( | { readonly kind: 'broken-file'; readonly type: EntryType; readonly source: string; readonly reason: string } | { readonly kind: 'duplicate'; readonly type: EntryType; readonly name: string; readonly source: string } | { @@ -191,7 +231,11 @@ export type EntryFailure = readonly first: string; readonly second: string; } - | { readonly kind: 'namespaces-unresolved'; readonly type: EntryType; readonly reason: string }; + | { readonly kind: 'namespaces-unresolved'; readonly type: EntryType; readonly reason: string } +) & { + /** How the failed reader's messages name what it reads. */ + readonly layout: EntryLayout; +}; /** A warning about an entry that still resolves, worded for the admin who can fix it. */ export interface EntryNotice { @@ -218,15 +262,16 @@ export type EntryResolution = */ export async function activeEntryNamespaces( localConfig: LocalConfig, - type: EntryType, + layout: EntryLayout, ): Promise<{ ok: true; active: string[] | null } | { ok: false; failure: EntryFailure }> { + const type = layout.activation; try { const resolved = await resolveResourceNamespaces(localConfig); return { ok: true, active: resolved ? resolved.activeNamespaces[type] ?? [] : null }; } catch (error) { return { ok: false, - failure: { kind: 'namespaces-unresolved', type, reason: error instanceof Error ? error.message : String(error) }, + failure: { kind: 'namespaces-unresolved', type, reason: error instanceof Error ? error.message : String(error), layout }, }; } } @@ -253,29 +298,30 @@ export async function resolveEntries( active: readonly string[] | null, ): Promise> { const { type } = reader; + const layout = asLayout(reader.layout ?? type); const repoPath = localConfig.repo.localPath; const places: (string | null)[] = [null, ...(active ?? [])]; const notices: EntryNotice[] = []; - const targets = new TargetFiles(repoPath, type); + const targets = new TargetFiles(repoPath, layout); - const dirs = active && active.length > 0 ? await listDirs(path.join(repoPath, type)) : []; + const dirs = active && active.length > 0 ? await listDirs(path.join(repoPath, layout.dir)) : []; const candidates: NamespaceCandidate[] = []; // Entries still scoped by the deprecated per-entry `roles:`. const roleScoped = new Set>(); for (const namespace of places) { const dir = namespace === null ? null : namespaceDir(dirs, namespace); - const source = entryFilePath(type, dir); - const read = await reader.read(entryFileAbsolutePath(repoPath, type, dir), source); + const source = entryFilePath(layout, dir); + const read = await reader.read(entryFileAbsolutePath(repoPath, layout, dir), source); if (read === null) continue; - if (!read.ok) return { kind: 'failed', failure: { kind: 'broken-file', type, source, reason: read.reason }, notices }; + if (!read.ok) return { kind: 'failed', failure: { kind: 'broken-file', type, source, reason: read.reason, layout }, notices }; for (const note of read.notes ?? []) notices.push({ kind: 'file-note', message: note }); for (const entry of read.entries) { const name = reader.nameOf(entry); const scope = reader.scopeOf(entry); const unknownKeys = read.unknownKeys?.get(entry) ?? []; - if (!await keepScopedEntry(type, name, source, scope, unknownKeys, localConfig, targets, notices)) continue; + if (!await keepScopedEntry(type, layout.noun, name, source, scope, unknownKeys, localConfig, targets, notices)) continue; const candidate = { name, source, namespace, value: entry }; candidates.push(candidate); if (scope.roles !== undefined) roleScoped.add(candidate); @@ -322,8 +368,10 @@ export async function resolveEntries( const resolution = resolveNamespacedItems(candidates.filter((candidate) => !laterCopies.has(candidate)), active); if (resolution.kind === 'conflict') { const failure: EntryFailure = resolution.reason === 'duplicate' - ? { kind: 'duplicate', type, name: resolution.name, source: resolution.first.source } - : { kind: 'two-namespaces', type, name: resolution.name, first: resolution.first.source, second: resolution.second.source }; + ? { kind: 'duplicate', type, name: resolution.name, source: resolution.first.source, layout } + : { + kind: 'two-namespaces', type, name: resolution.name, first: resolution.first.source, second: resolution.second.source, layout, + }; return { kind: 'failed', failure, notices }; } @@ -358,7 +406,8 @@ export async function resolveEntriesFor( reader: EntryReader, localConfig: LocalConfig, ): Promise> { - const namespaces = await activeEntryNamespaces(localConfig, reader.type); + const layout = asLayout(reader.layout ?? reader.type); + const namespaces = await activeEntryNamespaces(localConfig, layout); if (!namespaces.ok) return { kind: 'failed', failure: namespaces.failure, notices: [] }; return resolveEntries(reader, localConfig, namespaces.active); } @@ -377,6 +426,7 @@ export async function resolveEntriesFor( */ async function keepScopedEntry( type: EntryType, + noun: string, name: string, source: string, scope: EntryScopeKeys, @@ -385,7 +435,7 @@ async function keepScopedEntry( targets: TargetFiles, notices: EntryNotice[], ): Promise { - const label = `${source}: ${ENTRY_NOUN[type]} "${name}"`; + const label = `${source}: ${noun} "${name}"`; if (unknownKeys.length > 0) { const one = unknownKeys.length === 1; const keys = unknownKeys.map((key) => `\`${key}:\``).join(', '); @@ -444,17 +494,18 @@ export class TargetFiles { private roles: ReturnType | null = null; private projects: ReturnType | null = null; - constructor(private readonly repoPath: string, private readonly type: EntryType) {} + constructor(private readonly repoPath: string, private readonly layout: EntryLayout) {} async forIds(axis: 'roles' | 'projects', ids: readonly string[]): Promise { const files: string[] = []; for (const id of ids) { const declared = await this.declared(axis, id); if (declared.length > 0) { - files.push(...declared.map((namespace) => entryFilePath(this.type, namespace))); + files.push(...declared.map((namespace) => entryFilePath(this.layout, namespace))); } else { const owner = axis === 'roles' ? `role ${id}` : `project ${id}`; - files.push(`${entryFilePath(this.type, id)} (declare ${this.type}: [${id}] for ${owner} in manifest/${axis}.yaml)`); + const key = this.layout.activation; + files.push(`${entryFilePath(this.layout, id)} (declare ${key}: [${id}] for ${owner} in manifest/${axis}.yaml)`); } } return [...new Set(files)]; @@ -465,11 +516,11 @@ export class TargetFiles { if (axis === 'roles') { this.roles ??= loadRolesManifestIfPresent(this.repoPath); const manifest = await this.roles; - return (manifest ? findRole(manifest, id)?.resources[this.type] : undefined) ?? []; + return (manifest ? findRole(manifest, id)?.resources[this.layout.activation] : undefined) ?? []; } this.projects ??= loadProjectsManifest(this.repoPath); const manifest = await this.projects; - return (manifest ? findProject(manifest, id)?.resources[this.type] : undefined) ?? []; + return (manifest ? findProject(manifest, id)?.resources[this.layout.activation] : undefined) ?? []; } catch { // A manifest that does not load names no namespace; the fallback path // still tells the admin where the entry goes. @@ -496,8 +547,7 @@ async function isDeclaredNamespace(repoPath: string, type: EntryType, namespace: /** The failure as one actionable line: what happened, what it left alone, what to do. */ export function describeEntryFailure(failure: EntryFailure): string { - const kept = `${failure.type} was not applied this run, so your ${INSTALLED[failure.type]} are unchanged.`; - const noun = ENTRY_NOUN[failure.type]; + const { kept, noun } = failure.layout; switch (failure.kind) { case 'broken-file': // The reader's reason already names the file. @@ -568,17 +618,19 @@ export function describeOrigins(entries: readonly ResolvedEntry[]): str * contributes any, each override, and in legacy mode each name the root file * repeats. None is a problem, so none is a failing check. */ -export function describeEntryNotes(type: EntryType, resolution: EntryResolution): string[] { +export function describeEntryNotes(where: EntryType | EntryLayout, resolution: EntryResolution): string[] { if (resolution.kind !== 'resolved') return []; + const layout = asLayout(where); + const { label } = layout; const lines = resolution.entries.some((entry) => entry.namespace !== null) - ? [`${type}: ${resolution.entries.length} received here (${describeOrigins(resolution.entries)})`] + ? [`${label}: ${resolution.entries.length} received here (${describeOrigins(resolution.entries)})`] : []; for (const entry of resolution.entries) { - if (entry.replaces) lines.push(describeOverride(type, { name: entry.name, source: entry.source, replaces: entry.replaces })); + if (entry.replaces) lines.push(describeOverride(label, { name: entry.name, source: entry.source, replaces: entry.replaces })); } if (resolution.active === null) { for (const name of resolution.repeated) { - lines.push(`${type}: "${name}" is defined more than once in ${entryFilePath(type, null)} (legacy mode does not check this; keep one of them)`); + lines.push(`${label}: "${name}" is defined more than once in ${entryFilePath(layout, null)} (legacy mode does not check this; keep one of them)`); } } return lines; @@ -595,9 +647,11 @@ export function describeEntryNotes(type: EntryType, resolution: EntryResolution< */ export async function entryNamespaceFromFlags( repoPath: string, - type: EntryType, + where: EntryType | EntryLayout, flags: { role?: string; project?: string }, ): Promise<{ ok: true; namespace: string | null } | { ok: false; message: string }> { + const layout = asLayout(where); + const type = layout.activation; if (flags.role !== undefined && flags.project !== undefined) { return { ok: false, message: 'Use either --role or --project, not both.' }; } @@ -607,12 +661,12 @@ export async function entryNamespaceFromFlags( } if (await isDeclaredNamespace(repoPath, type, flags.role) === false) { log.warn( - `No role or project declares ${type} namespace "${flags.role}", so ${entryFilePath(type, flags.role)} reaches nobody. ` + `No role or project declares ${type} namespace "${flags.role}", so ${entryFilePath(layout, flags.role)} reaches nobody. ` + `Add \`${type}: [${flags.role}]\` to the resources of a role in manifest/roles.yaml or of a project in ` + 'manifest/projects.yaml.', ); } - return { ok: true, namespace: namespaceDir(await listDirs(path.join(repoPath, type)), flags.role) }; + return { ok: true, namespace: namespaceDir(await listDirs(path.join(repoPath, layout.dir)), flags.role) }; } if (flags.project === undefined) return { ok: true, namespace: null }; @@ -627,7 +681,7 @@ export async function entryNamespaceFromFlags( if (!project) return { ok: false, message: unknownProjectMessage(manifest, flags.project) }; const namespaces = project.resources[type] ?? []; if (namespaces.length === 1 && namespaces[0] !== undefined) { - return { ok: true, namespace: namespaceDir(await listDirs(path.join(repoPath, type)), namespaces[0]) }; + return { ok: true, namespace: namespaceDir(await listDirs(path.join(repoPath, layout.dir)), namespaces[0]) }; } return { ok: false, diff --git a/src/pull.ts b/src/pull.ts index 6604a59f4..39003ed74 100644 --- a/src/pull.ts +++ b/src/pull.ts @@ -49,9 +49,12 @@ import { } from './types.js'; import type { CultureFrontmatter } from './types.js'; import { deliversEveryNamespace } from './resource-namespaces.js'; -import { reportEntryResolution, resolveEntries } from './namespaced-entries.js'; +import { reportEntryResolution } from './namespaced-entries.js'; import { resetWarnOnce } from './utils/warn-once.js'; -import { envEntryReader } from './resources/env.js'; +import type { EnvVariable } from './resources/env.js'; +import { declaredSecretKeys } from './resources/secrets.js'; +import { envShVariables, resolveTeamEnv, variablesKeptWarning, type TeamEnv } from './env-resolution.js'; +import { describeEnvAdvisory, envAdvisories } from './env-advisories.js'; import { getUserHome } from './utils/home.js'; import { acquireLock, releaseLock } from './update.js'; import { mirrorLearnings } from './utils/learnings-mirror.js'; @@ -481,6 +484,43 @@ function activeEnvNamespaces(roleContext: RolePullContext | null): string[] | nu return roleContext ? roleContext.activeNamespaces.env ?? [] : null; } +/** + * This scope's env for this pull, in the role context's namespaces. Kept in + * `teamEnvs` so the MCP and advisory stages use the same resolution rather + * than read every file again. + */ +async function resolvePullEnv( + localConfig: LocalConfig, + roleContext: RolePullContext | null, + teamEnvs: Map | undefined, +): Promise { + const teamEnv = await resolveTeamEnv(localConfig, { active: activeEnvNamespaces(roleContext) }); + teamEnvs?.set(localConfig, teamEnv); + return teamEnv; +} + +/** + * The env variables to write to env.sh, or null to leave it as it is. A key + * the team also declares as a secret (#875) resolves as the secret, so its + * repo value is left out; secret declarations that cannot be used are + * reported and, like an env file that cannot be, keep env.sh as it is. A + * variable the member set for this team exports their value, so a new shell + * follows the order MCP does; one set with `--from-env` is left out, so env.sh + * holds no copy of a value the member keeps elsewhere. A values file that cannot be read + * keeps env.sh as it is too. + */ +function deliverableEnvVariables(teamEnv: TeamEnv): EnvVariable[] | null { + const { variables, declarations } = teamEnv; + reportEntryResolution(variables); + if (declarations.kind !== 'absent') reportEntryResolution(declarations); + if (variables.kind === 'failed' || !declaredSecretKeys(declarations)) return null; + if (teamEnv.variableValues.kind === 'store-unreadable') { + log.warn(variablesKeptWarning(teamEnv.variableValues.reason)); + return null; + } + return envShVariables(variables.entries, teamEnv.variableValues.values); +} + /** * Pull resources for a single scope. This is the core sync logic extracted * from the original pull() function to support both user and project scope. @@ -504,13 +544,12 @@ async function reconcileEnvForUnchangedRepo( freshConfig: TeamaiConfig, localConfig: LocalConfig, roleContext: RolePullContext | null, + teamEnvs: Map | undefined, ): Promise { try { - const resolution = await resolveEntries(envEntryReader, localConfig, activeEnvNamespaces(roleContext)); - reportEntryResolution(resolution); - if (resolution.kind === 'failed') return; - const envHandler = new EnvHandler(); - await envHandler.writeResolvedEnv(resolution.entries.map((entry) => entry.entry), freshConfig, localConfig); + const variables = deliverableEnvVariables(await resolvePullEnv(localConfig, roleContext, teamEnvs)); + if (!variables) return; + await new EnvHandler().writeResolvedEnv(variables, freshConfig, localConfig); } catch (e) { // Visible rather than debug-only, and still not rethrown. This is the path // that REMOVES a variable the member is no longer scoped to, so a failed @@ -733,6 +772,8 @@ async function pullForScope( } = {}, /** Set to `{ completed: true }` on a real (non-dry-run) sync. See pull(). */ result?: { completed: boolean; docsSyncFailed: boolean }, + /** Collects this scope's env resolution for the stages after it (see resolvePullEnv). */ + teamEnvs?: Map, ): Promise { const scopeLabel = localConfig.scope; const revisionField = policy.revisionField ?? 'lastPullRev'; @@ -1071,7 +1112,7 @@ async function pullForScope( // scope a variable this CLI version now withholds; the Step 2 env // branch below is unreachable from here. if (resourceTypes.includes('env')) { - await reconcileEnvForUnchangedRepo(freshConfig, localConfig, roleContext); + await reconcileEnvForUnchangedRepo(freshConfig, localConfig, roleContext, teamEnvs); } // The knowledge branch has its own history: a teammate's contribution // moves teamai-learnings without touching main, so main's revision is @@ -1144,10 +1185,8 @@ async function pullForScope( // even when the root file is absent or empty: rewriting env.sh from the // resolved set is what removes a deactivated namespace's variables. A // file that cannot be used, or a name defined twice, keeps env.sh as is. - const resolution = await resolveEntries(envEntryReader, localConfig, activeEnvNamespaces(roleContext)); - reportEntryResolution(resolution); - if (resolution.kind === 'failed') continue; - const variables = resolution.entries.map((entry) => entry.entry); + const variables = deliverableEnvVariables(await resolvePullEnv(localConfig, roleContext, teamEnvs)); + if (!variables) continue; const countLabel = `${variables.length} env variable(s)`; if (options.dryRun) { @@ -1891,6 +1930,8 @@ export async function pull( const reported = new Set(); // A later successful scope must not hide an earlier docs failure (or vice versa). const syncResult = { completed: false, docsSyncFailed: false }; + // Each scope's env, resolved once by its env stage (resolvePullEnv). + const teamEnvs = new Map(); // Whether HOME's settings.json still has the pre-dispatch hook format. Read now // (HOME-only, no shared clone), but the actual reinject runs later under the @@ -1992,7 +2033,7 @@ export async function pull( } else { activeUserConfig = loadedUserConfig; if (await lockScope(activeUserConfig)) { - await pullForScope(activeUserConfig, options, reported, {}, syncResult); + await pullForScope(activeUserConfig, options, reported, {}, syncResult, teamEnvs); } } } else if (inheritUserScope) { @@ -2009,7 +2050,7 @@ export async function pull( if (projectConfig) { try { if (await lockScope(projectConfig)) { - await pullForScope(projectConfig, options, reported, {}, syncResult); + await pullForScope(projectConfig, options, reported, {}, syncResult, teamEnvs); } } catch (e) { log.warn(`Project-scope pull error: ${(e as Error).message}`); @@ -2056,7 +2097,12 @@ export async function pull( // 3.6. Reconcile team MCP servers. Outside pullForScope for the same reason as // hooks. User-scope MCP remains isolated in project mode. - await reconcileMcpAllScopes(reconcileUser, reconcileProject, options); + await reconcileMcpAllScopes(reconcileUser, reconcileProject, options, teamEnvs); + + // 3.6b. What the member should run for a team secret with no value (#875). + // Not on the silent session-start pull: its output is discarded, and it runs + // on every session. + if (!options.silent) await reportEnvAdvisories(reconcileUser, reconcileProject, teamEnvs); // 3.7. Reconcile the team co-author policy (does an AI tool stamp a // Co-Authored-By / attribution trailer on its commits?). Outside pullForScope @@ -2330,6 +2376,7 @@ async function reconcileMcpAllScopes( userConfig: LocalConfig | null, projectConfig: LocalConfig | null, options: GlobalOptions, + teamEnvs: Map, ): Promise { // Same contract as the hooks stage: resolve and report the entry warnings on // a dry run, skip the writes. `reconcileMcpForConfig` already gates every @@ -2341,7 +2388,9 @@ async function reconcileMcpAllScopes( const teamConfig = await loadTeamConfig(localConfig.repo.localPath); if (!teamConfig) continue; const { reconcileMcpForConfig } = await import('./mcp-reconcile.js'); - const { changes } = await reconcileMcpForConfig(teamConfig, localConfig, { force: options.force, dryRun: options.dryRun }); + const { changes } = await reconcileMcpForConfig(teamConfig, localConfig, { + force: options.force, dryRun: options.dryRun, teamEnv: await scopeEnv(localConfig, teamEnvs), + }); const applied = changes.filter((c) => c.action !== 'skipped'); for (const c of changes) { @@ -2364,6 +2413,42 @@ async function reconcileMcpAllScopes( } } +/** + * Print each scope's env advisories (env-advisories.ts): a declared secret with + * no value and the command that sets it, an MCP entry kept for it, a key both + * declared as a secret and set in env.yaml. After the MCP reconcile, so a kept + * entry is the one this pull left. + */ +async function reportEnvAdvisories( + userConfig: LocalConfig | null, + projectConfig: LocalConfig | null, + teamEnvs: Map, +): Promise { + const scopes = [userConfig, projectConfig].filter((c): c is LocalConfig => !!c); + for (const localConfig of scopes) { + try { + const teamConfig = await loadTeamConfig(localConfig.repo.localPath); + const teamEnv = await scopeEnv(localConfig, teamEnvs); + for (const advisory of await envAdvisories(localConfig, teamConfig, teamEnv)) log.warn(describeEnvAdvisory(advisory)); + } catch (e) { + log.debug(`[${localConfig.scope}] Env advisories skipped: ${(e as Error).message}`); + } + } +} + +/** + * The env this pull resolved for the scope, or a fresh resolution when its env + * stage did not run (HTTP mode delivers none). Kept for the next stage. + */ +async function scopeEnv(localConfig: LocalConfig, teamEnvs: Map): Promise { + if (localConfig.repo.kind === 'http') return undefined; + const known = teamEnvs.get(localConfig); + if (known) return known; + const teamEnv = await resolveTeamEnv(localConfig); + teamEnvs.set(localConfig, teamEnv); + return teamEnv; +} + /** * Reconcile the co-author policy across active scopes. Mirrors * reconcileMcpAllScopes: loops the installed scopes, loads each team config, diff --git a/src/resources/env-key.ts b/src/resources/env-key.ts new file mode 100644 index 000000000..2e29f590b --- /dev/null +++ b/src/resources/env-key.ts @@ -0,0 +1,38 @@ +/** + * A key env.sh writes (`generateEnvFile`) and the only shape it reads back + * (`parseEnvFile`), in resources/env.ts. + * + * Shared by both on purpose: the write side has to reject exactly what the + * read side skips, or a variable can exist in env.sh that the CLI can never + * see again. Its own module so secrets.ts and secret-store.ts can import it + * without importing env.ts, which imports them. + */ +export const ENV_KEY_RE = /^[A-Za-z_][A-Za-z0-9_]*$/; + +/** + * A table keyed by env names, built without a prototype: `__proto__` passes + * ENV_KEY_RE, and on an ordinary object assigning it hits the inherited + * setter and reading it when unset returns `Object.prototype`. + */ +export function envTable(entries: Iterable = []): Record { + const table: Record = Object.create(null); + for (const [key, value] of entries) table[key] = value; + return table; +} + +/** `env[key]` when `key` is set: an unset `__proto__` would read `Object.prototype`. */ +export function envValue(env: NodeJS.ProcessEnv, key: string): string | undefined { + return Object.hasOwn(env, key) ? env[key] : undefined; +} + +/** `key` as the platform compares environment names: case-insensitively on Windows. */ +export function envName(key: string): string { + return process.platform === 'win32' ? key.toUpperCase() : key; +} + +/** The name in `names` that is the same environment variable as `key` (in any case on Windows), if any. */ +export function sameEnvName(names: Iterable, key: string): string | undefined { + const name = envName(key); + for (const other of names) if (envName(other) === name) return other; + return undefined; +} diff --git a/src/resources/env.ts b/src/resources/env.ts index 62d06396d..b827ecdac 100644 --- a/src/resources/env.ts +++ b/src/resources/env.ts @@ -7,9 +7,12 @@ import { TEAMAI_ENV_START, TEAMAI_ENV_END, getDataHome, getEnvBackupPath, getTea import { loadLocalConfigForScope } from '../config.js'; import { pathExists, readFileSafe, writeFile, ensureDir, fileContentEqual } from '../utils/fs.js'; import { log } from '../utils/logger.js'; +import { envShMarker, isEnvShMarker, recordEnvShExports } from '../env-sh-exports.js'; +import { ENV_KEY_RE } from './env-key.js'; +import { SECRETS_LAYOUT } from './secrets.js'; import { - entryFileAbsolutePath, listEntryFiles, readEntryFileText, reportEntryResolution, resolveEntriesFor, - unknownEntryKeys, writtenList, type EntryReader, + listEntryFiles, readEntryFileText, reportEntryResolution, resolveEntriesFor, + unknownEntryKeys, writtenList, type EntryFile, type EntryReader, } from '../namespaced-entries.js'; import { resolveActiveShellProfile, @@ -126,15 +129,6 @@ export function maskEnvValue(value: string): string { return `${value.slice(0, 2)}****`; } -/** - * A key this module will write into env.sh, and the only shape it reads back. - * - * Shared by `parseEnvFile` and `generateEnvFile` on purpose: the write side has - * to reject exactly what the read side skips, or a variable can exist in env.sh - * that the CLI can never see again. - */ -export const ENV_KEY_RE = /^[A-Za-z_][A-Za-z0-9_]*$/; - /** * Read back the assignments `generateEnvFile` writes, as key → value. * @@ -152,7 +146,8 @@ export function parseEnvFile(content: string): Map { while (i < content.length) { const eq = content.startsWith(PREFIX, i) ? content.indexOf('=', i + PREFIX.length) : -1; const key = eq === -1 ? '' : content.slice(i + PREFIX.length, eq); - if (eq === -1 || !ENV_KEY_RE.test(key) || content[eq + 1] !== "'") { + // The marker is not a team variable (env-sh-exports.ts), and fits on its line. + if (eq === -1 || !ENV_KEY_RE.test(key) || content[eq + 1] !== "'" || isEnvShMarker(key)) { const nl = content.indexOf('\n', i); if (nl === -1) break; i = nl + 1; @@ -232,10 +227,13 @@ export class EnvHandler extends ResourceHandler { readonly type = 'env' as const; /** - * Scan for local env changes that need to be pushed: `env/env.yaml` and every - * `env//env.yaml`, one item per changed file. + * Scan for local env changes that need to be pushed: `env/env.yaml`, every + * `env//env.yaml`, and the same for `secrets.yaml`, one item per changed file. */ async scanLocalForPush(_teamConfig: TeamaiConfig, localConfig: LocalConfig): Promise { + const listEnvFiles = async (root: string): Promise => + [...await listEntryFiles(root, 'env'), ...await listEntryFiles(root, SECRETS_LAYOUT)]; + // Single-repo mode: users edit team env directly at /.teamai/env/ // (it lives in their own repo). push runs in the knowledge worktree, so // localConfig.repo.localPath here is the origin/ checkout — diff the @@ -244,8 +242,8 @@ export class EnvHandler extends ResourceHandler { if (isSelfMode(localConfig) && localConfig.projectRoot) { const activeRoot = path.join(localConfig.projectRoot, '.teamai'); const items: ResourceItem[] = []; - for (const { namespace, relativePath, absolutePath: activeEnv } of await listEntryFiles(activeRoot, 'env')) { - const baseEnv = entryFileAbsolutePath(localConfig.repo.localPath, 'env', namespace); + for (const { relativePath, absolutePath: activeEnv } of await listEnvFiles(activeRoot)) { + const baseEnv = path.join(localConfig.repo.localPath, ...relativePath.split('/')); // Not in the baseline → new; present but different → modified; equal → skip. if (await pathExists(baseEnv) && await fileContentEqual(activeEnv, baseEnv)) continue; items.push(envPushItem(relativePath, activeEnv)); @@ -274,7 +272,7 @@ export class EnvHandler extends ResourceHandler { } const items: ResourceItem[] = []; - for (const { relativePath, absolutePath } of await listEntryFiles(repoPath, 'env')) { + for (const { relativePath, absolutePath } of await listEnvFiles(repoPath)) { if (changed && !changed.has(relativePath)) continue; items.push(envPushItem(relativePath, absolutePath)); } @@ -344,8 +342,15 @@ export class EnvHandler extends ResourceHandler { await ensureDir(teamaiHome); await writeFile(getEnvBackupPath(localConfig), backupLines.join('\n') + '\n'); - // /env.sh (sourceable export file) - await writeFile(envShPath, this.generateEnvFile(variables)); + // /env.sh (sourceable export file). What it exported before + // and after is recorded first: a shell opened before this rewrite still + // carries those values, and they are the team's, not the member's (#879 + // Conflict 10). The old ones are there too for an env.sh an older CLI wrote. + const previous = parseEnvFile(await readFileSafe(envShPath) ?? ''); + const recorded = await recordEnvShExports(envShPath, [...previous, ...variables.map((v): [string, string] => [v.key, v.value])]); + const envSh = this.generateEnvFile(variables); + const marker = envShMarker(teamaiHome, parseEnvFile(envSh), recorded); + await writeFile(envShPath, marker ? `${envSh}export ${marker[0]}='${marker[1]}'\n` : envSh); // Inject source line into shell profile if enabled const inject = teamConfig.sharing.env.injectShellProfile !== false; diff --git a/src/resources/mcp-format.ts b/src/resources/mcp-format.ts index e29345684..1c3257a00 100644 --- a/src/resources/mcp-format.ts +++ b/src/resources/mcp-format.ts @@ -1,5 +1,6 @@ import crypto from 'node:crypto'; import type { McpServerDef, McpTransport } from '../types.js'; +import { sameEnvName } from './env-key.js'; // ─── Per-tool rendering ────────────────────────────────────── // @@ -47,6 +48,12 @@ export const MCP_SERVER_KEY: Record, string> = { copilot: 'mcpServers', }; +/** Whether two formats keep their servers under one key of a shared file (Claude, Cursor and CodeBuddy all use `mcpServers`). */ +export function sameServerKey(a: McpFormat, b: McpFormat): boolean { + if (a === 'codex' || b === 'codex') return a === b; + return MCP_SERVER_KEY[a] === MCP_SERVER_KEY[b]; +} + /** Transports each format can actually express. */ const SUPPORTED_TRANSPORTS: Record> = { claude: new Set(['stdio', 'http', 'sse']), @@ -142,15 +149,23 @@ export interface ResolveResult { missing: string[]; } +/** The value `${name}` takes from `vars`: on Windows `${token}` names the `TOKEN` a table holds, one environment variable. */ +export function placeholderValue(vars: Record, name: string): string | undefined { + if (Object.hasOwn(vars, name)) return vars[name]; + const other = sameEnvName(Object.keys(vars), name); + return other === undefined ? undefined : vars[other]; +} + /** * Substitute ${VAR} throughout a def. Unresolved vars are left as-is and * reported, so the caller can skip the server rather than inject a broken one. */ export function resolvePlaceholders(def: McpServerDef, vars: Record): ResolveResult { const missing = new Set(); + const lookup = (name: string): string | undefined => placeholderValue(vars, name); const sub = (v: string): string => v.replace(PLACEHOLDER_RE, (whole, name: string) => { - const val = vars[name]; + const val = lookup(name); if (val === undefined || val === '') { missing.add(name); return whole; diff --git a/src/resources/secrets.ts b/src/resources/secrets.ts new file mode 100644 index 000000000..411ca58be --- /dev/null +++ b/src/resources/secrets.ts @@ -0,0 +1,160 @@ +/** + * Team secrets (#875): env keys a team declares without a value, in + * `env/secrets.yaml` and `env//secrets.yaml`, activated by `resources.env` + * like `env//env.yaml`. A namespace entry replaces the root entry with the + * same key. Each member supplies the value on their own machine; the repo only + * says which keys exist, what they are for and where to get one. + * + * The file is separate from env.yaml so an older CLI, which reads only + * env.yaml, ignores it, and `env add` / `env remove` on an older CLI cannot + * drop it by rewriting env.yaml. + */ +import path from 'node:path'; +import YAML from 'yaml'; +import { z } from 'zod'; +import { + entryLayout, missingTopLevelKeyReason, readEntryFileText, resolveEntries, resolveEntriesFor, unknownEntryKeys, + writtenList, type EntryLayout, type EntryReader, type EntryResolution, +} from '../namespaced-entries.js'; +import type { LocalConfig } from '../types.js'; +import { ensureDir, readFileSafe, writeFile } from '../utils/fs.js'; +import { ENV_KEY_RE, envName } from './env-key.js'; + +const SecretDeclarationSchema = z.object({ + key: z.string().regex(ENV_KEY_RE, 'must be a shell variable name: letters, digits and underscores, not starting with a digit'), + description: z.string().optional(), + /** Where a member gets a value. */ + url: z.string().optional(), +}); + +const SecretsYamlSchema = z.object({ + secrets: z.array(SecretDeclarationSchema).default([]), +}); + +export type SecretDeclaration = z.infer; + +/** The keys `declaration` was written with that secrets.yaml does not know: that secret is not declared. */ +export function unknownSecretDeclarationKeys(declaration: object): string[] { + return Object.keys(declaration).filter((key) => !Object.hasOwn(SecretDeclarationSchema.shape, key)); +} + +/** `env/secrets.yaml` and `env//secrets.yaml`, active through `resources.env`. */ +export const SECRETS_LAYOUT: EntryLayout = { + ...entryLayout('env'), + file: 'secrets.yaml', + label: 'secrets', + noun: 'secret', + kept: 'Team secrets were not resolved this run; env variables and MCP servers stay as they are.', +}; + +/** + * How the secrets files are read. A file without a top-level `secrets:` key + * is broken, not empty, as for env.yaml (#662). A key the schema does not + * know, `value:` included, keeps that secret from being declared: a value + * does not belong in the repo. + */ +export const secretsEntryReader: EntryReader = { + type: 'env', + layout: SECRETS_LAYOUT, + async read(absolutePath, relativePath) { + const file = await readEntryFileText(absolutePath, relativePath); + if (!file.ok) return file; + if (file.text === null) return null; + let raw: unknown; + try { + raw = YAML.parse(file.text); + } catch (e) { + return { ok: false, reason: `${relativePath} is not valid YAML: ${e instanceof Error ? e.message : String(e)}` }; + } + if (raw === null || raw === undefined) return { ok: true, entries: [] }; + const shapeProblem = missingTopLevelKeyReason(raw, SecretsYamlSchema); + if (shapeProblem) return { ok: false, reason: `${relativePath} declares no secrets: ${shapeProblem}` }; + const parsed = SecretsYamlSchema.safeParse(raw); + if (!parsed.success) { + const issues = parsed.error.issues.map((issue) => `${issue.path.join('.') || '(root)'}: ${issue.message}`).join('; '); + return { ok: false, reason: `${relativePath} does not match the secrets.yaml schema: ${issues}` }; + } + const entries = parsed.data.secrets; + return { ok: true, entries, unknownKeys: unknownEntryKeys(raw, 'secrets', entries, SecretDeclarationSchema) }; + }, + nameOf: (secret) => secret.key, + scopeOf: () => ({}), +}; + +/** + * A secrets file's declarations as written, for `env add --secret` and + * `env remove`: every key an entry was written with is kept, so a rewrite + * drops nothing the file had. A file that does not parse gives its reason: + * writing back what could be read would drop every declaration it has. + */ +export async function readSecretsForEdit( + absolutePath: string, + relativePath: string, +): Promise<{ ok: true; secrets: Record[] } | { ok: false; reason: string }> { + const read = await secretsEntryReader.read(absolutePath, relativePath); + if (read === null) return { ok: true, secrets: [] }; + if (!read.ok) return read; + const text = await readFileSafe(absolutePath); + const written = writtenList(text === null ? null : YAML.parse(text), 'secrets'); + const isEntry = (entry: unknown): entry is Record => entry !== null && typeof entry === 'object'; + return { ok: true, secrets: Array.isArray(written) ? written.filter(isEntry) : [] }; +} + +/** Write a secrets file with these declarations and nothing else. */ +export async function writeSecretsFile(absolutePath: string, secrets: readonly object[]): Promise { + await ensureDir(path.dirname(absolutePath)); + await writeFile(absolutePath, YAML.stringify({ secrets })); +} + +/** + * The secrets this member's scope declares: `absent` when none of the files it + * reads exists, else the resolution. A failed resolution is never "no + * secrets": a consumer that took it for none would drop what a member set. + */ +export type SecretDeclarations = { readonly kind: 'absent' } | EntryResolution; + +/** + * Env's active namespaces as a caller that already resolved them has them + * (pull's role context); `active` is null in legacy mode, which reads the root + * file alone. + */ +export interface KnownNamespaces { + readonly active: readonly string[] | null; +} + +/** + * Resolve the secret declarations for this member, in env's active namespaces: + * `namespaces` when the caller has them, else resolved from `resources.env`. + */ +export async function resolveSecretDeclarations( + localConfig: LocalConfig, + namespaces?: KnownNamespaces, +): Promise { + let found = false; + const reader: EntryReader = { + ...secretsEntryReader, + async read(absolutePath, relativePath) { + const read = await secretsEntryReader.read(absolutePath, relativePath); + if (read !== null) found = true; + return read; + }, + }; + const resolution = namespaces + ? await resolveEntries(reader, localConfig, namespaces.active) + : await resolveEntriesFor(reader, localConfig); + return resolution.kind === 'resolved' && !found ? { kind: 'absent' } : resolution; +} + +/** + * The keys `declarations` declares, or null when they failed: a failed file is + * never "no secrets". Its `has` matches a key in any case on Windows, where + * `token` in env.yaml is the same environment variable as a declared `TOKEN`. + */ +export function declaredSecretKeys(declarations: Exclude): ReadonlySet; +export function declaredSecretKeys(declarations: SecretDeclarations): ReadonlySet | null; +export function declaredSecretKeys(declarations: SecretDeclarations): ReadonlySet | null { + if (declarations.kind === 'failed') return null; + const keys = new Set(declarations.kind === 'resolved' ? declarations.entries.map((entry) => entry.name) : []); + const names = new Set([...keys].map(envName)); + return Object.assign(keys, { has: (key: string): boolean => names.has(envName(key)) }); +} diff --git a/src/secret-store.ts b/src/secret-store.ts new file mode 100644 index 000000000..2973a57bd --- /dev/null +++ b/src/secret-store.ts @@ -0,0 +1,221 @@ +/** + * The values a member keeps for their team's declared secrets (#875), on their + * own machine and never in the team repo: one file per team repo at + * `~/.teamai/secrets/teams/.json`, named by the hash of the configured + * team repo URL alone so renaming `team:` in `teamai.yaml` keeps the values, and + * one for every team on the machine at `~/.teamai/secrets/machine.json`. `~/.teamai/env` is not used: it is + * already the user scope's env backup file. + * + * Each entry is exactly one of a literal value or the name of a variable to + * read when the value is used (`--from-env`), so no copy of it is stored, and + * says what it is (`kind`): a secret's value, or the member's override of an + * env variable, as the scope declared the key when `env set` wrote it. A + * secret's value stays one after the team stops declaring the key, so it is + * never exported as a variable. An entry without `kind` (earlier builds) is a + * secret's. + */ +import crypto from 'node:crypto'; +import fs from 'node:fs'; +import path from 'node:path'; +import { z } from 'zod'; +import { ENV_KEY_RE, envTable, envValue } from './resources/env-key.js'; +import { isRepoReference } from './models/profile.js'; +import { getTeamaiHomeDir, type LocalConfig } from './types.js'; +import { acquireLock, releaseLock } from './update.js'; +import { writeJsonAtomic } from './utils/fs.js'; + +const StoredEntryKindSchema = z.enum(['secret', 'variable']); +export type StoredEntryKind = z.infer; + +const StoredSecretSchema = z.union([ + z.object({ value: z.string(), kind: StoredEntryKindSchema.optional() }).strict(), + z.object({ env: z.string().regex(ENV_KEY_RE), kind: StoredEntryKindSchema.optional() }).strict(), +]); +export type StoredSecret = z.infer; + +/** What an entry is; one without `kind` is a secret's, the side that never exports it. */ +export function storedEntryKind(entry: StoredSecret): StoredEntryKind { + return entry.kind ?? 'secret'; +} + +// Not z.record: it drops a `__proto__` key, which ENV_KEY_RE accepts. +const SecretStoreSchema = z + .custom((raw) => raw !== null && typeof raw === 'object' && !Array.isArray(raw)) + .transform((raw) => Object.entries(raw)) + .pipe(z.array(z.tuple([z.string().regex(ENV_KEY_RE), StoredSecretSchema]))) + .transform((entries) => envTable(entries)); +export type SecretStore = z.infer; + +/** A store file's entries, or why it cannot be used. A missing file holds none. */ +export type SecretStoreRead = + | { readonly ok: true; readonly values: SecretStore } + | { readonly ok: false; readonly reason: string }; + +/** + * This team's values file, named by the team repo URL this machine's config + * holds and never by `teamai.yaml`'s `repo:`: a copied team repo that claims + * another team's `repo:` must not get that team's values. See `repoIdentity` + * for which forms of the URL name one file. + */ +export function getTeamSecretsPath(localConfig: LocalConfig): string { + const { remote, url, localPath } = localConfig.repo; + // A remote names the repo only when it is a URL; an alias (origin, fork) names nothing, so the URL does. + const configured = remote && isRepoReference(remote) ? remote : url; + const identity = configured ? repoIdentity(configured) : localPath; + // The full digest: the file name is all that keeps one team's values from another's. + const hash = crypto.createHash('sha256').update(identity).digest('hex'); + return path.join(getTeamaiHomeDir(), 'secrets', 'teams', `${hash}.json`); +} + +/** Each scheme's default port, and the family whose URLs of one repo share a file: http and https are not one. */ +const SCHEMES: ReadonlyMap = new Map([ + ['ssh', { family: 'ssh', defaultPort: '22' }], + ['git+ssh', { family: 'ssh', defaultPort: '22' }], + ['ssh+git', { family: 'ssh', defaultPort: '22' }], + ['https', { family: 'https', defaultPort: '443' }], + ['http', { family: 'http', defaultPort: '80' }], + ['git', { family: 'git', defaultPort: '9418' }], +]); + +/** + * A team repo URL as the part of it that says which repo it is: scheme family + * (ssh, https or http), the ssh user, lowercased host, a port other than the + * scheme's default, and the path, query and fragment as written. An scp path + * that starts with neither `/` nor `~` is in the ssh user's home, so it is + * keyed as `~/path`, the path `ssh://host/~/path` names; `ssh://host/path` is + * from the root. Only http(s) credentials, trailing slashes on the path and, + * for a repo served over ssh or http(s), a trailing `.git` are dropped (a + * `file://` URL's `team` and `team.git` are two directories), so `git@host:acme/team.git` and + * `ssh://git@host:22/~/acme/team` name one file, while + * `ssh://git@host/acme/team` (from the root), two ssh users' repos on one + * host, two repos on one host with different ports or queries, or behind http + * and https, never share values. Not `normalizeRepoUrlForCompare`: it drops + * the port, and its callers compare loosely on purpose. + */ +function repoIdentity(url: string): string { + const trimmed = url.trim(); + const key = (family: string, user: string, host: string, port: string, repoPath: string, rest = '', served = true): string => { + const trimmedPath = repoPath.replace(/^\/+/, '').replace(/\/+$/, ''); + const name = served ? trimmedPath.replace(/\.git$/i, '') : trimmedPath; + return `${family}://${user ? `${user}@` : ''}${host.toLowerCase()}${port ? `:${port}` : ''}/${name}${rest}`; + }; + // `[user@]host:path`, as git reads it: no `/` before the `:`, no `//` after it, not a Windows drive. + const scp = /^[A-Za-z]:[\\/]/.test(trimmed) ? null : /^(?:([^/@]+)@)?([^:/]+):(?!\/\/)(.+)$/.exec(trimmed); + if (scp) { + const scpPath = scp[3] ?? ''; + return key('ssh', scp[1] ?? '', scp[2] ?? '', '', /^[/~]/.test(scpPath) ? scpPath : `~/${scpPath}`); + } + let parsed: URL; + try { + parsed = new URL(trimmed); + } catch { + return trimmed; + } + const scheme = parsed.protocol.slice(0, -1).toLowerCase(); + const known = SCHEMES.get(scheme); + const family = known?.family ?? scheme; + const user = family === 'ssh' ? decodeUser(parsed.username) : ''; + const port = parsed.port === known?.defaultPort ? '' : parsed.port; + return key(family, user, parsed.hostname, port, parsed.pathname, `${parsed.search}${parsed.hash}`, known !== undefined); +} + +/** A URL's percent-encoded user as the scp form writes it; one that does not decode stays as written. */ +function decodeUser(user: string): string { + try { + return decodeURIComponent(user); + } catch { + return user; + } +} + +/** The values file for every team on this machine (`teamai env set --global`). */ +export function getMachineSecretsPath(): string { + return path.join(getTeamaiHomeDir(), 'secrets', 'machine.json'); +} + +/** + * Read a store file. A file that does not parse is reported by its path only, + * one with an entry that is not one `value` or one `env` by the entry's + * number: the parser's own message quotes the text around the problem, which + * may be a value. + */ +export async function readSecretStore(filePath: string): Promise { + let content: string; + try { + content = await fs.promises.readFile(filePath, 'utf8'); + } catch (error) { + const code = error instanceof Error && 'code' in error && typeof error.code === 'string' ? error.code : undefined; + if (code === 'ENOENT') return { ok: true, values: {} }; + return { + ok: false, + reason: `Cannot read your secret values at ${filePath} (${code ?? 'unknown error'}). Check that the file is yours and ` + + `readable (\`ls -l ${filePath}\`), or delete it and set the values again with \`teamai env set\`.`, + }; + } + const fix = 'Fix the file, or delete it and set the values again with `teamai env set`.'; + let raw: unknown; + try { + raw = JSON.parse(content); + } catch { + return { ok: false, reason: `${filePath} is not valid JSON. ${fix}` }; + } + const parsed = SecretStoreSchema.safeParse(raw); + if (parsed.success) return { ok: true, values: parsed.data }; + const index = parsed.error.issues[0]?.path[0]; + const entry = typeof index === 'number' ? index + 1 : 0; + return { + ok: false, + reason: `${filePath} ${entry > 0 ? `has an invalid entry (entry ${entry})` : 'is not a JSON object'}: ` + + `each entry maps a variable name to {"value": "..."} or {"env": "VAR"}, with an optional "kind" of "secret" or "variable". ${fix}`, + }; +} + +/** Write a store file atomically, readable by this user only. */ +export async function writeSecretStore(filePath: string, values: SecretStore): Promise { + await writeJsonAtomic(filePath, SecretStoreSchema.parse(values), { mode: 0o600 }); +} + +/** What `updateSecretStore` did. */ +export type SecretStoreUpdate = + | { readonly kind: 'written' } + | { readonly kind: 'unchanged' } + | { readonly kind: 'failed'; readonly reason: string }; + +/** + * Change a store file under its lock (`.lock`), so two `env set` or + * `env unset` runs at once both land: the file is read inside the lock, and + * `change` returns the new entries, or null to leave it as it is. + */ +export async function updateSecretStore( + filePath: string, + change: (values: SecretStore) => SecretStore | null, +): Promise { + const lock = `${filePath}.lock`; + let held = false; + for (let attempt = 0; attempt < 100 && !held; attempt++) { + held = await acquireLock(lock); + if (!held) await new Promise((resolve) => setTimeout(resolve, 50)); + } + if (!held) { + return { + kind: 'failed', + reason: `Another teamai command is changing ${filePath}. Run the command again once it has finished.`, + }; + } + try { + const store = await readSecretStore(filePath); + if (!store.ok) return { kind: 'failed', reason: store.reason }; + const next = change(store.values); + if (!next) return { kind: 'unchanged' }; + await writeSecretStore(filePath, next); + return { kind: 'written' }; + } finally { + await releaseLock(lock); + } +} + +/** The value an entry stands for now: a `--from-env` reference is read from `env` each time. Empty is none. */ +export function storedSecretValue(entry: StoredSecret, env: NodeJS.ProcessEnv = process.env): string | undefined { + const value = 'value' in entry ? entry.value : envValue(env, entry.env); + return value === undefined || value === '' ? undefined : value; +} diff --git a/src/status.ts b/src/status.ts index d3c3a12d8..2af82cd57 100644 --- a/src/status.ts +++ b/src/status.ts @@ -18,10 +18,11 @@ import { } from './agent-skills.js'; import { RESOURCE_TYPES, LocalConfigSchema, getDataHome, type GlobalOptions, type ResourceType } from './types.js'; import { projectsRootDir, readAnchorFile, projectSlug, legacyProjectSlug } from './utils/partition.js'; -import { maskEnvValue } from './resources/env.js'; import { mcpEntryReader } from './resources/mcp.js'; import { resolveTeamHookEntries } from './resources/hooks.js'; import { envEntryReader } from './resources/env.js'; +import { envListing } from './env-listing.js'; +import { resolveTeamEnv } from './env-resolution.js'; import { describeEntryFailure, describeOrigin, describeOrigins, reportUndeliveredEntryNotices, resolveEntriesFor, type EntryResolution, type EntryType, @@ -326,29 +327,20 @@ async function printRepoSection( console.log(`=== REPO ${t.toUpperCase()} ===`); // Env, hooks and MCP list what reaches this directory, each with its - // namespace: root plus the active namespace files. + // namespace: root plus the active namespace files. Env is the listing + // `teamai env list` prints (env-listing.ts). if (t === 'env') { - const env = await resolveEntriesFor(envEntryReader, localConfig); - if (env.kind === 'failed') { - reportUndeliveredEntryNotices(env); - console.log(` ${describeEntryFailure(env.failure)}`); - } else { - reportUndeliveredEntryNotices(env); - if (env.entries.length === 0) { - console.log(' (none)'); - } else { - if (options.reveal) { - process.stderr.write('[warn] Env values will be shown in plaintext\n'); - } - for (const v of env.entries) { - const display = options.reveal ? v.entry.value : maskEnvValue(v.entry.value); - console.log(` ${v.name}=${display} (${describeOrigin(v)})`); - if (options.verbose && v.entry.description) { - console.log(` ${v.entry.description}`); - } - } - } + const teamEnv = await resolveTeamEnv(localConfig); + // An entry an unknown or removed key takes out of the delivered set never appears below (#822). + reportUndeliveredEntryNotices(teamEnv.variables); + const listing = envListing(teamEnv, options); + for (const problem of listing.problems) console.log(` ${problem}`); + if (listing.lines.length === 0) { + if (listing.problems.length === 0) console.log(' (none)'); + return; } + if (listing.revealed) process.stderr.write('[warn] Env values will be shown in plaintext\n'); + for (const line of listing.lines) console.log(line.text); return; } diff --git a/src/types.ts b/src/types.ts index 2125fd489..da5380877 100644 --- a/src/types.ts +++ b/src/types.ts @@ -898,6 +898,18 @@ export interface ManagedMcpRecord { name: string; /** sha1 (first 16 hex) of the rendered entry; drives idempotent rewrites. */ hash: string; + /** + * Project scope: whether the entry holds a `${VAR}` value teamai resolved + * (#882). Absent in records an older teamai wrote. + */ + resolved?: boolean; + /** + * Project scope: this record was rebuilt after it was lost, or written by a + * pull that found no managed-mcp.json, and the other servers in its file + * could not be noted in managed-mcp-files.json yet (#882). Until a pull + * notes them, the file counts as having no record. + */ + unnoted?: true; } /** ~/.teamai/managed-mcp.json — team MCP servers injected per tool+scope key. */ diff --git a/src/uninstall.ts b/src/uninstall.ts index 0f1e85a53..d6ca3289f 100644 --- a/src/uninstall.ts +++ b/src/uninstall.ts @@ -110,6 +110,8 @@ interface RemovalPlan { shellProfiles: string[]; /** Docs directory (null if doesn't exist). */ docsDir: string | null; + /** The .git/info/exclude files holding teamai's MCP config block (#882), each with its patterns and the paths each protects. */ + gitExcludes: Map>; /** The .teamai home directory path. */ teamaiHome: string; /** Whether teamaiHome exists on disk. */ @@ -616,6 +618,7 @@ async function buildRemovalPlan( mcpServers: [], shellProfiles: [], docsDir: null, + gitExcludes: new Map(), teamaiHome, teamaiHomeExists: includeShared && await pathExists(teamaiHome), unpublishedQueues: includeShared ? await listQueuesIn(teamaiHome) : [], @@ -710,6 +713,23 @@ async function buildRemovalPlan( if (await pathExists(docsDir)) { plan.docsDir = docsDir; } + + // (g) teamai's block in .git/info/exclude (#882): the project's own, and + // that of any nested repository an MCP config sits in. It counts on its + // own: a clone whose other resources are gone still gets it removed. + if (localConfig.scope === 'project') { + const { resolveMcpTargets, projectWorktreeConfigs } = await import('./mcp-reconcile.js'); + const { findMcpGitExcludes } = await import('./mcp-git-exclude.js'); + const { readResolvedMcpFiles } = await import('./mcp-resolved-files.js'); + const dirs: string[] = []; + for (const cfg of await projectWorktreeConfigs(localConfig)) { + if (cfg.projectRoot) dirs.push(cfg.projectRoot); + for (const target of await resolveMcpTargets(teamConfig, cfg, { includeUndetected: true })) dirs.push(path.dirname(target.file)); + // A file a pull wrote under a toolPaths mapping since changed. + for (const file of Object.keys((await readResolvedMcpFiles(cfg)).files)) dirs.push(path.dirname(file)); + } + plan.gitExcludes = await findMcpGitExcludes(dirs); + } } return plan; @@ -732,6 +752,7 @@ function isPlanEmpty(plan: RemovalPlan): boolean { plan.mcpServers.length === 0 && plan.shellProfiles.length === 0 && plan.docsDir === null && + plan.gitExcludes.size === 0 && !plan.teamaiHomeExists ); } @@ -846,6 +867,12 @@ function printSummary(plan: RemovalPlan, agentFilter?: string): void { console.log(''); } + if (plan.gitExcludes.size > 0) { + console.log(' Git exclude entries for MCP configs (teamai\'s block):'); + for (const [file, entries] of plan.gitExcludes) console.log(` ${file} (${entries.map((entry) => entry.pattern).join(', ')})`); + console.log(''); + } + if (plan.teamaiHomeExists) { console.log(' TeamAI home directory:'); console.log(` ${plan.teamaiHome}/`); @@ -1202,7 +1229,7 @@ export async function uninstall(opts: UninstallOptions): Promise { // must leave the remaining tools' MCP servers intact. if (plan.includeShared) { try { - const { reconcileMcpForConfig } = await import('./mcp-reconcile.js'); + const { reconcileMcpForConfig, projectWorktreeConfigs, mcpConfigsNotProvenClean } = await import('./mcp-reconcile.js'); // Project scope: the managed-mcp manifests are PER-WORKTREE under the // shared partition (#374 P1-2C), and each worktree's MCP config lives in // its own checkout. Since executeRemoval deletes the whole shared @@ -1210,23 +1237,43 @@ export async function uninstall(opts: UninstallOptions): Promise { // linked worktree — otherwise a sibling worktree is left with an injected // server whose ownership record just got deleted (orphaned). User scope // has a single global manifest, so the current config is enough. - const configs: LocalConfig[] = [localConfig]; - if (localConfig.scope === 'project' && localConfig.projectRoot) { - const { listWorktrees } = await import('./utils/git.js'); - const { resolveProjectDataHome } = await import('./config.js'); - const worktrees = await listWorktrees(localConfig.projectRoot); - for (const wt of worktrees) { - if (wt === localConfig.projectRoot) continue; - const dataHome = await resolveProjectDataHome(wt); - configs.push({ ...localConfig, projectRoot: wt, dataHome }); - } - } let removedTotal = 0; - for (const cfg of configs) { + for (const cfg of await projectWorktreeConfigs(localConfig)) { const { changes } = await reconcileMcpForConfig(teamConfig, cfg, { removeAll: true }); removedTotal += changes.filter((c) => c.action === 'removed').length; } if (removedTotal > 0) log.info(`Removed ${removedTotal} teamai-managed MCP server(s)`); + // Worktrees share one info/exclude, so it goes once they are all clean, + // judged by what the files hold, not by what the cleanup reported: a + // lost manifest cleans nothing and reports nothing. Without the block, + // `git add -A` would commit a value teamai resolved. + if (plan.gitExcludes.size > 0) { + const { removeMcpGitExclude } = await import('./mcp-git-exclude.js'); + const held = await mcpConfigsNotProvenClean(teamConfig, localConfig, [...plan.gitExcludes.values()].flat()); + for (const [excludeFile, entries] of plan.gitExcludes) { + const clean: string[] = []; + for (const { pattern, files } of entries) { + const still = files.flatMap((file) => { + const why = held.get(file); + return why ? [`${file} (${why})`] : []; + }); + if (still.length === 0) { + clean.push(pattern); + continue; + } + log.warn( + `Kept \`${pattern}\` in ${excludeFile}, so git still ignores ${still.join('; ')}: it may hold MCP values teamai resolved to plaintext. ` + + `Remove teamai's MCP servers from it (or delete the file), then delete that line from ${excludeFile} yourself, and the block's two marker lines with its last one.`, + ); + } + if (clean.length === 0) continue; + const result = await removeMcpGitExclude(excludeFile, clean); + if (result === 'written') log.info(`Removed teamai's MCP config entries ${clean.join(', ')} from ${excludeFile}`); + if (result === 'locked') { + log.warn(`Kept teamai's block in ${excludeFile}: another teamai command held it past the wait. Delete the block's ${clean.join(', ')} lines yourself.`); + } + } + } } catch (e) { log.warn(`Failed to remove MCP servers: ${(e as Error).message}`); } diff --git a/src/utils/fs.ts b/src/utils/fs.ts index 77ab90047..f298c83cc 100644 --- a/src/utils/fs.ts +++ b/src/utils/fs.ts @@ -114,7 +114,8 @@ export async function writeFileAtomic( } const tmp = `${expanded}.${process.pid}.${crypto.randomBytes(6).toString('hex')}.tmp`; try { - await fse.writeFile(tmp, content, 'utf-8'); + // Created with the mode, so it is never readable wider than the target; chmod undoes the umask. + await fse.writeFile(tmp, content, { encoding: 'utf-8', mode, flag: 'wx' }); await fse.chmod(tmp, mode); await fse.rename(tmp, expanded); } catch (error) { @@ -176,7 +177,8 @@ export async function writeJsonAtomic( } const tmp = `${expanded}.${process.pid}.${crypto.randomBytes(6).toString('hex')}.tmp`; try { - await fse.writeFile(tmp, content, 'utf-8'); + // Created with the mode, so it is never readable wider than the target; chmod undoes the umask. + await fse.writeFile(tmp, content, { encoding: 'utf-8', mode, flag: 'wx' }); await fse.chmod(tmp, mode); await fse.rename(tmp, expanded); } catch (error) { diff --git a/src/utils/git.ts b/src/utils/git.ts index b2b0bfebb..5baab5e10 100644 --- a/src/utils/git.ts +++ b/src/utils/git.ts @@ -906,15 +906,15 @@ async function readAnchors(cwd?: string): Promise { /** * List the realpath'd top-level directory of every worktree of the repo that * contains `cwd` (main checkout + all linked worktrees), from - * `git worktree list --porcelain`. Returns [] outside a git repo. Used by a - * project-wide uninstall to clean each worktree's managed resources before the - * shared partition is deleted (issue #374 P1-2C). + * `git worktree list --porcelain`. Returns [] outside a git repo, or when `cwd` + * does not exist. Used by a project-wide uninstall to clean each worktree's + * managed resources before the shared partition is deleted (issue #374 P1-2C). */ export async function listWorktrees(cwd?: string): Promise { - const git = createGit(cwd); let list: string; try { - list = await git.raw(['worktree', 'list', '--porcelain']); + // Inside the try: simple-git throws at once for a directory that does not exist. + list = await createGit(cwd).raw(['worktree', 'list', '--porcelain']); } catch { return []; } diff --git a/src/utils/prompt.ts b/src/utils/prompt.ts index 1a91d9928..3b0dd4247 100644 --- a/src/utils/prompt.ts +++ b/src/utils/prompt.ts @@ -144,6 +144,16 @@ export function askSecret(prompt: string): Promise { }); } +/** + * Read piped stdin to the end, without its trailing line breaks. Callers + * refuse a TTY first, in their own wording. + */ +export async function readStdin(): Promise { + let value = ''; + for await (const chunk of process.stdin) value += String(chunk); + return value.replace(/[\r\n]+$/, ''); +} + /** * Ask a yes/no confirmation question. *