From 6d489f9235e75c91a0626a0cc9c687fdc23c0e89 Mon Sep 17 00:00:00 2001 From: Andrew Anderson Date: Wed, 11 Nov 2026 16:55:11 -0700 Subject: [PATCH] =?UTF-8?q?=F0=9F=90=9B=20[v6]=20claude=20auth:=20share=20?= =?UTF-8?q?refreshed=20logins=20across=20per-agent=20.claude=20homes=20and?= =?UTF-8?q?=20harden=20shared-home=20permission=20handling?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Top-up of #17767 (v5 squash 0cf675d3a) onto v6, merged into the credential bridge v6 already carries from #17253/#17590: - keeps v6's dashboard surface (credentialBridgeBroken, resume after sign-in) and the legacy-dir-link rules; - brings the sharedfile-based promotion, owner read through su-exec, newest-expiresAt rule, 5s bridge cadence (HIVE_CLAUDE_CREDENTIAL_BRIDGE_INTERVAL), and the fail-safe handling of an unreadable private file (left in place except on sign-in); - hardens the shared-home permission guard and credential promotion against non-regular files (entrypoint guard, sticky shared dot-dirs, crux/sharedfile writer for the Claude login and Copilot config); - a dashboard sign-in still recovers a shared credential path that is a dangling or self-referencing link (#17590), by unlinking it first. Supersedes #17678 and #17748. (cherry picked from commit 0cf675d3a) Signed-off-by: Andrew Anderson --- .../fixed-claude-shared-credentials.md | 1 + src/crux/agent/claude_credential_bridge.go | 447 ++++++--- .../claude_credential_bridge_internal_test.go | 140 +-- .../agent/claude_credential_bridge_test.go | 930 ++++++++++++++---- src/crux/agent/copilottoken/copilottoken.go | 20 +- src/crux/agent/coverage_boost_test.go | 26 + src/crux/agent/credentials_watchdog.go | 30 + src/crux/agent/interactive_home.go | 6 + src/crux/agent/manager_credentials.go | 25 +- src/crux/agent/manager_tokens.go | 13 +- src/crux/agent/permissions_watcher.go | 4 +- src/crux/claude/oauth.go | 74 +- .../claude/oauth_write_error_branches_test.go | 62 +- src/crux/claude/usable_token_test.go | 65 ++ src/crux/sharedfile/sharedfile.go | 278 ++++++ src/crux/sharedfile/sharedfile_test.go | 480 +++++++++ src/deploy/entrypoint.sh | 108 +- src/deploy/test_entrypoint_perm_guard.sh | 84 ++ src/docs/agent-configuration.md | 10 +- src/docs/env-vars.md | 1 + 20 files changed, 2366 insertions(+), 438 deletions(-) create mode 100644 changelog.d/fixed-claude-shared-credentials.md create mode 100644 src/crux/sharedfile/sharedfile.go create mode 100644 src/crux/sharedfile/sharedfile_test.go diff --git a/changelog.d/fixed-claude-shared-credentials.md b/changelog.d/fixed-claude-shared-credentials.md new file mode 100644 index 0000000000..db3fa495ed --- /dev/null +++ b/changelog.d/fixed-claude-shared-credentials.md @@ -0,0 +1 @@ +- Claude auth on per-UID hives: a Claude Code token refresh writes a temp file and renames it over `~/.claude/.credentials.json`, which replaced the per-agent symlink to the shared login with a private 0600 file; the refreshed token (and the rotated refresh grant) never reached the shared credential and the fleet logged out about every eight hours. The hive now detects the replaced link at agent launch, every 5s from the credential watchdog (`HIVE_CLAUDE_CREDENTIAL_BRIDGE_INTERVAL`) and after a dashboard sign-in, reads the agent's private file as its owner through `su-exec` when the server cannot open it, publishes the newer usable token into the shared login (the previous shared file is kept as `.credentials.json.stale-`), moves the private copy aside and restores the link. Per-agent `~/.claude` memory stays isolated; only the login is shared, with no operator switch. Audited as `claude_credential_rebridged`. Also hardens the shared-home permission guard and credential promotion against non-regular files: shared logins and CLI configs are written through a randomly named temporary with fd-based mode/owner, the root guard only repairs regular files in regular directories, and the shared dot-dirs get the sticky bit. diff --git a/src/crux/agent/claude_credential_bridge.go b/src/crux/agent/claude_credential_bridge.go index 4ca94d05f0..1d9e4bc10e 100644 --- a/src/crux/agent/claude_credential_bridge.go +++ b/src/crux/agent/claude_credential_bridge.go @@ -1,6 +1,7 @@ package agent import ( + "encoding/json" "errors" "fmt" "io/fs" @@ -12,47 +13,61 @@ import ( "time" "github.com/hivecommons/hive/crux/claude" + "github.com/hivecommons/hive/crux/sharedfile" ) -// Claude credential bridge repair. +// Claude credential bridge: share the LOGIN across per-agent .claude homes +// without sharing anything else. // -// Each per-UID claude agent reaches the fleet's one Claude login through a -// symlink, $HOME/.claude/.credentials.json -> /data/home/.claude/.credentials.json -// (perAgentCLIHomes). Claude Code does not write through that link. When it -// refreshes the OAuth token (about every 8 hours) it writes a temp file in -// $HOME/.claude and renames it over .credentials.json, which REPLACES the -// symlink with a regular file owned by the agent's uid, mode 0600. From then -// on that agent runs on a private copy: a later dashboard sign-in rewrites the -// shared file and never reaches it, and when its private token is revoked the -// agent answers every kick with "Please run /login" while a perfectly good -// login sits one directory away. +// Since hivecommons/hive#17039 each per-UID claude agent has a REAL ~/.claude +// (its own transcripts, todos, history and auto-memory) with one symlink in +// it, ~/.claude/.credentials.json -> /data/home/.claude/.credentials.json, so +// one operator sign-in serves the fleet. Claude Code does not write through +// that link. When it refreshes the OAuth token (about every 8 hours) it writes +// a temp file in ~/.claude and renames it over .credentials.json, which +// REPLACES the symlink with a regular file owned by the agent's uid, mode +// 0600. The refresh also rotates the grant: the shared file now holds a dead +// refresh token, the fresh one is private to that agent, and every other +// agent fails its own refresh within the access token's lifetime. On per-UID +// hives the whole fleet logged out about every 8 hours. // -// The Claude Code CLI offers no supported way to keep the link: its documented -// knob, CLAUDE_CONFIG_DIR, moves the WHOLE config directory (settings, -// transcripts, auto-memory) and would undo the per-agent split of -// hivecommons/hive#17031; a bind mount cannot be renamed over (EBUSY) and -// needs mount privileges the pod does not have. So hive cannot prevent the -// replacement and instead DETECTS and REPAIRS it: at every agent launch, on -// every credential-watchdog cycle, and right after a dashboard sign-in. +// Claude Code offers no supported way to keep the link: CLAUDE_CONFIG_DIR +// moves the WHOLE config directory (settings, transcripts, auto-memory) and +// would undo the per-agent memory split; a bind mount cannot be renamed over +// (EBUSY) and needs mount privileges the pod does not have. So hive cannot +// prevent the replacement and instead DETECTS and PUBLISHES it: at every agent +// launch, on a short poll from the credential watchdog, and right after a +// dashboard sign-in. // // The repair never deletes a token. It moves the private file aside, keeps // whichever of the two credentials is newer AND usable in the shared file, and // restores the symlink: // -// - the private copy wins only when it is usable (claude.TokensUsable: a live -// access token, or a refresh grant that has not expired) and either the -// shared credential is not usable or the private access token expires -// LATER than the shared one. A refresh mints a later expiresAt, so this is -// what promotes one agent's fresh refresh to the whole fleet instead of -// stranding everyone else on the grant it just rotated; +// - the private copy wins only when it is complete and usable +// (claudeBridgePromotable: access and refresh tokens present, an expiresAt +// recorded, and claude.TokensUsable) and either the shared credential is +// not usable or the private access token expires LATER than the shared +// one. A refresh mints a later expiresAt, so this is what promotes one +// agent's fresh refresh to the whole fleet instead of stranding everyone +// else on the grant it just rotated. A failed refresh leaves an +// empty-token, expiresAt-0 file; it never qualifies; // - otherwise the shared file stays as it is (ties keep the shared file, so // a private copy identical to it is simply moved aside). // +// Who reads the private file. The hive server runs as its own uid (dev) and +// cannot open another uid's 0600 file directly. It CAN run a helper as that +// uid: the same su-exec seam seedClaudeSessionForAgent uses to adopt an +// agent-owned .claude.json (runAsAgentUser, gated to the hive-launch group). +// So the judgement happens in-process, as the file's owner, with no root +// helper and no widening of the private file's mode. When neither read works +// (no su-exec on this deployment) the repair fails SAFE: the private file is +// left exactly where it is, the agent keeps running on the token it already +// had, and one ERROR per agent and file version says so. +// // The private file is renamed to .credentials.json.stale- in // the agent's own .claude directory and keeps its owner and mode; when it is // promoted, the shared file it replaces is preserved the same way beside the -// shared credential. Both names fall outside sharedCredentialBases, so the -// permissions watcher never opens them up to the group. +// shared credential. // // The repair never acts on a link path that resolves to the shared credential // itself (an agent whose whole ~/.claude is still the legacy directory symlink @@ -76,13 +91,16 @@ const claudeCredentialStaleInfix = ".stale-" // reaches it through the node group. const claudeSharedCredentialMode = 0o660 -// Test seams. chownClaudeCredential applies the previous shared file's owner -// to the promoted replacement; it is a seam so tests run without root or real -// agent uids. claudeBridgeNow stamps the moved-aside file names. -var ( - chownClaudeCredential = os.Chown - claudeBridgeNow = time.Now -) +// claudeBridgeStaleNameAttempts bounds the search for a free +// .stale-[-N] name beside a credential. +const claudeBridgeStaleNameAttempts = 100 + +// claudeBridgeMaxCredentialBytes caps a credential file the repair will read. +// A real credentials file is well under 4 KiB. +const claudeBridgeMaxCredentialBytes = 64 << 10 + +// claudeBridgeNow stamps the moved-aside file names; a seam for tests. +var claudeBridgeNow = time.Now // claudeBridgeMu serializes repairs: the launch path, the watchdog and the // sign-in hook can all reach the same agent's link. It is never held while @@ -144,20 +162,39 @@ func claudeCredentialBridgeBroken(agentName string, uid int, backend string) boo // errClaudeBridgeLinkIsTarget: the agent's credential path IS the shared // credential, so there is no private copy to move aside and no link to -// restore. On a live v6 hive this was an agent whose whole ~/.claude was still -// the legacy directory symlink to the shared /data/home/.claude (the layout -// setupPerAgentCLIHome retires at launch, never reached for an agent that is -// paused or stopped). The repair then renamed the SHARED login aside and, -// for a token-less file, linked .credentials.json to itself, logging every -// agent out. +// restore (a legacy whole-directory ~/.claude symlink). Moving that file aside +// would strand the fleet. var errClaudeBridgeLinkIsTarget = errors.New("credential link path resolves to the shared credential it would link to") +// errClaudeBridgeAgentUnreadable: the agent's private credential exists but +// neither this process nor a helper running as the file's owner could read +// it. Its token cannot be judged, so the repair leaves it exactly where it is. +var errClaudeBridgeAgentUnreadable = errors.New("agent's private credential cannot be read") + +// errClaudeBridgeSourceChanged: the file renamed aside is not the regular +// file that was inspected a moment earlier. It is put back and not read. +var errClaudeBridgeSourceChanged = errors.New("private credential changed under the repair") + +// errClaudeBridgeNotPromotable: promoteClaudeCredential re-checks the bytes it +// is about to write and refuses an incomplete or unusable credential. +var errClaudeBridgeNotPromotable = errors.New("refusing to promote an incomplete or unusable credential over the shared login") + // claudeBridgeLegacyWarned holds the credential links the legacy-layout skip // has already been logged for, so the watchdog, which walks every managed // agent each cycle, logs it once per agent instead of every cycle. Keyed by // the link path, which names the agent. var claudeBridgeLegacyWarned sync.Map +// claudeBridgeUnreadableWarned holds, per credential link, the modification +// time of the unreadable private file last logged, so the poll logs once per +// agent and file version instead of every cycle. A new refresh (a new mtime) +// is logged again. +var claudeBridgeUnreadableWarned sync.Map + +// claudeBridgeUnsafeWarned holds the shared credential paths whose location +// sharedfile.Replace refused, so the refusal is logged once, not per cycle. +var claudeBridgeUnsafeWarned sync.Map + // claudeBridgeLinkIsTarget reports whether link names the shared credential // itself: the same path, or a file of the same name in a directory that // resolves to the target's directory (a legacy whole-directory ~/.claude @@ -193,10 +230,20 @@ func claudeBridgeLinkIsTarget(link, target string) (bool, error) { return os.SameFile(li, ti), nil } +// claudeBridgePromotable reports whether t may replace the shared login: a +// complete credential (access token, refresh token and a recorded expiresAt) +// that is usable now. A failed refresh leaves an empty-token, expiresAt-0 +// file, which never qualifies; neither does a token with no recorded expiry, +// which cannot be judged newer than anything. +func claudeBridgePromotable(t *claude.OAuthTokens, now time.Time) bool { + return t != nil && t.AccessToken != "" && t.RefreshToken != "" && t.ExpiresAt > 0 && + claude.TokensUsable(t, now) +} + // preferAgentClaudeCredential decides which credential the shared file keeps. // See the file comment for the rule. func preferAgentClaudeCredential(agent, shared *claude.OAuthTokens, now time.Time) bool { - if !claude.TokensUsable(agent, now) { + if !claudeBridgePromotable(agent, now) { return false } if !claude.TokensUsable(shared, now) { @@ -205,13 +252,64 @@ func preferAgentClaudeCredential(agent, shared *claude.OAuthTokens, now time.Tim return agent.ExpiresAt > shared.ExpiresAt } +// readClaudeCredentialAsOwner reads an agent's private credential: directly +// when this process may, otherwise through su-exec as the file's owner (the +// per-UID case: an agent-owned 0600 file and a server running as dev). +// claudeSessionOwnerSpec fails closed — a symlink, a root-owned or an +// oversized file never gets a helper — so a planted link at the credential +// path cannot make the helper read something else. +func readClaudeCredentialAsOwner(path string) ([]byte, error) { + data, err := sharedfile.ReadNoFollow(path, claudeBridgeMaxCredentialBytes) + if err == nil || !errors.Is(err, fs.ErrPermission) { + return data, err + } + spec, serr := claudeSessionOwnerSpec(path) + if serr != nil { + return nil, errors.Join(err, serr) + } + owned, oerr := runAsAgentUser(spec, nil, "cat", path) + if oerr != nil { + return nil, errors.Join(err, oerr) + } + if len(owned) > claudeBridgeMaxCredentialBytes { + return nil, fmt.Errorf("%w: %s returned %d bytes as owner", sharedfile.ErrTooLarge, path, len(owned)) + } + return owned, nil +} + +// canonicalClaudeCredential decodes data as a credentials file and re-encodes +// only its typed token block. What reaches the shared file is this encoding, +// never the bytes read from an agent's directory: a file that is not a +// credential in the shape claude.Credentials describes is not promotable. +func canonicalClaudeCredential(data []byte, now time.Time) ([]byte, *claude.OAuthTokens, error) { + if len(data) > claudeBridgeMaxCredentialBytes { + return nil, nil, errClaudeBridgeNotPromotable + } + tok := claude.ParseTokens(data) + if !claudeBridgePromotable(tok, now) { + return nil, nil, errClaudeBridgeNotPromotable + } + out, err := json.Marshal(claude.Credentials{ClaudeAIOAuth: tok}) + if err != nil { + return nil, nil, errClaudeBridgeNotPromotable + } + return out, tok, nil +} + // repairClaudeCredentialBridge re-bridges one agent's credential when a regular // file has replaced the symlink. It returns nil, nil when there is nothing to // repair (a symlink, an absent link, a directory, which is not this failure, // or a link or target directory that cannot be resolved). It returns an error // wrapping errClaudeBridgeLinkIsTarget, and touches nothing, when the link -// path resolves to the shared credential itself. -func (m *Manager) repairClaudeCredentialBridge(agentName, link, target string) (*claudeBridgeRepair, error) { +// path resolves to the shared credential itself, and, unless trigger is +// claudeBridgeTriggerSignIn, one wrapping errClaudeBridgeAgentUnreadable, +// again touching nothing, when the private file cannot be read. +// +// The sign-in trigger is the exception: a dashboard sign-in has just written +// a new shared login, newer than any private copy by construction, so on that +// trigger an unreadable private file is still moved aside (unread) and the +// agent relinked to the shared login. +func (m *Manager) repairClaudeCredentialBridge(agentName, link, target, trigger string) (*claudeBridgeRepair, error) { claudeBridgeMu.Lock() defer claudeBridgeMu.Unlock() @@ -226,8 +324,6 @@ func (m *Manager) repairClaudeCredentialBridge(agentName, link, target string) ( // symlink restored afterwards would point at its own path. same, err := claudeBridgeLinkIsTarget(link, target) if err != nil { - // Nothing safe to judge without both directories; never move a file - // aside on a guess. m.logger.Debug("claude credential bridge: cannot resolve the link or shared directory; not repairing", "agent", agentName, "link", link, "target", target, "error", err) return nil, nil @@ -235,26 +331,52 @@ func (m *Manager) repairClaudeCredentialBridge(agentName, link, target string) ( if same { return nil, fmt.Errorf("%w: %s", errClaudeBridgeLinkIsTarget, link) } + + // Never move aside a file whose token cannot be judged: it may be the + // only fresh token in the fleet. + if _, rerr := readClaudeCredentialAsOwner(link); rerr != nil { + if errors.Is(rerr, fs.ErrNotExist) { + // Gone since the Lstat: the CLI is mid-rename. Next cycle. + return nil, nil + } + if trigger == claudeBridgeTriggerSignIn && claude.HasUsableToken(target) { + return m.relinkUnreadableAfterSignIn(agentName, link, target) + } + return nil, fmt.Errorf("%w: %w", errClaudeBridgeAgentUnreadable, rerr) + } now := claudeBridgeNow() - // Move the private file aside FIRST. After the rename the CLI can no - // longer change what is being judged (its next write lands at link), so - // the decision below works on a stable snapshot. + // Move the private file aside FIRST, then judge the moved file: after the + // rename the CLI can no longer change what is being judged (its next + // write lands at link), so the decision works on a stable snapshot. staleAgent, err := moveAsideClaudeCredential(link, now) if err != nil { return nil, fmt.Errorf("move private credential aside: %w", err) } + agentData, rerr := readClaudeCredentialAsOwner(staleAgent) + if rerr != nil { + // The file renamed is not the one read above (the CLI replaced it in + // between) and this one cannot be read either. Put it back unless the + // CLI has already written yet another file there. + if _, lerr := os.Lstat(link); errors.Is(lerr, fs.ErrNotExist) { + if berr := os.Rename(staleAgent, link); berr != nil { + m.logger.Error("claude credential bridge: an unreadable private credential could not be put back; it is kept at the stale path", + "agent", agentName, "stale", staleAgent, "error", berr) + } + } + return nil, fmt.Errorf("%w: %w", errClaudeBridgeAgentUnreadable, rerr) + } rep := &claudeBridgeRepair{agent: agentName, kept: "shared", staleAgentPath: staleAgent} - if preferAgentClaudeCredential(claude.LoadTokens(staleAgent), claude.LoadTokens(target), now) { - staleShared, perr := promoteClaudeCredential(staleAgent, target, now) + if preferAgentClaudeCredential(claude.ParseTokens(agentData), claude.LoadTokens(target), now) { + staleShared, perr := promoteClaudeCredential(agentData, target, now) if perr != nil { // Put the private file back: the agent's copy was the only usable // token, and linking the agent to the older shared one would // strand it. The next cycle tries again. - if rerr := os.Rename(staleAgent, link); rerr != nil { + if berr := os.Rename(staleAgent, link); berr != nil { m.logger.Error("claude credential bridge: promotion failed and the private credential could not be restored; it is kept at the stale path", - "agent", agentName, "stale", staleAgent, "error", perr, "restore_error", rerr) + "agent", agentName, "stale", staleAgent, "error", perr, "restore_error", berr) } return nil, fmt.Errorf("promote private credential to shared: %w", perr) } @@ -270,68 +392,115 @@ func (m *Manager) repairClaudeCredentialBridge(agentName, link, target string) ( return rep, nil } -// moveAsideClaudeCredential renames path to a unique -// .stale-[-N] beside it and returns the new path. It skips -// any name that already exists. -func moveAsideClaudeCredential(path string, now time.Time) (string, error) { +// claudeBridgeStaleName returns a free .stale-[-N] name +// beside path, or "" when none is free. exists reports whether a candidate is +// taken; callers that create the file atomically pass a function that tries +// the create and reports EEXIST. +func claudeBridgeStaleName(path string, now time.Time, exists func(string) (bool, error)) (string, error) { base := path + claudeCredentialStaleInfix + now.UTC().Format("20060102T150405Z") - for i := 0; i < 100; i++ { + for i := 0; i < claudeBridgeStaleNameAttempts; i++ { dst := base if i > 0 { dst = fmt.Sprintf("%s-%d", base, i) } - if _, err := os.Lstat(dst); err == nil { - continue - } else if !errors.Is(err, fs.ErrNotExist) { + taken, err := exists(dst) + if err != nil { return "", err } - // A rename keeps the file's owner and mode: the private copy stays - // readable only by the agent that wrote it. - if err := os.Rename(path, dst); err != nil { - return "", err + if !taken { + return dst, nil } - return dst, nil } return "", fmt.Errorf("no free stale name for %s", path) } -// promoteClaudeCredential replaces the shared credential with the bytes of src, -// atomically and at claudeSharedCredentialMode, owned like the file it -// replaces. The replaced file is preserved beside it first; its path is -// returned ("" when there was no shared file). -func promoteClaudeCredential(src, target string, now time.Time) (string, error) { - data, err := os.ReadFile(src) +// moveAsideClaudeCredential renames path to a unique stale name beside it and +// returns the new path. A rename keeps the file's owner and mode: the private +// copy stays readable only by the agent that wrote it. The file is identified +// before the rename and checked after it (same device and inode, still a +// regular file); anything else is put back and reported as +// errClaudeBridgeSourceChanged, unread. +func moveAsideClaudeCredential(path string, now time.Time) (string, error) { + before, err := os.Lstat(path) if err != nil { return "", err } - uid, gid := -1, -1 - staleShared := "" - if info, err := os.Stat(target); err == nil { - if st, ok := info.Sys().(*syscall.Stat_t); ok { - uid, gid = int(st.Uid), int(st.Gid) + if !before.Mode().IsRegular() { + return "", fmt.Errorf("%w: %s is %s", errClaudeBridgeSourceChanged, path, before.Mode()) + } + dst, err := claudeBridgeStaleName(path, now, func(dst string) (bool, error) { + _, lerr := os.Lstat(dst) + if lerr == nil { + return true, nil } - staleShared, err = preserveClaudeCredential(target, now) - if err != nil { - return "", fmt.Errorf("preserve shared credential: %w", err) + if errors.Is(lerr, fs.ErrNotExist) { + return false, nil } + return false, lerr + }) + if err != nil { + return "", err } - tmp := target + ".tmp-rebridge" - _ = os.Remove(tmp) - if err := os.WriteFile(tmp, data, claudeSharedCredentialMode); err != nil { - return staleShared, err + if err := os.Rename(path, dst); err != nil { + return "", err } - // WriteFile honours the umask; assert the mode explicitly. - if err := os.Chmod(tmp, claudeSharedCredentialMode); err != nil { - _ = os.Remove(tmp) - return staleShared, err + after, err := os.Lstat(dst) + if err != nil { + return "", err } - if uid >= 0 || gid >= 0 { - // Best-effort: an unprivileged hive cannot chown, and the file it - // wrote is already group-readable by mode. - _ = chownClaudeCredential(tmp, uid, gid) + if !after.Mode().IsRegular() || !os.SameFile(before, after) { + if _, lerr := os.Lstat(path); errors.Is(lerr, fs.ErrNotExist) { + _ = os.Rename(dst, path) + } + return "", fmt.Errorf("%w: %s", errClaudeBridgeSourceChanged, path) } - if err := os.Rename(tmp, target); err != nil { - _ = os.Remove(tmp) + return dst, nil +} + +// relinkUnreadableAfterSignIn is the sign-in exception described on +// repairClaudeCredentialBridge: the private file is moved aside without being +// read and the agent is relinked to the shared login just written. +func (m *Manager) relinkUnreadableAfterSignIn(agentName, link, target string) (*claudeBridgeRepair, error) { + staleAgent, err := moveAsideClaudeCredential(link, claudeBridgeNow()) + if err != nil { + return nil, fmt.Errorf("move private credential aside: %w", err) + } + rep := &claudeBridgeRepair{agent: agentName, kept: "shared", staleAgentPath: staleAgent} + if err := os.Symlink(target, link); err != nil { + return rep, fmt.Errorf("restore credential symlink: %w", err) + } + return rep, nil +} + +// promoteClaudeCredential replaces the shared credential with the token block +// decoded from data, atomically and at claudeSharedCredentialMode, owned like +// the file it replaces. The replaced file is preserved beside it first; its +// path is returned ("" when there was no shared file). The bytes are decoded +// and re-checked here (canonicalClaudeCredential), so a credential that +// changed after it was judged can never put an incomplete token over the +// shared login, and nothing but a typed token block is ever written. The +// write itself is sharedfile.Replace: a random O_EXCL|O_NOFOLLOW temporary, +// metadata by descriptor, and a refusal (sharedfile.ErrUnsafeLocation) when +// the shared directory or file is not what it should be. +func promoteClaudeCredential(data []byte, target string, now time.Time) (string, error) { + canonical, _, err := canonicalClaudeCredential(data, now) + if err != nil { + return "", err + } + // Refuse before the backup copy, not only at the write: the copy lands in + // the same directory, and the directory is part of what is being checked. + info, err := sharedfile.CheckLocation(target) + if err != nil { + return "", err + } + staleShared := "" + if info != nil { + staleShared, err = preserveClaudeCredential(target, now) + if err != nil { + return "", fmt.Errorf("preserve shared credential: %w", err) + } + } + if err := sharedfile.Replace(target, canonical, claudeSharedCredentialMode, sharedfile.Options{PreserveOwner: true}); err != nil { return staleShared, err } return staleShared, nil @@ -340,43 +509,41 @@ func promoteClaudeCredential(src, target string, now time.Time) (string, error) // preserveClaudeCredential copies path to a unique stale name beside it, // leaving path itself in place (agents keep reading it until the rename that // replaces it). The copy is 0600: it is an archive, not a live credential. +// Neither the read nor the create follows a symlink. func preserveClaudeCredential(path string, now time.Time) (string, error) { - data, err := os.ReadFile(path) + data, err := sharedfile.ReadNoFollow(path, claudeBridgeMaxCredentialBytes) if err != nil { return "", err } - base := path + claudeCredentialStaleInfix + now.UTC().Format("20060102T150405Z") - for i := 0; i < 100; i++ { - dst := base - if i > 0 { - dst = fmt.Sprintf("%s-%d", base, i) - } - f, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600) - if errors.Is(err, fs.ErrExist) { - continue + var f *os.File + dst, err := claudeBridgeStaleName(path, now, func(dst string) (bool, error) { + var oerr error + f, oerr = os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_EXCL|syscall.O_NOFOLLOW, 0o600) + if errors.Is(oerr, fs.ErrExist) { + return true, nil } - if err != nil { - return "", err - } - _, werr := f.Write(data) - cerr := f.Close() - if werr != nil || cerr != nil { - _ = os.Remove(dst) - return "", errors.Join(werr, cerr) - } - return dst, nil + return false, oerr + }) + if err != nil { + return "", err } - return "", fmt.Errorf("no free stale name for %s", path) + _, werr := f.Write(data) + cerr := f.Close() + if werr != nil || cerr != nil { + _ = os.Remove(dst) + return "", errors.Join(werr, cerr) + } + return dst, nil } // rebridgeClaudeCredential runs the repair for one agent and records the -// outcome. It is the single entry point the launch path, the watchdog and the -// sign-in hook share, so all three log and audit identically. +// outcome. It is the single entry point the launch path, the watchdog poll and +// the sign-in hook share, so all three log and audit identically. func (m *Manager) rebridgeClaudeCredential(agentName, link, target, trigger string) *claudeBridgeRepair { - rep, err := m.repairClaudeCredentialBridge(agentName, link, target) + rep, err := m.repairClaudeCredentialBridge(agentName, link, target, trigger) if errors.Is(err, errClaudeBridgeLinkIsTarget) { // The legacy layout: skipped, and said once per agent rather than on - // every watchdog cycle. Not audited (nothing changed). + // every cycle. Not audited (nothing changed). if _, seen := claudeBridgeLegacyWarned.LoadOrStore(link, true); !seen { m.logger.Warn("claude credential bridge: this agent's .claude directory resolves to the shared login directory (a legacy whole-directory symlink), so its credential path IS the shared login; not re-bridging. The agent's next launch replaces the directory link with a per-agent directory", "agent", agentName, "trigger", trigger, "dir", filepath.Dir(link), "shared", target) @@ -386,11 +553,24 @@ func (m *Manager) rebridgeClaudeCredential(agentName, link, target, trigger stri // Any other outcome clears the mark, so a legacy layout that shows up // again later is logged again. claudeBridgeLegacyWarned.Delete(link) + if errors.Is(err, errClaudeBridgeAgentUnreadable) { + m.logClaudeBridgeUnreadable(agentName, link, trigger, err) + return nil + } + claudeBridgeUnreadableWarned.Delete(link) + if errors.Is(err, sharedfile.ErrUnsafeLocation) { + // The shared location is not what the repair expects; nothing was + // written and the agent keeps its private copy. Said once per target. + if _, seen := claudeBridgeUnsafeWarned.LoadOrStore(target, true); !seen { + m.logger.Error("claude credential bridge: the shared credential location was refused and nothing was written; the agent keeps its private copy", + "agent", agentName, "trigger", trigger, "shared", target, "error", err) + } + return nil + } + claudeBridgeUnsafeWarned.Delete(target) if err != nil { - // Logged, not audited: a repair that keeps failing (an unprivileged - // hive that cannot enter the agent's 0700 home) would otherwise add an - // audit row every watchdog cycle. The dashboard keeps showing the - // agent's bridge as broken, which is the standing signal. + // Logged, not audited: a repair that keeps failing would otherwise add + // an audit row every cycle. m.logger.Warn("claude credential bridge: repair failed", "agent", agentName, "link", link, "trigger", trigger, "error", err) return nil @@ -415,6 +595,21 @@ func (m *Manager) rebridgeClaudeCredential(agentName, link, target, trigger stri return rep } +// logClaudeBridgeUnreadable says, once per agent and private-file version, +// that the repair left an agent on a private credential it could not read. +// Not audited: nothing changed. +func (m *Manager) logClaudeBridgeUnreadable(agentName, link, trigger string, err error) { + var stamp int64 + if info, lerr := os.Lstat(link); lerr == nil { + stamp = info.ModTime().UnixNano() + } + if prev, loaded := claudeBridgeUnreadableWarned.Swap(link, stamp); loaded && prev == stamp { + return + } + m.logger.Error("claude credential bridge: this agent's private credential could not be read, directly or as its owner through su-exec, so its token cannot be judged; leaving the agent on its private copy (not moved aside, not re-linked). Other agents will not see this agent's token refresh until it is readable", + "agent", agentName, "path", link, "trigger", trigger, "error", err) +} + // RepairClaudeCredentialBridges checks every per-agent Claude credential // bridge and repairs each one a regular file has replaced. It returns the // names of the agents it re-bridged, sorted. Agents are snapshotted under m.mu @@ -448,3 +643,13 @@ func (m *Manager) RepairClaudeCredentialBridges(trigger string) []string { } return repaired } + +// rebridgeClaudeCredentialsAfterSignIn runs from ReloadClaudeToken: a sign-in +// wrote the shared file, and agents whose link a refresh replaced would never +// see it. Relink them now rather than on the next poll. A logout leaves +// nothing usable, so nothing is moved. +func (m *Manager) rebridgeClaudeCredentialsAfterSignIn() { + if claude.HasUsableToken(sharedClaudeBridgeTarget()) { + m.RepairClaudeCredentialBridges(claudeBridgeTriggerSignIn) + } +} diff --git a/src/crux/agent/claude_credential_bridge_internal_test.go b/src/crux/agent/claude_credential_bridge_internal_test.go index 9d4d9ce5e3..4c705474c2 100644 --- a/src/crux/agent/claude_credential_bridge_internal_test.go +++ b/src/crux/agent/claude_credential_bridge_internal_test.go @@ -3,87 +3,81 @@ package agent import ( "bytes" "errors" + "fmt" "log/slog" "os" "path/filepath" "strings" - "syscall" "testing" "time" - "github.com/hivecommons/hive/crux/claude" "github.com/hivecommons/hive/crux/config" ) -// TestRepairClaudeCredentialBridge_PromotionKeepsSharedOwner: the promoted -// shared credential is written by the hive process, so without a chown it would -// belong to whoever runs hive. It must keep the owner and group of the file it -// replaces. The chown is a seam so this runs without root or real agent uids. -func TestRepairClaudeCredentialBridge_PromotionKeepsSharedOwner(t *testing.T) { - shared := withSharedAgentHome(t) - m, _ := testManagerWithSink(t) - now := time.Now() - link, target := brokenBridgeFixture(t, shared, "coder", - fakeClaudeCredential("fake-old-access", now.Add(-time.Hour), "fake-old-refresh"), - fakeClaudeCredential("fake-new-access", now.Add(8*time.Hour), "fake-new-refresh")) - info, err := os.Stat(target) - if err != nil { +// Fixture helpers shared by the tests in this file (the v6 #17253 suite). +// fakeClaudeCredential is a credentials file in the on-disk shape Claude Code +// writes, with fake token strings. +func fakeClaudeCredential(access string, expiresAt time.Time, refresh string) string { + return fmt.Sprintf(`{"claudeAiOauth":{"accessToken":%q,"refreshToken":%q,"expiresAt":%d,"scopes":["user:inference"]}}`, + access, refresh, expiresAt.UnixMilli()) +} + +// brokenBridgeFixture lays out a per-agent home whose credential symlink a +// token refresh has replaced: the shared credential holds sharedJSON and the +// agent's .claude/.credentials.json is a REGULAR file holding agentJSON, mode +// 0600 as the CLI writes it. Returns (link, target). +func brokenBridgeFixture(t *testing.T, shared, agentName, sharedJSON, agentJSON string) (string, string) { + t.Helper() + target := filepath.Join(shared, ".claude", ".credentials.json") + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { t.Fatal(err) } - st := info.Sys().(*syscall.Stat_t) - - type chownCall struct { - path string - uid, gid int + if sharedJSON != "" { + if err := os.WriteFile(target, []byte(sharedJSON), 0o660); err != nil { + t.Fatal(err) + } } - var calls []chownCall - orig := chownClaudeCredential - chownClaudeCredential = func(path string, uid, gid int) error { - calls = append(calls, chownCall{path, uid, gid}) - return nil + link := filepath.Join(shared, "agents", agentName, ".claude", ".credentials.json") + if err := os.MkdirAll(filepath.Dir(link), 0o700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(link, []byte(agentJSON), 0o600); err != nil { + t.Fatal(err) } - t.Cleanup(func() { chownClaudeCredential = orig }) + return link, target +} - rep := m.rebridgeClaudeCredential("coder", link, target, claudeBridgeTriggerWatchdog) - if rep == nil || rep.kept != "agent" { - t.Fatalf("repair = %+v, want kept=agent", rep) +func assertBridgeRestored(t *testing.T, link, target string) { + t.Helper() + info, err := os.Lstat(link) + if err != nil { + t.Fatalf("credential link missing after repair: %v", err) } - if len(calls) != 1 || calls[0].uid != int(st.Uid) || calls[0].gid != int(st.Gid) { - t.Fatalf("chown calls = %+v, want one call with the previous shared owner %d:%d", calls, st.Uid, st.Gid) + if info.Mode()&os.ModeSymlink == 0 { + t.Fatalf("%s is still a regular file (mode %v); the bridge was not restored", link, info.Mode()) } - if filepath.Dir(calls[0].path) != filepath.Dir(target) { - t.Errorf("chown applied to %q, want the replacement written beside %q", calls[0].path, target) + if got, _ := os.Readlink(link); got != target { + t.Fatalf("credential link -> %q, want %q", got, target) } } -// TestRepairClaudeCredentialBridge_StaleNamesNeverCollide: two repairs in the -// same second must not overwrite the first moved-aside file. -func TestRepairClaudeCredentialBridge_StaleNamesNeverCollide(t *testing.T) { - shared := withSharedAgentHome(t) - m, _ := testManagerWithSink(t) - fixed := time.Date(2026, 11, 7, 14, 54, 0, 0, time.UTC) - orig := claudeBridgeNow - claudeBridgeNow = func() time.Time { return fixed } - t.Cleanup(func() { claudeBridgeNow = orig }) - - sharedJSON := fakeClaudeCredential("fake-shared-access", time.Now().Add(8*time.Hour), "fake-shared-refresh") - link, target := brokenBridgeFixture(t, shared, "tester", sharedJSON, - fakeClaudeCredential("fake-first-access", time.Now().Add(time.Hour), "fake-first-refresh")) - m.rebridgeClaudeCredential("tester", link, target, claudeBridgeTriggerWatchdog) - - // The CLI refreshes again within the same second. - if err := os.Remove(link); err != nil { - t.Fatal(err) - } - if err := os.WriteFile(link, []byte(fakeClaudeCredential("fake-second-access", time.Now().Add(time.Hour), "fake-second-refresh")), 0o600); err != nil { +// staleFiles lists the moved-aside credentials in dir. +func staleFiles(t *testing.T, dir string) []string { + t.Helper() + matches, err := filepath.Glob(filepath.Join(dir, ".credentials.json.stale-*")) + if err != nil { t.Fatal(err) } - m.rebridgeClaudeCredential("tester", link, target, claudeBridgeTriggerWatchdog) + return matches +} - if got := staleFiles(t, filepath.Dir(link)); len(got) != 2 { - t.Fatalf("want two distinct moved-aside files, got %v", got) +func readFileString(t *testing.T, path string) string { + t.Helper() + b, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) } - assertBridgeRestored(t, link, target) + return string(b) } // TestAgentSnapshot_ReportsCredentialBridgeBroken: the status the dashboard @@ -117,32 +111,6 @@ func TestAgentSnapshot_ReportsCredentialBridgeBroken(t *testing.T) { } } -// TestPreferAgentClaudeCredential pins the "newer AND usable" rule. -func TestPreferAgentClaudeCredential(t *testing.T) { - now := time.Now() - tok := func(exp time.Duration, refresh string) *claude.OAuthTokens { - return &claude.OAuthTokens{AccessToken: "fake", RefreshToken: refresh, ExpiresAt: now.Add(exp).UnixMilli()} - } - cases := []struct { - name string - agent, shared *claude.OAuthTokens - want bool - }{ - {"agent newer and live", tok(8*time.Hour, "r"), tok(time.Hour, "r"), true}, - {"shared newer", tok(time.Hour, "r"), tok(8*time.Hour, "r"), false}, - {"tie keeps shared", tok(time.Hour, "r"), tok(time.Hour, "r"), false}, - {"agent expired but refreshable, shared absent", tok(-time.Hour, "r"), nil, true}, - {"agent unusable, shared absent", tok(-time.Hour, ""), nil, false}, - {"agent usable, shared unusable", tok(-time.Minute, "r"), tok(-2*time.Hour, ""), true}, - {"agent absent", nil, tok(time.Hour, "r"), false}, - } - for _, c := range cases { - if got := preferAgentClaudeCredential(c.agent, c.shared, now); got != c.want { - t.Errorf("%s: got %v, want %v", c.name, got, c.want) - } - } -} - // legacyDirLinkFixture lays out the pre-#17031 layout that the launch path // retires but the credential watchdog still walks for a paused or stopped // agent: the agent's whole ~/.claude is a directory symlink to the shared @@ -213,7 +181,7 @@ func TestRepairClaudeCredentialBridge_LegacyDirLinkUsableSharedUntouched(t *test sharedJSON := fakeClaudeCredential("fake-shared-access", time.Now().Add(8*time.Hour), "fake-shared-refresh") link, target, before := legacyDirLinkFixture(t, shared, "reviewer", 2005, sharedJSON) - rep, err := m.repairClaudeCredentialBridge("reviewer", link, target) + rep, err := m.repairClaudeCredentialBridge("reviewer", link, target, claudeBridgeTriggerWatchdog) if rep != nil || !errors.Is(err, errClaudeBridgeLinkIsTarget) { t.Fatalf("repair = (%+v, %v), want (nil, errClaudeBridgeLinkIsTarget)", rep, err) } @@ -237,7 +205,7 @@ func TestRepairClaudeCredentialBridge_LegacyDirLinkTokenlessSharedNotSelfLinked( const sharedJSON = `{}` link, target, before := legacyDirLinkFixture(t, shared, "scanner", 2003, sharedJSON) - if _, err := m.repairClaudeCredentialBridge("scanner", link, target); !errors.Is(err, errClaudeBridgeLinkIsTarget) { + if _, err := m.repairClaudeCredentialBridge("scanner", link, target, claudeBridgeTriggerWatchdog); !errors.Is(err, errClaudeBridgeLinkIsTarget) { t.Fatalf("repair error = %v, want errClaudeBridgeLinkIsTarget", err) } assertSharedCredentialUntouched(t, target, before, sharedJSON) @@ -291,7 +259,7 @@ func TestRepairClaudeCredentialBridge_RefusesLinkEqualToTarget(t *testing.T) { t.Fatal(err) } - rep, err := m.repairClaudeCredentialBridge("coder", target, target) + rep, err := m.repairClaudeCredentialBridge("coder", target, target, claudeBridgeTriggerWatchdog) if rep != nil || !errors.Is(err, errClaudeBridgeLinkIsTarget) { t.Fatalf("repair = (%+v, %v), want (nil, errClaudeBridgeLinkIsTarget)", rep, err) } diff --git a/src/crux/agent/claude_credential_bridge_test.go b/src/crux/agent/claude_credential_bridge_test.go index e7bd0093f2..67928f5207 100644 --- a/src/crux/agent/claude_credential_bridge_test.go +++ b/src/crux/agent/claude_credential_bridge_test.go @@ -1,267 +1,853 @@ package agent import ( + "bytes" "context" + "encoding/json" + "errors" "fmt" + "io/fs" + "log/slog" "os" "path/filepath" "strings" + "sync" "testing" "time" + "github.com/hivecommons/hive/crux/claude" "github.com/hivecommons/hive/crux/config" + "github.com/hivecommons/hive/crux/internal/testutil" + "github.com/hivecommons/hive/crux/sharedfile" ) -// claude_credential_bridge_test.go: a Claude Code token refresh writes a temp -// file and renames it over $HOME/.claude/.credentials.json, which replaces the -// per-agent symlink to the shared login with a private regular file. These -// tests drive the three entry points that must repair it (agent launch, the -// credential watchdog, and the dashboard sign-in reload) through the same -// public paths production uses. Every token is a fake JSON blob written into a -// t.TempDir(). - -// fakeClaudeCredential is a credentials file in the on-disk shape Claude Code -// writes, with fake token strings. -func fakeClaudeCredential(access string, expiresAt time.Time, refresh string) string { - return fmt.Sprintf(`{"claudeAiOauth":{"accessToken":%q,"refreshToken":%q,"expiresAt":%d,"scopes":["user:inference"]}}`, - access, refresh, expiresAt.UnixMilli()) +// Fake tokens. Every token string carries this sentinel so a test can prove +// no log line or audit field ever leaks token material. +const bridgeTestTokenSentinel = "SENTINEL-TOKEN-MATERIAL" + +func bridgeTestCreds(t *testing.T, tag string, expiresAt int64) []byte { + t.Helper() + data, err := json.Marshal(claude.Credentials{ClaudeAIOAuth: &claude.OAuthTokens{ + AccessToken: "acc-" + tag + "-" + bridgeTestTokenSentinel, + RefreshToken: "ref-" + tag + "-" + bridgeTestTokenSentinel, + ExpiresAt: expiresAt, + }}) + if err != nil { + t.Fatal(err) + } + return data } -// brokenBridgeFixture lays out a per-agent home whose credential symlink a -// token refresh has replaced: the shared credential holds sharedJSON and the -// agent's .claude/.credentials.json is a REGULAR file holding agentJSON, mode -// 0600 as the CLI writes it. Returns (link, target). -func brokenBridgeFixture(t *testing.T, shared, agentName, sharedJSON, agentJSON string) (string, string) { +// bridgeTestFixture provisions a per-agent home for "scanner" with the shared +// login in place and the credential symlink bridged, and returns the manager, +// the audit sink, the link path and the shared path. +func bridgeTestFixture(t *testing.T) (*Manager, *fakeAuditSink, *AgentProcess, string, string) { t.Helper() + shared := withSharedAgentHome(t) + m, sink := testManagerWithSink(t) + ap := &AgentProcess{Name: "scanner", UID: 1001, Config: config.AgentConfig{Backend: "claude"}} + m.mu.Lock() + m.agents["scanner"] = ap + m.mu.Unlock() target := filepath.Join(shared, ".claude", ".credentials.json") if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { t.Fatal(err) } - if sharedJSON != "" { - if err := os.WriteFile(target, []byte(sharedJSON), 0o660); err != nil { - t.Fatal(err) - } + m.setupInteractiveHome(ap, "claude") + link := filepath.Join(interactiveHomePath("scanner"), ".claude", ".credentials.json") + if got, err := os.Readlink(link); err != nil || got != target { + t.Fatalf("fixture: credential bridge not provisioned: target=%q err=%v", got, err) + } + return m, sink, ap, link, target +} + +// simulateClaudeRefresh does what Claude Code's credential store does on a +// token refresh: write a temp file in the agent's .claude and rename it over +// .credentials.json, then chmod 0600. The rename replaces the symlink. +func simulateClaudeRefresh(t *testing.T, link string, data []byte) { + t.Helper() + tmp := link + ".tmp.12345" + if err := os.WriteFile(tmp, data, 0o600); err != nil { + t.Fatal(err) } - link := filepath.Join(shared, "agents", agentName, ".claude", ".credentials.json") - if err := os.MkdirAll(filepath.Dir(link), 0o700); err != nil { + if err := os.Rename(tmp, link); err != nil { t.Fatal(err) } - if err := os.WriteFile(link, []byte(agentJSON), 0o600); err != nil { + if err := os.Chmod(link, 0o600); err != nil { t.Fatal(err) } - return link, target + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Fatalf("simulated refresh did not replace the link: info=%v err=%v", info, err) + } } -func assertBridgeRestored(t *testing.T, link, target string) { +func readTokensAt(t *testing.T, path string) *claude.OAuthTokens { t.Helper() - info, err := os.Lstat(link) - if err != nil { - t.Fatalf("credential link missing after repair: %v", err) - } - if info.Mode()&os.ModeSymlink == 0 { - t.Fatalf("%s is still a regular file (mode %v); the bridge was not restored", link, info.Mode()) - } - if got, _ := os.Readlink(link); got != target { - t.Fatalf("credential link -> %q, want %q", got, target) + tok := claude.LoadTokens(path) + if tok == nil { + t.Fatalf("no tokens at %s", path) } + return tok } -// staleFiles lists the moved-aside credentials in dir. -func staleFiles(t *testing.T, dir string) []string { +func staleFilesBeside(t *testing.T, path string) []string { t.Helper() - matches, err := filepath.Glob(filepath.Join(dir, ".credentials.json.stale-*")) + matches, err := filepath.Glob(path + claudeCredentialStaleInfix + "*") if err != nil { t.Fatal(err) } return matches } -func readFileString(t *testing.T, path string) string { +func assertLinkRestored(t *testing.T, link, target string) { t.Helper() - b, err := os.ReadFile(path) - if err != nil { - t.Fatal(err) + got, err := os.Readlink(link) + if err != nil || got != target { + t.Fatalf("credential link not restored: target=%q err=%v", got, err) } - return string(b) } -// TestSetupInteractiveHome_RebridgesCredentialReplacedByRefresh_KeepsNewerShared -// is the hive-wild-mole state: the agent's private copy is an older refresh -// (whose grant was later revoked) and the shared file holds a newer dashboard -// sign-in. Launch must move the private file aside, keep the shared file -// as it is, and restore the symlink. On the v6 tip bridgeHomeEntry refused -// to touch the real file, so the agent ran on the dead copy forever. -func TestSetupInteractiveHome_RebridgesCredentialReplacedByRefresh_KeepsNewerShared(t *testing.T) { - shared := withSharedAgentHome(t) - m, sink := testManagerWithSink(t) +// TestSetupInteractiveHome_RefreshReplacedLinkIsPublishedToSharedLogin is the +// reported failure (Discord, 2026-11-11): on a per-UID hive one agent's token +// refresh replaced its credential link, the refresh never reached the shared +// file, and the fleet logged out when the shared token expired. The refreshed +// token must reach the shared login and the agent must be re-linked, with +// memory untouched. +func TestSetupInteractiveHome_RefreshReplacedLinkIsPublishedToSharedLogin(t *testing.T) { + m, sink, ap, link, target := bridgeTestFixture(t) now := time.Now() - sharedJSON := fakeClaudeCredential("fake-signin-access", now.Add(8*time.Hour), "fake-signin-refresh") - agentJSON := fakeClaudeCredential("fake-refreshed-access", now.Add(time.Hour), "fake-refreshed-refresh") - link, target := brokenBridgeFixture(t, shared, "architect", sharedJSON, agentJSON) + if err := os.WriteFile(target, bridgeTestCreds(t, "old", now.Add(10*time.Minute).UnixMilli()), 0o660); err != nil { + t.Fatal(err) + } + memory := filepath.Join(filepath.Dir(link), "projects", "-data-agents-scanner", "memory", "MEMORY.md") + if err := os.MkdirAll(filepath.Dir(memory), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(memory, []byte("# mine"), 0o644); err != nil { + t.Fatal(err) + } + + fresh := bridgeTestCreds(t, "fresh", now.Add(8*time.Hour).UnixMilli()) + simulateClaudeRefresh(t, link, fresh) + // The defect: the shared file did not move. + if tok := readTokensAt(t, target); !strings.HasPrefix(tok.AccessToken, "acc-old-") { + t.Fatalf("precondition: shared file should still hold the old token, got %q", tok.AccessToken) + } - ap := &AgentProcess{Name: "architect", UID: 2001, Config: config.AgentConfig{Backend: "claude"}} m.setupInteractiveHome(ap, "claude") - assertBridgeRestored(t, link, target) - if got := readFileString(t, target); got != sharedJSON { - t.Errorf("shared credential changed although it was the newer one:\n got %s\nwant %s", got, sharedJSON) + if tok := readTokensAt(t, target); !strings.HasPrefix(tok.AccessToken, "acc-fresh-") { + t.Errorf("refresh not published to the shared login: shared access token = %q", tok.AccessToken) } - stale := staleFiles(t, filepath.Dir(link)) - if len(stale) != 1 { - t.Fatalf("want exactly one moved-aside private file, got %v", stale) + assertLinkRestored(t, link, target) + info, err := os.Stat(target) + if err != nil || info.Mode().Perm() != claudeSharedCredentialMode { + t.Errorf("shared credential mode = %v err=%v, want %#o", info.Mode(), err, claudeSharedCredentialMode) } - if got := readFileString(t, stale[0]); got != agentJSON { - t.Errorf("moved-aside file does not hold the private credential: %s", got) + // Private copy moved aside, not deleted, still owner-only. + stale := staleFilesBeside(t, link) + if len(stale) != 1 { + t.Fatalf("stale agent files = %v, want exactly one", stale) } if info, err := os.Stat(stale[0]); err != nil || info.Mode().Perm() != 0o600 { - t.Errorf("moved-aside private file must keep its 0600 mode: info=%v err=%v", info, err) + t.Errorf("stale private copy mode = %v err=%v, want 0600", info.Mode(), err) + } + // Old shared file preserved beside the shared credential. + if got := staleFilesBeside(t, target); len(got) != 1 || !bytes.Contains(mustRead(t, got[0]), []byte("acc-old-")) { + t.Errorf("replaced shared file not preserved: %v", got) + } + if got, err := os.ReadFile(memory); err != nil || string(got) != "# mine" { + t.Errorf("per-agent memory disturbed: %q err=%v", got, err) + } + // Audited, kept=agent, no token material. + rec := sink.find(AuditClaudeCredentialRebridged) + if rec == nil { + t.Fatal("no claude_credential_rebridged audit entry") } - if got := staleFiles(t, filepath.Dir(target)); len(got) != 0 { - t.Errorf("shared credential was not replaced, so nothing beside it should be moved aside: %v", got) + if rec.Agent != "scanner" || rec.Fields["kept"] != "agent" || rec.Fields["trigger"] != claudeBridgeTriggerLaunch { + t.Errorf("audit entry = %+v", rec) } - ev := sink.find("claude_credential_rebridged") - if ev == nil { - t.Fatal("no claude_credential_rebridged audit entry for the repair") + if strings.Contains(fmt.Sprint(rec.Fields), bridgeTestTokenSentinel) { + t.Error("audit fields carry token material") } - if ev.Agent != "architect" || ev.Fields["trigger"] != "launch" || ev.Fields["kept"] != "shared" { - t.Errorf("audit entry = %+v, want agent=architect trigger=launch kept=shared", ev) + // Idempotent: a second launch finds a healthy link and changes nothing. + m.setupInteractiveHome(ap, "claude") + if sink.count() != 1 { + t.Errorf("second launch audited again: %d entries", sink.count()) + } +} + +func mustRead(t *testing.T, path string) []byte { + t.Helper() + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + return data +} + +// TestRepairClaudeCredentialBridge_KeepsNewerShared: a dashboard sign-in after +// the refresh wrote a newer shared credential; the agent's stale private copy +// is moved aside and the agent re-linked, the shared file untouched. +func TestRepairClaudeCredentialBridge_KeepsNewerShared(t *testing.T) { + m, sink, _, link, target := bridgeTestFixture(t) + now := time.Now() + simulateClaudeRefresh(t, link, bridgeTestCreds(t, "agent", now.Add(1*time.Hour).UnixMilli())) + sharedBytes := bridgeTestCreds(t, "signin", now.Add(8*time.Hour).UnixMilli()) + if err := os.WriteFile(target, sharedBytes, 0o660); err != nil { + t.Fatal(err) } - for k, v := range ev.Fields { - if s, ok := v.(string); ok && strings.Contains(s, "fake-") { - t.Errorf("audit field %s carries token material: %q", k, s) + + rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog) + if rep == nil || rep.kept != "shared" || rep.staleSharedPath != "" { + t.Fatalf("repair = %+v, want kept=shared with no preserved shared file", rep) + } + if !bytes.Equal(mustRead(t, target), sharedBytes) { + t.Error("shared credential was rewritten") + } + assertLinkRestored(t, link, target) + if got := staleFilesBeside(t, link); len(got) != 1 || !bytes.Contains(mustRead(t, got[0]), []byte("acc-agent-")) { + t.Errorf("private copy not moved aside: %v", got) + } + if rec := sink.find(AuditClaudeCredentialRebridged); rec == nil || rec.Fields["kept"] != "shared" { + t.Errorf("audit = %+v", rec) + } +} + +// TestRepairClaudeCredentialBridge_FailedRefreshNeverWins: an agent whose +// refresh failed on a rotated grant writes an empty-token, expiresAt-0 file. +// It must never replace the shared login, usable or not. +func TestRepairClaudeCredentialBridge_FailedRefreshNeverWins(t *testing.T) { + for _, sharedUsable := range []bool{true, false} { + t.Run(fmt.Sprintf("sharedUsable=%v", sharedUsable), func(t *testing.T) { + m, _, _, link, target := bridgeTestFixture(t) + exp := time.Now().Add(-time.Hour).UnixMilli() + if sharedUsable { + exp = time.Now().Add(time.Hour).UnixMilli() + } + sharedBytes := bridgeTestCreds(t, "shared", exp) + if err := os.WriteFile(target, sharedBytes, 0o660); err != nil { + t.Fatal(err) + } + simulateClaudeRefresh(t, link, []byte(`{"claudeAiOauth":{"accessToken":"","refreshToken":"","expiresAt":0}}`)) + + rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog) + if rep == nil || rep.kept != "shared" { + t.Fatalf("repair = %+v, want kept=shared", rep) + } + if !bytes.Equal(mustRead(t, target), sharedBytes) { + t.Error("empty-token file replaced the shared login") + } + assertLinkRestored(t, link, target) + }) + } +} + +// TestRepairClaudeCredentialBridge_ReadsAgentOwnedFileAsOwner is the per-UID +// case: the private file is the agent's, 0600, and the hive process cannot +// open it. The read must go through the su-exec seam as the file's owner, and +// the fresh token must still be promoted. +func TestRepairClaudeCredentialBridge_ReadsAgentOwnedFileAsOwner(t *testing.T) { + if os.Getuid() == 0 { + t.Skip("root can read a mode-000 file; the permission fallback cannot be exercised") + } + m, _, _, link, target := bridgeTestFixture(t) + now := time.Now() + if err := os.WriteFile(target, bridgeTestCreds(t, "old", now.Add(10*time.Minute).UnixMilli()), 0o660); err != nil { + t.Fatal(err) + } + fresh := bridgeTestCreds(t, "fresh", now.Add(8*time.Hour).UnixMilli()) + simulateClaudeRefresh(t, link, fresh) + if err := os.Chmod(link, 0); err != nil { + t.Fatal(err) + } + + var mu sync.Mutex + var calls []string + old := runAsAgentUser + runAsAgentUser = func(spec string, stdin []byte, argv ...string) ([]byte, error) { + mu.Lock() + defer mu.Unlock() + calls = append(calls, spec+" "+strings.Join(argv, " ")) + if len(argv) != 2 || argv[0] != "cat" { + return nil, fmt.Errorf("unexpected argv %v", argv) + } + // Stand in for the owner: the file is ours, only its mode blocks us. + if err := os.Chmod(argv[1], 0o600); err != nil { + return nil, err } + defer func() { _ = os.Chmod(argv[1], 0) }() + return os.ReadFile(argv[1]) + } + t.Cleanup(func() { runAsAgentUser = old }) + + rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog) + if rep == nil || rep.kept != "agent" { + t.Fatalf("repair = %+v, want kept=agent", rep) + } + if !bytes.Equal(mustRead(t, target), fresh) { + t.Error("agent-owned refresh was not promoted to the shared login") + } + assertLinkRestored(t, link, target) + mu.Lock() + defer mu.Unlock() + wantSpec := fmt.Sprintf("%d:%d", os.Getuid(), os.Getgid()) + if len(calls) < 1 || !strings.HasPrefix(calls[0], wantSpec+" cat "+link) { + t.Errorf("su-exec calls = %v, want first read as %s of %s", calls, wantSpec, link) + } +} + +// TestRepairClaudeCredentialBridge_UnreadablePrivateLeftInPlace: when neither +// a direct read nor the owner helper can read the private file, nothing is +// moved or relinked (the agent keeps the token it has), nothing is audited, +// and the ERROR is logged once per file version. +func TestRepairClaudeCredentialBridge_UnreadablePrivateLeftInPlace(t *testing.T) { + if os.Getuid() == 0 { + t.Skip("root can read a mode-000 file; the permission fallback cannot be exercised") + } + m, sink, _, link, target := bridgeTestFixture(t) + var logBuf bytes.Buffer + m.logger = slog.New(slog.NewTextHandler(&logBuf, nil)) + sharedBytes := bridgeTestCreds(t, "shared", time.Now().Add(time.Hour).UnixMilli()) + if err := os.WriteFile(target, sharedBytes, 0o660); err != nil { + t.Fatal(err) + } + private := bridgeTestCreds(t, "private", time.Now().Add(8*time.Hour).UnixMilli()) + simulateClaudeRefresh(t, link, private) + if err := os.Chmod(link, 0); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chmod(link, 0o600) }) + old := runAsAgentUser + runAsAgentUser = func(string, []byte, ...string) ([]byte, error) { + return nil, errors.New("su-exec unavailable") + } + t.Cleanup(func() { runAsAgentUser = old }) + + for i := 0; i < 3; i++ { + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep != nil { + t.Fatalf("cycle %d: repair = %+v, want none", i, rep) + } + } + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Errorf("private file was moved or relinked: info=%v err=%v", info, err) + } + if got := staleFilesBeside(t, link); len(got) != 0 { + t.Errorf("stale files created: %v", got) + } + if !bytes.Equal(mustRead(t, target), sharedBytes) { + t.Error("shared credential changed") + } + if sink.count() != 0 { + t.Errorf("audited %d entries, want none (nothing changed)", sink.count()) + } + logs := logBuf.String() + if n := strings.Count(logs, "cannot be judged"); n != 1 { + t.Errorf("unreadable ERROR logged %d times over three cycles, want once:\n%s", n, logs) + } + if strings.Contains(logs, bridgeTestTokenSentinel) { + t.Error("log carries token material") + } + if !strings.Contains(logs, link) { + t.Error("log does not name the path") + } + // A new file version is logged again. + if err := os.Chmod(link, 0o600); err != nil { + t.Fatal(err) + } + simulateClaudeRefresh(t, link, private) + future := time.Now().Add(2 * time.Second) + if err := os.Chtimes(link, future, future); err != nil { + t.Fatal(err) + } + if err := os.Chmod(link, 0); err != nil { + t.Fatal(err) + } + m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog) + if n := strings.Count(logBuf.String(), "cannot be judged"); n != 2 { + t.Errorf("new file version not logged again: %d", n) } } -// TestSetupInteractiveHome_RebridgePromotesNewerUsablePrivateCredential: when -// the agent's private copy is the NEWER usable one (it just refreshed, and the -// refresh rotated the grant the shared file still holds), the repair promotes -// it into the shared file so the rest of the fleet is not stranded, and -// preserves the old shared file instead of losing it. -func TestSetupInteractiveHome_RebridgePromotesNewerUsablePrivateCredential(t *testing.T) { +// TestRepairClaudeCredentialBridge_LegacyLayoutIsNeverTouched: an agent whose +// whole ~/.claude is still the legacy directory symlink has the SHARED login +// at its credential path. Moving it aside would log the fleet out. +func TestRepairClaudeCredentialBridge_LegacyLayoutIsNeverTouched(t *testing.T) { shared := withSharedAgentHome(t) m, sink := testManagerWithSink(t) - now := time.Now() - sharedJSON := fakeClaudeCredential("fake-old-access", now.Add(-time.Hour), "fake-old-refresh") - agentJSON := fakeClaudeCredential("fake-new-access", now.Add(8*time.Hour), "fake-new-refresh") - link, target := brokenBridgeFixture(t, shared, "coder", sharedJSON, agentJSON) + var logBuf bytes.Buffer + m.logger = slog.New(slog.NewTextHandler(&logBuf, nil)) + target := filepath.Join(shared, ".claude", ".credentials.json") + if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil { + t.Fatal(err) + } + sharedBytes := bridgeTestCreds(t, "shared", time.Now().Add(time.Hour).UnixMilli()) + if err := os.WriteFile(target, sharedBytes, 0o660); err != nil { + t.Fatal(err) + } + home := interactiveHomePath("scanner") + if err := os.MkdirAll(home, 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink(filepath.Join(shared, ".claude"), filepath.Join(home, ".claude")); err != nil { + t.Fatal(err) + } + link := filepath.Join(home, ".claude", ".credentials.json") - ap := &AgentProcess{Name: "coder", UID: 2002, Config: config.AgentConfig{Backend: "claude"}} - m.setupInteractiveHome(ap, "claude") + for i := 0; i < 2; i++ { + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep != nil { + t.Fatalf("repair = %+v, want none", rep) + } + } + if !bytes.Equal(mustRead(t, target), sharedBytes) { + t.Error("shared credential disturbed") + } + if got := staleFilesBeside(t, target); len(got) != 0 { + t.Errorf("shared login moved aside: %v", got) + } + if sink.count() != 0 { + t.Error("legacy skip must not audit") + } + if n := strings.Count(logBuf.String(), "legacy whole-directory symlink"); n != 1 { + t.Errorf("legacy warning logged %d times, want once", n) + } +} - assertBridgeRestored(t, link, target) - if got := readFileString(t, target); got != agentJSON { - t.Errorf("shared credential should now hold the newer private token:\n got %s\nwant %s", got, agentJSON) +// TestRepairClaudeCredentialBridge_NothingToRepair: a healthy link, an absent +// link and a directory are not this failure. +func TestRepairClaudeCredentialBridge_NothingToRepair(t *testing.T) { + m, sink, _, link, target := bridgeTestFixture(t) + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep != nil { + t.Errorf("healthy link: repair = %+v", rep) + } + if err := os.Remove(link); err != nil { + t.Fatal(err) + } + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep != nil { + t.Errorf("absent link: repair = %+v", rep) + } + if err := os.Mkdir(link, 0o755); err != nil { + t.Fatal(err) } - if info, err := os.Stat(target); err != nil || info.Mode().Perm() != 0o660 { - t.Errorf("promoted shared credential must be 0660 (group-readable by every agent uid): info=%v err=%v", info, err) + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep != nil { + t.Errorf("directory: repair = %+v", rep) } - oldShared := staleFiles(t, filepath.Dir(target)) - if len(oldShared) != 1 || readFileString(t, oldShared[0]) != sharedJSON { - t.Errorf("replaced shared credential must be preserved beside it, got %v", oldShared) + if sink.count() != 0 { + t.Error("nothing to repair must not audit") } - if got := staleFiles(t, filepath.Dir(link)); len(got) != 1 { - t.Errorf("private file must be moved aside, not deleted: %v", got) + // No per-agent home at all: no paths, no repair. + if _, _, ok := claudeCredentialBridgePaths("scanner", 0, "claude"); ok { + t.Error("uid 0 must have no bridge") } - if ev := sink.find("claude_credential_rebridged"); ev == nil || ev.Fields["kept"] != "agent" { - t.Errorf("audit entry = %+v, want kept=agent", ev) + if _, _, ok := claudeCredentialBridgePaths("scanner", 1001, "vllm"); ok { + t.Error("inference backend must have no bridge") + } + t.Setenv("HIVE_SHARED_AGENT_HOME", "1") + if _, _, ok := claudeCredentialBridgePaths("scanner", 1001, "claude"); ok { + t.Error("forced shared home must have no bridge") } } -// TestSetupInteractiveHome_RebridgeNeverPromotesUnusablePrivateCredential: a -// private copy with a later expiresAt but no usable grant must not overwrite -// the shared file; "newer" alone is not enough. -func TestSetupInteractiveHome_RebridgeNeverPromotesUnusablePrivateCredential(t *testing.T) { - shared := withSharedAgentHome(t) - m, _ := testManagerWithSink(t) +// TestRepairClaudeCredentialBridge_StaleNamesNeverCollide: two repairs within +// the same second get distinct stale names on both sides. +func TestRepairClaudeCredentialBridge_StaleNamesNeverCollide(t *testing.T) { + m, _, _, link, target := bridgeTestFixture(t) + fixed := time.Date(2026, 11, 11, 12, 0, 0, 0, time.UTC) + oldNow := claudeBridgeNow + claudeBridgeNow = func() time.Time { return fixed } + t.Cleanup(func() { claudeBridgeNow = oldNow }) + if err := os.WriteFile(target, bridgeTestCreds(t, "s0", fixed.Add(time.Hour).UnixMilli()), 0o660); err != nil { + t.Fatal(err) + } + for i := 1; i <= 2; i++ { + simulateClaudeRefresh(t, link, bridgeTestCreds(t, fmt.Sprintf("a%d", i), fixed.Add(time.Duration(i+1)*time.Hour).UnixMilli())) + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep == nil || rep.kept != "agent" { + t.Fatalf("repair %d = %+v", i, rep) + } + } + if got := staleFilesBeside(t, link); len(got) != 2 { + t.Errorf("agent stale files = %v, want 2 distinct", got) + } + got := staleFilesBeside(t, target) + if len(got) != 2 { + t.Fatalf("shared stale files = %v, want 2 distinct", got) + } + if !strings.HasSuffix(got[1], "-1") { + t.Errorf("second stale name = %q, want -1 suffix", got[1]) + } + if tok := readTokensAt(t, target); !strings.HasPrefix(tok.AccessToken, "acc-a2-") { + t.Errorf("shared token = %q, want the latest refresh", tok.AccessToken) + } +} + +func TestPreferAgentClaudeCredential(t *testing.T) { now := time.Now() - sharedJSON := fakeClaudeCredential("fake-shared-access", now.Add(-2*time.Hour), "fake-shared-refresh") - // Expired access token and no refresh grant: not usable. - agentJSON := fakeClaudeCredential("fake-dead-access", now.Add(-time.Minute), "") - link, target := brokenBridgeFixture(t, shared, "scanner", sharedJSON, agentJSON) + live := func(tag string, d time.Duration) *claude.OAuthTokens { + return &claude.OAuthTokens{AccessToken: "a" + tag, RefreshToken: "r" + tag, ExpiresAt: now.Add(d).UnixMilli()} + } + cases := []struct { + name string + agent, shared *claude.OAuthTokens + want bool + }{ + {"nil agent", nil, live("s", time.Hour), false}, + {"agent newer", live("a", 2*time.Hour), live("s", time.Hour), true}, + {"agent older", live("a", time.Hour), live("s", 2*time.Hour), false}, + {"tie keeps shared", live("a", time.Hour), live("a", time.Hour), false}, + {"shared unusable", live("a", time.Hour), &claude.OAuthTokens{AccessToken: "x", ExpiresAt: now.Add(-time.Hour).UnixMilli()}, true}, + {"shared absent", live("a", time.Hour), nil, true}, + {"agent no refresh token", &claude.OAuthTokens{AccessToken: "a", ExpiresAt: now.Add(time.Hour).UnixMilli()}, nil, false}, + {"agent no expiry", &claude.OAuthTokens{AccessToken: "a", RefreshToken: "r"}, nil, false}, + {"agent expired, no refresh expiry", &claude.OAuthTokens{AccessToken: "a", RefreshToken: "r", ExpiresAt: now.Add(-time.Hour).UnixMilli()}, nil, true}, + {"agent empty", &claude.OAuthTokens{}, nil, false}, + } + for _, tc := range cases { + if got := preferAgentClaudeCredential(tc.agent, tc.shared, now); got != tc.want { + t.Errorf("%s: prefer agent = %v, want %v", tc.name, got, tc.want) + } + } +} - ap := &AgentProcess{Name: "scanner", UID: 2003, Config: config.AgentConfig{Backend: "claude"}} - m.setupInteractiveHome(ap, "claude") +func TestPromoteClaudeCredential_RefusesUnusableBytes(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, ".credentials.json") + sharedBytes := bridgeTestCreds(t, "shared", time.Now().Add(time.Hour).UnixMilli()) + if err := os.WriteFile(target, sharedBytes, 0o660); err != nil { + t.Fatal(err) + } + for _, bad := range [][]byte{[]byte("not json"), []byte(`{}`), []byte(`{"claudeAiOauth":{"accessToken":"","refreshToken":"","expiresAt":0}}`)} { + stale, err := promoteClaudeCredential(bad, target, time.Now()) + if !errors.Is(err, errClaudeBridgeNotPromotable) || stale != "" { + t.Errorf("promote(%q) = (%q, %v), want errClaudeBridgeNotPromotable", bad, stale, err) + } + } + if !bytes.Equal(mustRead(t, target), sharedBytes) { + t.Error("shared credential changed") + } + if got := staleFilesBeside(t, target); len(got) != 0 { + t.Errorf("preserved copies made for refused promotions: %v", got) + } + // Promotion with no shared file: nothing to preserve, chown seam called + // only when an owner was read. + fresh := filepath.Join(dir, "fresh", ".credentials.json") + if err := os.MkdirAll(filepath.Dir(fresh), 0o755); err != nil { + t.Fatal(err) + } + stale, err := promoteClaudeCredential(sharedBytes, fresh, time.Now()) + if err != nil || stale != "" { + t.Fatalf("promote to absent target = (%q, %v)", stale, err) + } + if info, err := os.Stat(fresh); err != nil || info.Mode().Perm() != claudeSharedCredentialMode { + t.Errorf("promoted mode = %v err=%v", info.Mode(), err) + } +} + +func TestPromoteClaudeCredential_WritesOnlyTheTokenBlock(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, ".credentials.json") + if err := os.WriteFile(target, bridgeTestCreds(t, "shared", time.Now().Add(time.Hour).UnixMilli()), 0o660); err != nil { + t.Fatal(err) + } + exp := time.Now().Add(8 * time.Hour).UnixMilli() + // A credential with extra material around and inside the token block. + raw := fmt.Sprintf(`{"claudeAiOauth":{"accessToken":"acc-%s","refreshToken":"ref-%s","expiresAt":%d,"scopes":["user:inference"],"extraField":{"deep":"EXTRA-MATERIAL"}},"otherTopLevel":"EXTRA-MATERIAL"}`, + bridgeTestTokenSentinel, bridgeTestTokenSentinel, exp) + stale, err := promoteClaudeCredential([]byte(raw), target, time.Now()) + if err != nil || stale == "" { + t.Fatalf("promote = (%q, %v)", stale, err) + } + got := mustRead(t, target) + if bytes.Contains(got, []byte("EXTRA-MATERIAL")) { + t.Errorf("shared file carries bytes outside the typed token block: %s", got) + } + want, _ := json.Marshal(claude.Credentials{ClaudeAIOAuth: &claude.OAuthTokens{ + AccessToken: "acc-" + bridgeTestTokenSentinel, RefreshToken: "ref-" + bridgeTestTokenSentinel, + ExpiresAt: exp, Scopes: []string{"user:inference"}, + }}) + if !bytes.Equal(got, want) { + t.Errorf("shared file = %s, want canonical %s", got, want) + } + if info, err := os.Lstat(target); err != nil || !info.Mode().IsRegular() || info.Mode().Perm() != claudeSharedCredentialMode { + t.Errorf("shared file mode = %v err=%v", info.Mode(), err) + } + if info, err := os.Stat(stale); err != nil || info.Mode().Perm() != 0o600 { + t.Errorf("preserved shared copy mode = %v err=%v, want 0600", info.Mode(), err) + } + if _, err := os.Lstat(target + ".tmp-rebridge"); !errors.Is(err, fs.ErrNotExist) { + t.Error("a predictable temporary name was used") + } + if left, _ := filepath.Glob(target + ".*.tmp"); len(left) != 0 { + t.Errorf("temporary files left: %v", left) + } + if _, _, err := canonicalClaudeCredential(bytes.Repeat([]byte("x"), claudeBridgeMaxCredentialBytes+1), time.Now()); !errors.Is(err, errClaudeBridgeNotPromotable) { + t.Errorf("oversized input: err = %v", err) + } +} + +// TestRepairClaudeCredentialBridge_SharedTargetSymlinkIsRefused: the shared +// credential path holds a symlink. Nothing is written through it, the agent +// keeps its private copy in place, and the refusal is logged once. +func TestRepairClaudeCredentialBridge_SharedTargetSymlinkIsRefused(t *testing.T) { + m, sink, _, link, target := bridgeTestFixture(t) + var logBuf bytes.Buffer + m.logger = slog.New(slog.NewTextHandler(&logBuf, nil)) + victim := filepath.Join(t.TempDir(), "victim") + if err := os.WriteFile(victim, []byte("keep"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.Symlink(victim, target); err != nil { + t.Fatal(err) + } + simulateClaudeRefresh(t, link, bridgeTestCreds(t, "fresh", time.Now().Add(8*time.Hour).UnixMilli())) + + for i := 0; i < 2; i++ { + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep != nil { + t.Fatalf("repair = %+v, want refusal", rep) + } + } + if got, _ := os.ReadFile(victim); string(got) != "keep" { + t.Errorf("victim rewritten: %q", got) + } + if info, _ := os.Lstat(victim); info.Mode().Perm() != 0o600 { + t.Errorf("victim mode changed: %v", info.Mode()) + } + if info, err := os.Lstat(target); err != nil || info.Mode()&fs.ModeSymlink == 0 { + t.Errorf("symlink at the shared path was replaced: %v %v", info, err) + } + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Errorf("agent's private file not left in place: %v %v", info, err) + } + if got := staleFilesBeside(t, link); len(got) != 0 { + t.Errorf("private copy left aside: %v", got) + } + if sink.count() != 0 { + t.Error("a refusal must not audit") + } + if n := strings.Count(logBuf.String(), "location was refused"); n != 1 { + t.Errorf("refusal logged %d times over two cycles, want once:\n%s", n, logBuf.String()) + } + if strings.Contains(logBuf.String(), bridgeTestTokenSentinel) { + t.Error("log carries token material") + } +} + +// TestRepairClaudeCredentialBridge_SharedDirSymlinkIsRefused: the shared +// .claude directory itself is a symlink. Nothing is written into what it +// points at. +func TestRepairClaudeCredentialBridge_SharedDirSymlinkIsRefused(t *testing.T) { + m, _, _, link, target := bridgeTestFixture(t) + sharedBytes := bridgeTestCreds(t, "shared", time.Now().Add(time.Hour).UnixMilli()) + if err := os.WriteFile(target, sharedBytes, 0o660); err != nil { + t.Fatal(err) + } + dir := filepath.Dir(target) + elsewhere := filepath.Join(t.TempDir(), "elsewhere") + if err := os.Rename(dir, elsewhere); err != nil { + t.Fatal(err) + } + if err := os.Symlink(elsewhere, dir); err != nil { + t.Fatal(err) + } + simulateClaudeRefresh(t, link, bridgeTestCreds(t, "fresh", time.Now().Add(8*time.Hour).UnixMilli())) + + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep != nil { + t.Fatalf("repair = %+v, want refusal", rep) + } + if !bytes.Equal(mustRead(t, filepath.Join(elsewhere, ".credentials.json")), sharedBytes) { + t.Error("wrote through the directory symlink") + } + entries, _ := os.ReadDir(elsewhere) + for _, e := range entries { + if e.Name() != ".credentials.json" { + t.Errorf("unexpected entry behind the directory symlink: %s", e.Name()) + } + } + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Errorf("agent's private file not left in place: %v %v", info, err) + } +} - assertBridgeRestored(t, link, target) - if got := readFileString(t, target); got != sharedJSON { - t.Errorf("an unusable private credential replaced the shared one: %s", got) +// TestMoveAsideClaudeCredential_NonRegularSourceIsRefused: a symlink where the +// private file should be is neither moved nor read. +func TestMoveAsideClaudeCredential_NonRegularSourceIsRefused(t *testing.T) { + dir := t.TempDir() + victim := filepath.Join(dir, "victim") + if err := os.WriteFile(victim, bridgeTestCreds(t, "v", time.Now().Add(time.Hour).UnixMilli()), 0o600); err != nil { + t.Fatal(err) + } + src := filepath.Join(dir, ".credentials.json") + if err := os.Symlink(victim, src); err != nil { + t.Fatal(err) + } + if _, err := moveAsideClaudeCredential(src, time.Now()); !errors.Is(err, errClaudeBridgeSourceChanged) { + t.Errorf("err = %v, want errClaudeBridgeSourceChanged", err) + } + if info, err := os.Lstat(src); err != nil || info.Mode()&fs.ModeSymlink == 0 { + t.Errorf("symlink moved: %v %v", info, err) + } + if got := staleFilesBeside(t, src); len(got) != 0 { + t.Errorf("stale files created: %v", got) + } + // A directory is refused the same way. + d := filepath.Join(dir, "d") + if err := os.Mkdir(d, 0o700); err != nil { + t.Fatal(err) + } + if _, err := moveAsideClaudeCredential(d, time.Now()); !errors.Is(err, errClaudeBridgeSourceChanged) { + t.Errorf("directory: err = %v", err) + } + // The owner-read path refuses a symlink too, even one to a readable file. + if _, err := readClaudeCredentialAsOwner(src); !errors.Is(err, sharedfile.ErrNotRegular) { + t.Errorf("read through symlink: err = %v, want ErrNotRegular", err) } } -// TestCredentialWatchdog_RebridgesWithinOneCycle: a refresh that replaces the -// link while the agent is running must be caught by the next watchdog tick, -// not only at the next launch. +// TestRepairClaudeCredentialBridge_UnreadableIsRelinkedOnlyAfterSignIn: an +// unreadable private file stays put on the watchdog, but a dashboard sign-in +// (a shared login newer than any private copy) moves it aside unread and +// relinks the agent. +func TestRepairClaudeCredentialBridge_UnreadableIsRelinkedOnlyAfterSignIn(t *testing.T) { + if os.Getuid() == 0 { + t.Skip("root can read a mode-000 file; the permission fallback cannot be exercised") + } + m, sink, _, link, target := bridgeTestFixture(t) + simulateClaudeRefresh(t, link, bridgeTestCreds(t, "private", time.Now().Add(8*time.Hour).UnixMilli())) + if err := os.Chmod(link, 0); err != nil { + t.Fatal(err) + } + old := runAsAgentUser + runAsAgentUser = func(string, []byte, ...string) ([]byte, error) { return nil, errors.New("no su-exec") } + t.Cleanup(func() { runAsAgentUser = old }) + + // No shared login yet: even the sign-in trigger moves nothing. + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerSignIn); rep != nil { + t.Fatalf("sign-in without a shared login: repair = %+v", rep) + } + sharedBytes := bridgeTestCreds(t, "signin", time.Now().Add(8*time.Hour).UnixMilli()) + if err := os.WriteFile(target, sharedBytes, 0o660); err != nil { + t.Fatal(err) + } + if rep := m.rebridgeClaudeCredential("scanner", link, target, claudeBridgeTriggerWatchdog); rep != nil { + t.Fatalf("watchdog: repair = %+v, want none", rep) + } + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Fatalf("watchdog moved the unreadable file: %v %v", info, err) + } + + m.ReloadClaudeToken() + assertLinkRestored(t, link, target) + stale := staleFilesBeside(t, link) + if len(stale) != 1 { + t.Fatalf("stale files = %v, want the unreadable copy moved aside", stale) + } + if info, err := os.Lstat(stale[0]); err != nil || info.Mode().Perm() != 0 { + t.Errorf("moved-aside copy mode = %v err=%v, want unchanged (000)", info.Mode(), err) + } + _ = os.Chmod(stale[0], 0o600) + if !bytes.Equal(mustRead(t, target), sharedBytes) { + t.Error("shared login changed") + } + rec := sink.find(AuditClaudeCredentialRebridged) + if rec == nil || rec.Fields["kept"] != "shared" || rec.Fields["trigger"] != claudeBridgeTriggerSignIn { + t.Errorf("audit = %+v", rec) + } +} + +// TestCredentialWatchdog_RebridgesWithinOneCycle: the bridge poll repairs a +// replaced link without waiting for the 5-minute probe. func TestCredentialWatchdog_RebridgesWithinOneCycle(t *testing.T) { - shared := withSharedAgentHome(t) - t.Setenv(CredentialWatchdogIntervalEnv, "20ms") - m, sink := testManagerWithSink(t) - m.mu.Lock() - m.agents["planner"] = &AgentProcess{Name: "planner", UID: 2004, Config: config.AgentConfig{Backend: "claude"}} - m.mu.Unlock() + t.Setenv(ClaudeBridgeIntervalEnv, "20ms") + t.Setenv(CredentialWatchdogIntervalEnv, "1h") + m, _, _, link, target := bridgeTestFixture(t) now := time.Now() - link, target := brokenBridgeFixture(t, shared, "planner", - fakeClaudeCredential("fake-shared-access", now.Add(8*time.Hour), "fake-shared-refresh"), - fakeClaudeCredential("fake-private-access", now.Add(time.Hour), "fake-private-refresh")) + if err := os.WriteFile(target, bridgeTestCreds(t, "old", now.Add(10*time.Minute).UnixMilli()), 0o660); err != nil { + t.Fatal(err) + } + fresh := bridgeTestCreds(t, "fresh", now.Add(8*time.Hour).UnixMilli()) + simulateClaudeRefresh(t, link, fresh) ctx, cancel := context.WithCancel(context.Background()) + defer cancel() done := make(chan struct{}) go func() { m.StartCredentialWatchdog(ctx); close(done) }() - t.Cleanup(func() { cancel(); <-done }) - - // One cycle is 20ms; allow generous scheduling slack but far less than a - // second cycle would need to matter. - deadline := time.Now().Add(2 * time.Second) - for time.Now().Before(deadline) { - if info, err := os.Lstat(link); err == nil && info.Mode()&os.ModeSymlink != 0 { - break - } - time.Sleep(10 * time.Millisecond) + + testutil.Eventually(t, 5*time.Second, func() bool { + got, err := os.Readlink(link) + return err == nil && got == target + }, "watchdog did not re-bridge the credential within 5s") + cancel() + <-done + if !bytes.Equal(mustRead(t, target), fresh) { + t.Error("watchdog did not publish the refresh to the shared login") + } +} + +func TestClaudeBridgeInterval(t *testing.T) { + t.Setenv(ClaudeBridgeIntervalEnv, "") + if got := claudeBridgeInterval(); got != defaultClaudeBridgeInterval { + t.Errorf("default = %v", got) } - assertBridgeRestored(t, link, target) - if ev := sink.find("claude_credential_rebridged"); ev == nil || ev.Fields["trigger"] != "watchdog" { - t.Errorf("audit entry = %+v, want trigger=watchdog", ev) + t.Setenv(ClaudeBridgeIntervalEnv, "250ms") + if got := claudeBridgeInterval(); got != 250*time.Millisecond { + t.Errorf("override = %v", got) + } + for _, bad := range []string{"nope", "-1s", "0"} { + t.Setenv(ClaudeBridgeIntervalEnv, bad) + if got := claudeBridgeInterval(); got != defaultClaudeBridgeInterval { + t.Errorf("%q: = %v, want default", bad, got) + } } } -// TestReloadClaudeToken_RebridgesAfterSignIn: the dashboard sign-in writes the -// shared credential and calls ReloadClaudeToken. That must reach the agents a -// refresh had cut off, or the sign-in cannot help them (the hive-wild-mole -// 14:31Z and 14:50Z sign-ins). +// TestReloadClaudeToken_RebridgesAfterSignIn: a dashboard sign-in reaches an +// agent whose link a refresh had replaced, at once rather than on the next +// poll; a logout (nothing usable) moves nothing. func TestReloadClaudeToken_RebridgesAfterSignIn(t *testing.T) { - shared := withSharedAgentHome(t) - m, sink := testManagerWithSink(t) - m.mu.Lock() - m.agents["reviewer"] = &AgentProcess{Name: "reviewer", UID: 2005, Config: config.AgentConfig{Backend: "claude"}} - m.mu.Unlock() - now := time.Now() - link, target := brokenBridgeFixture(t, shared, "reviewer", - fakeClaudeCredential("fake-signin-access", now.Add(8*time.Hour), "fake-signin-refresh"), - fakeClaudeCredential("fake-revoked-access", now.Add(-10*time.Minute), "fake-revoked-refresh")) + m, sink, _, link, target := bridgeTestFixture(t) + simulateClaudeRefresh(t, link, bridgeTestCreds(t, "agent", time.Now().Add(-time.Hour).UnixMilli())) + // Logout: no shared file. Nothing moves. m.ReloadClaudeToken() - - assertBridgeRestored(t, link, target) - if ev := sink.find("claude_credential_rebridged"); ev == nil || ev.Fields["trigger"] != "sign-in" || ev.Fields["kept"] != "shared" { - t.Errorf("audit entry = %+v, want trigger=sign-in kept=shared", ev) + if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { + t.Fatalf("logout must not touch the private file: info=%v err=%v", info, err) + } + if sink.count() != 0 { + t.Fatal("logout audited a rebridge") } -} - -// TestReloadClaudeToken_LogoutDoesNotRebridge: after a logout the shared file -// is gone; moving a private credential aside then would leave the agent with -// nothing at all. -func TestReloadClaudeToken_LogoutDoesNotRebridge(t *testing.T) { - shared := withSharedAgentHome(t) - m, _ := testManagerWithSink(t) - m.mu.Lock() - m.agents["reviewer"] = &AgentProcess{Name: "reviewer", UID: 2005, Config: config.AgentConfig{Backend: "claude"}} - m.mu.Unlock() - link, _ := brokenBridgeFixture(t, shared, "reviewer", "", - fakeClaudeCredential("fake-private-access", time.Now().Add(time.Hour), "fake-private-refresh")) + sharedBytes := bridgeTestCreds(t, "signin", time.Now().Add(8*time.Hour).UnixMilli()) + if err := os.WriteFile(target, sharedBytes, 0o660); err != nil { + t.Fatal(err) + } m.ReloadClaudeToken() + assertLinkRestored(t, link, target) + rec := sink.find(AuditClaudeCredentialRebridged) + if rec == nil || rec.Fields["trigger"] != claudeBridgeTriggerSignIn || rec.Fields["kept"] != "shared" { + t.Errorf("audit = %+v", rec) + } +} - if info, err := os.Lstat(link); err != nil || !info.Mode().IsRegular() { - t.Errorf("private credential must stay in place when there is no shared login: info=%v err=%v", info, err) +// TestRepairClaudeCredentialBridge_MidRenameIsNextCycle: the link vanishes +// between the Lstat and the read (the CLI is mid-rename). Nothing to do. +func TestRepairClaudeCredentialBridge_MidRenameIsNextCycle(t *testing.T) { + m, _, _, link, target := bridgeTestFixture(t) + simulateClaudeRefresh(t, link, bridgeTestCreds(t, "agent", time.Now().Add(time.Hour).UnixMilli())) + if err := os.Chmod(link, 0); err != nil { + t.Fatal(err) + } + old := runAsAgentUser + runAsAgentUser = func(spec string, stdin []byte, argv ...string) ([]byte, error) { + _ = os.Remove(link) + return nil, &fs.PathError{Op: "open", Path: link, Err: fs.ErrNotExist} + } + t.Cleanup(func() { runAsAgentUser = old }) + if os.Getuid() == 0 { + // Root reads the file directly; remove it from under the Lstat another way. + runAsAgentUser = old + t.Skip("root cannot exercise the owner-read path") + } + rep, err := m.repairClaudeCredentialBridge("scanner", link, target, claudeBridgeTriggerWatchdog) + if rep != nil || err != nil { + t.Errorf("mid-rename: (%+v, %v), want nothing", rep, err) } } diff --git a/src/crux/agent/copilottoken/copilottoken.go b/src/crux/agent/copilottoken/copilottoken.go index 480036a3d3..de66bcbfb7 100644 --- a/src/crux/agent/copilottoken/copilottoken.go +++ b/src/crux/agent/copilottoken/copilottoken.go @@ -11,6 +11,7 @@ import ( "strings" "github.com/hivecommons/hive/crux/claude" + "github.com/hivecommons/hive/crux/sharedfile" ) // ConfigHasTokens returns true if either the Copilot config or Claude @@ -134,19 +135,18 @@ func ReadConfig(path string) (map[string]interface{}, error) { } // WriteConfig marshals cfg back to config.json with the CLI's // header -// preserved and the CLI-expected mode. Written via a temp file + rename so a -// concurrent CLI read never sees a half-written file. +// preserved and the CLI-expected mode. Written through sharedfile.Replace (a +// random temporary beside path, renamed into place) so a concurrent CLI read +// never sees a half-written file, and so a non-regular file at path, or a +// directory that is not this process's own, is refused rather than written +// through: config.json lives in a directory every agent uid can write to. func WriteConfig(path string, cfg map[string]interface{}, mode os.FileMode) error { out, err := json.MarshalIndent(cfg, "", " ") if err != nil { return err } content := ConfigHeader + string(out) - tmp := path + ".tmp" - if err := os.WriteFile(tmp, []byte(content), mode); err != nil { - return err - } - return os.Rename(tmp, path) + return sharedfile.Replace(path, []byte(content), mode, sharedfile.Options{PreserveOwner: true}) } // ClearExpiredTokens removes stored copilot tokens from config.json. @@ -361,9 +361,5 @@ func WriteDurableToken(durablePath, token string) error { if token == "" { return nil } - tmp := durablePath + ".tmp" - if err := os.WriteFile(tmp, []byte(token), 0o600); err != nil { - return err - } - return os.Rename(tmp, durablePath) + return sharedfile.Replace(durablePath, []byte(token), 0o600, sharedfile.Options{}) } diff --git a/src/crux/agent/coverage_boost_test.go b/src/crux/agent/coverage_boost_test.go index 972bbef9b5..e71f441268 100644 --- a/src/crux/agent/coverage_boost_test.go +++ b/src/crux/agent/coverage_boost_test.go @@ -1883,6 +1883,32 @@ func TestClearExpiredTokens_MissingFile(t *testing.T) { // fixSharedConfigPerms — with actual path // --------------------------------------------------------------------------- +// A symlink at the shared config path is left alone: the file it points at +// keeps its mode. +func TestFixSharedConfigPerms_SymlinkIsNotFollowed(t *testing.T) { + cleanup := configTestHelper(t) + defer cleanup() + victim := filepath.Join(t.TempDir(), "victim") + if err := os.WriteFile(victim, []byte(`{}`), 0o600); err != nil { + t.Fatal(err) + } + if err := os.Symlink(victim, sharedCopilotConfigPath); err != nil { + t.Fatal(err) + } + m := NewManager(map[string]config.AgentConfig{ + "scanner": {Backend: "claude"}, + }, discardLogger(), ProjectContext{}) + m.mu.RLock() + agent := m.agents["scanner"] + m.mu.RUnlock() + + m.fixSharedConfigPerms(agent) + + if info, _ := os.Stat(victim); info.Mode().Perm() != 0o600 { + t.Fatalf("victim mode = %v, want 0600 untouched", info.Mode()) + } +} + func TestFixSharedConfigPerms_FixesPerms(t *testing.T) { cleanup := configTestHelper(t) defer cleanup() diff --git a/src/crux/agent/credentials_watchdog.go b/src/crux/agent/credentials_watchdog.go index 61eb65ecbe..502316fce7 100644 --- a/src/crux/agent/credentials_watchdog.go +++ b/src/crux/agent/credentials_watchdog.go @@ -25,6 +25,19 @@ const ( // only for overrides) — disabling is intentionally not offered so the // safety net cannot be silently turned off. CredentialWatchdogIntervalEnv = "HIVE_CREDENTIAL_WATCHDOG_INTERVAL" + + // defaultClaudeBridgeInterval is how often the watchdog checks each + // per-agent Claude credential link for a token refresh that replaced it + // (claude_credential_bridge.go). Short on purpose: every agent holds the + // SAME access token, so they all reach expiry together and refresh within + // a minute or two of each other; the first refresh rotates the grant, and + // each agent that tries before that refresh is published fails and signs + // itself out. The check is one Lstat per claude agent, so 5s costs + // nothing; the su-exec read happens only when a link has been replaced. + defaultClaudeBridgeInterval = 5 * time.Second + // ClaudeBridgeIntervalEnv overrides the bridge poll interval with a Go + // duration string. + ClaudeBridgeIntervalEnv = "HIVE_CLAUDE_CREDENTIAL_BRIDGE_INTERVAL" ) // credentialWatchdogInterval resolves the watchdog interval from @@ -38,6 +51,17 @@ func credentialWatchdogInterval() time.Duration { return defaultCredentialWatchdogInterval } +// claudeBridgeInterval resolves the bridge poll interval from +// ClaudeBridgeIntervalEnv, falling back to the default. +func claudeBridgeInterval() time.Duration { + if v := os.Getenv(ClaudeBridgeIntervalEnv); v != "" { + if d, err := time.ParseDuration(v); err == nil && d > 0 { + return d + } + } + return defaultClaudeBridgeInterval +} + // credentialWatch describes one CLI backend whose usable credential lives in a // durable file on the hive PVC that the hive relies on but does NOT itself keep // alive. Only backends of this shape belong here: bob resolves its key at @@ -205,6 +229,8 @@ func credentialWatches() []credentialWatch { func (m *Manager) StartCredentialWatchdog(ctx context.Context) { ticker := time.NewTicker(credentialWatchdogInterval()) defer ticker.Stop() + bridgeTicker := time.NewTicker(claudeBridgeInterval()) + defer bridgeTicker.Stop() // Per-backend transition tracker so we log/audit on TRANSITIONS (usable -> // unusable and back) rather than every tick — a standing condition would // otherwise flood the Audit Log at the tick rate. @@ -213,6 +239,10 @@ func (m *Manager) StartCredentialWatchdog(ctx context.Context) { select { case <-ctx.Done(): return + case <-bridgeTicker.C: + if m.backendInUse("claude") { + m.RepairClaudeCredentialBridges(claudeBridgeTriggerWatchdog) + } case <-ticker.C: // Re-bridge BEFORE probing: a Claude token refresh can leave one // agent on a private copy holding the only live grant, and the diff --git a/src/crux/agent/interactive_home.go b/src/crux/agent/interactive_home.go index 38b903f66c..ee7ed1e9a1 100644 --- a/src/crux/agent/interactive_home.go +++ b/src/crux/agent/interactive_home.go @@ -429,6 +429,12 @@ func (m *Manager) setupPerAgentCLIHome(agentName, home string, uid int, spec per for _, name := range spec.bridges { link := filepath.Join(dir, name) target := filepath.Join(shared, name) + if spec.dir == claudeHomeDirName && name == claudeCredentialsFileName { + // A token refresh since the last launch replaced the link with a + // private file; publish it to the shared login and relink BEFORE + // bridgeHomeEntry, which refuses to clobber a regular file. + m.rebridgeClaudeCredential(agentName, link, target, claudeBridgeTriggerLaunch) + } if _, err := os.Stat(target); err != nil && isPerAgentCLIBridgeDir(spec.dir, name) { // Absent target: a dangling FILE link goes live on first login; a // dangling DIRECTORY link would make the CLI's own mkdir -p fail, diff --git a/src/crux/agent/manager_credentials.go b/src/crux/agent/manager_credentials.go index 3b80f264ba..37651cf1d5 100644 --- a/src/crux/agent/manager_credentials.go +++ b/src/crux/agent/manager_credentials.go @@ -2,14 +2,16 @@ package agent import ( "context" + "errors" "fmt" - "os" + "io/fs" "strings" "time" "github.com/hivecommons/hive/crux" "github.com/hivecommons/hive/crux/claude" "github.com/hivecommons/hive/crux/logscrub" + "github.com/hivecommons/hive/crux/sharedfile" ) // sharedCopilotConfigPath and sharedClaudeCredentialPath are vars (not consts) @@ -331,19 +333,22 @@ func compactPaneTail(output string, maxLines int) string { // fixSharedConfigPerms ensures /data/home/.copilot/config.json is group-readable // before launching an agent. Copilot CLI rewrites this file with 600 perms on // token refresh, locking out other agent UIDs that share the same HOME. +// +// The chmod goes through an open descriptor (sharedfile.Chmod), so anything +// other than a regular file at that path is left alone and logged. func (m *Manager) fixSharedConfigPerms(agent *AgentProcess) { - info, err := os.Stat(sharedCopilotConfigPath) - if err != nil { + prev, changed, err := sharedfile.Chmod(sharedCopilotConfigPath, sharedConfigDesiredMode) + switch { + case errors.Is(err, fs.ErrNotExist): return - } - if info.Mode().Perm() == sharedConfigDesiredMode { + case err != nil: + m.logger.Warn("failed to fix config.json perms", "agent", agent.Name, "error", err) + return + case !changed: return } - m.logger.Warn("fixing shared config.json perms before launch", + m.logger.Warn("fixed shared config.json perms before launch", "agent", agent.Name, - "was", fmt.Sprintf("%04o", info.Mode().Perm()), + "was", fmt.Sprintf("%04o", prev), "fix", fmt.Sprintf("%04o", sharedConfigDesiredMode)) - if err := os.Chmod(sharedCopilotConfigPath, sharedConfigDesiredMode); err != nil { - m.logger.Warn("failed to fix config.json perms", "error", err) - } } diff --git a/src/crux/agent/manager_tokens.go b/src/crux/agent/manager_tokens.go index fefc1655e1..e5694a82ba 100644 --- a/src/crux/agent/manager_tokens.go +++ b/src/crux/agent/manager_tokens.go @@ -24,20 +24,17 @@ type ( // ReloadClaudeToken re-reads the Claude credentials file and updates the // cached token (dashboard sign-in or logout). When the shared credential is // usable afterwards, it re-bridges agents whose link a token refresh replaced -// with a private file, so the new login reaches them. +// with a private file (after m.mu is released), so the new login reaches them. func (m *Manager) ReloadClaudeToken() { + defer m.rebridgeClaudeCredentialsAfterSignIn() // LIFO: runs after Unlock m.mu.Lock() + defer m.mu.Unlock() m.claudeAuthToken = claude.ReadAccessToken(claude.CredentialsPath) - m.mu.Unlock() - if claude.HasUsableToken(sharedClaudeBridgeTarget()) { - m.RepairClaudeCredentialBridges(claudeBridgeTriggerSignIn) - } } // Copilot token sources, as reported by CopilotTokenSource. Operator-facing -// phrases: they are spliced verbatim into the model-picker notice and the -// "rejected by upstream" log line, so each one has to read as an answer to -// "which credential did GitHub just reject?". +// phrases, spliced verbatim into the model-picker notice and the "rejected by +// upstream" log line: each must answer "which credential did GitHub reject?". const ( // Defined in the crux root package; see crux.CopilotTokenSourceEnv. CopilotTokenSourceEnv = crux.CopilotTokenSourceEnv diff --git a/src/crux/agent/permissions_watcher.go b/src/crux/agent/permissions_watcher.go index d8b5fbb394..648b60fb3d 100644 --- a/src/crux/agent/permissions_watcher.go +++ b/src/crux/agent/permissions_watcher.go @@ -620,7 +620,9 @@ func fixEntry(path string, fi os.FileInfo, logger *slog.Logger) { if fi.IsDir() { // Directories need u+rwx to be usable. if mode.Perm()&DirPerms != DirPerms { - newMode := mode.Perm() | DirPerms + // Keep setgid and sticky: the entrypoint sets both on the shared + // dot-dirs, and a repair of the group bits must not drop them. + newMode := mode&(os.ModePerm|os.ModeSetgid|os.ModeSticky) | DirPerms if err := chmod(newMode); err != nil { warnDeduped(logger, "chmod dir", "permissions watcher: chmod dir failed", path, err) } else { diff --git a/src/crux/claude/oauth.go b/src/crux/claude/oauth.go index 8b262a88fa..550b7d45c6 100644 --- a/src/crux/claude/oauth.go +++ b/src/crux/claude/oauth.go @@ -6,14 +6,19 @@ import ( "crypto/sha256" "encoding/base64" "encoding/json" + "errors" "fmt" "io" + "io/fs" "net/http" "net/url" "os" "path/filepath" "strings" + "syscall" "time" + + "github.com/hivecommons/hive/crux/sharedfile" ) const ( @@ -342,8 +347,27 @@ func ApplyProfile(tokens *OAuthTokens, prof Profile, fetchErr error, previous *O return tokens.SubscriptionType != "" || tokens.RateLimitTier != "" } -// WriteCredentials atomically writes tokens to the credentials file. +// ErrNoAccessToken is returned by WriteCredentials for a token set with no +// access token. Writing one would replace whatever login is on file with a +// signed-out credential. +var ErrNoAccessToken = errors.New("refusing to write a Claude credential with no access token") + +// CredentialsFileMode is the mode of the shared credentials file: owner and +// group read/write, so every agent uid reaches it through the shared group. +const CredentialsFileMode = 0o660 + +// WriteCredentials atomically writes tokens to the credentials file. It +// refuses a nil token set or one with an empty access token +// (ErrNoAccessToken), so no hive caller can replace the shared login with a +// signed-out credential. The write is sharedfile.Replace: the credentials +// directory is shared with every agent uid, so the temporary name is random, +// metadata is set by descriptor, and a non-regular file at the destination or +// a directory that is not this process's own is refused rather than written +// through. func WriteCredentials(tokens *OAuthTokens, path string) error { + if tokens == nil || tokens.AccessToken == "" { + return ErrNoAccessToken + } dir := filepath.Dir(path) if err := os.MkdirAll(dir, 0o2775); err != nil { return fmt.Errorf("create credentials dir: %w", err) @@ -354,13 +378,30 @@ func WriteCredentials(tokens *OAuthTokens, path string) error { if err != nil { return fmt.Errorf("marshal credentials: %w", err) } + if err := removeDeadCredentialLink(path); err != nil { + return fmt.Errorf("write credentials: %w", err) + } + if err := sharedfile.Replace(path, data, CredentialsFileMode, sharedfile.Options{PreserveOwner: true}); err != nil { + return fmt.Errorf("write credentials: %w", err) + } + return nil +} - tmpPath := path + ".tmp" - if err := os.WriteFile(tmpPath, data, 0o660); err != nil { - return fmt.Errorf("write temp credentials: %w", err) +// removeDeadCredentialLink unlinks a symlink at path that resolves to nothing +// (dangling, or a loop such as a link to itself, which an older bridge repair +// once left on live hives) so a sign-in can recover the shared login. A link +// that resolves to an existing file is left alone: sharedfile.Replace refuses +// it, because nothing hive writes may land through a link another uid planted. +func removeDeadCredentialLink(path string) error { + info, err := os.Lstat(path) + if err != nil || info.Mode()&os.ModeSymlink == 0 { + return nil + } + if _, err := os.Stat(path); err == nil || !(errors.Is(err, fs.ErrNotExist) || errors.Is(err, syscall.ELOOP)) { + return nil } - if err := os.Rename(tmpPath, path); err != nil { - return fmt.Errorf("rename credentials: %w", err) + if err := os.Remove(path); err != nil && !errors.Is(err, fs.ErrNotExist) { + return fmt.Errorf("remove dead credential link: %w", err) } return nil } @@ -369,10 +410,29 @@ func WriteCredentials(tokens *OAuthTokens, path string) error { // nil when the file is absent, malformed, or carries no token block. It never // applies an expiry rule — callers decide what a stale token means to them. func LoadTokens(path string) *OAuthTokens { - data, err := os.ReadFile(path) + tokens, err := ReadTokens(path) if err != nil { return nil } + return tokens +} + +// ReadTokens is LoadTokens with the read error kept, for callers that must +// tell "this process cannot read the file" (fs.ErrNotExist, or +// fs.ErrPermission on another uid's 0600 credential) apart from "the file +// holds no usable token". A file that was read but is malformed, or carries +// no token block, returns (nil, nil). The error never carries file content. +func ReadTokens(path string) (*OAuthTokens, error) { + data, err := os.ReadFile(path) + if err != nil { + return nil, err + } + return ParseTokens(data), nil +} + +// ParseTokens decodes a credentials file already in memory. Nil for malformed +// bytes or a file with no token block, like LoadTokens. +func ParseTokens(data []byte) *OAuthTokens { var creds Credentials if err := json.Unmarshal(data, &creds); err != nil { return nil diff --git a/src/crux/claude/oauth_write_error_branches_test.go b/src/crux/claude/oauth_write_error_branches_test.go index b8b9e8881b..97a1e9a2dc 100644 --- a/src/crux/claude/oauth_write_error_branches_test.go +++ b/src/crux/claude/oauth_write_error_branches_test.go @@ -1,10 +1,14 @@ package claude import ( + "errors" "os" "path/filepath" "strings" "testing" + "time" + + "github.com/hivecommons/hive/crux/sharedfile" ) // tokenEndpoint must fall back to the real Claude TokenURL when no test @@ -35,8 +39,11 @@ func TestWriteCredentials_TempWriteError(t *testing.T) { if err == nil { t.Fatalf("WriteCredentials into read-only dir: err = nil, want error") } - if !strings.Contains(err.Error(), "write temp credentials") { - t.Fatalf("error = %q, want it to mention write temp credentials", err) + if !strings.Contains(err.Error(), "write credentials") { + t.Fatalf("error = %q, want it to mention write credentials", err) + } + if left, _ := filepath.Glob(filepath.Join(dir, "creds.json.*")); len(left) != 0 { + t.Fatalf("temporary files left behind: %v", left) } } @@ -51,9 +58,54 @@ func TestWriteCredentials_RenameError(t *testing.T) { err := WriteCredentials(&OAuthTokens{AccessToken: "at"}, path) if err == nil { - t.Fatalf("WriteCredentials over non-empty dir: err = nil, want error") + t.Fatalf("WriteCredentials over a directory: err = nil, want error") + } + if !errors.Is(err, sharedfile.ErrUnsafeLocation) { + t.Fatalf("error = %q, want sharedfile.ErrUnsafeLocation", err) + } + if left, _ := filepath.Glob(path + ".*"); len(left) != 0 { + t.Fatalf("temporary files left behind: %v", left) + } +} + +// A symlink at the credentials path is refused, and the file it points at is +// left exactly as it was. +func TestWriteCredentials_SymlinkAtPathIsRefused(t *testing.T) { + dir := t.TempDir() + victim := filepath.Join(t.TempDir(), "victim") + if err := os.WriteFile(victim, []byte("keep"), 0o600); err != nil { + t.Fatal(err) + } + path := filepath.Join(dir, ".credentials.json") + if err := os.Symlink(victim, path); err != nil { + t.Fatal(err) + } + err := WriteCredentials(&OAuthTokens{AccessToken: "at"}, path) + if !errors.Is(err, sharedfile.ErrUnsafeLocation) { + t.Fatalf("error = %v, want sharedfile.ErrUnsafeLocation", err) + } + if got, _ := os.ReadFile(victim); string(got) != "keep" { + t.Fatalf("victim rewritten: %q", got) + } + if info, _ := os.Lstat(victim); info.Mode().Perm() != 0o600 { + t.Fatalf("victim mode changed: %v", info.Mode()) + } +} + +// TestWriteCredentialsRefusesSignedOutTokens: no hive writer may put a +// credential with no access token over the shared login. +func TestWriteCredentialsRefusesSignedOutTokens(t *testing.T) { + path := filepath.Join(t.TempDir(), ".credentials.json") + if err := WriteCredentials(&OAuthTokens{AccessToken: "live", ExpiresAt: time.Now().Add(time.Hour).UnixMilli()}, path); err != nil { + t.Fatal(err) + } + before, _ := os.ReadFile(path) + for _, tok := range []*OAuthTokens{nil, {}, {RefreshToken: "r", ExpiresAt: 1}} { + if err := WriteCredentials(tok, path); !errors.Is(err, ErrNoAccessToken) { + t.Errorf("WriteCredentials(%+v) = %v, want ErrNoAccessToken", tok, err) + } } - if !strings.Contains(err.Error(), "rename credentials") { - t.Fatalf("error = %q, want it to mention rename credentials", err) + if after, _ := os.ReadFile(path); string(after) != string(before) { + t.Error("the shared login was replaced by a signed-out credential") } } diff --git a/src/crux/claude/usable_token_test.go b/src/crux/claude/usable_token_test.go index bde265b4cc..720c474a5b 100644 --- a/src/crux/claude/usable_token_test.go +++ b/src/crux/claude/usable_token_test.go @@ -2,6 +2,8 @@ package claude import ( "encoding/json" + "errors" + "io/fs" "os" "path/filepath" "testing" @@ -132,3 +134,66 @@ func TestRefreshTokenExpiresAtRoundTrips(t *testing.T) { t.Fatalf("refreshTokenExpiresAt lost on round trip: got %+v", got) } } + +// TestReadTokens_KeepsTheReadError: the credential bridge must tell "this +// process may not open another uid's 0600 file" apart from "no token here"; +// LoadTokens flattens both to nil, ReadTokens does not. +func TestReadTokens_KeepsTheReadError(t *testing.T) { + if _, err := ReadTokens(filepath.Join(t.TempDir(), "missing")); !errors.Is(err, fs.ErrNotExist) { + t.Errorf("missing file: err = %v, want ErrNotExist", err) + } + bad := filepath.Join(t.TempDir(), ".credentials.json") + if err := os.WriteFile(bad, []byte("{not json"), 0o600); err != nil { + t.Fatal(err) + } + if tok, err := ReadTokens(bad); err != nil || tok != nil { + t.Errorf("malformed: (%v, %v), want nil, nil (readable, nothing usable)", tok, err) + } + if tok := LoadTokens(bad); tok != nil { + t.Errorf("LoadTokens malformed = %v, want nil", tok) + } + empty := filepath.Join(t.TempDir(), ".credentials.json") + if err := os.WriteFile(empty, []byte(`{}`), 0o600); err != nil { + t.Fatal(err) + } + if tok, err := ReadTokens(empty); err != nil || tok != nil { + t.Errorf("no token block: (%v, %v), want nil, nil", tok, err) + } + if os.Getuid() != 0 { + locked := writeCredFile(t, &OAuthTokens{AccessToken: "a", ExpiresAt: ms(time.Hour)}) + if err := os.Chmod(locked, 0); err != nil { + t.Fatal(err) + } + if _, err := ReadTokens(locked); !errors.Is(err, fs.ErrPermission) { + t.Errorf("unreadable: err = %v, want ErrPermission", err) + } + } +} + +func TestParseTokens(t *testing.T) { + if tok := ParseTokens([]byte("nope")); tok != nil { + t.Errorf("malformed = %v", tok) + } + if tok := ParseTokens([]byte(`{}`)); tok != nil { + t.Errorf("no block = %v", tok) + } + tok := ParseTokens([]byte(`{"claudeAiOauth":{"accessToken":"a","refreshToken":"r","expiresAt":42}}`)) + if tok == nil || tok.AccessToken != "a" || tok.RefreshToken != "r" || tok.ExpiresAt != 42 { + t.Errorf("parsed = %+v", tok) + } +} + +func TestTokensUsable_AtAnInstant(t *testing.T) { + now := time.Now() + later := now.Add(2 * time.Hour) + tok := &OAuthTokens{AccessToken: "a", ExpiresAt: now.Add(time.Hour).UnixMilli()} + if !TokensUsable(tok, now) { + t.Error("live access token at now should be usable") + } + if TokensUsable(tok, later) { + t.Error("the same token judged after its expiry should be unusable") + } + if TokensUsable(nil, now) { + t.Error("nil is never usable") + } +} diff --git a/src/crux/sharedfile/sharedfile.go b/src/crux/sharedfile/sharedfile.go new file mode 100644 index 0000000000..9d4fd05ce0 --- /dev/null +++ b/src/crux/sharedfile/sharedfile.go @@ -0,0 +1,278 @@ +// Package sharedfile replaces and reads files that live in directories other +// uids can write to: the shared CLI credential and config files under +// /data/home that the hive maintains for a fleet of per-UID agents. +// +// A plain "write .tmp, chmod, rename" in such a directory acts on +// whatever is at those names when each call runs. Here every step acts on the +// inode this process created, by descriptor, and the name of that inode is +// unpredictable; the directory and the destination are checked to be what the +// caller expects (a directory this process owns, a regular file or nothing) +// right before the rename, and a symlink at any of those places is a refusal, +// never something to write through. +package sharedfile + +import ( + "crypto/rand" + "encoding/hex" + "errors" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "syscall" +) + +// ErrUnsafeLocation is wrapped by Replace when the destination directory or +// the destination itself is not in a state this package will write through: +// the directory is missing, a symlink, world-writable, or not owned by this +// process, or the destination exists and is not a regular file. Nothing is +// written when it is returned. Callers that poll should log it once, not per +// cycle. +var ErrUnsafeLocation = errors.New("refusing to replace a file at an unsafe location") + +// ErrTooLarge is returned by ReadNoFollow for a file over the caller's cap. +var ErrTooLarge = errors.New("file is larger than the read cap") + +// ErrNotRegular is returned by ReadNoFollow when the path is a symlink or any +// other non-regular file. +var ErrNotRegular = errors.New("not a regular file") + +// createAttempts bounds the search for a free temporary name. A collision on +// a 64-bit random suffix is not expected; the retries make an EEXIST from a +// deliberately pre-created name a failure rather than a write-through. +const createAttempts = 8 + +// chownFile applies an owner to the inode behind an open descriptor. A seam: +// tests run as one uid and cannot change an owner for real. +var chownFile = func(f *os.File, uid, gid int) error { return f.Chown(uid, gid) } + +// randRead fills the temporary-name suffix; a seam so tests can force a +// collision or a failure. +var randRead = rand.Read + +// syncFile flushes the written temporary; a seam so tests can fail it. +var syncFile = func(f *os.File) error { return f.Sync() } + +// Options tune Replace. +type Options struct { + // PreserveOwner re-applies the previous destination's owner to the new + // file, best-effort: an unprivileged process cannot chown, and the file it + // wrote is already readable to the group by mode. + PreserveOwner bool +} + +// Replace atomically replaces target with data at mode. The data is written +// to a new file with a random name beside target (O_EXCL|O_NOFOLLOW, 0600), +// chmodded, optionally chowned, fsynced and verified by descriptor, and then +// renamed over target; the directory is fsynced afterwards. On any failure the +// temporary file is removed and target is untouched. +// +// Before the rename, the directory must be a real directory (not a symlink) +// owned by this process and not world-writable, and target must be absent or +// a regular file (not a symlink); otherwise ErrUnsafeLocation. +func Replace(target string, data []byte, mode fs.FileMode, opts Options) error { + dir := filepath.Dir(target) + prev, err := checkLocation(dir, target) + if err != nil { + return err + } + f, tmp, err := createTemp(target) + if err != nil { + return err + } + err = fill(f, data, mode, prev, opts) + if cerr := f.Close(); err == nil { + err = cerr + } + if err != nil { + _ = os.Remove(tmp) + return err + } + // The location can have changed while the file was being written. + if _, err := checkLocation(dir, target); err != nil { + _ = os.Remove(tmp) + return err + } + if err := os.Rename(tmp, target); err != nil { + _ = os.Remove(tmp) + return err + } + syncDir(dir) + return nil +} + +// CheckLocation verifies target's directory and target as documented on +// Replace, and returns the current target's Lstat (nil when absent). Callers +// that do preparatory work beside target (a backup copy, say) use it to refuse +// before that work, not only at the final write. +func CheckLocation(target string) (os.FileInfo, error) { + return checkLocation(filepath.Dir(target), target) +} + +// checkLocation verifies dir and target as documented on Replace and returns +// the current target's stat (nil when absent). +func checkLocation(dir, target string) (os.FileInfo, error) { + dinfo, err := os.Lstat(dir) + if err != nil { + return nil, fmt.Errorf("%w: %s: %w", ErrUnsafeLocation, dir, err) + } + if !dinfo.IsDir() { + return nil, fmt.Errorf("%w: %s is not a directory (mode %s)", ErrUnsafeLocation, dir, dinfo.Mode()) + } + if dinfo.Mode().Perm()&0o002 != 0 { + return nil, fmt.Errorf("%w: %s is world-writable (mode %s)", ErrUnsafeLocation, dir, dinfo.Mode()) + } + if st, ok := dinfo.Sys().(*syscall.Stat_t); ok { + if euid := os.Geteuid(); euid != 0 && int(st.Uid) != euid { + return nil, fmt.Errorf("%w: %s is owned by uid %d, this process is uid %d", ErrUnsafeLocation, dir, st.Uid, euid) + } + } + tinfo, err := os.Lstat(target) + if errors.Is(err, fs.ErrNotExist) { + return nil, nil + } + if err != nil { + return nil, fmt.Errorf("%w: %s: %w", ErrUnsafeLocation, target, err) + } + if !tinfo.Mode().IsRegular() { + return nil, fmt.Errorf("%w: %s is not a regular file (mode %s)", ErrUnsafeLocation, target, tinfo.Mode()) + } + return tinfo, nil +} + +// createTemp creates a new 0600 file with a random name beside target. The +// name is ".<16 hex>.tmp", so sweeps that match ".*.tmp" still +// recognise a leftover. +func createTemp(target string) (*os.File, string, error) { + for i := 0; i < createAttempts; i++ { + var b [8]byte + if _, err := randRead(b[:]); err != nil { + return nil, "", err + } + tmp := target + "." + hex.EncodeToString(b[:]) + ".tmp" + f, err := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_EXCL|syscall.O_NOFOLLOW, 0o600) + if errors.Is(err, fs.ErrExist) { + continue + } + if err != nil { + return nil, "", err + } + return f, tmp, nil + } + return nil, "", fmt.Errorf("no free temporary name beside %s after %d attempts", target, createAttempts) +} + +// fill writes data to f and sets its metadata, all through the descriptor, +// then verifies by fstat that the inode is a regular file with exactly the +// mode (and, when a chown succeeded, the owner) asked for. +func fill(f *os.File, data []byte, mode fs.FileMode, prev os.FileInfo, opts Options) error { + if _, err := f.Write(data); err != nil { + return err + } + if err := f.Chmod(mode); err != nil { + return err + } + wantUID, wantGID := -1, -1 + if opts.PreserveOwner && prev != nil { + if st, ok := prev.Sys().(*syscall.Stat_t); ok { + if err := chownFile(f, int(st.Uid), int(st.Gid)); err == nil { + wantUID, wantGID = int(st.Uid), int(st.Gid) + } + } + } + if err := syncFile(f); err != nil { + return err + } + info, err := f.Stat() + if err != nil { + return err + } + if !info.Mode().IsRegular() || info.Mode().Perm() != mode.Perm() { + return fmt.Errorf("%s: written file is mode %s, want regular %s", f.Name(), info.Mode(), mode.Perm()) + } + if st, ok := info.Sys().(*syscall.Stat_t); ok { + if st.Nlink != 1 { + return fmt.Errorf("%s: written file has %d links, want 1", f.Name(), st.Nlink) + } + if wantUID >= 0 && (int(st.Uid) != wantUID || int(st.Gid) != wantGID) { + return fmt.Errorf("%s: written file is owned by %d:%d, want %d:%d", f.Name(), st.Uid, st.Gid, wantUID, wantGID) + } + } + return nil +} + +// syncDir makes the rename durable; best-effort, like every writer before it. +func syncDir(dir string) { + d, err := os.Open(dir) + if err != nil { + return + } + _ = d.Sync() + _ = d.Close() +} + +// openRegularNoFollow opens path without following a symlink and confirms the +// opened file is regular. ErrNotRegular for a symlink or any other kind. +func openRegularNoFollow(path string, flag int) (*os.File, os.FileInfo, error) { + f, err := os.OpenFile(path, flag|syscall.O_NOFOLLOW, 0) + if err != nil { + var pe *fs.PathError + if errors.As(err, &pe) && (errors.Is(pe.Err, syscall.ELOOP) || errors.Is(pe.Err, syscall.EMLINK)) { + return nil, nil, fmt.Errorf("%w: %s is a symlink", ErrNotRegular, path) + } + return nil, nil, err + } + info, err := f.Stat() + if err == nil && !info.Mode().IsRegular() { + err = fmt.Errorf("%w: %s (mode %s)", ErrNotRegular, path, info.Mode()) + } + if err != nil { + f.Close() + return nil, nil, err + } + return f, info, nil +} + +// Chmod sets mode on the regular file at path through an open descriptor, so +// a symlink planted at path is refused (ErrNotRegular) instead of followed. +// Returns the mode the file had before, and changed=false when it already +// matched (nothing is done in that case). +func Chmod(path string, mode fs.FileMode) (prev fs.FileMode, changed bool, err error) { + f, info, err := openRegularNoFollow(path, os.O_RDONLY) + if err != nil { + return 0, false, err + } + defer f.Close() + prev = info.Mode().Perm() + if prev == mode.Perm() { + return prev, false, nil + } + if err := f.Chmod(mode.Perm()); err != nil { + return prev, false, err + } + return prev, true, nil +} + +// ReadNoFollow reads a regular file of at most max bytes without following a +// symlink at path. ErrNotRegular for a symlink or other non-regular file, +// ErrTooLarge over the cap; the open error (fs.ErrNotExist, fs.ErrPermission) +// otherwise. +func ReadNoFollow(path string, max int64) ([]byte, error) { + f, info, err := openRegularNoFollow(path, os.O_RDONLY) + if err != nil { + return nil, err + } + defer f.Close() + if info.Size() > max { + return nil, fmt.Errorf("%w: %s is %d bytes, cap %d", ErrTooLarge, path, info.Size(), max) + } + data, err := io.ReadAll(io.LimitReader(f, max+1)) + if err != nil { + return nil, err + } + if int64(len(data)) > max { + return nil, fmt.Errorf("%w: %s grew past %d bytes while being read", ErrTooLarge, path, max) + } + return data, nil +} diff --git a/src/crux/sharedfile/sharedfile_test.go b/src/crux/sharedfile/sharedfile_test.go new file mode 100644 index 0000000000..d8307e37bc --- /dev/null +++ b/src/crux/sharedfile/sharedfile_test.go @@ -0,0 +1,480 @@ +package sharedfile + +import ( + "bytes" + "errors" + "io/fs" + "os" + "path/filepath" + "strings" + "syscall" + "testing" +) + +func leftovers(t *testing.T, target string) []string { + t.Helper() + m, err := filepath.Glob(target + ".*.tmp") + if err != nil { + t.Fatal(err) + } + return m +} + +func modeOf(t *testing.T, path string) fs.FileMode { + t.Helper() + info, err := os.Lstat(path) + if err != nil { + t.Fatal(err) + } + return info.Mode() +} + +func TestReplace_WritesAtModeAndLeavesNoTemp(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, "config.json") + if err := Replace(target, []byte("one"), 0o660, Options{}); err != nil { + t.Fatal(err) + } + if got, _ := os.ReadFile(target); string(got) != "one" { + t.Errorf("content = %q", got) + } + if m := modeOf(t, target); !m.IsRegular() || m.Perm() != 0o660 { + t.Errorf("mode = %v, want regular 0660", m) + } + // Replacing an existing regular file works and keeps the new mode. + if err := Replace(target, []byte("two"), 0o640, Options{PreserveOwner: true}); err != nil { + t.Fatal(err) + } + if got, _ := os.ReadFile(target); string(got) != "two" { + t.Errorf("content after replace = %q", got) + } + if m := modeOf(t, target); m.Perm() != 0o640 { + t.Errorf("mode after replace = %v", m) + } + if l := leftovers(t, target); len(l) != 0 { + t.Errorf("temporary files left: %v", l) + } +} + +// Two writes never share a temporary name, and the name is not derivable +// from the target alone. +func TestReplace_TempNameIsUnpredictable(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, "f") + var names []string + old := chownFile + chownFile = func(f *os.File, uid, gid int) error { + names = append(names, filepath.Base(f.Name())) + return nil + } + t.Cleanup(func() { chownFile = old }) + if err := os.WriteFile(target, []byte("seed"), 0o600); err != nil { + t.Fatal(err) + } + for i := 0; i < 2; i++ { + if err := Replace(target, []byte("x"), 0o600, Options{PreserveOwner: true}); err != nil { + t.Fatal(err) + } + } + if len(names) != 2 || names[0] == names[1] { + t.Fatalf("temp names = %v, want two distinct", names) + } + for _, n := range names { + if !strings.HasPrefix(n, "f.") || !strings.HasSuffix(n, ".tmp") || len(n) != len("f.")+16+len(".tmp") { + t.Errorf("temp name %q is not .<16 hex>.tmp", n) + } + } + // The sentinel a fixed-name writer would use is never created. + if _, err := os.Lstat(target + ".tmp"); !errors.Is(err, fs.ErrNotExist) { + t.Error("a predictable .tmp exists") + } +} + +func TestReplace_RefusesSymlinkAtTarget(t *testing.T) { + dir := t.TempDir() + victim := filepath.Join(t.TempDir(), "victim") + if err := os.WriteFile(victim, []byte("keep"), 0o600); err != nil { + t.Fatal(err) + } + target := filepath.Join(dir, "cred") + if err := os.Symlink(victim, target); err != nil { + t.Fatal(err) + } + err := Replace(target, []byte("evil"), 0o660, Options{PreserveOwner: true}) + if !errors.Is(err, ErrUnsafeLocation) { + t.Fatalf("err = %v, want ErrUnsafeLocation", err) + } + if got, _ := os.ReadFile(victim); string(got) != "keep" { + t.Errorf("victim rewritten: %q", got) + } + if m := modeOf(t, victim); m.Perm() != 0o600 { + t.Errorf("victim mode changed: %v", m) + } + if m := modeOf(t, target); m&fs.ModeSymlink == 0 { + t.Error("symlink at target was replaced") + } + if l := leftovers(t, target); len(l) != 0 { + t.Errorf("temporary files left: %v", l) + } +} + +func TestReplace_RefusesUnsafeDirectory(t *testing.T) { + real := t.TempDir() + // A symlink where the directory should be. + linkDir := filepath.Join(t.TempDir(), "home") + if err := os.Symlink(real, linkDir); err != nil { + t.Fatal(err) + } + if err := Replace(filepath.Join(linkDir, "f"), []byte("x"), 0o600, Options{}); !errors.Is(err, ErrUnsafeLocation) { + t.Errorf("symlinked dir: err = %v, want ErrUnsafeLocation", err) + } + if entries, _ := os.ReadDir(real); len(entries) != 0 { + t.Errorf("wrote through the directory symlink: %v", entries) + } + // A missing directory. + if err := Replace(filepath.Join(real, "nope", "f"), []byte("x"), 0o600, Options{}); !errors.Is(err, ErrUnsafeLocation) { + t.Errorf("missing dir: err = %v", err) + } + // A world-writable directory. + ww := filepath.Join(t.TempDir(), "ww") + if err := os.Mkdir(ww, 0o777); err != nil { + t.Fatal(err) + } + if err := os.Chmod(ww, 0o1777); err != nil { + t.Fatal(err) + } + if err := Replace(filepath.Join(ww, "f"), []byte("x"), 0o600, Options{}); !errors.Is(err, ErrUnsafeLocation) { + t.Errorf("world-writable dir: err = %v", err) + } + // A file where the directory should be. + plain := filepath.Join(t.TempDir(), "plain") + if err := os.WriteFile(plain, nil, 0o600); err != nil { + t.Fatal(err) + } + if err := Replace(filepath.Join(plain, "f"), []byte("x"), 0o600, Options{}); !errors.Is(err, ErrUnsafeLocation) { + t.Errorf("file as dir: err = %v", err) + } + // A directory where the target should be. + dir := t.TempDir() + if err := os.Mkdir(filepath.Join(dir, "d"), 0o700); err != nil { + t.Fatal(err) + } + if err := Replace(filepath.Join(dir, "d"), []byte("x"), 0o600, Options{}); !errors.Is(err, ErrUnsafeLocation) { + t.Errorf("dir as target: err = %v", err) + } +} + +func TestReplace_ChownFailureIsBestEffortAndVerified(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, "f") + if err := os.WriteFile(target, []byte("seed"), 0o600); err != nil { + t.Fatal(err) + } + old := chownFile + chownFile = func(*os.File, int, int) error { return errors.New("EPERM") } + t.Cleanup(func() { chownFile = old }) + if err := Replace(target, []byte("new"), 0o660, Options{PreserveOwner: true}); err != nil { + t.Fatalf("a failed chown must not fail the write: %v", err) + } + if got, _ := os.ReadFile(target); string(got) != "new" { + t.Errorf("content = %q", got) + } + // A chown that "succeeds" but leaves the wrong owner is caught by fstat. + chownFile = func(*os.File, int, int) error { return nil } + victimOwner := filepath.Join(dir, "g") + if err := os.WriteFile(victimOwner, []byte("seed"), 0o600); err != nil { + t.Fatal(err) + } + // Pretend the previous file belonged to someone else by faking prev. + prev := fakeInfo{FileInfo: must(os.Lstat(victimOwner)), uid: 12345, gid: 12345} + f, tmp, err := createTemp(victimOwner) + if err != nil { + t.Fatal(err) + } + err = fill(f, []byte("x"), 0o660, prev, Options{PreserveOwner: true}) + _ = f.Close() + _ = os.Remove(tmp) + if err == nil || !strings.Contains(err.Error(), "owned by") { + t.Errorf("owner mismatch not detected: %v", err) + } +} + +type fakeInfo struct { + os.FileInfo + uid, gid uint32 +} + +func (f fakeInfo) Sys() any { + st, _ := f.FileInfo.Sys().(*syscall.Stat_t) + cp := *st + cp.Uid, cp.Gid = f.uid, f.gid + return &cp +} + +func must(info os.FileInfo, err error) os.FileInfo { + if err != nil { + panic(err) + } + return info +} + +func TestCreateTemp_SkipsTakenNames(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, "f") + f, tmp, err := createTemp(target) + if err != nil { + t.Fatal(err) + } + _ = f.Close() + if m := modeOf(t, tmp); m.Perm() != 0o600 { + t.Errorf("temp mode = %v, want 0600", m) + } + // A second create beside the first gets another name. + g, tmp2, err := createTemp(target) + if err != nil { + t.Fatal(err) + } + _ = g.Close() + if tmp == tmp2 { + t.Error("same temp name twice") + } +} + +func TestReadNoFollow(t *testing.T) { + dir := t.TempDir() + p := filepath.Join(dir, "f") + if err := os.WriteFile(p, []byte("hello"), 0o600); err != nil { + t.Fatal(err) + } + got, err := ReadNoFollow(p, 64) + if err != nil || !bytes.Equal(got, []byte("hello")) { + t.Errorf("read = %q, %v", got, err) + } + if _, err := ReadNoFollow(p, 4); !errors.Is(err, ErrTooLarge) { + t.Errorf("over cap: err = %v", err) + } + link := filepath.Join(dir, "l") + if err := os.Symlink(p, link); err != nil { + t.Fatal(err) + } + if _, err := ReadNoFollow(link, 64); !errors.Is(err, ErrNotRegular) { + t.Errorf("symlink: err = %v, want ErrNotRegular", err) + } + if _, err := ReadNoFollow(dir, 64); !errors.Is(err, ErrNotRegular) { + t.Errorf("directory: err = %v, want ErrNotRegular", err) + } + if _, err := ReadNoFollow(filepath.Join(dir, "missing"), 64); !errors.Is(err, fs.ErrNotExist) { + t.Errorf("missing: err = %v", err) + } + if os.Getuid() != 0 { + if err := os.Chmod(p, 0); err != nil { + t.Fatal(err) + } + if _, err := ReadNoFollow(p, 64); !errors.Is(err, fs.ErrPermission) { + t.Errorf("unreadable: err = %v, want ErrPermission", err) + } + } +} + +func TestChmod(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "f") + if err := os.WriteFile(path, []byte("x"), 0o600); err != nil { + t.Fatal(err) + } + prev, changed, err := Chmod(path, 0o660) + if err != nil || !changed || prev != 0o600 { + t.Fatalf("Chmod = (%o, %v, %v)", prev, changed, err) + } + if info, _ := os.Stat(path); info.Mode().Perm() != 0o660 { + t.Fatalf("mode = %v", info.Mode()) + } + if _, changed, err := Chmod(path, 0o660); err != nil || changed { + t.Fatalf("already matching: changed=%v err=%v", changed, err) + } + if _, _, err := Chmod(filepath.Join(dir, "absent"), 0o660); !errors.Is(err, fs.ErrNotExist) { + t.Fatalf("absent: %v", err) + } + + victim := filepath.Join(dir, "victim") + if err := os.WriteFile(victim, []byte("x"), 0o600); err != nil { + t.Fatal(err) + } + link := filepath.Join(dir, "link") + if err := os.Symlink(victim, link); err != nil { + t.Fatal(err) + } + if _, _, err := Chmod(link, 0o666); !errors.Is(err, ErrNotRegular) { + t.Fatalf("symlink: %v, want ErrNotRegular", err) + } + if info, _ := os.Stat(victim); info.Mode().Perm() != 0o600 { + t.Fatalf("victim mode changed through the symlink: %v", info.Mode()) + } + if _, _, err := Chmod(dir, 0o700); !errors.Is(err, ErrNotRegular) { + t.Fatalf("directory: %v, want ErrNotRegular", err) + } +} + +func TestCheckLocation(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, "f") + if info, err := CheckLocation(target); err != nil || info != nil { + t.Fatalf("absent: (%v, %v)", info, err) + } + if err := os.WriteFile(target, []byte("x"), 0o600); err != nil { + t.Fatal(err) + } + if info, err := CheckLocation(target); err != nil || info == nil || !info.Mode().IsRegular() { + t.Fatalf("regular: (%v, %v)", info, err) + } + if os.Geteuid() != 0 { + // A directory that cannot be searched: the target's Lstat fails with + // something other than "not exist", and that is refused too. + closed := filepath.Join(dir, "closed") + if err := os.Mkdir(closed, 0o600); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chmod(closed, 0o700) }) + if _, err := CheckLocation(filepath.Join(closed, "f")); !errors.Is(err, ErrUnsafeLocation) { + t.Errorf("unsearchable dir: %v", err) + } + } +} + +func TestCreateTemp_RandomFailures(t *testing.T) { + dir := t.TempDir() + target := filepath.Join(dir, "f") + old := randRead + t.Cleanup(func() { randRead = old }) + + randRead = func([]byte) (int, error) { return 0, errors.New("no entropy") } + if _, _, err := createTemp(target); err == nil || !strings.Contains(err.Error(), "no entropy") { + t.Errorf("entropy failure: %v", err) + } + if left := leftovers(t, target); len(left) != 0 { + t.Errorf("leftovers: %v", left) + } + + // Every name the generator can produce is already taken: refused after + // createAttempts, never written through the existing file. + randRead = func(b []byte) (int, error) { + for i := range b { + b[i] = 0xAB + } + return len(b), nil + } + taken := target + "." + strings.Repeat("ab", 8) + ".tmp" + if err := os.WriteFile(taken, []byte("keep"), 0o600); err != nil { + t.Fatal(err) + } + if err := Replace(target, []byte("new"), 0o600, Options{}); err == nil || !strings.Contains(err.Error(), "no free temporary name") { + t.Errorf("all names taken: %v", err) + } + if got, _ := os.ReadFile(taken); string(got) != "keep" { + t.Errorf("the taken name was written through: %q", got) + } + if _, err := os.Lstat(target); !errors.Is(err, fs.ErrNotExist) { + t.Errorf("target appeared: %v", err) + } +} + +func TestReplace_FailuresLeaveNoTemp(t *testing.T) { + if os.Geteuid() == 0 { + t.Skip("root is not stopped by directory modes") + } + old := chownFile + t.Cleanup(func() { chownFile = old }) + + // Cannot create the temporary: the directory is read-only. + ro := t.TempDir() + if err := os.Chmod(ro, 0o500); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chmod(ro, 0o700) }) + if err := Replace(filepath.Join(ro, "f"), []byte("x"), 0o600, Options{}); err == nil || errors.Is(err, ErrUnsafeLocation) { + t.Errorf("read-only dir: err = %v, want a create error", err) + } + + // The written file fails verification (mode changed under us). + dir := t.TempDir() + target := filepath.Join(dir, "f") + if err := os.WriteFile(target, []byte("seed"), 0o600); err != nil { + t.Fatal(err) + } + chownFile = func(f *os.File, _, _ int) error { return f.Chmod(0o644) } + if err := Replace(target, []byte("new"), 0o660, Options{PreserveOwner: true}); err == nil || !strings.Contains(err.Error(), "written file is mode") { + t.Errorf("mode drift: %v", err) + } + if got, _ := os.ReadFile(target); string(got) != "seed" { + t.Errorf("target replaced despite failed verification: %q", got) + } + if left := leftovers(t, target); len(left) != 0 { + t.Errorf("leftovers after failed fill: %v", left) + } + + // The flush fails: no rename, no leftover. + oldSync := syncFile + syncFile = func(*os.File) error { return errors.New("EIO") } + err := Replace(target, []byte("new"), 0o660, Options{}) + syncFile = oldSync + if err == nil || !strings.Contains(err.Error(), "EIO") { + t.Errorf("sync failure: %v", err) + } + if got, _ := os.ReadFile(target); string(got) != "seed" { + t.Errorf("target replaced despite failed sync: %q", got) + } + if left := leftovers(t, target); len(left) != 0 { + t.Errorf("leftovers after failed sync: %v", left) + } + + // The temporary gained a second name while being written. + chownFile = func(f *os.File, _, _ int) error { + hl := f.Name() + ".hl" + t.Cleanup(func() { _ = os.Remove(hl) }) + return os.Link(f.Name(), hl) + } + if err := Replace(target, []byte("new"), 0o660, Options{PreserveOwner: true}); err == nil || !strings.Contains(err.Error(), "links, want 1") { + t.Errorf("extra link: %v", err) + } + if got, _ := os.ReadFile(target); string(got) != "seed" { + t.Errorf("target replaced despite extra link: %q", got) + } + + // The location changes while the temporary is being written. + chownFile = func(*os.File, int, int) error { + _ = os.Remove(target) + return os.Symlink("/nonexistent", target) + } + if err := Replace(target, []byte("new"), 0o660, Options{PreserveOwner: true}); !errors.Is(err, ErrUnsafeLocation) { + t.Errorf("location changed mid-write: %v", err) + } + if info, err := os.Lstat(target); err != nil || info.Mode()&fs.ModeSymlink == 0 { + t.Errorf("the planted link was replaced: %v %v", info, err) + } + if left := leftovers(t, target); len(left) != 0 { + t.Errorf("leftovers after refused rename: %v", left) + } + _ = os.Remove(target) + if err := os.WriteFile(target, []byte("seed"), 0o600); err != nil { + t.Fatal(err) + } + + // The rename itself fails: the directory went read-only after the + // temporary was created. + chownFile = func(*os.File, int, int) error { return os.Chmod(dir, 0o500) } + t.Cleanup(func() { _ = os.Chmod(dir, 0o700) }) + err = Replace(target, []byte("new"), 0o660, Options{PreserveOwner: true}) + _ = os.Chmod(dir, 0o700) + if err == nil || errors.Is(err, ErrUnsafeLocation) { + t.Errorf("rename failure: err = %v", err) + } + if got, _ := os.ReadFile(target); string(got) != "seed" { + t.Errorf("target changed on failed rename: %q", got) + } + // The temporary could not be removed from the read-only directory at + // the time; it is the one leftover this path can produce, and it is + // removable now. + for _, l := range leftovers(t, target) { + _ = os.Remove(l) + } +} diff --git a/src/deploy/entrypoint.sh b/src/deploy/entrypoint.sh index 111d79e2dd..99c3c5a53f 100644 --- a/src/deploy/entrypoint.sh +++ b/src/deploy/entrypoint.sh @@ -150,7 +150,9 @@ hive_run_uid_isolation_migration() { else echo "[entrypoint] UID isolation: repairing ${_agent_name} (uid ${_agent_uid}) in background" hive_reown_agent_paths_if_needed "$_agent_name" "$_agent_uid" - chmod g+rwX "$_agent_dir" 2>/dev/null || true + if [ -d "$_agent_dir" ] && ! [ -L "$_agent_dir" ]; then + chmod g+rwX "$_agent_dir" 2>/dev/null || true + fi hive_publish_uid_isolation_marker "$_agent_marker" "$_agent_expected" || true echo "[entrypoint] UID isolation: ${_agent_name} pass complete" fi @@ -685,6 +687,9 @@ hive_sweep_root_phase_paths() { _sweep_failed="" for _p in $HIVE_DATA_ROOT_PHASE_PATHS; do [ -e "$_p" ] || continue + # Never through a link: chown without -h acts on what the link names, and + # /data/home is writable by every agent uid. + [ -L "$_p" ] && continue # Already owned by the runtime user — the steady state on almost every # boot. Skip the syscall entirely so the common path costs one stat. [ "$(stat -c '%u' "$_p" 2>/dev/null || echo)" = "1001" ] && continue @@ -1421,6 +1426,9 @@ if [ "$(id -u)" = "0" ]; then fi for _seeded_dir in /data/agents/*/; do [ -d "$_seeded_dir" ] || continue + # The trailing-slash glob also matches a link to a directory; a + # recursive chown/chmod must never start from one. + [ -L "${_seeded_dir%/}" ] && continue if [ "$(stat -c '%u' "$_seeded_dir" 2>/dev/null || echo 0)" = "0" ]; then chown -R dev:node "$_seeded_dir" 2>/dev/null || true chmod -R g+rwX "$_seeded_dir" 2>/dev/null || true @@ -1489,6 +1497,27 @@ if [ "$(id -u)" = "0" ]; then chown dev:node /data/home/.copilot 2>/dev/null || true chmod 2775 /data/home/.claude /data/home/.claude/session-env 2>/dev/null || true chown dev:node /data/home/.claude /data/home/.claude/session-env 2>/dev/null || true + # hive_shared_dir_sticky DIR... — add the sticky bit to the shared CLI + # directories whose files the root-run perm guard and the in-process Go + # writers rename over: in a group-writable directory only the sticky bit + # keeps a dev-owned entry from being removed or replaced by another uid. + # Under the per-agent home layout every CLI rewrites its own copy inside + # /data/home/agents/, so nothing legitimate renames over a dev-owned + # file here. The legacy whole-directory layout (HIVE_SHARED_AGENT_HOME=1) + # does — each agent uid refreshes the shared file in place — so the bit is + # left off there. Group bits are untouched; the existing 2770/2775 stays. + # Only directories that are real directories get it: a chmod through a + # symlink would land on whatever the link names. + hive_shared_dir_sticky() { + [ "${HIVE_SHARED_AGENT_HOME:-0}" = "1" ] && return 0 + for _d; do + if [ -d "$_d" ] && ! [ -L "$_d" ]; then + chmod +t "$_d" 2>/dev/null || true + fi + done + return 0 + } + hive_shared_dir_sticky /data/home /data/home/.claude /data/home/.copilot # Codex (newer method) writes a local sqlite state db under $HOME/.codex — # pre-create it group-writable + setgid so every agent UID (node group) can # init it, matching .claude/.copilot. Without this, switching an agent to the @@ -1536,6 +1565,9 @@ if [ "$(id -u)" = "0" ]; then # process touches .bob first. 2770: no world access, these hold auth settings. chmod 2770 /data/home/.bob /data/home/.bob/settings 2>/dev/null || true chown dev:node /data/home/.bob /data/home/.bob/settings 2>/dev/null || true + # Not antigravity-cli/ or its cache/: those are bridged into each agent home + # as directories, so agy renames its token there under the agent's own uid. + hive_shared_dir_sticky /data/home/.gemini /data/home/.bob ln -sfn /data/config/github-copilot /home/dev/.config/github-copilot ln -sfn /data/config/github-copilot /data/home/.config/github-copilot ln -sfn /data/home/.copilot /home/dev/.copilot @@ -1625,18 +1657,52 @@ if [ "$(id -u)" = "0" ]; then # this is an OAuth token. The chown restores dev ownership as well, because # that is the invariant the in-process Go permissions watcher depends on — it # runs as dev after the privilege drop and can only chmod what dev owns. + # + # Both single-file repairs go through hive_fix_shared_file, which acts only + # on a regular file inside a regular directory: chmod and chown (without -h) + # act on whatever a path resolves to, and these directories are writable by + # every agent uid. The invariant is that a root-run repair never changes the + # mode or owner of anything an agent uid could have pointed the path at. hive_fix_shared_credential() { - [ -f "$1" ] || return 0 - chmod g+r "$1" 2>/dev/null || true - chown dev:node "$1" 2>/dev/null || true + hive_fix_shared_file "$1" g+r return 0 } # hive_fix_copilot_config — copilot rewrites config.json 0600 on refresh. hive_fix_copilot_config() { - [ -f /data/home/.copilot/config.json ] || return 0 - chmod 660 /data/home/.copilot/config.json 2>/dev/null || true - chown dev:node /data/home/.copilot/config.json 2>/dev/null || true + hive_fix_shared_file /data/home/.copilot/config.json 660 + return 0 + } + + # hive_fix_shared_file PATH MODE — chmod MODE and chown dev:node the regular + # file at PATH. Nothing is done when PATH is absent. When PATH or its + # directory is anything other than a regular file in a regular directory, + # nothing is done either, and a WARN is logged once per path per guard + # process (HIVE_PERM_GUARD_REFUSED is that process's memory of it). + # + # The `[ -L ]` tests are the decision; the find runs the chmod and chown + # from the same enumeration (`-type f` never matches a link) and `chown -h` + # never follows, so the time between the check and each action is as short + # as the shell can make it. The sticky bit set on the shared dot-dirs at + # boot is what keeps a dev-owned entry from being swapped out from under it. + hive_fix_shared_file() { + _p="$1"; _m="$2" + _dir="${_p%/*}"; _base="${_p##*/}" + if [ -d "$_dir" ] && ! [ -L "$_dir" ] && [ -f "$_p" ] && ! [ -L "$_p" ]; then + find "$_dir" -maxdepth 1 -name "$_base" -type f ! -type l -exec chmod "$_m" {} + -exec chown -h dev:node {} + 2>/dev/null || true + return 0 + fi + # Absent (and not a dangling link): nothing to repair, nothing to say. + if ! [ -e "$_p" ] && ! [ -L "$_p" ]; then + return 0 + fi + case " ${HIVE_PERM_GUARD_REFUSED:-} " in + *" $_p "*) ;; + *) + HIVE_PERM_GUARD_REFUSED="${HIVE_PERM_GUARD_REFUSED:-} $_p" + echo "[entrypoint] WARN: perm guard: $_p is not a regular file in a regular directory; left untouched" >&2 + ;; + esac return 0 } @@ -1747,7 +1813,9 @@ if [ "$(id -u)" = "0" ]; then # sweep still happens, on the 5-minute cycle. hive_fix_claude_instant() { hive_fix_shared_credential /data/home/.claude/.credentials.json - chmod g+rwx /data/home/.claude 2>/dev/null || true + if [ -d /data/home/.claude ] && ! [ -L /data/home/.claude ]; then + chmod g+rwx /data/home/.claude 2>/dev/null || true + fi return 0 } @@ -1765,7 +1833,11 @@ if [ "$(id -u)" = "0" ]; then hive_fix_gemini_instant() { hive_fix_shared_credential /data/home/.gemini/antigravity-cli/antigravity-oauth-token hive_fix_shared_credential /data/home/.gemini/antigravity-cli/cache/onboarding.json - chmod g+rwx /data/home/.gemini /data/home/.gemini/antigravity-cli /data/home/.gemini/antigravity-cli/cache 2>/dev/null || true + for _d in /data/home/.gemini /data/home/.gemini/antigravity-cli /data/home/.gemini/antigravity-cli/cache; do + if [ -d "$_d" ] && ! [ -L "$_d" ]; then + chmod g+rwx "$_d" 2>/dev/null || true + fi + done return 0 } @@ -1898,6 +1970,11 @@ if [ "$(id -u)" = "0" ]; then # launch. Falling back to the proxy CA alone if the combined bundle isn't there # yet keeps proxied HTTPS working; the combined bundle is strictly safer because # it also trusts real public roots for any non-proxied endpoint (advisory mode). + # Root writes this file into a directory every agent uid can write to, so + # it is only ever written when the path is absent or already a regular file. + if [ -L /data/home/.bashrc ]; then + echo "[entrypoint] WARN: /data/home/.bashrc is not a regular file; left untouched" + else cat > /data/home/.bashrc <<'BASHRC' 2>/dev/null || true # Hive agent shell environment if [ -f /data/proxy-ca-bundle.pem ]; then @@ -1914,15 +1991,16 @@ BASHRC # Written by root via `cat >` above, so it is created root:root. Hand it to # dev at the point of creation (#5369): the recursive /data chown is guarded # off on every normal boot and cannot be relied on to fix it later. - chown dev:node /data/home/.bashrc 2>/dev/null || true + chown -h dev:node /data/home/.bashrc 2>/dev/null || true + fi # Login shells (tmux default-command) read ~/.profile, not ~/.bashrc — chain # them so both shell flavors get the same environment. - if [ ! -f /data/home/.profile ]; then + if [ ! -f /data/home/.profile ] && ! [ -L /data/home/.profile ]; then # shellcheck disable=SC2016 printf '[ -f "$HOME/.bashrc" ] && . "$HOME/.bashrc"\n' > /data/home/.profile 2>/dev/null || true chmod 644 /data/home/.profile 2>/dev/null || true # Same as .bashrc above — created by root, so hand it over here (#5369). - chown dev:node /data/home/.profile 2>/dev/null || true + chown -h dev:node /data/home/.profile 2>/dev/null || true fi # ── #5369: targeted post-phase ownership sweep ──────────────────────── @@ -2054,8 +2132,10 @@ if spek.get('enabled') and hub.get('enabled') is not False: fi # Ensure agent dirs are group-writable so the dev user (same node group) # can clean up stale workspaces at runtime without root privileges. - chmod g+rwX "/data/agents/${agent_name}" 2>/dev/null || true - if [ -d "/data/beads/${agent_name}" ]; then + if [ -d "/data/agents/${agent_name}" ] && ! [ -L "/data/agents/${agent_name}" ]; then + chmod g+rwX "/data/agents/${agent_name}" 2>/dev/null || true + fi + if [ -d "/data/beads/${agent_name}" ] && ! [ -L "/data/beads/${agent_name}" ]; then BEAD_DIR_OWNER=$(stat -c '%u' "/data/beads/${agent_name}" 2>/dev/null || echo "0") if [ "$BEAD_DIR_OWNER" != "$AGENT_UID" ]; then chown -R "hive-${agent_name}:node" "/data/beads/${agent_name}" 2>/dev/null || true diff --git a/src/deploy/test_entrypoint_perm_guard.sh b/src/deploy/test_entrypoint_perm_guard.sh index 857dfce8dd..9b34f8ca07 100755 --- a/src/deploy/test_entrypoint_perm_guard.sh +++ b/src/deploy/test_entrypoint_perm_guard.sh @@ -594,6 +594,90 @@ for repair in hive_fix_tree_recent hive_fix_tree; do done rm -f "$SETTINGS" "$SIBLING" +# ── 14. The single-file repairs act only on a regular file in a regular dir ── +# chmod and chown (without -h) act on whatever a path resolves to. The shared +# dot-dirs are writable by every agent uid, so the file-level repairs must do +# nothing at all when the path, or the directory it is in, is not what the +# guard expects — and say so once, not once per 5-second poll. +mode_of() { stat -c '%a' "$1" 2>/dev/null || stat -f '%Lp' "$1"; } +PLANT_HOME="$WORK/plant-home" +PLANT_OUT="$WORK/plant-outside" +mkdir -p "$PLANT_HOME/.claude" "$PLANT_HOME/.copilot" +sed "s#/data/home#${PLANT_HOME}#g" "$GUARDS" > "$WORK/guards.plant.sh" +printf 'outside\n' > "$PLANT_OUT" +chmod 0600 "$PLANT_OUT" + +# A link where the credential should be. +ln -s "$PLANT_OUT" "$PLANT_HOME/.claude/.credentials.json" +ln -s "$PLANT_OUT" "$PLANT_HOME/.copilot/config.json" +PLANT_LOG="$WORK/plant.log" +sh -c 'set -e; . "$1"; hive_fix_credentials_fast; hive_fix_credentials_fast; hive_fix_claude_instant' \ + sh "$WORK/guards.plant.sh" >/dev/null 2>"$PLANT_LOG" +if [ "$(mode_of "$PLANT_OUT")" = "600" ]; then + ok "a link planted at the credential path is not acted on (target stays 0600)" +else + bad "a link planted at the credential path is not acted on" \ + "target went to mode $(mode_of "$PLANT_OUT")" + chmod 0600 "$PLANT_OUT" +fi +if [ -L "$PLANT_HOME/.claude/.credentials.json" ] && [ -L "$PLANT_HOME/.copilot/config.json" ]; then + ok "the planted links themselves are left in place" +else + bad "the planted links themselves are left in place" +fi +warn_count="$(grep -c 'WARN: perm guard' "$PLANT_LOG" || true)" +if [ "$warn_count" = "2" ]; then + ok "a refused path is reported once per guard process (2 paths, 3 passes, 2 WARNs)" +else + bad "a refused path is reported once per guard process" \ + "expected 2 WARN lines, got $warn_count: $(cat "$PLANT_LOG")" +fi +rm -f "$PLANT_HOME/.claude/.credentials.json" "$PLANT_HOME/.copilot/config.json" + +# A link where the directory should be, with a real file behind it. +rmdir "$PLANT_HOME/.copilot" +mkdir -p "$WORK/plant-elsewhere" +printf '{}\n' > "$WORK/plant-elsewhere/config.json" +chmod 0600 "$WORK/plant-elsewhere/config.json" +ln -s "$WORK/plant-elsewhere" "$PLANT_HOME/.copilot" +sh -c 'set -e; . "$1"; hive_fix_copilot_config' sh "$WORK/guards.plant.sh" >/dev/null 2>&1 +if [ "$(mode_of "$WORK/plant-elsewhere/config.json")" = "600" ]; then + ok "a link planted at the directory is not acted through (file behind it stays 0600)" +else + bad "a link planted at the directory is not acted through" \ + "file behind the link went to mode $(mode_of "$WORK/plant-elsewhere/config.json")" +fi +rm -f "$PLANT_HOME/.copilot" +mkdir -p "$PLANT_HOME/.copilot" + +# A regular file in a regular directory is still repaired, and an absent one +# is silent. +printf '{}\n' > "$PLANT_HOME/.copilot/config.json" +chmod 0600 "$PLANT_HOME/.copilot/config.json" +sh -c 'set -e; . "$1"; hive_fix_copilot_config' sh "$WORK/guards.plant.sh" >/dev/null 2>"$PLANT_LOG" +if group_rw "$(mode_of "$PLANT_HOME/.copilot/config.json")"; then + ok "a regular config.json in a regular .copilot is still reopened (mode $(mode_of "$PLANT_HOME/.copilot/config.json"))" +else + bad "a regular config.json in a regular .copilot is still reopened" \ + "mode $(mode_of "$PLANT_HOME/.copilot/config.json")" +fi +rm -f "$PLANT_HOME/.copilot/config.json" +sh -c 'set -e; . "$1"; hive_fix_credentials_fast' sh "$WORK/guards.plant.sh" >/dev/null 2>>"$PLANT_LOG" +if [ -s "$PLANT_LOG" ]; then + bad "an absent shared file is silent" "$(cat "$PLANT_LOG")" +else + ok "an absent shared file is silent" +fi + +# The guard that survives a planted link also survives the next loop: the +# refusal returns 0 under set -e. +if sh -c 'set -e; . "$1"; ln -s "$2" "$3/.claude/.credentials.json"; hive_fix_shared_credential "$3/.claude/.credentials.json"; echo alive' \ + sh "$WORK/guards.plant.sh" "$PLANT_OUT" "$PLANT_HOME" 2>/dev/null | grep -q alive; then + ok "a refusal does not end the guard under set -e" +else + bad "a refusal does not end the guard under set -e" +fi + echo "" echo "Results: $PASS passed, $FAIL failed" [ "$FAIL" -eq 0 ] || exit 1 diff --git a/src/docs/agent-configuration.md b/src/docs/agent-configuration.md index af61e9df89..3368abee17 100644 --- a/src/docs/agent-configuration.md +++ b/src/docs/agent-configuration.md @@ -692,19 +692,25 @@ The `.claude/.credentials.json` bridge is a symlink, and Claude Code does not wr Claude Code has no supported way to keep the link. `CLAUDE_CONFIG_DIR` moves the whole config directory (settings, transcripts, auto-memory), which would undo the per-agent split above, and a bind mount cannot be renamed over. So hive detects and repairs the replacement instead, in three places: - at every agent launch; -- on every credential-watchdog cycle (`HIVE_CREDENTIAL_WATCHDOG_INTERVAL`, default 5 minutes), so a refresh that happens while the agent runs is caught within one cycle; +- every `HIVE_CLAUDE_CREDENTIAL_BRIDGE_INTERVAL` (default `5s`) from the credential watchdog, and again before each 5-minute probe (`HIVE_CREDENTIAL_WATCHDOG_INTERVAL`). The short cadence matters: every agent holds the same token and refreshes at about the same time, and the first refresh rotates the grant, so publishing it quickly is what keeps the siblings from failing on the old one; - right after a successful dashboard Claude sign-in. +When the server cannot open the private file (per-UID hives: it is owned by the agent's uid, mode `0600`, and hive runs as `dev`), it reads it as the agent's own uid through the same `su-exec` seam that adopts `.claude.json` sessions. + The repair never deletes a token: 1. It moves the private file aside to `.credentials.json.stale-` in the agent's `~/.claude`. The file keeps its owner and `0600` mode. -2. It keeps whichever credential is newer **and** usable in the shared `/data/home/.claude/.credentials.json`. "Usable" is the rule the rest of hive uses: a live access token, or a refresh grant that has not expired. "Newer" means the access token's `expiresAt` is later. A fresh refresh in one agent therefore becomes the fleet's login, instead of stranding every other agent on the grant it just rotated. When the agent's copy wins, the old shared file is kept beside it as `.credentials.json.stale-` and the new one is written `0660` with the old file's owner and group. On a tie, or when the agent's copy is not usable, the shared file stays as it is. +2. It keeps whichever credential is newer **and** usable in the shared `/data/home/.claude/.credentials.json`. "Usable" is the rule the rest of hive uses: a live access token, or a refresh grant that has not expired. "Newer" means the access token's `expiresAt` is later. The agent's copy wins only when it is complete (access token, refresh token, recorded `expiresAt`); a failed refresh (empty tokens, `expiresAt` 0) never wins. A fresh refresh in one agent therefore becomes the fleet's login, instead of stranding every other agent on the grant it just rotated. When the agent's copy wins, the old shared file is kept beside it as `.credentials.json.stale-` and the new one is written `0660` with the old file's owner and group. On a tie, or when the agent's copy is not usable, the shared file stays as it is. 3. It restores the symlink. +When the file cannot be read either way (a deployment without `su-exec`), nothing is moved or relinked, the agent keeps the token it already had, and one `ERROR` per agent and file version names the path. The one exception is a dashboard sign-in: a fresh shared login is newer than anything an unreadable private copy could hold, so that trigger moves the unreadable file aside unread and relinks the agent. + Each repair writes a `claude_credential_rebridged` Audit Log entry naming the agent, the trigger (`launch`, `watchdog` or `sign-in`), which credential was kept (`shared` or `agent`) and where the moved-aside file went. It never records token material. Until the repair runs, the agent card shows **credential bridge broken (private stale file)** instead of a generic "needs login". Moved-aside files are not cleaned up automatically; remove them by hand once you no longer need them. The repair skips an agent whose whole `~/.claude` is still the old directory symlink to `/data/home/.claude`. That happens to an agent that has been paused or stopped since before the per-agent split, because only a launch replaces the directory link. Through that link the agent's `.credentials.json` is the shared login itself, so there is nothing private to move aside. The hive log says so once per agent (`claude credential bridge: this agent's .claude directory resolves to the shared login directory`), the card does not show a broken bridge, and the agent's next launch replaces the directory link with its own `~/.claude`. +Every hive-side write to a shared login or CLI config (`.claude/.credentials.json`, `.copilot/config.json`, the durable Copilot token) goes through `crux/sharedfile`: a randomly named temporary beside the target, mode and owner set on the open descriptor, verified, then renamed into place, and only when the target is absent or a regular file inside a regular directory the server owns. Anything else at those paths is refused and logged once. The entrypoint's root permission guard applies the same rule to the files it repairs, and the shared dot-dirs carry the sticky bit so a dev-owned entry cannot be replaced by another uid. Memory, transcripts and todos stay per agent throughout; only the credential file is published. + ### CLI auto-memory (`agents.cli_memory`) Subscription CLIs keep their own long-lived "memory" next to the transcripts: Claude Code's auto-memory (`~/.claude/projects/-data-agents-/memory/MEMORY.md`), Codex memories (`$CODEX_HOME/memories/`), Copilot CLI's `memory` setting, and Gemini's `~/.gemini/GEMINI.md`. Hive agents already get their durable context from beads, the knowledge store, and the kick prompt, so a hidden per-CLI notebook mostly accumulates stale one-off inferences ([#17032](https://github.com/hivecommons/hive/issues/17032)). The `agents.cli_memory` block controls it fleet-wide, and each agent may override it with its own `cli_memory:` block: diff --git a/src/docs/env-vars.md b/src/docs/env-vars.md index a1a09b6868..a50d5bc805 100644 --- a/src/docs/env-vars.md +++ b/src/docs/env-vars.md @@ -313,6 +313,7 @@ and [network/credential controls](security-model.md#5-network-and-credentials). | `CODEX_API_KEY` | No | none | API key read by `crux/agent/authprobe.go` for the Codex CLI backend; either this or `OPENAI_API_KEY` makes Codex API-key mode count as configured. | | `HIVE_AGENT_TOKEN_REFRESH_INTERVAL` | No | `40m` | Go duration overriding the per-agent token refresh interval. Invalid or non-positive values fall back to the default. | | `HIVE_CREDENTIAL_WATCHDOG_INTERVAL` | No | `5m` | Go duration overriding how often the credential watchdog verifies each in-use backend credential file. `0` does NOT disable the watchdog — disabling is intentionally not offered. | +| `HIVE_CLAUDE_CREDENTIAL_BRIDGE_INTERVAL` | No | `5s` | Go duration overriding how often the credential watchdog checks each per-UID claude agent's `~/.claude/.credentials.json` link for a token refresh that replaced it with a private file, and publishes that refresh to the shared login (see [Per-agent homes](agent-configuration.md)). One `lstat` per claude agent per tick. | | `HIVE_PROVIDER_ERROR_BACKOFF_BASE` | No | `2m` | Go duration for the first inference-provider error backoff before another kick is allowed. Invalid or non-positive values fall back to the default. | | `HIVE_PROVIDER_ERROR_BACKOFF_MAX` | No | `30m` | Go duration cap for exponential inference-provider error backoff. Invalid or non-positive values fall back to the default. | | `HIVE_START_FAILURE_BLOCK_THRESHOLD` | No | `3` | Consecutive identical start failures (positive integer) before an agent is blocked from automatic relaunch (`crux/agent/start_failure.go`). Backoff is applied from the first failure regardless; invalid values fall back to the default. |