diff --git a/README.md b/README.md index e69280d..c2826b5 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,7 @@ Eight concepts, in the order you meet them: | **Handoff** | A structured continuation record (done / remaining / exact commit / blockers) that transfers work to a successor instead of abandoning it. | | **Approval** | A sign-off by eligible agents, hash-bound to one exact operation (a specific push, takeover, or deletion), consumed on use. | | **Approval policy** | Owned by each repository: how many votes an approval needs, which roles may cast them, and whether the requester may vote. The agent asking for permission never chooses its own quorum. | +| **Content governance** | In the hub model, what a push *actually changes* is derived from the Git objects it carries and checked against the pushing claim's scopes and the repository's protected paths. | Everything an agent does — claim, renew, checkpoint, hand off, release — writes an audit event in the same database transaction. The audit table is append-only, enforced by a trigger. @@ -59,7 +60,7 @@ Then initialize the store once: ```bash $ quorumgit init store: /home/you/.quorumgit/quorumgit.db -migrations applied: ['001_core.sql', '002_approval_identities.sql', '004_approval_authority.sql'] +migrations applied: ['001_core.sql', '002_approval_identities.sql', '004_approval_authority.sql', '005_content_governance.sql'] contract: ok ``` @@ -169,7 +170,7 @@ QUORUMGIT_AGENT=agent-two git push origin feat/x # rejected — branch is cla git push origin feat/x # rejected — unidentified ``` -**Scopes are enforced at push time.** When a branch has a live claim, every commit the push introduces must only touch paths inside that claim's scopes (`**` spans directories, `*` stays within one; a scope without wildcards covers that file or directory). Merge commits count only the paths the merge itself changed, so merging the base branch in is not attributed to the claim holder. In the local model agents commit directly in their worktree, so no hook runs and scopes remain coordination metadata. +Owning the branch is necessary but not sufficient: when the branch has a live claim, every commit the push introduces must only touch paths inside that claim's scopes, or carry an `out_of_scope_push` approval for exactly the paths outside them (see [Content governance](#content-governance)). In the local model agents commit directly in their worktree, so no hook runs and scopes remain coordination metadata. The hook learns who is pushing from `QUORUMGIT_AGENT` in its own environment, which Git passes through only when the remote is a **local path** (as `origin` is above). Over SSH or HTTP the variable does not reach the hook, so every push is rejected as unidentified. The hook also opens the store under its own `QUORUMGIT_DATA_DIR`, so pushers must use the same data directory as the rest of the deployment. @@ -177,9 +178,57 @@ Accepting a handoff in the hub model prints `worktree: (none — create manually Checkpoints in the hub model take an explicit `--commit `. Continuation points are **verified, not trusted**: the commit must exist in the registered repository, and when the claimed branch exists, be reachable from it. A typo'd or fabricated OID is rejected. +## Content governance + +A claim's scopes say what an agent *intends* to modify. In the hub model the pre-receive hook also checks what the push *actually* modifies, derived only from the Git objects being received: + +``` +push → identify pusher → branch reservations → allowed ref namespace + → derive changed paths from pushed objects + → compare with the claim's scopes and the repository's protected paths + → ALLOW, or REQUIRE the matching approval(s) +``` + +- **Changed paths** are the paths introduced by commits that are new to the repository — reachable from the pushed revision and from no existing ref. Content already in the hub was governed when it first arrived, so merging `main` into a feature branch does not attribute `main`'s changes to the pusher; a merge contributes only the paths it changes relative to *every* parent (conflict resolutions and hand edits). Root commits are compared with the empty tree. Deletions carry no content. +- **Out-of-scope changes need an approval, not a new claim.** When a claimed branch receives changes to paths none of its scopes cover, the push requires an `out_of_scope_push` approval naming the claim and exactly those paths. Scope globs use `**` (any number of directories), `*` and `?` (within one path segment), and `[...]`; a glob with no wildcard, or ending in `/`, covers a file or everything beneath a directory. Branches with no live claim carry no scopes, so only the other checks apply to them. +- **Protected paths** need a `protected_path_update` approval whenever a push changes them, on any ref and whoever owns the branch: + + ```bash + quorumgit repo protect-path myproject '.github/workflows/**' --agent lead + quorumgit repo protect-path myproject 'migrations/**' --agent lead + quorumgit repo add other /path/to/other.git --protected-path 'infrastructure/**' + ``` +- **Allowed ref namespaces.** Every repository accepts pushes to `refs/heads/` only. Tags, notes, and custom refs are refused until an operator opts in: `quorumgit repo allow-ref myproject refs/tags/ --agent lead` (`--remove` withdraws it). Branch claims, reservations, and scopes apply to `refs/heads/`; protected refs and protected paths apply to every allowed namespace. + +One push can need several approvals — a direct push to protected `main` that edits a protected path needs both `protected_ref_update` and `protected_path_update`. They are all-or-nothing: the hook lists every missing approval at once, and consumes none unless all are usable. + +### Planning a push: `approve prepare` + +Rather than pushing to learn what governance requires, ask first. `approve prepare` reads objects from your clone and ref state from the registered repository and runs the **same derivation the hook uses**, so the operations and hashes it prints are exactly the ones the hook will demand: + +```bash +$ quorumgit approve prepare --repo myproject --ref feat/api # --new defaults to HEAD +repository: myproject +refname: refs/heads/feat/api +oldrev: 8e72… +newrev: a31f… +claim: 3 by agent-one (scopes: src/api/**) +changed paths (3): + infrastructure/prod.tf + src/api/routes.py + src/database/schema.py +approvals required (1): + out_of_scope_push sha256:5c1d… + infrastructure/prod.tf + src/database/schema.py +re-run with --request to open these approvals. +``` + +`--request` opens an approval for each required operation (as `--agent`/`QUORUMGIT_AGENT`), `--json` prints the plan for tooling, `--delete` plans a ref deletion, and `-C ` names a clone other than the current directory. Nothing is pushed or changed without `--request`. It exits non-zero when the push would be refused outright (outside the allowed namespaces, a frozen branch, or a branch claimed by someone else). Fetch first: the plan is bound to the hub's current ref values, so `prepare` refuses to plan from a clone missing any commit a hub ref points at (a single-branch clone, say), and a plan made before the hub moved simply won't match. + ## Protected operations and approvals -Updates to protected refs, force pushes, ref deletions, and lease takeovers all require an approval. An approval is bound by SHA-256 hash to the **exact operation payload** (canonical JSON) — approving one push authorizes that push at those exact revisions, and nothing else. +Updates to protected refs, force pushes, ref deletions, out-of-scope pushes, protected-path changes, and lease takeovers all require an approval. An approval is bound by SHA-256 hash to the **exact operation payload** (canonical JSON) — approving one push authorizes that push at those exact revisions, and nothing else. The flow, driven by the rejection messages themselves: @@ -188,8 +237,9 @@ The flow, driven by the rejection messages themselves: # protected_ref_update on refs/heads/main requires an approval # bound to this exact update (hash sha256:ab12…). -# 2. The agent requests that exact operation; an operator votes on the returned -# approval instance ID. The threshold comes from the repository's policy: +# 2. The agent requests that exact operation (`quorumgit approve prepare --repo myproject +# --ref main --request` does this for every operation the push needs); an operator +# votes on the returned approval instance ID. The threshold comes from the policy: QUORUMGIT_AGENT=agent-one quorumgit approve request '{"type":"protected_ref_update","repository":"myproject","refname":"refs/heads/main","oldrev":"","newrev":""}' # approval 17 hash=sha256:ab12… status=pending threshold=1 quorumgit approve vote 17 --agent lead @@ -213,6 +263,8 @@ Rules that hold no matter what: Registering a `reviewer` or `operator`, changing a role (`quorumgit agent role `), and changing repository policy are themselves authority decisions, so each requires an operator acting through `--agent` or `QUORUMGIT_AGENT`, and each is audited. The single exception is bootstrap: while no operator exists, the first one can be designated by anyone. The last remaining operator cannot be demoted, so a store never falls back into bootstrap mode by accident. +> **Upgrading to content governance.** Migration `005_content_governance.sql` restricts every existing repository to `refs/heads/`. If your agents push tags or other refs, allow those namespaces with `quorumgit repo allow-ref` after running `quorumgit init`. Claimed branches now also enforce their scopes at push time, so a claim declared too narrowly will ask for `out_of_scope_push` approvals. + > **Upgrading.** Migration `004_approval_authority.sql` makes every existing agent a `worker` and gives every existing repository the default policy. Approvals granted under the old rules (including self-approvals) stop authorizing anything until an eligible operator votes on a fresh request. After running `quorumgit init`, designate an operator with `quorumgit agent role operator`. Takeovers follow the same pattern: claiming a task someone else holds (`claim --takeover`) prints the takeover operation to approve. Its payload includes the incumbent claim ID, so an unused approval cannot displace a later claim by the same agent. The takeover is atomic — the incumbent is released, the replacement claim created, and the approval consumed in one transaction, or none of it happens. A refused takeover leaves the incumbent untouched and the approval unconsumed. @@ -227,10 +279,12 @@ The incumbent's checkout is not duplicated. When a claim supersedes an earlier c | `quorumgit status` | Store health, contract check, row counts | | `quorumgit doctor [--repair]` | Detect and conservatively reconcile recorded managed-worktree drift | | `quorumgit destroy --yes` | Delete the database file (managed worktrees under `QUORUMGIT_DATA_DIR/worktrees` are left in place) | -| `quorumgit repo add [--protected-ref ]…` | Register a repository | +| `quorumgit repo add [--protected-ref ]… [--protected-path ]…` | Register a repository | | `quorumgit agent add [--role worker\|reviewer\|operator]` | Register an agent identity (non-workers need an operator once one exists) | | `quorumgit agent role ` | Change an agent's role (operator only, after bootstrap) | -| `quorumgit repo policy [--threshold ] [--role ]… [--[no-]requester-may-vote]` | Show or change a repository's approval policy (changes are operator only) | +| `quorumgit repo policy [--threshold ] [--role ]… [--[no-]requester-may-vote]` | Show or change a repository's approval policy (changes are operator only); also shows protected refs, protected paths, and allowed ref namespaces | +| `quorumgit repo protect-path [--remove]` | Require an approval for any push changing matching paths (operator only) | +| `quorumgit repo allow-ref [--remove]` | Allow pushes to a ref namespace such as `refs/tags/` (operator only) | | `quorumgit task add --repo --title [--objective ]` | Create a task | | `quorumgit task done [--note ] [--remove-worktree]` | Close a task for good — live claim holder only; releases the claim | | `quorumgit claim --branch --scope … [--no-worktree] [--takeover] [--override-overlap] [--lease-hours ]` | Claim a task | @@ -244,11 +298,12 @@ The incumbent's checkout is not duplicated. When a claim supersedes an earlier c | `quorumgit handoff list [--status ] / show ` | Inspect handoffs | | `quorumgit approve request ` | Open an approval for an exact operation, at the repository's threshold | | `quorumgit approve vote [--deny]` | Vote on one approval instance | +| `quorumgit approve prepare --repo --ref [--new \| --delete] [-C ] [--request] [--json]` | Show every approval a push would need, with the hook's exact hashes; `--request` opens them | | `quorumgit approve hash ` | Compute an operation's hash | | `quorumgit hook install --repo ` | Install the pre-receive hook (hub model) | | `quorumgit audit [--entity ] [--entity-id ] [--limit ]` | Read the audit trail | -`repo list`, `agent list`, and `task list [--repo ]` enumerate what's registered. Commands that act as an agent (`claim`, `renew`, `release`, `task done`, `checkpoint`, `handoff create/accept/decline/cancel`, `approve request/vote`, and the operator actions `agent add`, `agent role`, `repo policy`) also take `--agent `, which overrides `QUORUMGIT_AGENT`. At least one `--scope` is required to claim. Exit codes: `0` success, `1` refused/violation/error, `2` usage error. +`repo list`, `agent list`, and `task list [--repo ]` enumerate what's registered. Commands that act as an agent (`claim`, `renew`, `release`, `task done`, `checkpoint`, `handoff create/accept/decline/cancel`, `approve request/vote`, `approve prepare`, and the operator actions `agent add`, `agent role`, `repo policy`, `repo protect-path`, `repo allow-ref`) also take `--agent `, which overrides `QUORUMGIT_AGENT`. At least one `--scope` is required to claim. Exit codes: `0` success, `1` refused/violation/error, `2` usage error. `quorumgit doctor` only checks worktree paths already recorded by QuorumGit. It reports: @@ -283,6 +338,7 @@ QuorumGit v1 coordinates **cooperating agents**; the adversary is *accident, not - Agent identity is asserted (`QUORUMGIT_AGENT`), not cryptographically authenticated. Any local process can claim to be any agent. - The trust root is write access to the database and the filesystem. An actor with either can bypass governance. - The pre-receive hook governs `git push` only. Direct ref manipulation inside a repository bypasses it. +- Content is governed when it first reaches the hub. Scopes are checked only on claimed branches, so changes pushed to an unclaimed branch (subject to protected paths) and later merged into a claimed one are not re-attributed to the claim. - Approval hashes provide exact-payload binding and tamper-evidence, not approver authentication. - Roles separate duties between cooperating agents. Because identity is asserted, a process willing to impersonate an operator can still do so; role separation stops an agent from authorizing its own work by accident or by default, not a determined impersonator. @@ -305,6 +361,8 @@ pyright Supported interpreters: Python 3.11 through 3.14; `ruff` and `pyright` target the 3.11 floor so post-3.11 syntax and APIs fail the checks. CI runs `ruff check` once on Linux, and `pyright` and the test suite on Linux, macOS, and Windows with Python 3.11 and 3.14. `ruff` is pinned to an exact version in the `dev` extra because new ruff releases enable new rules; bump it deliberately and fix any new findings in the same change. Tests run against a real SQLite database and real Git repositories — nothing is mocked. The suite covers the full acceptance workflow (register → claim → isolate → block overlap → parallel work → checkpoint → handoff → accept → approval-gated takeover → audit → restart survival), real-push hook enforcement, concurrent voting/consumption/resolution races, and a branding gate that keeps application code free of any identity other than QuorumGit. +Operating guidance for agents (the work loop, pushing through the hook, and keeping secrets out of the permanent record): [`docs/agent-guide.md`](docs/agent-guide.md). + Design notes: [`docs/parallel-code-review.md`](docs/parallel-code-review.md) records which ideas from the Parallel Code project were adopted (the checkout-identity checks in `doctor`) and which were rejected. ## License diff --git a/docs/agent-guide.md b/docs/agent-guide.md new file mode 100644 index 0000000..fadd5e4 --- /dev/null +++ b/docs/agent-guide.md @@ -0,0 +1,34 @@ +# Operating guide for agents + +This is the short version of how a coding agent should behave in a repository governed by QuorumGit. The [README](../README.md) is the reference; this page is the checklist. + +## Identity + +- Always act as your own registered agent: set `QUORUMGIT_AGENT` (or pass `--agent`) once per session and never change it to another agent's name. Identity is asserted, not authenticated — QuorumGit trusts you to be who you say you are. +- Never vote on an approval you requested, benefit from, or intend to carry out. The repository's policy refuses most of these; don't go looking for the ones it doesn't. + +## The work loop + +1. **Find work.** `quorumgit task list --repo `; pick an `open` task. +2. **Claim it with honest scopes.** `quorumgit claim --branch --scope ''…`. Declare every path you expect to touch. In a hub deployment the hook compares every push with these scopes, so a scope that is too narrow turns routine pushes into approval requests, and one that is too broad blocks other agents (`OVERLAPPING`). Never use `--override-overlap` to get around another agent's live claim without being told to. +3. **Work only in your checkout.** Local model: the worktree the claim printed. Hub model: your own clone, with the claim made `--no-worktree`. +4. **Checkpoint** at meaningful, verified points: `quorumgit checkpoint --note '…'` (hub model: `--commit ` of a pushed commit). +5. **Renew before the lease runs out** (`quorumgit renew `). An expired lease cannot be renewed; the task becomes claimable by anyone. +6. **Finish with a trail.** Hand off with what was done, what remains, blockers, and how it was validated (`quorumgit handoff create …`), or `release` if nobody needs to continue. Never simply stop. + +## Pushing (hub model) + +- **Plan before you push.** `quorumgit approve prepare --repo --ref ` shows the paths your push changes and every approval it needs, with the exact hashes the hook will demand. If it says `approvals required: none`, push. +- If approvals are needed, first ask whether the push should be different: changes outside your scopes usually mean the work belongs to another task, or your claim should have declared more. Only when the change is right, run `prepare … --request` and wait for an eligible reviewer or operator to vote. +- Fetch before planning. A plan is bound to the hub's current ref value; if the hub moves, re-plan. +- Never force-push, delete refs, or push to protected refs as a workaround. Each needs its own approval, and each approval authorizes one exact update once. +- A rejected push is information, not an obstacle: read the reason and act on it. Don't retry the same push hoping for a different answer. + +## Secrets + +QuorumGit's records are deliberately permanent. The audit table is append-only (enforced by a database trigger), and handoff notes, checkpoint notes, approval operations, and task text are kept for good. Anything you write there cannot be taken back. + +- Never put credentials, tokens, private keys, connection strings, or personal data in task titles/objectives, checkpoint notes, handoff fields (`--completed`, `--remaining`, `--blockers`, `--validation`), or approval payloads. Describe *where* a secret lives ("the deploy key in the CI secret store"), never its value. +- Never commit secrets. A committed secret reaches every clone and every later reader of the hub; removing it later needs history rewriting, which is itself a governed, approval-gated force push. If you do commit one, stop, tell a human, and treat the secret as compromised — rotating it matters more than rewriting history. +- Ask your operator to protect secret-adjacent paths (`quorumgit repo protect-path ''`), for example deployment configuration or CI workflow files, so that changes there always get a second pair of eyes. +- Don't paste command output containing environment variables or configuration into handoffs without reading it first. diff --git a/src/quorumgit/cli.py b/src/quorumgit/cli.py index 5cd4bf0..5ed33e5 100644 --- a/src/quorumgit/cli.py +++ b/src/quorumgit/cli.py @@ -10,7 +10,18 @@ import json import sys -from . import __version__, audit, config, gate, handoff, registry, store, trees, work +from . import ( + __version__, + audit, + config, + gate, + git_objects, + handoff, + registry, + store, + trees, + work, +) def _agent(args, cfg) -> str: @@ -104,7 +115,11 @@ def cmd_doctor(args, cfg) -> int: def cmd_repo_add(args, cfg) -> int: with store.session(cfg) as conn: repo_id = registry.add_repository( - conn, args.name, args.path, protected_refs=args.protected_ref + conn, + args.name, + args.path, + protected_refs=args.protected_ref, + protected_paths=args.protected_path, ) conn.commit() print(f"repository {args.name} registered (id {repo_id}).") @@ -143,6 +158,39 @@ def cmd_repo_policy(args, cfg) -> int: print(f"approval threshold: {policy['threshold']}") print(f"approving roles: {', '.join(policy['roles'])}") print(f"requester may vote: {'yes' if policy['requester_may_vote'] else 'no'}") + with store.session(cfg) as conn: + repo = registry.get_repository(conn, args.name) + print(f"protected refs: {', '.join(repo['protected_refs']) or '-'}") + print(f"protected paths: {', '.join(repo['protected_paths']) or '-'}") + print(f"allowed ref namespaces: {', '.join(repo['allowed_ref_namespaces']) or '-'}") + return 0 + + +def cmd_repo_protect_path(args, cfg) -> int: + with store.session(cfg) as conn: + globs = registry.set_protected_path( + conn, + args.name, + args.glob, + actor=getattr(args, "agent", None) or cfg.agent, + remove=args.remove, + ) + conn.commit() + print(f"protected paths for {args.name}: {', '.join(globs) or '-'}") + return 0 + + +def cmd_repo_allow_ref(args, cfg) -> int: + with store.session(cfg) as conn: + prefixes = registry.set_allowed_ref_namespace( + conn, + args.name, + args.prefix, + actor=getattr(args, "agent", None) or cfg.agent, + remove=args.remove, + ) + conn.commit() + print(f"allowed ref namespaces for {args.name}: {', '.join(prefixes) or '-'}") return 0 @@ -451,6 +499,66 @@ def cmd_approve_vote(args, cfg) -> int: return 0 +def cmd_approve_prepare(args, cfg) -> int: + pusher = getattr(args, "agent", None) or cfg.agent + if args.request and not pusher: + raise SystemExit( + "Agent identity required to --request: pass --agent or set " + "QUORUMGIT_AGENT." + ) + with store.session(cfg) as conn: + plan = gate.prepare_push( + conn, + args.repo, + args.ref, + args.clone, + None if args.delete else args.new, + pusher=pusher, + ) + if args.request and not plan["refusals"]: + for entry in plan["operations"]: + approval = gate.request_approval( + conn, entry["operation"], requested_by=pusher + ) + entry["approval"] = { + key: approval[key] for key in ("id", "status", "threshold") + } + conn.commit() + if args.json: + print(json.dumps(plan, indent=2, sort_keys=True)) + return 1 if plan["refusals"] else 0 + + print(f"repository: {plan['repository']}") + print(f"refname: {plan['refname']}") + print(f"oldrev: {plan['oldrev']}") + print(f"newrev: {plan['newrev']}") + claim = plan["claim"] + if claim is not None: + print(f"claim: {claim['id']} by {claim['agent']} " + f"(scopes: {', '.join(claim['scopes'])})") + print(f"changed paths ({len(plan['paths'])}):") + for path in plan["paths"]: + print(f" {path}") + for refusal in plan["refusals"]: + print(f"REFUSED: {refusal}") + if not plan["operations"]: + print("approvals required: none") + else: + print(f"approvals required ({len(plan['operations'])}):") + for entry in plan["operations"]: + operation = entry["operation"] + print(f" {operation['type']} {entry['hash']}") + for path in operation.get("paths", []): + print(f" {path}") + if "approval" in entry: + approval = entry["approval"] + print(f" approval {approval['id']} status={approval['status']} " + f"threshold={approval['threshold']}") + if plan["operations"] and not args.request and not plan["refusals"]: + print("re-run with --request to open these approvals.") + return 1 if plan["refusals"] else 0 + + def cmd_approve_hash(args, cfg) -> int: print(gate.operation_hash(_operation_from_args(args))) return 0 @@ -512,6 +620,8 @@ def add(name, fn, configure=None, parent=None): sp.add_argument("name"), sp.add_argument("path"), sp.add_argument("--protected-ref", action="append", default=[]), + sp.add_argument("--protected-path", action="append", default=[], + help="path glob whose changes always need an approval"), ), parent=repo) add("list", cmd_repo_list, parent=repo) add("policy", cmd_repo_policy, lambda sp: ( @@ -526,6 +636,18 @@ def add(name, fn, configure=None, parent=None): sp.add_argument("--requester-may-vote", dest="requester_may_vote", action=argparse.BooleanOptionalAction, default=None), ), parent=repo) + add("protect-path", cmd_repo_protect_path, lambda sp: ( + sp.add_argument("name"), + sp.add_argument("glob"), + sp.add_argument("--remove", action="store_true"), + sp.add_argument("--agent", help="operator making the change"), + ), parent=repo) + add("allow-ref", cmd_repo_allow_ref, lambda sp: ( + sp.add_argument("name"), + sp.add_argument("prefix", help="ref namespace such as refs/tags/"), + sp.add_argument("--remove", action="store_true"), + sp.add_argument("--agent", help="operator making the change"), + ), parent=repo) agent = sub.add_parser("agent").add_subparsers(dest="sub", required=True) add("add", cmd_agent_add, lambda sp: ( @@ -632,6 +754,21 @@ def add(name, fn, configure=None, parent=None): sp.add_argument("--agent"), sp.add_argument("--deny", action="store_true"), ), parent=ap) + add("prepare", cmd_approve_prepare, lambda sp: ( + sp.add_argument("--repo", required=True), + sp.add_argument("--ref", required=True, + help="ref to push (a bare name means refs/heads/)"), + sp.add_argument("--new", default="HEAD", + help="revision in your clone to push (default HEAD)"), + sp.add_argument("--delete", action="store_true", + help="plan deleting the ref instead"), + sp.add_argument("-C", dest="clone", default=".", + help="your clone (default: current directory)"), + sp.add_argument("--request", action="store_true", + help="open an approval for every required operation"), + sp.add_argument("--json", action="store_true"), + sp.add_argument("--agent"), + ), parent=ap) add("hash", cmd_approve_hash, lambda sp: sp.add_argument("operation"), parent=ap) @@ -662,6 +799,7 @@ def main(argv: list[str] | None = None) -> int: gate.GateError, handoff.HandoffError, config.ConfigError, + git_objects.GitObjectError, ) as exc: print(f"[quorumgit] ERROR: {exc}", file=sys.stderr) return 1 diff --git a/src/quorumgit/gate.py b/src/quorumgit/gate.py index 8304a15..c6ff86d 100644 --- a/src/quorumgit/gate.py +++ b/src/quorumgit/gate.py @@ -1,7 +1,10 @@ """Approval gate and pre-receive enforcement. Protected operations require an approval whose hash binds to the exact -operation payload. Who may authorize is owned by the operation's repository: +operation payload. What a push touches is derived from the Git objects it +carries (see git_objects), never from what an agent declared it would modify: +changes outside the pushing claim's scopes and changes to protected paths are +approval-governed just like protected refs, force pushes, and deletions. Who may authorize is owned by the operation's repository: its approval policy names the threshold, the roles whose votes count, and whether the requester may vote. An agent that approved an operation may never be the one that carries it out. Enforcement is fail-closed: any hook error @@ -19,21 +22,24 @@ from pathlib import Path from typing import Any -from . import audit +from . import audit, git_objects from .canonical import stable_hash +from .git_objects import is_zero, zero_oid_like from .registry import ( RegistryError, approval_policy, assert_repository_identity_unique, get_agent, get_repository, + git_common_dir, ) from .store import Connection, begin_immediate, json_dumps, json_loads from .work import ( claim_scopes, live_claim_for_branch, open_handoff_for_branch, - path_in_scopes, + paths_outside, + paths_within, ) HOOK_MARKER = "# quorumgit-managed-pre-receive v1" @@ -436,90 +442,6 @@ def consume_approval( # --------------------------------------------------------------- hook logic -ZERO_OID = "0" * 40 - - -def _is_zero(oid: str) -> bool: - return set(oid) == {"0"} - - -def _is_fast_forward(git_dir: str, oldrev: str, newrev: str) -> bool: - result = subprocess.run( - [ - "git", - "--git-dir", - git_dir, - "merge-base", - "--is-ancestor", - oldrev, - newrev, - ], - capture_output=True, - check=False, - ) - if result.returncode not in (0, 1): - raise PushRejected("Unable to determine fast-forward status.") - return result.returncode == 0 - - -MAX_REPORTED_PATHS = 10 - - -def _pushed_paths(git_dir: str, newrev: str) -> list[str]: - """Paths changed by the commits this push introduces to the repository. - - `newrev --not --all` is exactly the set of new commits: in pre-receive no - ref points at them yet. `--cc` lists only the paths a merge commit itself - authored (differing from every parent), so merging the base branch in does - not attribute the base's files to the pusher. Renames count as both paths. - """ - result = subprocess.run( - [ - "git", - "--git-dir", - git_dir, - "log", - "--format=", - "--name-only", - "--no-renames", - "--cc", - "-z", - newrev, - "--not", - "--all", - ], - capture_output=True, - check=False, - ) - if result.returncode != 0: - raise PushRejected("Unable to determine the paths changed by this push.") - paths = ( - entry.lstrip("\n") - for entry in result.stdout.decode("utf-8", "surrogateescape").split("\0") - ) - return list(dict.fromkeys(path for path in paths if path)) - - -def enforce_claim_scopes( - conn: Connection, git_dir: str, claim: dict, branch: str, newrev: str -) -> None: - """Reject new commits that touch paths outside the claim's declared scopes.""" - scopes = claim_scopes(conn, claim["id"]) - outside = [ - path for path in _pushed_paths(git_dir, newrev) - if not path_in_scopes(path, scopes) - ] - if outside: - shown = ", ".join(outside[:MAX_REPORTED_PATHS]) - more = len(outside) - MAX_REPORTED_PATHS - if more > 0: - shown += f", and {more} more" - raise PushRejected( - f"Push to {branch!r} changes paths outside claim {claim['id']}'s " - f"scopes {scopes}: {shown}. Claim a scope that covers them first." - ) - - def _invoking_git_common_dir(git_dir: str) -> Path: result = subprocess.run( ["git", "--git-dir", git_dir, "rev-parse", "--git-common-dir"], @@ -556,6 +478,98 @@ def _verify_repository_binding( return repo +def ref_namespace_refusal(repo: dict, refname: str) -> str | None: + """Why this repository refuses any push to refname, or None.""" + allowed = repo["allowed_ref_namespaces"] + if any(refname.startswith(prefix) for prefix in allowed): + return None + return ( + f"{refname} is outside {repo['name']!r}'s allowed ref namespaces " + f"({', '.join(allowed) or 'none'}); an operator can allow it with " + f"`quorumgit repo allow-ref {repo['name']} `." + ) + + +def governed_operations( + conn: Connection, + repo: dict, + refname: str, + oldrev: str, + newrev: str, + git_dir: str | Path, + paths: list[str], +) -> list[dict]: + """Every approval one ref update needs, derived from what it carries. + + ``paths`` are the paths the update brings in (git_objects.changed_paths). + Ref-level governance (protected ref, force push, deletion) comes first, + then content governance: paths outside the scopes of the claim on the + branch, and paths under the repository's protected paths. Each operation + binds the exact revisions, so its hash is reproducible by anyone who can + see the same objects — the hook and ``approve prepare`` alike. + """ + base = { + "repository": repo["name"], + "refname": refname, + "oldrev": oldrev, + "newrev": newrev, + } + operations: list[dict] = [] + deletion = is_zero(newrev) + forced = ( + not deletion + and not is_zero(oldrev) + and not git_objects.is_ancestor(git_dir, oldrev, newrev) + ) + if refname in repo["protected_refs"]: + operations.append({"type": "protected_ref_update", **base}) + elif deletion: + operations.append({"type": "ref_delete", **base}) + elif forced: + operations.append({"type": "force_update", **base}) + + if refname.startswith("refs/heads/") and paths: + branch = refname.removeprefix("refs/heads/") + claim = live_claim_for_branch(conn, repo["id"], branch) + if claim is not None: + outside = paths_outside(paths, claim_scopes(conn, claim["id"])) + if outside: + operations.append({ + "type": "out_of_scope_push", + **base, + "claim_id": claim["id"], + "paths": outside, + }) + protected = paths_within(paths, repo["protected_paths"]) + if protected: + operations.append({"type": "protected_path_update", **base, "paths": protected}) + return operations + + +def _describe_paths(paths: list[str], limit: int = 10) -> str: + shown = ", ".join(paths[:limit]) + if len(paths) > limit: + shown += f", … ({len(paths) - limit} more)" + return shown + + +def _requirement_message(operation: dict, refusal: str | None) -> str: + message = ( + f"{operation['type']} on {operation['refname']} requires an approval " + f"bound to this exact update (hash {operation_hash(operation)})." + ) + if operation["type"] == "out_of_scope_push": + message += ( + f" The push changes paths outside claim {operation['claim_id']}'s " + f"scopes: {_describe_paths(operation['paths'])}." + ) + elif operation.get("paths"): + message += f" Paths: {_describe_paths(operation['paths'])}." + if refusal: + message += f" {refusal}" + return message + + def check_ref_update( conn: Connection, repository: str, @@ -567,6 +581,9 @@ def check_ref_update( ) -> None: """Enforce governance for one ref update. Raises PushRejected.""" repo = _verify_repository_binding(conn, repository, git_dir) + refusal = ref_namespace_refusal(repo, refname) + if refusal is not None: + raise PushRejected(refusal) branch = refname.removeprefix("refs/heads/") if refname.startswith("refs/heads/"): @@ -583,45 +600,46 @@ def check_ref_update( f"(claim {claim['id']}); pusher is " f"{pusher or 'unidentified — set QUORUMGIT_AGENT'}." ) - if claim and not _is_zero(newrev): - enforce_claim_scopes(conn, git_dir, claim, branch, newrev) - protected = refname in repo["protected_refs"] - deletion = _is_zero(newrev) - forced = ( - not deletion - and not _is_zero(oldrev) - and not _is_fast_forward(git_dir, oldrev, newrev) - ) - - if protected or deletion or forced: - operation = { - "type": "protected_ref_update" - if protected - else ("ref_delete" if deletion else "force_update"), - "repository": repository, - "refname": refname, - "oldrev": oldrev, - "newrev": newrev, - } - approval = approved_instance(conn, operation, consumer=pusher) - if approval is None: - refusal = approval_refusal(conn, operation, consumer=pusher) - raise PushRejected( - f"{operation['type']} on {refname} requires an approval " - f"bound to this exact update (hash {operation_hash(operation)})." - + (f" {refusal}" if refusal else "") - ) - assert pusher is not None - consume_approval(conn, approval["id"], operation, agent=pusher) - audit.record( - conn, - "gate.protected_update_allowed", - "repository", - repo["id"], - agent=pusher, - detail=operation, + try: + paths = git_objects.changed_paths( + git_dir, oldrev, newrev, git_objects.ref_tips(git_dir) ) + operations = governed_operations( + conn, repo, refname, oldrev, newrev, git_dir, paths + ) + except git_objects.GitObjectError as exc: + raise PushRejected(f"Unable to inspect pushed objects: {exc}") from exc + + if operations: + granted: list[tuple[dict, dict]] = [] + missing: list[str] = [] + for operation in operations: + approval = approved_instance(conn, operation, consumer=pusher) + if approval is None: + missing.append(_requirement_message( + operation, approval_refusal(conn, operation, consumer=pusher) + )) + else: + granted.append((approval, operation)) + if missing: + if len(operations) > 1: + missing.append( + "Run `quorumgit approve prepare` from your clone to list " + "and request every approval this push needs." + ) + raise PushRejected(" ".join(missing)) + assert pusher is not None + for approval, operation in granted: + consume_approval(conn, approval["id"], operation, agent=pusher) + audit.record( + conn, + "gate.protected_update_allowed", + "repository", + repo["id"], + agent=pusher, + detail=operation, + ) return audit.record( @@ -630,10 +648,108 @@ def check_ref_update( "repository", repo["id"], agent=pusher, - detail={"refname": refname, "oldrev": oldrev, "newrev": newrev}, + detail={ + "refname": refname, + "oldrev": oldrev, + "newrev": newrev, + "paths": len(paths), + }, ) +# ---------------------------------------------------------- push planning + + +def _full_refname(ref: str) -> str: + return ref if ref.startswith("refs/") else f"refs/heads/{ref}" + + +def prepare_push( + conn: Connection, + repository: str, + ref: str, + local_path: str | Path, + new: str | None = "HEAD", + *, + pusher: str | None = None, +) -> dict: + """What pushing ``new`` to ``ref`` would require, without pushing anything. + + Objects are read from the agent's clone at ``local_path`` and ref state + from the registered repository, and the same derivation the hook uses + produces the operations, so their hashes match what the hook will + demand. ``new=None`` plans a deletion. Nothing is written. + """ + repo = get_repository(conn, repository) + refname = _full_refname(ref) + hub_dir = git_common_dir(repo["path"]) + local_dir = git_objects.absolute_git_dir(local_path) + oldrev = git_objects.ref_value(hub_dir, refname) + if new is None: + if is_zero(oldrev): + raise GateError(f"{refname} does not exist in {repository!r}.") + newrev = zero_oid_like(oldrev) + else: + newrev = git_objects.resolve_object(local_dir, new) + if is_zero(oldrev): + oldrev = zero_oid_like(newrev) + if not is_zero(oldrev) and not git_objects.object_exists(local_dir, oldrev): + raise GateError( + f"{refname} is at {oldrev} in {repository!r}, which your clone does " + "not have; fetch before preparing this push." + ) + known = git_objects.ref_tips(hub_dir) + missing = [ + tip for tip in known if not git_objects.object_exists(local_dir, tip) + ] + if missing and not is_zero(newrev): + # Commits reachable from a tip the clone lacks would look new here but + # not to the hook, so the derived paths (and hashes) would differ. + raise GateError( + f"Your clone lacks {len(missing)} commit(s) that refs in " + f"{repository!r} point at (for example {missing[0]}); fetch every " + "ref you can push to before preparing this push, so the plan " + "matches what the hook will see." + ) + paths = git_objects.changed_paths(local_dir, oldrev, newrev, known) + + refusals: list[str] = [] + namespace = ref_namespace_refusal(repo, refname) + if namespace is not None: + refusals.append(namespace) + claim = None + if refname.startswith("refs/heads/"): + branch = refname.removeprefix("refs/heads/") + pending = open_handoff_for_branch(conn, repo["id"], branch) + if pending: + refusals.append( + f"Branch {branch!r} is frozen pending handoff {pending['id']}." + ) + claim = live_claim_for_branch(conn, repo["id"], branch) + if claim is not None: + claim = {**claim, "scopes": claim_scopes(conn, claim["id"])} + if pusher is not None and claim["agent"] != pusher: + refusals.append( + f"Branch {branch!r} is claimed by {claim['agent']} " + f"(claim {claim['id']}), not {pusher}." + ) + operations = governed_operations( + conn, repo, refname, oldrev, newrev, local_dir, paths + ) + return { + "repository": repo["name"], + "refname": refname, + "oldrev": oldrev, + "newrev": newrev, + "paths": paths, + "claim": claim, + "refusals": refusals, + "operations": [ + {"operation": op, "hash": operation_hash(op)} for op in operations + ], + } + + def run_pre_receive( conn: Connection, repository: str, stdin_lines: Iterable[str] ) -> int: diff --git a/src/quorumgit/git_objects.py b/src/quorumgit/git_objects.py new file mode 100644 index 0000000..53eea4f --- /dev/null +++ b/src/quorumgit/git_objects.py @@ -0,0 +1,203 @@ +"""What a ref update actually carries, derived from Git objects alone. + +Governance decisions about content must come from the objects Git is +receiving, not from what an agent said it planned to modify. Everything here +is a pure function of a Git object database: the pre-receive hook calls it +against the hub (quarantined objects included, since Git exports the +quarantine through the environment), and ``approve prepare`` calls the same +code against an agent's clone before pushing. Both therefore derive the same +changed paths, and so the same operation hashes, for the same update. +""" + +from __future__ import annotations + +import subprocess +from pathlib import Path + + +class GitObjectError(RuntimeError): + pass + + +ZERO_OID = "0" * 40 + + +def is_zero(oid: str) -> bool: + return set(oid) == {"0"} + + +def zero_oid_like(oid: str) -> str: + """The null OID in the same object format (SHA-1 or SHA-256) as oid. + + Git spells "no object" with as many zeros as its hash has hex digits, and + the hook receives it that way, so a planned operation must match. + """ + return "0" * len(oid) + + +def _text_path(path: str) -> str: + """A path as stable, encodable text. + + Git paths are bytes. Valid UTF-8 is kept as is; any other byte is written + as a ``\\xNN`` escape, so the path can be hashed, stored, and shown, and + every caller derives the same text for the same bytes. + """ + try: + path.encode("utf-8") + except UnicodeEncodeError: + raw = path.encode("utf-8", "surrogateescape") + return raw.decode("utf-8", "backslashreplace") + return path + + +def _git(git_dir: str | Path, *args: str, stdin: str | None = None) -> str: + result = subprocess.run( + ["git", "--git-dir", str(git_dir), *args], + input=stdin, + capture_output=True, + text=True, + encoding="utf-8", + errors="surrogateescape", + check=False, + ) + if result.returncode != 0: + detail = (result.stderr or result.stdout).strip() + suffix = f": {detail}" if detail else "" + raise GitObjectError(f"git {args[0]} failed{suffix}") + return result.stdout + + +def absolute_git_dir(path: str | Path) -> Path: + """The Git directory of a checkout, linked worktree, or bare repository.""" + result = subprocess.run( + ["git", "-C", str(path), "rev-parse", "--absolute-git-dir"], + capture_output=True, + text=True, + check=False, + ) + if result.returncode != 0 or not result.stdout.strip(): + detail = (result.stderr or result.stdout).strip() + suffix = f": {detail}" if detail else "" + raise GitObjectError(f"Not a git repository: {path}{suffix}") + return Path(result.stdout.strip()) + + +def _refuse_option(value: str) -> None: + if value.startswith("-"): + raise GitObjectError(f"Refusing revision that looks like an option: {value!r}") + + +def resolve_object(git_dir: str | Path, rev: str) -> str: + _refuse_option(rev) + try: + return _git(git_dir, "rev-parse", "--verify", f"{rev}^{{object}}").strip() + except GitObjectError as exc: + raise GitObjectError(f"Cannot resolve {rev!r} to a Git object.") from exc + + +def ref_value(git_dir: str | Path, refname: str) -> str: + """The OID a ref currently names, or ZERO_OID when it does not exist.""" + _refuse_option(refname) + result = subprocess.run( + ["git", "--git-dir", str(git_dir), "rev-parse", "--verify", "--quiet", + refname], + capture_output=True, + text=True, + check=False, + ) + oid = result.stdout.strip() + if result.returncode == 0 and oid: + return oid + if result.returncode == 1: + return ZERO_OID + detail = (result.stderr or result.stdout).strip() + raise GitObjectError(f"Cannot read {refname}: {detail}") + + +def object_exists(git_dir: str | Path, oid: str) -> bool: + result = subprocess.run( + ["git", "--git-dir", str(git_dir), "cat-file", "-e", oid], + capture_output=True, + check=False, + ) + return result.returncode == 0 + + +def is_ancestor(git_dir: str | Path, ancestor: str, descendant: str) -> bool: + result = subprocess.run( + ["git", "--git-dir", str(git_dir), "merge-base", "--is-ancestor", + ancestor, descendant], + capture_output=True, + check=False, + ) + if result.returncode not in (0, 1): + raise GitObjectError( + f"Unable to determine whether {ancestor} is an ancestor of {descendant}." + ) + return result.returncode == 0 + + +def ref_tips(git_dir: str | Path) -> list[str]: + """Commits every existing ref points at (annotated tags peeled). + + Inside pre-receive these are the values from *before* the push: Git + updates no ref until every hook has accepted. + """ + out = _git( + git_dir, + "for-each-ref", + "--format=%(objecttype) %(objectname) %(*objecttype) %(*objectname)", + ) + tips: set[str] = set() + for line in out.splitlines(): + parts = line.split() + if len(parts) == 2 and parts[0] == "commit": + tips.add(parts[1]) + elif len(parts) == 4 and parts[2] == "commit": + tips.add(parts[3]) + return sorted(tips) + + +def new_commits( + git_dir: str | Path, newrev: str, known_tips: list[str] +) -> list[str]: + """Commits reachable from newrev and from none of known_tips.""" + lines = [newrev, *(f"^{tip}" for tip in known_tips)] + out = _git(git_dir, "rev-list", "--stdin", stdin="\n".join(lines) + "\n") + return out.split() + + +def commit_paths(git_dir: str | Path, commit: str) -> list[str]: + """Paths one commit introduces. + + Root commits are compared with the empty tree. Merges use Git's combined + diff, so a clean merge introduces nothing of its own and only paths that + differ from every parent (conflict resolutions, evil merges) are counted. + """ + out = _git( + git_dir, + "diff-tree", "-r", "-c", "--root", "--no-renames", "--name-only", + "--no-commit-id", "-z", commit, + ) + return [_text_path(path) for path in out.split("\0") if path] + + +def changed_paths( + git_dir: str | Path, oldrev: str, newrev: str, known_tips: list[str] +) -> list[str]: + """Paths this update brings into the repository, sorted and de-duplicated. + + Content already reachable from an existing ref was governed when it first + arrived, so only commits new to the repository are inspected. Merging + another branch in therefore does not re-attribute that branch's changes + to the pusher. A deletion carries no content. + """ + if is_zero(newrev): + return [] + known = list(known_tips) + if not is_zero(oldrev): + known.append(oldrev) + paths: set[str] = set() + for commit in new_commits(git_dir, newrev, known): + paths.update(commit_paths(git_dir, commit)) + return sorted(paths) diff --git a/src/quorumgit/migrations/005_content_governance.sql b/src/quorumgit/migrations/005_content_governance.sql new file mode 100644 index 0000000..1826285 --- /dev/null +++ b/src/quorumgit/migrations/005_content_governance.sql @@ -0,0 +1,35 @@ +-- Content governance: what a push may touch, and where it may land. +-- +-- Protected paths name files that need an approval whenever a push brings +-- changes to them, whoever owns the branch. Allowed ref namespaces name the +-- ref prefixes a repository accepts pushes to at all; everything else is +-- refused. Every repository, existing or new, starts with branches only +-- (refs/heads/); tags, notes, and custom refs must be opted into explicitly. + +CREATE TABLE protected_paths ( + id INTEGER PRIMARY KEY, + repository_id INTEGER NOT NULL REFERENCES repositories(id) ON DELETE CASCADE, + path_glob TEXT NOT NULL CHECK (path_glob <> ''), + UNIQUE (repository_id, path_glob) +); + +-- quorumgit-statement +CREATE TABLE allowed_ref_namespaces ( + id INTEGER PRIMARY KEY, + repository_id INTEGER NOT NULL REFERENCES repositories(id) ON DELETE CASCADE, + prefix TEXT NOT NULL + CHECK (prefix LIKE 'refs/%/' AND length(prefix) > length('refs//')), + UNIQUE (repository_id, prefix) +); + +-- quorumgit-statement +INSERT INTO allowed_ref_namespaces (repository_id, prefix) +SELECT id, 'refs/heads/' FROM repositories; + +-- quorumgit-statement +CREATE TRIGGER repositories_default_ref_namespaces +AFTER INSERT ON repositories +BEGIN + INSERT INTO allowed_ref_namespaces (repository_id, prefix) + VALUES (NEW.id, 'refs/heads/'); +END; diff --git a/src/quorumgit/registry.py b/src/quorumgit/registry.py index 1dd329b..21436a5 100644 --- a/src/quorumgit/registry.py +++ b/src/quorumgit/registry.py @@ -63,6 +63,39 @@ def _protected_refs(conn: Connection, repository_id: int) -> list[str]: ] +def _protected_paths(conn: Connection, repository_id: int) -> list[str]: + return [ + row[0] + for row in conn.execute( + "SELECT path_glob FROM protected_paths WHERE repository_id = ? " + "ORDER BY id", + (repository_id,), + ).fetchall() + ] + + +def _allowed_ref_namespaces(conn: Connection, repository_id: int) -> list[str]: + return [ + row[0] + for row in conn.execute( + "SELECT prefix FROM allowed_ref_namespaces WHERE repository_id = ? " + "ORDER BY prefix", + (repository_id,), + ).fetchall() + ] + + +def _repository_dict(conn: Connection, row) -> dict: + return { + "id": row[0], + "name": row[1], + "path": row[2], + "protected_refs": _protected_refs(conn, row[0]), + "protected_paths": _protected_paths(conn, row[0]), + "allowed_ref_namespaces": _allowed_ref_namespaces(conn, row[0]), + } + + def git_common_dir(path: str | Path) -> Path: """Return the canonical Git common directory for a repository path. @@ -147,6 +180,7 @@ def add_repository( name: str, path: str | Path, protected_refs: list[str] | None = None, + protected_paths: list[str] | None = None, ) -> int: begin_immediate(conn) repo_path = Path(path).resolve() @@ -177,6 +211,11 @@ def add_repository( "INSERT INTO protected_refs (repository_id, refname) VALUES (?, ?)", (repo_id, refname), ) + for glob in protected_paths or []: + conn.execute( + "INSERT INTO protected_paths (repository_id, path_glob) VALUES (?, ?)", + (repo_id, _validate_path_glob(glob)), + ) audit.record( conn, "repository.registered", @@ -198,12 +237,7 @@ def get_repository(conn: Connection, name: str) -> dict: ).fetchone() if row is None: raise RegistryError(f"Repository is not registered: {name}") - return { - "id": row[0], - "name": row[1], - "path": row[2], - "protected_refs": _protected_refs(conn, row[0]), - } + return _repository_dict(conn, row) def approval_policy(conn: Connection, repository_id: int) -> dict: @@ -282,19 +316,105 @@ def set_approval_policy( return after +def _validate_path_glob(glob: str) -> str: + if not glob.strip() or glob.startswith("-"): + raise RegistryError(f"Invalid protected path glob: {glob!r}") + return glob + + +def _validate_ref_namespace(prefix: str) -> str: + if ( + not prefix.startswith("refs/") + or not prefix.endswith("/") + or len(prefix) <= len("refs//") + or "//" in prefix + ): + raise RegistryError( + f"Ref namespace must look like 'refs//', got {prefix!r}." + ) + return prefix + + +def set_protected_path( + conn: Connection, + repository: str, + glob: str, + *, + actor: str | None, + remove: bool = False, +) -> list[str]: + """Add or remove a path glob whose changes always need an approval.""" + begin_immediate(conn) + repo = get_repository(conn, repository) + require_operator(conn, actor, "Changing protected paths") + _validate_path_glob(glob) + if remove: + cur = conn.execute( + "DELETE FROM protected_paths WHERE repository_id = ? AND path_glob = ?", + (repo["id"], glob), + ) + else: + cur = conn.execute( + "INSERT INTO protected_paths (repository_id, path_glob) VALUES (?, ?) " + "ON CONFLICT DO NOTHING", + (repo["id"], glob), + ) + if cur.rowcount: + audit.record( + conn, + "repository.protected_path_removed" if remove + else "repository.protected_path_added", + "repository", + repo["id"], + agent=actor, + detail={"path_glob": glob}, + ) + return _protected_paths(conn, repo["id"]) + + +def set_allowed_ref_namespace( + conn: Connection, + repository: str, + prefix: str, + *, + actor: str | None, + remove: bool = False, +) -> list[str]: + """Allow (or stop allowing) pushes to refs under a namespace prefix.""" + begin_immediate(conn) + repo = get_repository(conn, repository) + require_operator(conn, actor, "Changing allowed ref namespaces") + _validate_ref_namespace(prefix) + if remove: + cur = conn.execute( + "DELETE FROM allowed_ref_namespaces " + "WHERE repository_id = ? AND prefix = ?", + (repo["id"], prefix), + ) + else: + cur = conn.execute( + "INSERT INTO allowed_ref_namespaces (repository_id, prefix) " + "VALUES (?, ?) ON CONFLICT DO NOTHING", + (repo["id"], prefix), + ) + if cur.rowcount: + audit.record( + conn, + "repository.ref_namespace_disallowed" if remove + else "repository.ref_namespace_allowed", + "repository", + repo["id"], + agent=actor, + detail={"prefix": prefix}, + ) + return _allowed_ref_namespaces(conn, repo["id"]) + + def list_repositories(conn: Connection) -> list[dict]: rows = conn.execute( "SELECT id, name, path FROM repositories ORDER BY name" ).fetchall() - return [ - { - "id": r[0], - "name": r[1], - "path": r[2], - "protected_refs": _protected_refs(conn, r[0]), - } - for r in rows - ] + return [_repository_dict(conn, r) for r in rows] def add_agent( diff --git a/src/quorumgit/store.py b/src/quorumgit/store.py index 3625c2e..7501cbb 100644 --- a/src/quorumgit/store.py +++ b/src/quorumgit/store.py @@ -42,6 +42,8 @@ "conflict_events", "audit_events", "repository_approval_roles", + "protected_paths", + "allowed_ref_namespaces", ) # Engine-level governance rules. A store missing any of these would still @@ -58,6 +60,7 @@ "votes_require_eligible_voter", "vote_updates_require_eligible_voter", "approvals_consumer_is_not_approver", + "repositories_default_ref_namespaces", ) diff --git a/src/quorumgit/work.py b/src/quorumgit/work.py index fc06bdf..b107870 100644 --- a/src/quorumgit/work.py +++ b/src/quorumgit/work.py @@ -399,6 +399,16 @@ def path_in_scopes(path: str, scopes: list[str]) -> bool: return any(_glob_regex(normalize_scope(s)).match(path) for s in scopes) +def paths_outside(paths: list[str], scopes: list[str]) -> list[str]: + """The paths no scope covers, in their original order.""" + return [p for p in paths if not path_in_scopes(p, scopes)] + + +def paths_within(paths: list[str], scopes: list[str]) -> list[str]: + """The paths some scope covers, in their original order.""" + return [p for p in paths if path_in_scopes(p, scopes)] + + def verify_commit( repo_path: str, commit_oid: str, branch: str | None = None ) -> None: diff --git a/tests/test_approval_authority.py b/tests/test_approval_authority.py index 063b4ce..f60fa7b 100644 --- a/tests/test_approval_authority.py +++ b/tests/test_approval_authority.py @@ -359,7 +359,10 @@ def test_upgrade_leaves_no_agent_with_authority(tmp_path): conn.commit() conn.close() - assert store.migrate(local) == ["004_approval_authority.sql"] + assert store.migrate(local) == [ + "004_approval_authority.sql", + "005_content_governance.sql", + ] upgraded = store.connect(local) try: assert registry.get_agent(upgraded, "self-approver")["role"] == "worker" diff --git a/tests/test_authorization_binding.py b/tests/test_authorization_binding.py index a19faaa..5f5523b 100644 --- a/tests/test_authorization_binding.py +++ b/tests/test_authorization_binding.py @@ -39,6 +39,7 @@ def _apply_001(cfg: Config): UPGRADE_MIGRATIONS = [ "002_approval_identities.sql", "004_approval_authority.sql", + "005_content_governance.sql", ] @@ -205,7 +206,8 @@ def test_contract_rejects_001_only_store_until_all_migrations_apply(tmp_path): message = ( "Missing required migrations: " - "['002_approval_identities.sql', '004_approval_authority.sql']" + "['002_approval_identities.sql', '004_approval_authority.sql', " + "'005_content_governance.sql']" ) with pytest.raises(store.ContractViolation, match=re.escape(message)): store.verify_contract(cfg) diff --git a/tests/test_content_governance.py b/tests/test_content_governance.py new file mode 100644 index 0000000..099dc25 --- /dev/null +++ b/tests/test_content_governance.py @@ -0,0 +1,434 @@ +"""Governance derived from pushed Git objects: scopes, protected paths, refs. + +Every push here is a real `git push` through the installed pre-receive hook. +""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +import uuid +from pathlib import Path + +import pytest + +from quorumgit import cli, gate, git_objects, registry, work +from tests.conftest import OPERATOR, approve, ensure_agent +from tests.test_gate import _push, _setup + +GIT_ENV = { + "GIT_AUTHOR_NAME": "t", + "GIT_AUTHOR_EMAIL": "t@localhost", + "GIT_COMMITTER_NAME": "t", + "GIT_COMMITTER_EMAIL": "t@localhost", +} + + +def _git(repo: Path, *args: str) -> str: + return subprocess.run( + ["git", "-C", str(repo), *args], + check=True, + capture_output=True, + text=True, + env={**os.environ, **GIT_ENV}, + ).stdout.strip() + + +def _edit(clone: Path, *paths: str, message: str = "edit") -> str: + for path in paths: + target = clone / path + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(f"{message}\n") + _git(clone, "add", "-A") + _git(clone, "commit", "-m", message) + return _git(clone, "rev-parse", "HEAD") + + +def _claimed_branch(conn, repo_name, agent, branch, scopes): + task = work.create_task(conn, repo_name, f"work on {branch}") + claim_id, _, _ = work.claim_task( + conn, task, agent, branch=branch, scope_globs=scopes + ) + conn.commit() + return claim_id + + +def _required(plan: dict) -> dict[str, dict]: + return {entry["operation"]["type"]: entry for entry in plan["operations"]} + + +# ------------------------------------------------------------ scope matching + + +@pytest.mark.parametrize( + ("path", "glob", "expected"), + [ + ("src/api/routes.py", "src/api/**", True), + ("src/database/schema.py", "src/api/**", False), + ("src/app.py", "src/*.py", True), + ("src/pkg/app.py", "src/*.py", False), + ("app.py", "**/*.py", True), + ("a/b/app.py", "**/*.py", True), + ("src/api/routes.py", "src/api", True), + ("src/apiary.py", "src/api", False), + ("src/api/routes.py", "src/api/", True), + ("SECURITY.md", "SECURITY.md", True), + (".github/workflows/ci.yml", ".github/workflows/**", True), + ("x/y", "x/**/y", True), + ("x/a/b/y", "x/**/y", True), + ("v1", "v[0-9]", True), + ("vx", "v[!0-9]", True), + ], +) +def test_path_in_scope(path, glob, expected): + assert work.path_in_scopes(path, [glob]) is expected + + +# ------------------------------------------------------- changed-path derivation + + +def test_changed_paths_count_only_content_new_to_the_repository(tmp_path): + repo = tmp_path / "repo" + repo.mkdir() + _git(repo, "init", "-b", "main") + base = _edit(repo, "README.md", message="base") + _git(repo, "checkout", "-b", "feat") + feat = _edit(repo, "src/api/routes.py", message="feature") + _git(repo, "checkout", "main") + main = _edit(repo, "infra/prod.tf", message="mainline") + _git(repo, "checkout", "feat") + _git(repo, "merge", "--no-edit", "main") + merged = _git(repo, "rev-parse", "HEAD") + git_dir = git_objects.absolute_git_dir(repo) + + # Merging already-governed mainline content in introduces nothing new. + assert git_objects.changed_paths(git_dir, feat, merged, [main, feat]) == [] + # A new branch carries only the commits the repository does not have. + assert git_objects.changed_paths( + git_dir, git_objects.ZERO_OID, feat, [base] + ) == ["src/api/routes.py"] + # A root commit is compared with the empty tree. + assert git_objects.changed_paths( + git_dir, git_objects.ZERO_OID, base, [] + ) == ["README.md"] + assert git_objects.changed_paths(git_dir, feat, git_objects.ZERO_OID, []) == [] + + +# ------------------------------------------------------------ claimed scopes + + +def test_push_outside_claimed_scope_requires_approval(committed_conn, tmp_path, cfg): + conn = committed_conn + repo_name, _hub, clone, a, _b = _setup(conn, tmp_path) + claim_id = _claimed_branch(conn, repo_name, a, "feat/api", ["src/api/**"]) + _git(clone, "checkout", "-b", "feat/api") + + _edit(clone, "src/api/routes.py", message="in scope") + inside = _push(clone, a, "feat/api", cfg=cfg) + assert inside.returncode == 0, inside.stderr + + _edit( + clone, + "src/api/routes.py", + "src/database/schema.py", + "infrastructure/prod.tf", + message="strays", + ) + stray = _push(clone, a, "feat/api", cfg=cfg) + assert stray.returncode != 0 + assert "out_of_scope_push" in stray.stderr + assert "src/database/schema.py" in stray.stderr + assert "infrastructure/prod.tf" in stray.stderr + + plan = gate.prepare_push(conn, repo_name, "feat/api", clone, pusher=a) + assert plan["refusals"] == [] + required = _required(plan) + assert set(required) == {"out_of_scope_push"} + operation = required["out_of_scope_push"]["operation"] + assert operation["claim_id"] == claim_id + assert operation["paths"] == ["infrastructure/prod.tf", "src/database/schema.py"] + # The planner derives exactly the hash the hook demanded. + assert required["out_of_scope_push"]["hash"] in stray.stderr + + approve(conn, operation, requested_by=a) + conn.commit() + accepted = _push(clone, a, "feat/api", cfg=cfg) + assert accepted.returncode == 0, accepted.stderr + consumed = gate.get_approval(conn, required["out_of_scope_push"]["hash"]) + assert consumed["status"] == "consumed" + + +def test_merging_mainline_into_a_claimed_branch_stays_in_scope( + committed_conn, tmp_path, cfg +): + conn = committed_conn + repo_name, hub, clone, a, _b = _setup(conn, tmp_path) + _claimed_branch(conn, repo_name, a, "feat/api", ["src/api/**"]) + _git(clone, "checkout", "-b", "feat/api") + _edit(clone, "src/api/routes.py", message="feature") + assert _push(clone, a, "feat/api", cfg=cfg).returncode == 0 + + # Mainline moves on elsewhere, outside the claim's scope. + other = tmp_path / "other" + _git(tmp_path, "clone", str(hub), str(other)) + _edit(other, "infra/prod.tf", message="mainline") + op = { + "type": "protected_ref_update", + "repository": repo_name, + "refname": "refs/heads/main", + "oldrev": _git(hub, "rev-parse", "refs/heads/main"), + "newrev": _git(other, "rev-parse", "HEAD"), + } + approve(conn, op, requested_by=a) + conn.commit() + assert _push(other, a, "main", cfg=cfg).returncode == 0 + + _git(clone, "fetch", "origin") + _git(clone, "merge", "--no-edit", "origin/main") + merged = _push(clone, a, "feat/api", cfg=cfg) + assert merged.returncode == 0, merged.stderr + + +# ----------------------------------------------------------- protected paths + + +def test_protected_paths_need_approval_on_any_branch(committed_conn, tmp_path, cfg): + conn = committed_conn + repo_name, _hub, clone, _a, b = _setup(conn, tmp_path) + ensure_agent(conn, OPERATOR, "operator") + registry.set_protected_path( + conn, repo_name, ".github/workflows/**", actor=OPERATOR + ) + conn.commit() + + _git(clone, "checkout", "-b", "feat/ci") + _edit(clone, ".github/workflows/ci.yml", "src/app.py", message="ci") + rejected = _push(clone, b, "feat/ci", cfg=cfg) + assert rejected.returncode != 0 + assert "protected_path_update" in rejected.stderr + + plan = gate.prepare_push(conn, repo_name, "feat/ci", clone) + required = _required(plan) + assert set(required) == {"protected_path_update"} + assert required["protected_path_update"]["operation"]["paths"] == [ + ".github/workflows/ci.yml" + ] + + approve(conn, required["protected_path_update"]["operation"], requested_by=b) + conn.commit() + accepted = _push(clone, b, "feat/ci", cfg=cfg) + assert accepted.returncode == 0, accepted.stderr + + +def test_protected_path_changes_are_operator_only(conn, approval_repo): + ensure_agent(conn, OPERATOR, "operator") + ensure_agent(conn, "a-worker") + with pytest.raises(registry.RegistryError, match="requires an operator"): + registry.set_protected_path(conn, approval_repo, "infra/**", actor="a-worker") + with pytest.raises(registry.RegistryError, match="requires an operator"): + registry.set_allowed_ref_namespace( + conn, approval_repo, "refs/tags/", actor="a-worker" + ) + assert registry.set_protected_path( + conn, approval_repo, "infra/**", actor=OPERATOR + ) == ["infra/**"] + assert registry.set_protected_path( + conn, approval_repo, "infra/**", actor=OPERATOR, remove=True + ) == [] + with pytest.raises(registry.RegistryError, match="refs//"): + registry.set_allowed_ref_namespace( + conn, approval_repo, "refs/tags", actor=OPERATOR + ) + + +# ------------------------------------------------------------- ref namespaces + + +def test_refs_outside_allowed_namespaces_are_refused(committed_conn, tmp_path, cfg): + conn = committed_conn + repo_name, _hub, clone, a, _b = _setup(conn, tmp_path) + assert registry.get_repository(conn, repo_name)["allowed_ref_namespaces"] == [ + "refs/heads/" + ] + _git(clone, "tag", "v1") + + refused = _push(clone, a, "refs/tags/v1", cfg=cfg) + assert refused.returncode != 0 + assert "allowed ref namespaces" in refused.stderr + custom = _push(clone, a, "HEAD:refs/custom/x", cfg=cfg) + assert custom.returncode != 0 + + ensure_agent(conn, OPERATOR, "operator") + registry.set_allowed_ref_namespace(conn, repo_name, "refs/tags/", actor=OPERATOR) + conn.commit() + allowed = _push(clone, a, "refs/tags/v1", cfg=cfg) + assert allowed.returncode == 0, allowed.stderr + + +# ------------------------------------------------------------- push planning + + +def test_prepare_cli_requests_every_required_approval( + committed_conn, tmp_path, cfg, monkeypatch, capsys +): + conn = committed_conn + repo_name, hub, clone, a, _b = _setup(conn, tmp_path) + ensure_agent(conn, OPERATOR, "operator") + registry.set_protected_path(conn, repo_name, "SECURITY.md", actor=OPERATOR) + conn.commit() + _edit(clone, "SECURITY.md", message="policy") + + monkeypatch.setenv("QUORUMGIT_DATA_DIR", str(cfg.data_dir)) + monkeypatch.setenv("QUORUMGIT_AGENT", a) + code = cli.main([ + "approve", "prepare", "--repo", repo_name, "--ref", "main", + "-C", str(clone), "--request", "--json", + ]) + assert code == 0 + plan = json.loads(capsys.readouterr().out) + assert plan["oldrev"] == _git(hub, "rev-parse", "refs/heads/main") + assert plan["newrev"] == _git(clone, "rev-parse", "HEAD") + assert plan["paths"] == ["SECURITY.md"] + required = _required(plan) + assert set(required) == {"protected_ref_update", "protected_path_update"} + + # One push needs both approvals; with only one granted it is refused and + # nothing is consumed. + for entry in required.values(): + assert entry["approval"]["status"] == "pending" + gate.vote(conn, required["protected_ref_update"]["approval"]["id"], OPERATOR, True) + conn.commit() + partial = _push(clone, a, "main", cfg=cfg) + assert partial.returncode != 0 + assert "protected_path_update" in partial.stderr + assert gate.get_approval( + conn, required["protected_ref_update"]["hash"] + )["status"] == "approved" + + gate.vote(conn, required["protected_path_update"]["approval"]["id"], OPERATOR, True) + conn.commit() + accepted = _push(clone, a, "main", cfg=cfg) + assert accepted.returncode == 0, accepted.stderr + for entry in required.values(): + assert gate.get_approval(conn, entry["hash"])["status"] == "consumed" + + +def test_prepare_reports_refusals_without_writing(committed_conn, tmp_path, cfg): + conn = committed_conn + repo_name, _hub, clone, a, b = _setup(conn, tmp_path) + _claimed_branch(conn, repo_name, a, "feat/owned", ["src/**"]) + _git(clone, "checkout", "-b", "feat/owned") + _edit(clone, "src/app.py", message="work") + + plan = gate.prepare_push(conn, repo_name, "feat/owned", clone, pusher=b) + assert any("claimed by" in refusal for refusal in plan["refusals"]) + assert plan["operations"] == [] + + tag_plan = gate.prepare_push(conn, repo_name, "refs/tags/v9", clone) + assert any("allowed ref namespaces" in r for r in tag_plan["refusals"]) + + deletion = gate.prepare_push(conn, repo_name, "main", clone, None) + assert [e["operation"]["type"] for e in deletion["operations"]] == [ + "protected_ref_update" + ] + assert deletion["paths"] == [] + + +# ------------------------------------------------- review regressions (#23) + + +def test_prepare_refuses_a_clone_missing_hub_tips(committed_conn, tmp_path, cfg): + """A tip the clone lacks would make already-governed commits look new.""" + conn = committed_conn + repo_name, hub, clone, a, _b = _setup(conn, tmp_path) + _git(clone, "checkout", "-b", "feat/other") + _edit(clone, "docs/other.md", message="other") + assert _push(clone, a, "feat/other", cfg=cfg).returncode == 0 + + narrow = tmp_path / "narrow" + _git( + tmp_path, "clone", "--no-local", "--single-branch", "--branch", "main", + str(hub), str(narrow), + ) + _git(narrow, "checkout", "-b", "feat/mine") + _edit(narrow, "src/mine.py", message="mine") + with pytest.raises(gate.GateError, match="lacks 1 commit"): + gate.prepare_push(conn, repo_name, "feat/mine", narrow) + + _git(narrow, "fetch", "origin", "feat/other") + plan = gate.prepare_push(conn, repo_name, "feat/mine", narrow) + assert plan["paths"] == ["src/mine.py"] + + +def _sha256_supported(tmp_path: Path) -> bool: + probe = subprocess.run( + ["git", "init", "-q", "--object-format=sha256", str(tmp_path / "probe")], + capture_output=True, + check=False, + ) + return probe.returncode == 0 + + +def test_prepare_matches_the_hook_in_sha256_repositories( + committed_conn, tmp_path, cfg +): + if not _sha256_supported(tmp_path): + pytest.skip("git without SHA-256 object format support") + conn = committed_conn + seed = tmp_path / "seed256" + _git(tmp_path, "init", "-q", "-b", "main", "--object-format=sha256", str(seed)) + _edit(seed, "README.md", message="seed") + hub = tmp_path / "hub256.git" + _git(tmp_path, "clone", "--bare", str(seed), str(hub)) + repo_name = f"hub256-{uuid.uuid4().hex[:8]}" + registry.add_repository( + conn, repo_name, hub, protected_refs=["refs/heads/main"], + protected_paths=["secrets/**"], + ) + agent = f"agent-256-{uuid.uuid4().hex[:8]}" + registry.add_agent(conn, agent) + gate.install_hook(conn, repo_name) + conn.commit() + clone = tmp_path / "clone256" + _git(tmp_path, "clone", str(hub), str(clone)) + + _git(clone, "checkout", "-b", "feat/new") + _edit(clone, "secrets/key.txt", message="new ref") + rejected = _push(clone, agent, "feat/new", cfg=cfg) + assert rejected.returncode != 0 + plan = gate.prepare_push(conn, repo_name, "feat/new", clone) + assert plan["oldrev"] == "0" * 64 + entry = _required(plan)["protected_path_update"] + assert entry["hash"] in rejected.stderr + + deletion = gate.prepare_push(conn, repo_name, "main", clone, None) + assert deletion["newrev"] == "0" * 64 + + +@pytest.mark.skipif( + sys.platform != "linux", reason="needs a filesystem that accepts non-UTF-8 names" +) +def test_non_utf8_paths_can_be_approved(committed_conn, tmp_path, cfg): + conn = committed_conn + repo_name, _hub, clone, a, _b = _setup(conn, tmp_path) + _claimed_branch(conn, repo_name, a, "feat/bytes", ["src/**"]) + _git(clone, "checkout", "-b", "feat/bytes") + with open(os.fsencode(clone) + b"/caf\xe9.txt", "wb") as handle: + handle.write(b"bytes\n") + _git(clone, "add", "-A") + _git(clone, "commit", "-m", "non-utf8 name") + + rejected = _push(clone, a, "feat/bytes", cfg=cfg) + assert rejected.returncode != 0 + assert "out_of_scope_push" in rejected.stderr + plan = gate.prepare_push(conn, repo_name, "feat/bytes", clone) + entry = _required(plan)["out_of_scope_push"] + assert entry["operation"]["paths"] == ["caf\\xe9.txt"] + assert entry["hash"] in rejected.stderr + + approve(conn, entry["operation"], requested_by=a) + conn.commit() + accepted = _push(clone, a, "feat/bytes", cfg=cfg) + assert accepted.returncode == 0, accepted.stderr