Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@
| Provider-neutral agent protocol, Paseo boundary, and local runtime release | Accepted | `.agents/decisions/0013-agent-protocol-and-local-runtime-release.md` |
| Stable latest-release Unix installer asset | Accepted | `.agents/decisions/0015-stable-latest-release-installer.md` |
| Local runtime wiring without embedded scheduling | Accepted | `.agents/decisions/0016-runtime-wiring-boundary.md` |
| Declarative QA profiles around the canonical gate | Accepted | `.agents/decisions/0017-declarative-qa-profiles.md` |
4 changes: 4 additions & 0 deletions .agents/PROGRESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ Phase 9 — runtime wiring in progress.
- Defined product goal, normative domain model, and implementation roadmap.
- Resolved revision identity, canonical content, exact composition, review targeting, and integration semantics.
- Established the evidence-driven development, review, CI, and specification-release harness.
- Added a Weft-specific declarative QA harness with dependency-aware profiles,
bounded reports, explicit PASS/FAIL/BLOCKED/SKIPPED evidence, process-group
timeout cleanup, release archive smoke composition, and optional live-provider
observations; it preserves `make check` as the canonical gate.
- Added a passing Native Git Phase 0 spike for canonical reconstruction, provider
rewrite survival, candidate composition, target guarding, conflict capture, and
external-ref reconciliation.
Expand Down
14 changes: 14 additions & 0 deletions .agents/agent-harness/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,20 @@ Provider names, branches, commands, and successful happy paths do not prove iden
- Add a project skill only for a repeated workflow with deterministic inputs, outputs, and validation.
- Keep historical task reports out of normative specifications.

## Declarative QA profiles

`qa/qa.config.json` declares additive QA profiles around existing Weft commands;
it does not replace `make check`. `python3 qa/runner/harness.py` runs the default
smoke profile, resolves each suite's dependencies once, runs commands without a
shell, terminates timed-out process groups, and writes ignored bounded reports
under `qa/reports/`. Results are explicit: `PASS` ran successfully, `FAIL` ran
and failed, `BLOCKED` lacks a mandatory local capability, and `SKIPPED` is
inapplicable or depends on unsatisfied proof. `full` adds Native Git feasibility,
`release --version vMAJOR.MINOR.PATCH` builds and smoke-tests the matching archive,
and `provider-observation` exposes version-gated GitButler evidence without
claiming it is deterministic. The runner's unit/lifecycle self-test is part of
`make check`.

## Orchestrator boundary

Agent runtimes use the provider-neutral process contract in
Expand Down
22 changes: 22 additions & 0 deletions .agents/decisions/0017-declarative-qa-profiles.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# ADR-0017: Declarative QA profiles around the canonical gate

- **Status:** Accepted
- **Date:** 2026-09-06

## Decision

Weft adopts a small standard-library QA runner with declarative smoke, full,
release, and opt-in provider-observation profiles. It resolves suite dependencies,
preflights local capabilities, bounds output, cleans up timed-out process groups,
emits ignored reports, and distinguishes pass, failure, blocking, and skipped
coverage. `make check` remains the canonical local and CI gate; the QA runner
does not replace or recursively redefine it.

## Consequences

- QA configuration contains only Rust/CLI/release-relevant checks.
- Generated reports remain local under `qa/reports/`.
- `SKIPPED` is disclosed evidence, never a passing provider/platform claim.
- DBMS, CMake, compatibility-container, and branch-name rules from the source
project are deliberately excluded.
- The runner self-test is included in `make check`.
1 change: 1 addition & 0 deletions .agents/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@ Each ADR records status, context, decision, alternatives, consequences, migratio
- [ADR-0014: Minimal runtime archive and release metadata](0014-minimal-runtime-archive-and-release-metadata.md)
- [ADR-0015: Stable latest-release Unix installer asset](0015-stable-latest-release-installer.md)
- [ADR-0016: Local runtime wiring without embedded scheduling](0016-runtime-wiring-boundary.md)
- [ADR-0017: Declarative QA profiles around the canonical gate](0017-declarative-qa-profiles.md)
56 changes: 56 additions & 0 deletions .agents/tasks/qa-harness-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Task Record: Weft declarative QA harness integration

## Outcome and scope

- **User/operator result:** Weft has a local, declarative QA entrypoint that
composes existing proof commands and reports their exact outcome without
replacing the repository gate.
- **In scope:** Dependency-aware suite planning, preflight capability checks,
bounded reports, timeout cleanup, stable result/exit semantics, runner tests,
and Weft-specific smoke/full/release profiles.
- **Out of scope:** Agent scheduling, provider mutation, CMake/DBMS/container
policies from the source project, and claiming unavailable platform coverage.
- **Affected domain invariants:** None directly; harness evidence must never
overstate provider, release, or recovery proof.
- **Provider/runtime scope:** Local Rust CLI, Native Git feasibility evidence,
optional GitButler live evidence, and release archives.
- **Compatibility surface:** CLI, harness, release.

## Acceptance criteria

1. Profiles resolve dependency order once and fail safely on invalid config or cycles.
2. Every suite reports PASS, FAIL, BLOCKED, or SKIPPED with stable exit behavior.
3. Commands run directly (without a shell), time out as a process group, and retain bounded output.
4. `make check` validates the runner contract; smoke and release profiles are exercised end to end.

## Risks

- **Data/security:** Commands must not interpolate through a shell or capture unbounded output.
- **Concurrency/crash recovery:** Timeout must terminate suite descendants; reports must disclose interruption/blocking.
- **Provider divergence/compatibility:** Optional provider/platform evidence remains skipped, never passed.
- **Upgrade/rollback:** Release profile uses the declared version and existing archive proof.

## Evidence and plan

- Relevant paths: `qa/`, `Makefile`, `.agents/agent-harness/`, `docs/DEPLOYMENT.md`.
- Source comparison: retain declarative planning/status/reporting from ezis-nexus;
exclude CMake, DBMS, compatibility-container, and branch-target rules.

1. Build pure config/planning/report helpers — unit tests for invalid configs, ordering, and exit codes.
2. Build Weft runner lifecycle — tests for preflight, dependency skip, and timeout cleanup.
3. Define Weft profiles — end-to-end smoke/release evidence and full local gate.

## Validation record

| Check | Command/test | Result | Evidence |
| --- | --- | --- | --- |
| Focused | `python3 qa/runner/harness.py self-test` | Passed | Config, dependency, status, preflight, and descendant-timeout tests |
| Harness/docs | `python3 qa/runner/harness.py --profile smoke` | Passed | Canonical gate executed through declarative profile |
| Release | `python3 qa/runner/harness.py --profile release --version v0.2.1` | Passed | Gate, package, clean archive install/restart/uninstall |
| Provider observation | `python3 qa/runner/harness.py --profile provider-observation` | Passed | Explicit live GitButler adapter proof in this environment |

## Decision and follow-up

- **Decision and alternatives rejected:** Retain additive orchestration, not a replacement build system or agent scheduler.
- **Residual risks:** Optional live-provider evidence remains environment-dependent and explicitly reported; it is not part of `make check`.
- **Follow-up:** Add a suite only when its proof contract is stable and directly relevant to Weft.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ tmp/
build/
dist/
coverage/
qa/reports/
__pycache__/
target/
.env
.env.*
Expand Down
7 changes: 5 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
.PHONY: check docs-check harness-check rust-check phase0-spike phase0-native-git-spike phase0-gitbutler-spike phase3-gitbutler-live package-release test-release test-upgrade-rollback test-release-reproducibility
.PHONY: check docs-check harness-check qa-self-test rust-check phase0-spike phase0-native-git-spike phase0-gitbutler-spike phase3-gitbutler-live package-release test-release test-upgrade-rollback test-release-reproducibility

RUST_HOST := $(shell rustc -vV | sed -n 's/^host: //p')

check: harness-check docs-check rust-check
check: harness-check docs-check qa-self-test rust-check

harness-check:
./scripts/verify-harness.sh

docs-check:
python3 scripts/check_docs.py

qa-self-test:
python3 qa/runner/harness.py self-test

rust-check:
cargo fmt --all --check
cargo test --workspace --all-targets --target $(RUST_HOST)
Expand Down
55 changes: 55 additions & 0 deletions qa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Weft QA Harness

The QA harness is a small standard-library orchestration layer around Weft's
existing Make and script entrypoints. It does not replace `make check`, provider
tests, or release scripts; it declares when and how those existing proofs are
combined, then produces bounded local evidence.

## Commands

```bash
# Canonical deterministic repository gate
python3 qa/runner/harness.py --profile smoke

# Add Native Git provider feasibility evidence
python3 qa/runner/harness.py --profile full

# Gate, package, and clean archive install/restart/uninstall proof
python3 qa/runner/harness.py --profile release --version v0.2.1

# Explicit, environment-dependent GitButler adapter observation
python3 qa/runner/harness.py --profile provider-observation

# Runner configuration, planning, lifecycle, timeout, and report tests
python3 qa/runner/harness.py self-test
```

Profiles and suites live in `qa.config.json`. Commands execute as argument arrays,
not shell strings. Dependencies resolve once in order; a dependent suite is
`SKIPPED` if its prerequisite did not establish the required evidence.

## Result contract

| Status | Meaning |
| --- | --- |
| `PASS` | The declared command ran and completed successfully. |
| `FAIL` | The command ran but failed or exceeded its deadline. |
| `BLOCKED` | A required option or mandatory local capability is unavailable. |
| `SKIPPED` | The suite is inapplicable, optional capability is unavailable, or a dependency was not satisfied. |

Exit code `0` means no executed suite failed and at least one proof passed; `1`
means an executed suite failed; `2` means blocked, all-skipped, or invalid
configuration. A skipped provider/platform suite is disclosure, not coverage.

Reports are written to the ignored `qa/reports/<profile>/` directory. `latest.json`
is machine-readable and the timestamped Markdown companion is human-readable.
Output tails are capped at 24,000 characters. On POSIX, a timeout terminates the
suite process group so descendants cannot outlive the result.

## Weft-specific boundaries

The harness adopts declarative planning, lifecycle status, bounded reporting, and
standard-library portability from ezis-nexus. It deliberately excludes that
project's CMake build assumptions, DBMS branch rules, compatibility containers,
and self-hosted runner checks. It never launches or schedules agents; Weft's
durable agent/process boundary remains defined in `.agents/AGENT_PROTOCOL.md`.
52 changes: 52 additions & 0 deletions qa/qa.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
{
"schemaVersion": 1,
"profiles": {
"smoke": ["repository-gate"],
"full": ["repository-gate", "native-git-spike"],
"release": ["repository-gate", "release-archive-smoke"],
"provider-observation": ["gitbutler-live"]
},
"suites": {
"repository-gate": {
"description": "Run the canonical local repository gate",
"kind": "deterministic",
"command": ["make", "check"],
"requiredCommands": ["make"],
"timeoutSeconds": 900
},
"native-git-spike": {
"description": "Reproduce the native Git feasibility evidence",
"kind": "deterministic",
"command": ["make", "phase0-native-git-spike"],
"requiredCommands": ["make", "git"],
"timeoutSeconds": 300
},
"package-release": {
"description": "Build the release archive for the declared version",
"kind": "deterministic",
"command": ["make", "package-release", "VERSION={version}"],
"requiresVariables": ["version"],
"requiredCommands": ["make"],
"timeoutSeconds": 900
},
"release-archive-smoke": {
"description": "Install, restart, and uninstall the declared release archive",
"kind": "deterministic",
"requires": ["package-release"],
"requiresVariables": ["version"],
"command": ["make", "test-release", "ARCHIVE=dist/weft-{release_version}-x86_64-unknown-linux-musl.tar.gz"],
"requiredCommands": ["make"],
"platforms": ["linux"],
"timeoutSeconds": 300
},
"gitbutler-live": {
"description": "Observe the explicitly version-gated GitButler adapter proof",
"kind": "observation",
"optional": true,
"command": ["make", "phase3-gitbutler-live"],
"requiredCommands": ["make", "but"],
"platforms": ["linux"],
"timeoutSeconds": 300
}
}
}
Loading