diff --git a/DEBATES.md b/DEBATES.md index 9b60e99..4754a86 100644 --- a/DEBATES.md +++ b/DEBATES.md @@ -1159,7 +1159,7 @@ Selected: a bounded precedence-climbing expression tree, explicit named module e ## Recovery architecture alternatives (2026-11-02; unresolved) -Accepted ADRs were locally mirrored through DEC-037 after a detected register drift, but this does **not** decide canonical ownership or provide cross-repository CI parity. Candidate A: research-owned accepted ADR register with generated app mirror. Candidate B: application-owned register with research carrying immutable references. Owner selection and a failing cross-repo drift test are required before treating R1 as complete. +Research PR #7 repaired the research accepted index through DEC-037 and merged into main. Application Draft PR #7 now runs a cross-repository metadata parity gate (37 matching accepted decisions, all three CI jobs green). This repairs observed drift but does **not** formally decide canonical ownership or require identical historical ADR prose. Candidate A: research-owned accepted ADR register with a verified application metadata mirror. Candidate B: application-owned register with immutable research references. Owner acceptance remains required before R1 governance is closed. For Web Studio, either make a versioned client of authoritative World execution with independent trace fixtures, or retain only a clearly labeled seeded single-path illustration that rejects unsupported graphs. Local browser containment is not a research result, semantic engine parity, or a predictive market model. See application `docs/ARCHITECTURE_RECOVERY_AUDIT.md`; no new ADR was accepted. @@ -1167,3 +1167,57 @@ For Web Studio, either make a versioned client of authoritative World execution ## R3 resource envelope: counted queue admission versus streaming execution (2026-11-02) A finite processed-event cap does not bound the number of future events in the calendar or retained `RunResult` fields. Direct CPython 3.12/arm64 measurements reached 100,000 pending events and about 22 MB traced peak with `max_events=1`. Option A adds opt-in queue/history/intent admission quotas to the existing ordered heap, retaining v0 semantics whenever the limits do not bind. Option B introduces a separately typed streaming/aggregate result and checkpoint contract to reduce retained history for long jobs, but requires backpressure, restoration, and sink failure semantics. Neither option supplies a hard process-memory quota for arbitrary user payloads; neither is accepted yet. Proposed hypotheses H-RQ1/RQ2/RT1/RT2/RP1 and evidence: `spec/runtime-resource-envelope-v0.md`, `evals/runtime-resource-baseline-2026-11-02.md`. + + +## R1 architectural decision authority and lifecycle (2026-11-02; PROPOSED) + +**Research anchor:** DEC-011 (living decision, debate and speculation ledgers), +DEC-023 (docs as code), and Michael Nygard, +[Documenting Architecture Decisions](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions) +(2011). The current research index is self-consistent after Research PR #7; +the application mirror is checked in Application Draft PR #7. These are verified +repository facts, not evidence that formal authority was accepted. + +**Option A, recommended for owner review:** The research repository owns the +immutable accepted ADR register and semantic research proposals; the application +retains the historical detailed ledger but treats matching index IDs, titles, +dates and statuses as a checked replica. The application never interprets a +passing metadata check as equal detailed prose or matching runtime behavior. +Acceptance is an explicit owner-reviewed research decision, not a side effect +of an implementation PR becoming green. Implementation state belongs to +conformance evidence and the tracked recovery index, not to an overwritten ADR +status. Supersession creates a new decision referencing the old one. + +**Option B:** The application owns the accepted ADRs and research keeps +immutable references. This aligns the register with working code, but makes +scientific and grammar authority dependent on an implementation repository +and reverses the existing hypothesis-first review dependency. Either option +requires a coordinated edit procedure across two Git histories. + +**Falsifiable H-GOV1:** A missing, duplicate or non-Accepted indexed record, +or an unindexed detailed DEC record in the canonical research ledger, fails +research CI. The five new independent research tests include adversarial +copies of the real ledger and verify these rejection paths. +The invariant is +`set(index_ids) = set(detailed_ids)` with unique IDs and Accepted index status. + +**Falsifiable H-GOV2:** On an application PR or main push, differing accepted +metadata across repositories fails the app parity job. After that workflow is +merged into application main, a daily scheduled parity-only run also probes +research-only drift. This is eventual detection, not a cross-repository atomic +merge or an instant block on research PRs. GitHub scheduled events can be +delayed or dropped, so the operating procedure must inspect failing checks +and not advertise a strict daily recovery guarantee. + +**Acceptance sequence proposed:** (1) write the hypothesis and competing +alternatives in research DEBATES/spec; (2) owner reviews and accepts the +chosen governance rule in a new immutable DEC record; (3) update matching +application index metadata in a coordinated PR; (4) require independent +research-ledger, cross-repository parity and application conformance tests; +(5) track Implemented separately from Accepted. No automated merge is +authorized by test success alone. + +**Stop conditions:** Owner has not selected A or B; an accepted decision +changes without traceable review; existing detailed historical prose is +silently overwritten; accepted status is inferred from completed code; or +the only drift detector is falsely represented as an atomic safety gate. diff --git a/evals/runtime-resource-baseline-2026-11-02.md b/evals/runtime-resource-baseline-2026-11-02.md index 6e2f9bc..e81e221 100644 --- a/evals/runtime-resource-baseline-2026-11-02.md +++ b/evals/runtime-resource-baseline-2026-11-02.md @@ -72,3 +72,44 @@ policies under a new reviewed contract, replay exact v0 ordering when limits do not bind, and measure 1k/10k/100k events and agent identities with RSS and adversarial fan-out. No production code, runtime semantics, public API or accepted ADR was changed in this diagnostic. + + +## Reproducible fresh-process workload matrix (follow-on, 2026-11-02) + +Application remediation commit `2741aaa` adds +`scripts/benchmark_event_resources.py` with four targeted tests. It measures +the **unchanged v0 EventMachine**, not an implementation of resource limits. +Each command launches a separate CPython process: + +```bash +uv run python scripts/benchmark_event_resources.py --scenario initial --count 100000 +uv run python scripts/benchmark_event_resources.py --scenario fanout --count 100000 +uv run python scripts/benchmark_event_resources.py --scenario due --count 100000 +``` + +Identical commands were run with `--count 1000` and `10000`. Platform: +macOS 27.0.1, arm64, CPython 3.12.13. `elapsed_seconds` measures just +`EventMachine.run`, excluding interpreter startup; RSS is the fresh process +high-water mark, not incremental queue allocation. Traced peaks include +objects allocated during the run. One sample per workload, no warmup. + +| Scenario | Events | Processed | Pending | Seconds | Traced peak bytes | Process peak RSS bytes | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| Future-only admission | 1,000 | 0 | 1,000 | 0.003348 | 219,964 | 30,851,072 | +| Future-only admission | 10,000 | 0 | 10,000 | 0.034992 | 2,223,468 | 36,012,032 | +| Future-only admission | 100,000 | 0 | 100,000 | 0.391962 | 22,128,956 | 89,243,648 | +| Single-event fan-out | 1,000 | 1 | 1,000 | 0.003898 | 228,160 | 30,556,160 | +| Single-event fan-out | 10,000 | 1 | 10,000 | 0.041737 | 2,303,664 | 36,323,328 | +| Single-event fan-out | 100,000 | 1 | 100,000 | 0.473330 | 22,928,928 | 88,326,144 | +| All events due | 1,000 | 1,000 | 0 | 0.006927 | 205,660 | 30,638,080 | +| All events due | 10,000 | 10,000 | 0 | 0.071033 | 2,045,980 | 36,421,632 | +| All events due | 100,000 | 100,000 | 0 | 0.885116 | 20,402,228 | 88,621,056 | + +**Interpretation:** The full future calendar and full processed history both +retain memory proportional to count in these three limited workloads. +The observed process peaks near 88-89 MB at 100k events are not a hard bound: +arbitrary payload size, reducer side effects, concurrent trials and 100k +independently executed World agents are **not measured**. This cannot decide +between the proposed bounded-heap and streaming/aggregate alternatives. +A future limit must first preserve exact default replay and return typed, +explicit failure rather than incomplete successful results. diff --git a/spec/runtime-resource-envelope-v0.md b/spec/runtime-resource-envelope-v0.md index 0891ee3..c663a8b 100644 --- a/spec/runtime-resource-envelope-v0.md +++ b/spec/runtime-resource-envelope-v0.md @@ -16,7 +16,10 @@ therefore does not imply constant space. Observed on CPython 3.12.13/arm64: 100,000 future events, `until=0`, `max_events=1` produced 100,000 pending events at a 22.1 MB traced peak. A single processed event emitting 100,000 future events also returned 100,000 pending at a 22.9 MB traced peak. -See [measured baseline](../evals/runtime-resource-baseline-2026-11-02.md). +A follow-on 1k/10k/100k matrix measured 88-89 MB process peak RSS for +all three 100k-event cases on the same platform. This is not a guaranteed +bound, and population-scale World behavior remains unmeasured. See the +[measured baseline](../evals/runtime-resource-baseline-2026-11-02.md). `PopulationExecutor.run` executes one isolated `WorldExecutor.run` per enrolled agent and retains every `RunResult`, identity partition and optional trace. diff --git a/tests/test_accepted_decision_index.py b/tests/test_accepted_decision_index.py new file mode 100644 index 0000000..4e8b5b6 --- /dev/null +++ b/tests/test_accepted_decision_index.py @@ -0,0 +1,84 @@ +"""Fail closed when the research-owned accepted decision index drifts.""" + +import re +import unittest +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +ROW = re.compile( + r"^\| \*\*(DEC-\d{3})\*\* \| \[[^\]]+\]\(#[^)]+\) " + r"\| \d{4}-\d{2}-\d{2} \| \*\*Accepted\*\* \| [^|]+\|$" +) +DETAIL = re.compile(r"^### (DEC-\d{3}): .+$", re.MULTILINE) + + +def accepted_index_size(body: str) -> int: + try: + index = body.split("## Decision Index", 1)[1].split( + "## Detailed Architecture Decision Records", 1 + )[0] + details = body.split("## Detailed Architecture Decision Records", 1)[1] + except IndexError as exc: + raise ValueError("decision index or detail section missing") from exc + + rows = [line for line in index.splitlines() if line.startswith("| **DEC-")] + if not rows: + raise ValueError("accepted decision index is empty") + + indexed = [] + for row in rows: + match = ROW.fullmatch(row) + if match is None: + raise ValueError(f"invalid accepted index row: {row}") + indexed.append(match.group(1)) + + detailed = DETAIL.findall(details) + if len(indexed) != len(set(indexed)): + raise ValueError("duplicate indexed decision") + if len(detailed) != len(set(detailed)): + raise ValueError("duplicate detailed decision") + if set(indexed) != set(detailed): + raise ValueError( + f"unindexed details: {sorted(set(detailed) - set(indexed))}; " + f"missing details: {sorted(set(indexed) - set(detailed))}" + ) + return len(indexed) + + +class AcceptedDecisionIndexTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.body = (ROOT / "DECISIONS.md").read_text(encoding="utf-8") + cls.first_row = next( + line for line in cls.body.splitlines() if line.startswith("| **DEC-") + ) + + def test_canonical_index_and_details_match(self): + self.assertGreater(accepted_index_size(self.body), 0) + + def test_removed_index_row_is_rejected(self): + mutated = self.body.replace(self.first_row + "\n", "", 1) + with self.assertRaisesRegex(ValueError, "unindexed details"): + accepted_index_size(mutated) + + def test_duplicate_index_row_is_rejected(self): + mutated = self.body.replace(self.first_row, self.first_row + "\n" + self.first_row, 1) + with self.assertRaisesRegex(ValueError, "duplicate indexed"): + accepted_index_size(mutated) + + def test_proposed_entry_cannot_enter_accepted_index(self): + mutated = self.body.replace(self.first_row, self.first_row.replace("**Accepted**", "**Proposed**"), 1) + with self.assertRaisesRegex(ValueError, "invalid accepted index"): + accepted_index_size(mutated) + + def test_duplicate_detail_is_rejected(self): + first_detail = DETAIL.search(self.body) + self.assertIsNotNone(first_detail) + mutated = self.body + "\n" + first_detail.group(0) + "\n" + with self.assertRaisesRegex(ValueError, "duplicate detailed"): + accepted_index_size(mutated) + + +if __name__ == "__main__": + unittest.main()