Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 55 additions & 1 deletion DEBATES.md
Original file line number Diff line number Diff line change
Expand Up @@ -1159,11 +1159,65 @@ Selected: a bounded precedence-climbing expression tree, explicit named module e

## Recovery architecture alternatives (2026-11-02; unresolved)

Accepted ADRs were locally mirrored through DEC-037 after a detected register drift, but this does **not** decide canonical ownership or provide cross-repository CI parity. Candidate A: research-owned accepted ADR register with generated app mirror. Candidate B: application-owned register with research carrying immutable references. Owner selection and a failing cross-repo drift test are required before treating R1 as complete.
Research PR #7 repaired the research accepted index through DEC-037 and merged into main. Application Draft PR #7 now runs a cross-repository metadata parity gate (37 matching accepted decisions, all three CI jobs green). This repairs observed drift but does **not** formally decide canonical ownership or require identical historical ADR prose. Candidate A: research-owned accepted ADR register with a verified application metadata mirror. Candidate B: application-owned register with immutable research references. Owner acceptance remains required before R1 governance is closed.

For Web Studio, either make a versioned client of authoritative World execution with independent trace fixtures, or retain only a clearly labeled seeded single-path illustration that rejects unsupported graphs. Local browser containment is not a research result, semantic engine parity, or a predictive market model. See application `docs/ARCHITECTURE_RECOVERY_AUDIT.md`; no new ADR was accepted.


## R3 resource envelope: counted queue admission versus streaming execution (2026-11-02)

A finite processed-event cap does not bound the number of future events in the calendar or retained `RunResult` fields. Direct CPython 3.12/arm64 measurements reached 100,000 pending events and about 22 MB traced peak with `max_events=1`. Option A adds opt-in queue/history/intent admission quotas to the existing ordered heap, retaining v0 semantics whenever the limits do not bind. Option B introduces a separately typed streaming/aggregate result and checkpoint contract to reduce retained history for long jobs, but requires backpressure, restoration, and sink failure semantics. Neither option supplies a hard process-memory quota for arbitrary user payloads; neither is accepted yet. Proposed hypotheses H-RQ1/RQ2/RT1/RT2/RP1 and evidence: `spec/runtime-resource-envelope-v0.md`, `evals/runtime-resource-baseline-2026-11-02.md`.


## R1 architectural decision authority and lifecycle (2026-11-02; PROPOSED)

**Research anchor:** DEC-011 (living decision, debate and speculation ledgers),
DEC-023 (docs as code), and Michael Nygard,
[Documenting Architecture Decisions](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions)
(2011). The current research index is self-consistent after Research PR #7;
the application mirror is checked in Application Draft PR #7. These are verified
repository facts, not evidence that formal authority was accepted.

**Option A, recommended for owner review:** The research repository owns the
immutable accepted ADR register and semantic research proposals; the application
retains the historical detailed ledger but treats matching index IDs, titles,
dates and statuses as a checked replica. The application never interprets a
passing metadata check as equal detailed prose or matching runtime behavior.
Acceptance is an explicit owner-reviewed research decision, not a side effect
of an implementation PR becoming green. Implementation state belongs to
conformance evidence and the tracked recovery index, not to an overwritten ADR
status. Supersession creates a new decision referencing the old one.

**Option B:** The application owns the accepted ADRs and research keeps
immutable references. This aligns the register with working code, but makes
scientific and grammar authority dependent on an implementation repository
and reverses the existing hypothesis-first review dependency. Either option
requires a coordinated edit procedure across two Git histories.

**Falsifiable H-GOV1:** A missing, duplicate or non-Accepted indexed record,
or an unindexed detailed DEC record in the canonical research ledger, fails
research CI. The five new independent research tests include adversarial
copies of the real ledger and verify these rejection paths.
The invariant is
`set(index_ids) = set(detailed_ids)` with unique IDs and Accepted index status.

**Falsifiable H-GOV2:** On an application PR or main push, differing accepted
metadata across repositories fails the app parity job. After that workflow is
merged into application main, a daily scheduled parity-only run also probes
research-only drift. This is eventual detection, not a cross-repository atomic
merge or an instant block on research PRs. GitHub scheduled events can be
delayed or dropped, so the operating procedure must inspect failing checks
and not advertise a strict daily recovery guarantee.

**Acceptance sequence proposed:** (1) write the hypothesis and competing
alternatives in research DEBATES/spec; (2) owner reviews and accepts the
chosen governance rule in a new immutable DEC record; (3) update matching
application index metadata in a coordinated PR; (4) require independent
research-ledger, cross-repository parity and application conformance tests;
(5) track Implemented separately from Accepted. No automated merge is
authorized by test success alone.

**Stop conditions:** Owner has not selected A or B; an accepted decision
changes without traceable review; existing detailed historical prose is
silently overwritten; accepted status is inferred from completed code; or
the only drift detector is falsely represented as an atomic safety gate.
41 changes: 41 additions & 0 deletions evals/runtime-resource-baseline-2026-11-02.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,44 @@ policies under a new reviewed contract, replay exact v0 ordering when limits do
not bind, and measure 1k/10k/100k events and agent identities with RSS and
adversarial fan-out. No production code, runtime semantics, public API or
accepted ADR was changed in this diagnostic.


## Reproducible fresh-process workload matrix (follow-on, 2026-11-02)

Application remediation commit `2741aaa` adds
`scripts/benchmark_event_resources.py` with four targeted tests. It measures
the **unchanged v0 EventMachine**, not an implementation of resource limits.
Each command launches a separate CPython process:

```bash
uv run python scripts/benchmark_event_resources.py --scenario initial --count 100000
uv run python scripts/benchmark_event_resources.py --scenario fanout --count 100000
uv run python scripts/benchmark_event_resources.py --scenario due --count 100000
```

Identical commands were run with `--count 1000` and `10000`. Platform:
macOS 27.0.1, arm64, CPython 3.12.13. `elapsed_seconds` measures just
`EventMachine.run`, excluding interpreter startup; RSS is the fresh process
high-water mark, not incremental queue allocation. Traced peaks include
objects allocated during the run. One sample per workload, no warmup.

| Scenario | Events | Processed | Pending | Seconds | Traced peak bytes | Process peak RSS bytes |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Future-only admission | 1,000 | 0 | 1,000 | 0.003348 | 219,964 | 30,851,072 |
| Future-only admission | 10,000 | 0 | 10,000 | 0.034992 | 2,223,468 | 36,012,032 |
| Future-only admission | 100,000 | 0 | 100,000 | 0.391962 | 22,128,956 | 89,243,648 |
| Single-event fan-out | 1,000 | 1 | 1,000 | 0.003898 | 228,160 | 30,556,160 |
| Single-event fan-out | 10,000 | 1 | 10,000 | 0.041737 | 2,303,664 | 36,323,328 |
| Single-event fan-out | 100,000 | 1 | 100,000 | 0.473330 | 22,928,928 | 88,326,144 |
| All events due | 1,000 | 1,000 | 0 | 0.006927 | 205,660 | 30,638,080 |
| All events due | 10,000 | 10,000 | 0 | 0.071033 | 2,045,980 | 36,421,632 |
| All events due | 100,000 | 100,000 | 0 | 0.885116 | 20,402,228 | 88,621,056 |

**Interpretation:** The full future calendar and full processed history both
retain memory proportional to count in these three limited workloads.
The observed process peaks near 88-89 MB at 100k events are not a hard bound:
arbitrary payload size, reducer side effects, concurrent trials and 100k
independently executed World agents are **not measured**. This cannot decide
between the proposed bounded-heap and streaming/aggregate alternatives.
A future limit must first preserve exact default replay and return typed,
explicit failure rather than incomplete successful results.
5 changes: 4 additions & 1 deletion spec/runtime-resource-envelope-v0.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,10 @@ therefore does not imply constant space. Observed on CPython 3.12.13/arm64:
100,000 future events, `until=0`, `max_events=1` produced 100,000 pending
events at a 22.1 MB traced peak. A single processed event emitting 100,000
future events also returned 100,000 pending at a 22.9 MB traced peak.
See [measured baseline](../evals/runtime-resource-baseline-2026-11-02.md).
A follow-on 1k/10k/100k matrix measured 88-89 MB process peak RSS for
all three 100k-event cases on the same platform. This is not a guaranteed
bound, and population-scale World behavior remains unmeasured. See the
[measured baseline](../evals/runtime-resource-baseline-2026-11-02.md).

`PopulationExecutor.run` executes one isolated `WorldExecutor.run` per enrolled
agent and retains every `RunResult`, identity partition and optional trace.
Expand Down
84 changes: 84 additions & 0 deletions tests/test_accepted_decision_index.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
"""Fail closed when the research-owned accepted decision index drifts."""

import re
import unittest
from pathlib import Path


ROOT = Path(__file__).resolve().parents[1]
ROW = re.compile(
r"^\| \*\*(DEC-\d{3})\*\* \| \[[^\]]+\]\(#[^)]+\) "
r"\| \d{4}-\d{2}-\d{2} \| \*\*Accepted\*\* \| [^|]+\|$"
)
DETAIL = re.compile(r"^### (DEC-\d{3}): .+$", re.MULTILINE)


def accepted_index_size(body: str) -> int:
try:
index = body.split("## Decision Index", 1)[1].split(
"## Detailed Architecture Decision Records", 1
)[0]
details = body.split("## Detailed Architecture Decision Records", 1)[1]
except IndexError as exc:
raise ValueError("decision index or detail section missing") from exc

rows = [line for line in index.splitlines() if line.startswith("| **DEC-")]
if not rows:
raise ValueError("accepted decision index is empty")

indexed = []
for row in rows:
match = ROW.fullmatch(row)
if match is None:
raise ValueError(f"invalid accepted index row: {row}")
indexed.append(match.group(1))

detailed = DETAIL.findall(details)
if len(indexed) != len(set(indexed)):
raise ValueError("duplicate indexed decision")
if len(detailed) != len(set(detailed)):
raise ValueError("duplicate detailed decision")
if set(indexed) != set(detailed):
raise ValueError(
f"unindexed details: {sorted(set(detailed) - set(indexed))}; "
f"missing details: {sorted(set(indexed) - set(detailed))}"
)
return len(indexed)


class AcceptedDecisionIndexTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.body = (ROOT / "DECISIONS.md").read_text(encoding="utf-8")
cls.first_row = next(
line for line in cls.body.splitlines() if line.startswith("| **DEC-")
)

def test_canonical_index_and_details_match(self):
self.assertGreater(accepted_index_size(self.body), 0)

def test_removed_index_row_is_rejected(self):
mutated = self.body.replace(self.first_row + "\n", "", 1)
with self.assertRaisesRegex(ValueError, "unindexed details"):
accepted_index_size(mutated)

def test_duplicate_index_row_is_rejected(self):
mutated = self.body.replace(self.first_row, self.first_row + "\n" + self.first_row, 1)
with self.assertRaisesRegex(ValueError, "duplicate indexed"):
accepted_index_size(mutated)

def test_proposed_entry_cannot_enter_accepted_index(self):
mutated = self.body.replace(self.first_row, self.first_row.replace("**Accepted**", "**Proposed**"), 1)
with self.assertRaisesRegex(ValueError, "invalid accepted index"):
accepted_index_size(mutated)

def test_duplicate_detail_is_rejected(self):
first_detail = DETAIL.search(self.body)
self.assertIsNotNone(first_detail)
mutated = self.body + "\n" + first_detail.group(0) + "\n"
with self.assertRaisesRegex(ValueError, "duplicate detailed"):
accepted_index_size(mutated)


if __name__ == "__main__":
unittest.main()
Loading