Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,116 @@ All notable changes to `views-frames` are documented here. The format is based o
[Keep a Changelog](https://keepachangelog.com/), and this project adheres to
[Semantic Versioning](https://semver.org/) as governed in `GOVERNANCE.md`.

## [2.0.0] — 2026-08-18

**A frame's index must actually be an index.** Until now you could hand a frame anything as
its `index` — a different frame, or a bare object with an `n_rows` attribute — and it was
accepted. The frame was built, it looked fine, and the published conformance suite said it
was fine. That now raises `TypeError`.

**If you pass a `SpatioTemporalIndex`, you will not notice this release.** That is what the
type hint has always said and what every example does. Change your version constraint,
re-lock, and carry on:

```diff
- views-frames = ">=1.10.2,<2"
+ views-frames = ">=2.0.0,<3"
```

This is the package's **first MAJOR** and the first move of `CONFORMANCE_FLOOR` since the
v1.0.0 freeze (`1.0.0` → `2.0.0`). The decision, the full migration table and the reasoning
are in **ADR-028**.

### What actually breaks

| If your code… | Before | Now |
|---|---|---|
| passes a `SpatioTemporalIndex` as `index` | works | **works, unchanged** |
| passes a frame or other object as `index` | silently constructed | `TypeError` |
| passes a non-index to `reindex`/`reindex_fill` | `AttributeError` on `_level` | `TypeError`, naming what was expected |
| mutates `frame.values` in place | silently corrupted buffer-sharing frames | `ValueError: assignment destination is read-only` |
| calls `map_estimate` on `inf`/`NaN` draws | bare `IndexError` | `ValueError` naming the cause |

Only the fourth row can plausibly stop working code, and ADR-025 already documented that
operation as unsupported.

**No `from_legacy_*` shim.** GOVERNANCE's MAJOR process asks for one where a consumer format
changes. Nothing here changes a wire format, a serialized layout or a signature — only the
set of inputs that were never valid narrows, so a shim would have nothing to translate.

### Fixed

- **The three frame constructors never type-checked `index`** (register C-93, Tier 2). They
validated `y_pred` four times — coerced it, checked dtype, ndim, row count — and read
exactly one attribute off `index`: `n_rows`. A frame has one. So
`PredictionFrame(values, another_frame)` constructed silently, as did any object exposing
`n_rows`.

The half that made it Tier 2: every summarizer reads `.values` and `.n_rows` and none
reads `.index`, so a malformed frame returned **numerically correct** answers from
`collapse`, `map_estimate` and `hdi` — and **`assert_summarizer_contract` certified it**.
That checker is published under ADR-016 and consumers run it in their own CI, and
`docs/CICs/Conformance.md` names "a checker that cannot detect a violation" as the failure
mode it must never have. A false pass is worse than no checker, because it is evidence a
consumer is entitled to rely on.

The checker keeps its own assertion even though construction now makes it unreachable by
ordinary means: consumers run the suite against their own frame factories, and
`with_metadata` already builds frames through `__new__` rather than `__init__`.

Found by a falsification audit run against the claim that this package was finished.

- **Same-level alignment leaked a private attribute** (register C-94).
`reindex`, `reindex_fill`, `is_superset_of` and `intersect` raised
`AttributeError: '...' object has no attribute '_level'` — naming a private attribute of a
class the caller never mentioned — where ADR-008 requires `ValueError`/`TypeError`. All
four route through one private helper, so one guard fixes the family.

- **`map_estimate` crashed obscurely on non-finite draws** (register C-57). An `inf` draw
produced a `nan` bin index whose `astype(intp)` cast overflowed to the int-min sentinel,
giving `IndexError: index -9223372036854775808 is out of bounds`. It now raises
`ValueError` naming the cause, like `exceedance` and `expected_shortfall` have since
v1.5.0 and v1.6.0.

### Changed

- **`frame.values` is write-protected** (register C-66) — the enforce ADR-025 deferred,
riding this MAJOR exactly as planned.

It did not ship as the one-liner ADR-025 recorded. `self._values.setflags(write=False)`
would have been a defect: `coerce_values` returns the caller's *own* array when it is
already `float32` — the zero-copy guarantee — so it silently makes **the caller's** array
read-only too. Measured, not reasoned about: the naive form does flip
`caller.flags.writeable` to `False`. The frames take a read-only **view** instead, which
locks the frame's buffer, leaves your array alone, and still shares memory. A `np.memmap`
keeps its subclass and its zero-copy.

- **`CONFORMANCE_FLOOR` `1.0.0` → `2.0.0`.** The first move since the freeze, because
`assert_summarizer_contract` now rejects a frame that misreports its own index. Every
consumer's CI will assert a new contract version — that is what the constant is for, but
it has never happened before.

### Changed — governance

- **ADR-028** records the decision, the migration and the alternatives — including the three
that were rejected and why (deferring it as C-66 was deferred; fixing only the checker;
using a `Protocol` check, which fails because `SpatioTemporalIndexed` describes a *frame*
and `SpatioTemporalIndex` does not even satisfy it).
- **ADR-018 amended** — the freeze it declares has now been broken once, deliberately.
- **ADR-025 amended** — its "by convention" title is history, and the one-line fix it
recorded would have been wrong.
- **C-43 declined as a rider, in writing.** C-13's pre-tag checklist names it, so passing it
over silently would have been the failure this register has recorded three times. The two
binning functions are not two implementations of one concept: one is a deliberately
approximate clipped-linear bucket for a heuristic flag, the other reproduces
`numpy.histogram`'s edge-exact path bit-for-bit and is ulp-sensitive across numpy
versions. Its precondition is rewritten to `#89` alone, because "#89 or a MAJOR" became
false the moment this released.
- **New: register C-95** — `SpatioTemporalIndex` still write-protects the caller's
identifier arrays in place, the hazard C-66 had to avoid for the frames. Left alone
deliberately: that is ADR-025's own reasoning applied consistently rather than an
exception made because a MAJOR was already open.

## [1.11.0] — 2026-08-18

**The published MAP-containment law was wrong on tied draws, and the governance documents
Expand Down
16 changes: 11 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,20 @@ at the **root of the platform dependency DAG**, plus two sibling operation packa
in the same wheel. numpy only; depends on nothing internal; every other repo
depends *toward* it.

> **Status:** **released — v1.11.x on PyPI**, public API **frozen since v1.0.0**
> (ADR-018; everything after is additive, `CONFORMANCE_FLOOR` stays `1.0.0`).
> **Status:** **released — v2.x on PyPI**. The v1 API was frozen from v1.0.0 (ADR-018)
> and **broken once, deliberately, in 2.0.0** (ADR-028): a frame's `index` must now
> actually be a `SpatioTemporalIndex`, and `frame.values` is write-protected.
> `CONFORMANCE_FLOOR` moved `1.0.0` → `2.0.0` — its first move. Everything else is
> additive, and the bar for a further MAJOR stays deliberately high.
> Consumers install `views-frames` and validate against the published conformance
> suite (`views_frames.conformance`, ADR-016). See `CHANGELOG.md` for the release
> history and `README.md` §status for the version chronicle.

## Maintenance mode

**This package is finished.** It is released, its public API has been frozen since v1.0.0,
**This package is finished.** It is released, its public API was frozen at v1.0.0 and has been
broken exactly once since (2.0.0, ADR-028 — a validation hole found by attacking the claim
that it was finished),
and its governance documents have been checked against the code and are now asserted by CI.
Work here should be rare, small, and caused by something outside this repository.

Expand Down Expand Up @@ -102,8 +107,9 @@ CI additionally gates 100% line+branch coverage
*only* under `io/`. Enforced by `tests/test_import_enforcement.py` (ADR-002).
2. **Immutable value objects.** Operations return new frames; structural ops share
the buffer (zero-copy); only reductions allocate (register C-07). Enforced for the
*index*; **by convention** for the value buffer (writeable on purpose to preserve
zero-copy/mmap — mutating `.values` in place is unsupported; ADR-025, C-66 rider).
*index* and, since 2.0.0, for the value buffer too — a read-only **view**, so
zero-copy and mmap survive and the caller's own array stays writeable (ADR-028,
register C-66; convention-only until then, ADR-025).
3. **Fail loud.** Invariants raise `ValueError`/`TypeError` at construction and at
every validation guard; the guarantee is *structural*, not temporal (`time` is
opaque; register C-11).
Expand Down
5 changes: 4 additions & 1 deletion GOVERNANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,10 @@ for name, mod in (('views_frames', c), ('views_frames_summarize', s), ('views_fr
"
```

- **Conformance-floor version:** `1.0.0` (`views_frames.conformance.CONFORMANCE_FLOOR`).
- **Conformance-floor version:** `2.0.0` (`views_frames.conformance.CONFORMANCE_FLOOR`).
Moved from `1.0.0` on 2026-08-18 with the 2.0.0 MAJOR (ADR-028) — the first move since
the freeze, because `assert_summarizer_contract` now rejects a frame that misreports
its own index.
- The floor is a **single governed version every consumer runs regardless of its
runtime pin** — this is what makes the suite test "all consumers agree," not
"my adapter vs my pin" (closes register C-10). The floor is bumped deliberately,
Expand Down
19 changes: 10 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
> containers (`FeatureFrame`, `PredictionFrame`, and their anticipated siblings)
> that every other repo depends on and that depends on nothing internal.
>
> **Status:** **v1.11.0 — frozen API, published to PyPI** (frozen since v1.0.0, ADR-018; the
> **Status:** **v2.0.0 — published to PyPI** (frozen at v1.0.0, ADR-018, and broken once in 2.0.0, ADR-028; the
> v1.1 surface is
> purely additive — the coherent posterior summary, ADR-019; v1.2.0 rebuilt the tower
> `outside-in`, C-44; v1.3.0 makes the tower summary distribution-agnostic — no magnitude
Expand All @@ -21,7 +21,7 @@
> `research/figures/` tower-figure toolkit; v1.10.0 adds the **dense-grid fill**
> primitive — `reindex_fill(other, *, fill_value)` on all three frames +
> `SpatioTemporalIndex.cartesian` + the published `assert_reindex_fill_law`, ADR-026,
> unblocking pandas-free FAO ingestion; v1.10.1 makes `io.arrow.load` fail loudly on a parquet whose row order breaks the wire contract, #199; v1.10.2 changes no behaviour — it arms the documentation check in CI, turns eleven falsification tests from README-wording checks into behaviour checks, and records ADR-027 declining the construction shortcut; v1.11.0 corrects the published MAP-containment law, which was wrong on tied draws and so failed ~6% of rows on integer count posteriors — register C-88 — and makes the two IO codecs agree on a non-JSON metadata value, C-90; the rest of that release makes the governance documents match the code and adds the CI checks that keep them matching). This
> unblocking pandas-free FAO ingestion; v1.10.1 makes `io.arrow.load` fail loudly on a parquet whose row order breaks the wire contract, #199; v1.10.2 changes no behaviour — it arms the documentation check in CI, turns eleven falsification tests from README-wording checks into behaviour checks, and records ADR-027 declining the construction shortcut; v1.11.0 corrects the published MAP-containment law, which was wrong on tied draws and so failed ~6% of rows on integer count posteriors — register C-88 — and makes the two IO codecs agree on a non-JSON metadata value, C-90; the rest of that release makes the governance documents match the code and adds the CI checks that keep them matching; **v2.0.0 is the first MAJOR** — a falsification audit found that a frame's `index` was never type-checked, so a frame (or any object with an `n_rows`) was accepted silently and the published `assert_summarizer_contract` then certified the result, and it now raises `TypeError`; riding it, `frame.values` becomes read-only (C-66) and `map_estimate` rejects non-finite draws (C-57), and `CONFORMANCE_FLOOR` moves to `2.0.0` — a correct consumer changes its version constraint and re-locks, nothing else, ADR-028). This
> README is the design
> bible; the contract it specifies is realised in `src/views_frames/` (index, frames,
> io, conformance suite) plus the `src/views_frames_summarize/` sibling package
Expand Down Expand Up @@ -233,13 +233,14 @@ makes it safe to depend on from everywhere (SDP).
3. **Immutable value objects.** A frame is validated at construction and then
treated as read-only. Operations (`collapse`, `select`, `with_metadata`)
**return new frames**; they never mutate in place. (Directly forbids the
C-184 cross-repo-mutation anti-pattern.) **Enforced for the index, by
convention for the value buffer:** the identifier arrays (`time`/`unit`) are
write-protected (`setflags(write=False)`); the `values` buffer is left
**writeable on purpose** — so structural ops can share it zero-copy — and is
immutable *by convention*: mutating `.values` in place is unsupported and may
silently corrupt buffer-sharing frames (ADR-025 / register C-63; enforcing it
is a deferred MAJOR-rider). **Copy-vs-view:** structural and
C-184 cross-repo-mutation anti-pattern.) **Enforced for the index, and since 2.0.0
for the value buffer too:** the identifier arrays (`time`/`unit`) are
write-protected (`setflags(write=False)`); the `values` buffer was left
**writeable on purpose** — so structural ops can share it zero-copy — and was
immutable *by convention* **until 2.0.0**, which enforced it: `values` is now a
read-only **view**, so an in-place mutation raises instead of silently corrupting
buffer-sharing frames, while zero-copy, mmap and the caller's own array are all
preserved (ADR-028 / register C-66; the convention-only era is ADR-025 / C-63). **Copy-vs-view:** structural and
metadata-only operations (`with_metadata`, contiguous `select`) return frames
that **share** the underlying `values` buffer (numpy view / zero-copy), and a
`mmap`-backed frame stays `mmap`-backed — a new frame must never copy a
Expand Down
22 changes: 21 additions & 1 deletion docs/ADRs/018_api_freeze_v1.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,31 @@
# ADR-018: API freeze at v1.0.0

**Status:** Accepted
**Status:** Accepted (amended 2026-08-18 — see *Amendment* below)
**Date:** 2026-06-21
**Deciders:** VIEWS platform maintainers (keystone owner)
**Consulted:** two rounds of consumer review
**Informed:** all `views_frames` consumers

> **Amendment (2026-08-18, ADR-028 / register C-13).** **The freeze has been broken once,
> deliberately, in 2.0.0** — the first MAJOR since this ADR was written.
>
> What changed: the three frames' constructors now reject an `index` argument that is not a
> `SpatioTemporalIndex`. That is a **tightening of an invariant** on the constructor shapes
> frozen below, and therefore a MAJOR under `GOVERNANCE.md`. Before 2.0.0 the constructors
> read one attribute off `index` — `n_rows` — so a frame, or any object with an `n_rows`,
> was accepted; the published `assert_summarizer_contract` then certified the result.
>
> Shipping alongside it, as riders on the same bump: `frame.values` is now write-protected
> (register C-66, the enforce ADR-025 deferred), the same-level alignment ops raise
> `TypeError` rather than leaking a private attribute, and `map_estimate` rejects non-finite
> draws (register C-57). `CONFORMANCE_FLOOR` moved `1.0.0` → `2.0.0`, its first move.
>
> **The frozen list below is otherwise unchanged** — no name was removed, renamed or
> re-signatured, and no consumer passing a real `SpatioTemporalIndex` is affected. What
> narrowed is the set of inputs that were never valid. ADR-028 carries the decision, the
> full migration table and the reasoning; read it before treating this as licence for a
> second MAJOR.

---

## Context
Expand Down
23 changes: 22 additions & 1 deletion docs/ADRs/025_value_buffer_immutability_by_convention.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,33 @@

# ADR-025: Value-buffer immutability is by convention; only the index is enforced

**Status:** Accepted
**Status:** Accepted — superseded in part by ADR-028 (2026-08-18); see *Amendment*
**Date:** 2026-06-28
**Deciders:** VIEWS platform maintainers
**Consulted:** repo-assimilation + test-review (2026-06-27, the audit that surfaced C-63)
**Informed:** views-pipeline-core, views-datafactory (consumers that hold frames)

> **Amendment (2026-08-18, ADR-028 / register C-66).** **The deferred enforce has shipped.**
> `frame.values` is write-protected as of 2.0.0, so the "by convention" half of this ADR's
> title now describes history rather than current behaviour. The reasoning below for *why*
> it was deferred — that tightening frozen surface is a MAJOR disproportionate to a hole no
> code in the ecosystem hits — was correct and is why it waited; 2.0.0 happened for another
> reason and carried it for free, exactly as C-66 planned.
>
> **It did not ship as the one-liner recorded below.** This ADR names the fix as
> `self._values.setflags(write=False)`. That would have been wrong: `coerce_values` returns
> the caller's own array when it is already `float32` — the C-07 zero-copy guarantee — so
> the one-liner silently makes the **caller's** array read-only too. Measured, not assumed:
> the naive form flips `caller.flags.writeable` to `False`. The frames take a **read-only
> view** instead, which locks the frame's buffer, leaves the caller's array writeable, and
> still shares memory; the memmap path keeps its subclass and zero-copy through `.view()`.
>
> One inconsistency is left standing on purpose: `SpatioTemporalIndex` still write-protects
> the identifier arrays in place, so it *does* reach back into the caller's arrays, and its
> comment called them "views" when `ascontiguousarray` returns the same object for
> already-contiguous input. The comment now says what the code does; the behaviour is
> long-standing and out of 2.0.0's scope.

---

## Context
Expand Down
Loading
Loading