Replace the transformer quadratic attention with a geometric operator stack: the data geometry is recomputed and jammed in every forward pass, a multi-scale band bank reads the transient, and a sparse route head restores sharp retrieval.
A transformer layer spends its budget on dense weight matrices and quadratic attention — the part that makes long context expensive and retrieval possible. Superfluid replaces the attention with a geometric operator:
X(t) = exp(-t L_F) X(0), L_F = D_abs(α) − A # signed sheaf Laplacian, PSD by construction
The field converges exponentially to ker(L_F) — the Hodge limit, the frozen geometry. A smooth diffusion cannot select one neighbour sharply, so retrieval is re-introduced explicitly as a sparse top-k route head. Global geometry from the fluid, sharp selection from the head — that is the architecture.
| Stage | What happens | Why |
|---|---|---|
| ① 🧭 Router | top-k content routing → semantic edges | the needle path |
| ② 🕸️ Topology | semantic edges ∪ temporal chain | order becomes 1-dimensional topology |
| ③ 🔗 Sheaf | signed, tanh-bounded restriction maps α | content-adaptive geometry, PSD by construction |
| ④ ❄️ Freeze | band filter bank in spectral time (Chebyshev, shared basis) | multi-scale reading of the transient |
| ⑤ 🧊 Harmonic | exact Hodge projection = per-component mean, O(n+m) | the frozen state — without it the block cannot aggregate globally (measured: 49.3% = chance) |
| ⑥ 📖 Read | mix → gelu → residual | the single nonlinearity |
| ⑦ 🎯 Route head | hard top-k gather, winner-take-all | sharp retrieval |
Five lemmas carry the architecture, and every one is verified numerically on every commit:
- ✅ Jamming bound — ‖e^{−tL}u‖ ≤ e^{−λ₂t}‖u‖ for u ⊥ ker(L)
- ✅ Jam-time bound — t_jam = ln(1/ε)/λ₂ upper-bounds the freezing time
- ✅ Chebyshev degree — m = O(√κ · ln(1/ε)) suffices; the recurrence carries the factor 2
- ✅ Linear collapse — a purely linear fluid cannot fit XOR; one pointwise gelu can
- ✅ Rank-c trap — the frozen state has rank = #components; the band bank restores the signal
- ✅ G-balance phase transition, contraction certificate, frame property
- ✅ End-to-end gradients — checked against finite differences, including the full block
Run them yourself: python -m superfluid.selftest → 21/21 pass in under a second.
Input: scalars. Target: the sequence mean — a global statistic a strictly local model cannot compute.
| aggregator | sign accuracy | RMSE |
|---|---|---|
| 🌊 Superfluid (the fluid) | 97.3% | 0.015 |
| ⚡ Attention (causal) | 100.0% | 0.000 |
| 📍 Local, R = 4 | 65.6% | 0.113 |
| 📍 Local, R = 8 | 71.0% | 0.102 |
The first version scored 49.3% — chance — and that failure redesigned the architecture (see the engineering ledger).
Char-level LM on Alice in Wonderland, identical tokens / optimizer / schedule:
| model | params | final val | best val | wall clock |
|---|---|---|---|---|
| 🌊 Superfluid | 71,408 | 2.7397 | 2.6181 | 340 s |
| ⚡ Transformer | 116,544 | 2.6331 | 2.5401 | 95 s |
At this tiny scale the transformer is better, and the spec says so. The wall-clock gap is an implementation artefact (dense O(n²) Laplacian + Python loop); the quality gap is exactly what the planned FLOP-matched ablations (A9/A10) exist to measure.
| context | Transformer KV cache | Superfluid (static topology) |
|---|---|---|
| 4,096 | 2.0 GB | 16 KB |
| 131,072 | 64.0 GB | 16 KB |
| 1,000,000 | 488 GB | 16 KB |
The fluid itself costs 384 FLOPs/token/layer — 0.00016% of a transformer layer.
pip install -e . # numpy >= 1.23 is the only dependency
pytest # 59 regression tests
python -m superfluid.selftest # 21/21 mathematical checksTrain both models on the deterministic bundled corpus:
from superfluid import (SuperfluidLM, TinyTransformer, train_steps,
char_dataset, synthetic_corpus)
corpus = synthetic_corpus(seed=0)
ds = char_dataset(corpus, seq_len=16)
fluid = SuperfluidLM(len(ds[vocab]), d_model=32, n_layers=2, seed=0)
curve = train_steps(fluid, corpus, steps=50, lr=3e-3) # loss curve| path | what |
|---|---|
| 📄 FRAMEWORK.md | the specification of record — lemmas, improvement ledger, cost model, kill criteria, roadmap |
| 🛠️ docs/ENGINEERING_LEDGER.md | every bug the code found, the experiment that changed the architecture, the honest A/B |
| 📊 index.html | the generated report — specification and measurements, self-contained |
| 📦 src/superfluid/ | the package: autodiff, sheaf, diffusion, topology, blocks, model, losses, optim, data, selftest |
| ✅ tests/ | 59 regression tests, re-run by CI on every commit |
| 📋 ci_logs/pytest.log | the test log, published by CI after every run |
- v1.1 — sparse Laplacian backend, Torch port, ablation runner (A1–A12)
- v1.2 — scaling-law harness, learned band placement around the measured 1/λ₂
- v2.0 — multi-sheaf superposition, differentiable topology, domain ports (molecules, meshes, proteins)
See CONTRIBUTING.md. Every numerical claim needs a test; every bug gets a row in the engineering ledger. Security issues: SECURITY.md.
MIT — see LICENSE. Cite via CITATION.cff.