Skip to content
UOR-FoundationPublic

Latest commit

 

History

3,510 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

UOR-R4 Geometric Language Model

An open research project: can geometry replace a transformer's run-time machinery in a language model that runs on a laptop?

License: MIT Rust 1.97.1 Status: pre-alpha Milestones: 1 of 8 complete Last commit Hugging Face

Four panels read left to right: a sentence split into numbered pieces; each piece placed as a point on a geometric object; the newest piece looking at only a few nearby earlier points; a few drawers opening near what was read, with the nearest word inside them picked. A band underneath reads "Served as whole numbers: add, subtract, integer multiply/divide, compare, table look-up. No floats.".

How UOR-R4 is designed to predict the next word. The picture shows the design, not a claim of useful conversation.

What this is

Chat assistants run on transformers: programs that spend much of their effort multiplying very large grids of decimal numbers. UOR-R4 bets that a different toolkit can do the same job more cheaply: geometric rotations, whole-number addresses built from primes, small lookup tables and an exact memory. The goal is a fast language model that trains quickly and runs on your own laptop or in your browser, computing locally by default. The web Studio alone has an optional mode that sends prompts to an OpenAI-compatible endpoint the user configures; the desktop shell has none.

The project is open research and pre-alpha. One of eight milestones is complete, but no model built under its rules holds a useful conversation yet. What UOR-R4 is explains the bet in five minutes, and the glossary defines every term used below.

Where we are

Milestone 1 is complete (30 Oct 2026). A saved discrete model, exported as two integer artifacts (one per training seed), predicts the next token better than two fixed baselines built from its own training text:

  • the unigram table, which always guesses by overall word frequency;
  • the T2 table, which looks at the previous word and guesses the word most often seen after it.
Test panel Saved model, seed 1 / seed 2 Unigram T2 Outcome
Development panel, 14,287 rows 7,002 / 7,001 1,226 6,300 Beats both at both seeds
A fresh panel, 14,479 rows 7,215 / 7,228 1,279 6,582 Beats both beyond the paired noise band
Prompts in phrasings absent from training, 15,953 rows 7,850 / 7,872 1,409 7,375 Beats both beyond the paired noise band

The integer artifacts match the reference model exactly on the development panel and decode at a median of about 10,900 / 10,600 tokens per second on an Apple M1 laptop.

What this does not show. These panels come from a templated conversation corpus, and 62.5% of the held-out panel's reference replies also appear verbatim in training. M1 shows a model reading new prompts into known kinds of reply. It does not show a model inventing new replies or carrying a conversation. The evidence: M1 issue and M1 close.

The open research questions (14 Nov 2026)

Does the model learn human-written text? M1's panels are templated. On real human text, a held-out panel of 1,460,697 rows from public conversation and encyclopedia text, counted as correct next-token guesses:

  • First runs (6 Nov): training loss stays flat over 8,192 steps at two model widths, and every run trails the unigram table (report).
  • More data and more steps (6 Nov): the best run gets 272,867 and 234,892 rows right at its two seeds against T2's 256,660, inside the noise band (report).
  • A batch of 64 with twice the training text (6 Nov): 331,698 and 342,250 rows right against T2's 256,660, ahead of T2 at both seeds and on every sub-panel. The gain is the larger batch with twice the tokens, not the batch alone, and whether the seeds' spread is training noise is undecided at two seeds (report).
  • Width (8 Nov): width 256 gets about 125,000 more rows right than width 128; width 512 did not fit the GPU's memory (notebook row, claude-317).

Can the model answer in a conversation? Next-token counts are not replies. Since 10 Nov the work has been recall in controlled conversations. The current models state a value told earlier in the conversation on 314, 320 and 325 of 578 recall questions at three seeds (report). The chat program has an optional table counted from the training text, the pair experts: when its experts agree, it replaces the model's reply with a stored training reply, so a reply served that way comes from the table and not from the model (report). What the model trains and serves today is on the mechanism board.

Every result, negative ones included, is one row in the research notebook.

The three products

None ships today. A trained integer artifact exists; an integrated, qualified product does not (products).

  • The model: one whole-number model file for local serving, with integer multiply and divide allowed by D80/D81. The saved discrete M1 pair is real; so is a separate lut-stack/5 FBS export with exact rotations and its DEV export gap inside the bar. New FBS exports are lut-stack/10, which binds the stack contract and the tokenizer. Neither is a released conversational product.
  • Web client: the live Studio can load the publicly released M1 model after an explicit click and serves top-8 next-token prediction—not chat. The synthetic parity harness passes; M7's accepted-model parity and a qualified product bundle remain open.
  • Desktop client: the Tauri shell exists and verifies durable session behavior on a synthetic artifact. It has no qualified trained product bundle.

Where we're going

M2 asks for complete correct replies grounded in exact memory; M3 then qualifies durable memory. After that come one served model (M4), laptop cost (M5), reasoning and coding (M6), browser/API distribution (M7) and an alpha release (M8). The immediate work is the second open question above: replies that answer from the conversation and from what the model knows, in a form the integer artifact serves (#3407). This is a plan, not a promise: where we're going and PLAN.md.

How it works

Each transformer part gets a geometric or exact replacement that avoids softmax and dense matrix multiplication. Statuses come from the mechanism board: ACTIVE is on a current model path, READY is built and tested but not trained in, and PARKED means the tested forms failed or broke a rule while the mechanism stays open.

Transformer part Geometric replacement in UOR-R4 Status
Embedding table (a starting vector per token) Prime-zeta codebook and address cells: whole-number addresses from primes and zeta-zero phases ACTIVE
Attention (which earlier tokens matter) Route, Flock and Relation reads: walk an address tree, rotate a few neighbours into one frame, follow copy links ACTIVE; independent effects remain unisolated
Feed-forward layers (most of the multiplying) Hamilton rotation gate and SpiralCore/FBS routes Hamilton gate is in M1 and FBS; FBS routes are trained and served on FBS only; dated float projection form READY
Softmax output and cross-entropy loss Admission (open a few address "drawers", called cells) and a rank-margin hinge ACTIVE
Backpropagation optimizer Discrete learning: try each legal rotation of one entry, keep it only if the score strictly improves ACTIVE
Matrix-multiply runtime Integer serving (D11): whole-number adds, shifts, compares and table reads; integer multiply and divide are allowed ACTIVE; the discrete M1 pair and the separate FBS lut-stack/5 export both serve

Position and normalization have replacements on the FBS trunk: SpiralCore transport spin and RMS norm with an integer square root. M1 instead uses Flock FrozenZeroV1, DiscreteHamiltonGateV1, DiscreteQuaternionRecurrenceV1, PositionHolonomyV1, OrbitRootSnapNormV1, an orbit-root causal pointer and post-rotation OuterL1V1—not FBS projection or the FBS RMS normalization. The current FBS models (claude-358) read with the Hamming read, the XOR/popcount form of Ternary/XOR, not the Route, Flock or Relation reads (board). M1 does not establish every replacement's independent effect. The board lists 28 mechanisms: 12 ACTIVE, 5 READY and 11 PARKED. Part by part: transformer vs geometry.

Historical K16 result — 19 Oct 2026

Before the M1 closing artifacts, the selected K16-A discrete model scored 65 and 463 correct next-token guesses out of 14,287, below the unigram table's 1,226 and T2's 6,300. K16-R collapsed to 19 or 20 occupied cells. These remain dated negative results, not the current best model: K16 outcome.

Four older 214-million-parameter checkpoints on Hugging Face predate today's rules (they used softmax and matrix multiplication) and are kept as historical evidence, not as the D11 product.

Explore

Contribute

cargo check --locked -p uor-r4-discrete --lib
cargo test --locked -p uor-r4-discrete --lib
cargo run -q --locked -p uor-r4-claim-wording -- .

These checks run no model. Documentation fixes, reproductions of a recorded result and tooling are welcome; model work is gated by the milestones. Start with CONTRIBUTING.md. AI research labs read CHARTER.md and AGENTS.md.

Acknowledgements

  • Alex Flom (UOR Foundation, @afflom): the upstream UOR framework, addressing and Lean sources this project builds on, and review of its early design.
  • Ari (UOR, @auser): migrated the transformerless engine into this workspace and authored the R4G1/graph-compiler design and early certify measurement work.
  • Maura (UOR, @maurathat): UOR contributions including uor-addr, and a README correction here (PR #235).
  • Mark (N3mesis): geometry research in NEMESIS-Theory, the octonion/Fano and structure-carrying criteria that informed the substrate reviews.
  • Matthew: author of SpiralCore (the v63 Cl(0,6) octonion operator convention and the v68/v69 material), whose signed transport is now the trunk's projection.
  • Gerrit (DarkUnicorn, LinkedIn): author of GoldSnnail and goldworm-coder, reviewed as external sources for state-layout patterns and evaluation ideas.
  • Wil Dahn (W33-Theory): theory discussions on the high-level geometry and its allowed pathing; author of the W(3,3) corpus reviewed by the labs.
  • John Haines: theory discussions on the high-level geometry and its allowed pathing.
  • Ron Brazil: theory discussions, engineering design, and research and development work.
  • Adam De Delva (LinkedIn): vision, relationships, unending support, product analysis, promotion, and commercialization efforts.
  • Yassine Amraoui (LinkedIn): research and development work.
  • Andrew Kadziolka: development integration, assessment, analysis and conceptual design.
  • Yazan Barakat (@Cyborg-Hawk-AI): research and development work on the WebAssembly D11 integer serving runtime (#1172, PR #2654).
  • sharmis-tp (@sharmis-tp): the bounded rustc --test verifier for the frozen M6 CodeTasks panel (PR #3754, adopted and completed by the DeepSeek lab).
  • The AI research labs (Kimi, Claude, Antigravity, DeepSeek and Codex; Codex left the registry on 27 Oct 2026 and rejoined on 12 Nov, PR #3942), which do most implementation under the owner's direction; owner and principal investigator Casey Allard.

Citation and license

Cite the software with CITATION.cff (GitHub's "Cite this repository" button reads it). No paper is claimed. MIT; third-party research and dependencies keep their own licenses.

Releases

Packages

Contributors

Languages