A bitemporal graph ledger for knowledge management — embedded, single-file, no server.
Macrame stores concepts linked by typed, weighted relationships — where both concepts and relationships change over time, and the history of those changes is itself a first-class asset. Everything lives in one .db file on disk. No database server, no network protocol, no external service.
| Strength | What it means |
|---|---|
| Bitemporal by design | Two independent clocks per row — valid time (when a fact held in the world) and transaction time (when the database learned it). as_of(ts) answers "what did the world look like?" and reconstruct(ts) answers "what did we believe?" — both correct, both different. |
| Single file, embedded | The entire database is one file on the local filesystem. Link it directly into your application. Run on Windows desktop, Linux, or macOS — the Rust suite runs on all three in CI. |
| Graph + vectors + search | Recursive CTE traversal, native DiskANN vector search, FTS5 keyword search, and hybrid RRF fusion — all in one crate, no external graph library. |
| Five in-memory analytics | Dijkstra, A*, SCC, k-core, and Louvain — operating on a typed Subgraph with zero external dependencies. |
| Rebuildable materialization | links_current is a cache of current belief, always rebuildable from the append-only transaction_log. Drift is detectable by audit, recoverable by atomic or chunked rebuild. |
| Archival path | Closed intervals move to a cold database inside atomic sessions. Point-in-time reconstruction composes from snapshots plus anchored folds — fast because it doesn't fold from genesis. |
| Runtime safety | One Write Actor serialises all writes; read connections carry PRAGMA query_only = ON enforced at the engine level. No raw SQL escapes the guard. |
[dependencies]
macrame-db = "0.9"use macrame::prelude::*;
async fn main() {
let db = Database::open("knowledge.db").await?;
db.upsert_concept(ConceptUpsert::new("quantum", "Quantum Computing")
.valid_from("2026-01-01T00:00:00.000000Z"))
.await?;
db.upsert_concept(ConceptUpsert::new("entanglement", "Quantum Entanglement")
.valid_from("2026-01-01T00:00:00.000000Z"))
.await?;
db.assert_edge(EdgeAssertion::new("quantum", "entanglement", "ENTAILS")
.valid_from("2026-01-01T00:00:00.000000Z")
.weight(1.0))
.await?;
let subgraph = db.traverse()
.start_node("quantum")
.max_depth(3)
.execute(db.read_conn(), None)
.await?;
}pip install macrame-dbimport macrame
T0 = "2026-01-01T00:00:00.000000Z"
with macrame.Database.open("knowledge.db") as db:
db.write_concepts([
macrame.ConceptUpsert("quantum", "Quantum Computing", valid_from=T0),
macrame.ConceptUpsert("entanglement", "Quantum Entanglement", valid_from=T0),
])
db.assert_edge(
macrame.EdgeAssertion("quantum", "entanglement", "ENTAILS", valid_from=T0)
)
graph = db.load_subgraph("quantum", 3, 1 << 20)
print(graph.dijkstra("quantum"))Every design decision derives from these invariants:
- The boundary is sacred — Everything above libSQL is ours; everything below it is upstream. Never patch the engine.
- Two clocks, never mixed — Valid time and transaction time are independent axes. No code path derives one from the other.
- Assertions are immutable — Rows in
linksare never updated in place. The past is never rewritten; it is only ever superseded. - The ledger is a table, not the log — Transaction-time reconstruction reads
transaction_log, not WAL or CDC frames. - No physical deletion in hot tables — Rows leave through the archive path only. Ad-hoc
DELETEaborts at the trigger layer. - Derivative state is disposable —
links_currentis a rebuildable materialization. Drift is detectable, recoverable by rebuild. - Embeddings are immutable per version, excluded from the ledger — Vectors live in per-model tables; they never appear in
transaction_logpayloads. - Fidelity is a parameter, never a silent default —
as_of(ts)andreconstruct(ts)say what they mean in their signatures.
- One writer — a dedicated Tokio task holds the sole write-capable connection
- Many readers — WAL journaling; readers never block on writer
- Two-tier priority channels — high-priority (user-driven) preempts low-priority (background)
- Cooperative chunking — bounded to ~3 ms per chunk, sized from each chunk's measured hold rather than from a constant, under four per-path ceilings (90 edges, 70 concepts, 600 annotations, 30 embeddings) and a 35-row floor
| Version | Feature |
|---|---|
| v2 | Legacy-free baseline |
| v3 | analytics_annotations table |
| v4 | FTS5 external-content index |
| v5 | Overlap guard index |
| v6 | Overlapping closed intervals refused in actor |
| v7 | CHECK (weight >= 0.0) on links.weight |
| v8 | concepts.rowid_pk, the third FTS trigger, and the two unread indices dropped |
| v9 | trg_concepts_guard_delete becomes conditional on an archive session, so concepts can be archived (D-129) |
| v10 | trg_concepts_log_insert becomes conditional on the same marker, so rehydration mints no transaction-time facts (D-131) — current |
v8 is the last rung that could change a primary key before the 1.0 freeze: rowid_pk INTEGER PRIMARY KEY costs id the primary key, and D-036 forbids a primary-key change after 1.0 (D-119). It also drops idx_annotations_label and idx_lc_tgt_active, which shipped in the v7 baseline with no query that seeks on them — measured at −7.9% off assert_edge (D-089, D-118).
| Detail | Value |
|---|---|
| Edition | Rust 2021 |
| MSRV | 1.88 (verified, not declared) |
| Runtime | tokio async, single process |
| Engine | libSQL 0.9.30 (MIT, unmodified) |
| Schema version | 10 |
| Test suite | 348 Rust · 357 with metrics · 355 Python — all green (measured 2026-08-08, 0.12.0). The three property-tests binaries (23 tests) are run as their own step — see below. --all-features is not a supported configuration, see below. Regenerate rather than trust this line: python scripts/run_rust_suite.py --features metrics |
| Dependencies | tokio, serde, bincode, zstd, thiserror, tracing, ulid |
| Module | Responsibility |
|---|---|
schema |
DDL, triggers, migrations |
graph |
CTE compilation, subgraph loading, vector filters |
temporal |
as_of(), reconstruct(), snapshots, archive, rehydrate |
vector |
Model registration, embedding upsert, DiskANN search, hybrid RRF |
integrity |
Audit, atomic rebuild, chunked shadow-swap rebuild |
connection |
Database handle, Write Actor, priority channels |
error |
DbError enum, error classification |
| Detail | Value |
|---|---|
| Engine | pyo3 0.29 + maturin |
| Surface | Synchronous (Write Actor serialises all writes) |
| GIL | Released via Python::detach around Runtime::block_on |
| Distribution | macrame-db on PyPI, import macrame |
| Wheels | abi3-py310 — one per platform (Linux x86_64/aarch64, macOS universal2, Windows x86_64) |
| Python | CPython 3.10+ |
| Type stubs | Ship with wheel, py.typed set, mypy --strict in CI |
- Synchronous surface — The Write Actor serialises every write through one channel, so exposing
awaitadvertises concurrency the architecture does not grant. - Opaque
Subgraph— A#[pyclass]with forwarded accessors;.to_dict()for callers who want the copy. It paid for itself in 0.8.0: the crate re-representedEdgeRefand no binding signature moved, because there is no converted copy whose layout had to follow (D-101, D-123). - Open intervals cross as
None— Not a sentinel datetime, becausedatetime.maxcannot survive.astimezone()east of UTC. - Absent
contentcrosses asNone—load_subgraphdoes not fetch document text unless asked (content=True).""cannot mark not loaded, because it is a valid value of the type (D-116, D-123). - Every error is typed — 35 exception classes under
MacrameError, with six intermediate groups for catching sets:IntegrityError,ValidationError,VectorError,TemporalError,WriterError,BudgetError. metricsshipped on — The wheel ships with themetricsfeature enabled because feature flags do not survive into binary artifacts.
Re-measured at 0.8.0, because B2 changed how a
Subgraph is represented, B3 changed what a
load carries, and B4 dropped an index — three
reasons a table of 0.7.0 numbers would have been describing a different crate.
| Operation | Budget | 0.7.0 | 0.8.0 | 0.9.0 | 0.10.0 |
|---|---|---|---|---|---|
| Single assertion | ≤ 5 ms | — | 258 µs, published with an O(out-degree) caveat (D-059) | 224 µs, and the caveat is retired on measurement (D-134) | 220 µs |
| Single concept upsert | ≤ 3 ms | — | — | 198 µs | 193 µs |
| Chunk commit (edges, 90 rows) | ≤ 3 ms | 2.39 ms | 2.40 ms | 2.38 ms | 2.71 ms — see below |
| Three-hop traversal | ≤ 10 ms | 2.1 ms | 1.66 ms | 1.61 ms | 1.72 ms |
| Vector top-10 | ≤ 20 ms | 294 µs | 246 µs | 248 µs | 264 µs |
| Hybrid top-10 | ≤ 50 ms | 2.0 ms | 1.77 ms | 1.77 ms | 1.79 ms |
| Full fold (reconstruct) | ≤ 100 ms | 21 ms | 16.9 ms | 17.1 ms | 16.5 ms |
| Composition (snapshot + delta) | ≤ 100 ms | 3.4 ms | 2.18 ms | 2.22 ms | 2.06 ms |
| Rehydrate, 1 concept | ≤ 5 ms | — | n/a | 3.71 ms | 3.41 ms |
| Rehydrate, per concept after the 1st | ≤ 300 µs | — | n/a | ~74 µs to n=1,000; 114 µs at n=10,000 | ~71 µs to n=1,000; 105 µs at n=10,000 |
There is no 0.11.0 or 0.12.0 column, and the second absence needs a word. 0.11.0 changed no
code that runs, so a column would have been a second measurement of the same crate. 0.12.0 does
change one of these rows — bulk_import no longer commits 90-row chunks. 90 is now the
ceiling it starts from and it settles at 35 within a chunk or two
(D-146), so "chunk commit, edges, 90 rows"
still names a real measurement of a real transaction and no longer names what that method does.
A 0.12.0 column is not published rather than half-published: this table is a per-release series
measured as a whole under one control, and adding one row measured in a different session is the
practice D-070 and
D-145 exist to prevent. What 0.12.0 measured
instead is in §5.1.5,
against the fixed size rather than against previous releases.
0.10.0's column is a full re-measurement, median of three sessions, controls published below.
Every row is inside its budget. Eight of the ten are within ±8% of 0.9.0 — below the ~11%
single-arm variance D-134 measured and far
below D-070's ~29% session spread — which is
the expected answer, because 0.10.0 changed no traversal, no search, no fold and no write path.
control/select_1 reads 1.55–1.69 µs per group against
D-090's recorded 1.589–1.639 µs, so the
machine is where it was.
One row read high, and 0.12.0 explained it: the machine, not the chunk
(D-145). Chunk commit has published
2.39 / 2.40 / 2.38 ms for three releases and this column reads 2.71 ms — five measurements at a
1.1% spread, which was taken at the time as evidence that a 14% rise could not be session variance.
It was not evidence about that at all: a tight spread within a session says nothing about the
spread between sessions, which D-070 had
already measured at ~29%. Re-run over six sessions, the arm tracks control/select_1
monotonically — with the control at or below D-090's recorded band it reads 2.356 / 2.358 /
2.365 ms, a 0.4% spread agreeing with 2.39; with an elevated control it reads 2.54-2.73.
The cell is left at 2.71 because that is what was measured here, and this table is a
per-release series rather than a statement of current cost. The current cost is 2.39 ms.
It never overturned chunk_rows::EDGES, and by 0.11.0 it could not have.
D-058 solved the 90-row constant against the
3 ms bound from the 2.39 ms figure, which is why a 14% move in that figure looked consequential.
It is not any more: D-143 re-derived all four
constants against the fixture matrix, on grounds that never mention this number. The constant stays
at 90 for those reasons, and would have whichever way this row resolved.
Two controls, or the read-path numbers would mean nothing. A uniform improvement across
unrelated paths is what a faster machine looks like, so: the fixed control/select_1 row reads
1.51–1.62 µs against the 1.589–1.639 µs
D-090 recorded, and the chunk-commit path —
which 0.8.0 did not touch — is 2.39 → 2.40 ms. The machine has not moved and an untouched path
has not moved, so the 12–36% on the read paths is the code.
0.9.0 re-measured the same rows and the answer is "nothing moved", which is the result rather
than the absence of one. 0.9.0 changed the archive path and two triggers; it touched no traversal,
no search and no fold, so a table that showed a change would be evidence of a problem. Every
carried-over row but one is within 3.2% of its 0.8.0 figure — the largest being three-hop
traversal at −3.1% — with control/select_1 at 1.51–1.54 µs across every group.
The new row is the one 0.9.0 could plausibly have cost something. The v9 → v10 rung puts a
WHEN NOT EXISTS (SELECT 1 FROM sqlite_master …) clause on the concepts insert log trigger, and it
is evaluated on every concept write, not only during an archive. At 198 µs against a 3 ms
budget the gating is not measurable on this fixture — worth stating, because "we added a subquery to
the hot write path" is the kind of change that is usually paid for somewhere.
The single-assertion row reads 13% lower and that is not claimed as an improvement. Nothing in
0.9.0 touches the links write path, and no mechanism explains it. It is reported as measured and
attributed to nothing. The 0.9.0 text added a second reason to distrust the figure — that the row is
complexity-bound rather than a stable constant, "since it remains linear in out-degree" — and that
half is now withdrawn: it was never measured, and
D-134 measured it. What remains is an
unexplained 13%, which is the smaller and more honest claim.
Figures are the median of three runs, and the reason is a 21% excursion that the control did not
catch. The first pass read the full fold at 20.4 ms — with control/select_1 sitting normal at
1.59 µs — and two repeats returned 16.96 and 17.09 ms. A SELECT 1 round trip bounds machine,
scheduler and engine-overhead noise; it does not bound page-cache state or fsync variance, so an
I/O-bound row needs repetition as well as a control. D-070
put this project's session-to-session noise at ~29%, which is exactly the size of the thing that
almost got written down here as a regression.
The single-assertion row's caveat is retired, and it was wrong for four minor versions. This
paragraph used to say the row "remains linear in out-degree, so a high-degree hub still exceeds it".
overlap_guard now measures the assertion into tables of 0, 2,000 and 8,000 edges — hub out-degree
0, 666 and 2,666 — at 983 / 920 / 882 µs, median of three sessions against a 1.52 µs control, so
out-degree rises by thousands and latency does not move
(D-134). The claim described the access
path as it stood in 0.5.5 and has been false since the v5 → v6 rung shipped idx_lc_open_interval
(D-059) — it outlived the defect by four
releases because nothing measured it. The real cost is O(version count per edge key), which
archival caps. Dropping idx_lc_tgt_active bought −7.9% on that path
(D-118); the complexity claim it was said not to
change was not there to change.
Those figures are a shape, not a decimal: session-to-session spread on this path is ~11%, and normalising by the control does not remove it (D-070).
All budgets measured on named reference hardware, and deliberately not CI gates
(D-055) — an absolute ≤ 5 ms on a shared
runner is an assertion about whichever machine picked up the job. Regression detection uses
criterion baselines, machine against itself. See §9 of the architecture docs for full table.
| Risk | Mitigation |
|---|---|
| R15: Concurrent open → access violation (libSQL 0.9.30) | One open per database; R15 reproduces transparently through Python. --features property-tests is run as its own step, not folded into the suite: integrity_property_tests needs a database per case, and inside the full run it faults often enough that the classifier's three retries are routinely exhausted. Alone it crashes on 93 of 100 attempts and is green when it completes — measured 2026-08-08 under sustained load, and it is the engine rather than the tests. This row said "~50/50" until then, on no measurement of this quantity; .cargo/config.toml is where the rate lives and what to read before quoting it (D-147) |
| Property test binaries fault mid-suite | property-tests feature gate; serialised runs; CI classifies each run rather than counting failures, and retries only a crash |
| Covering index wins over selective | EXPLAIN QUERY PLAN assertions on every index-sensitive query |
| Snapshot chain divergence | verify_snapshot_chain() reports but does not repair (snapshots are disposable) |
--all-features is not a configuration this project supports or gates, and 0.10.0 stopped publishing a test count for it. --all-features is metrics + property-tests together, which puts the R15-prone binaries back inside the main run — the exact arrangement the step above exists to avoid. Measured 2026-08-07: 4 of 4 runs crashed at one attempt, and 4 of 5 still went red at the six-attempt retry budget the quarantined step uses. A required job that fails four times in five is not a gate, it is noise that teaches people to re-run CI without reading it. Run --features metrics and --features property-tests as the two separate steps CI does (D-140).
1.88, verified rather than declared — cargo +1.88.0 check --all-features --all-targets passes and 1.85 does not. The constraint comes from libsql-ffi's build dependency chain (bindgen → which → home), not from this crate's own code (which needs only 1.73).
- Architecture specification — normative surfaces: §4 (schema) and Appendix A (API)
- Architecture Quick Reference — v0.9.0 reference: API, schema, decisions, performance
- Python bindings — §14: async→sync boundary, error tree, stubs
- Decision register — D-001…D-109 with rationale
Distribution macrame-db, import macrame — on both crates.io and PyPI. The Rust side has no caveat: a crate's [lib] name is namespaced per build graph, so macrame-db providing macrame collides with nothing. site-packages is flat.
The PyPI package macrame is an unrelated, effectively abandoned build tool (0.0.1, 2021). If it installs a top-level macrame/, then installing both leaves two distributions contending for one directory — pip warns on file conflicts, so this is a known and non-silent risk. Importing as macrame_db is the fallback if it ever matters.
See LICENSE for details.