Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: build validate audit eval site graph track synthesize recap-dry clockbench fair check help
.PHONY: build validate audit eval site graph track synthesize recap-dry clockbench fair repos check help

help:
@echo "longevity-loop — an AI-native compounding loop for aging science"
Expand All @@ -9,7 +9,8 @@ help:
@echo " make synthesize Draft frontier+roadmap proposals into data/_synthesis.md (human-gated)"
@echo " make clockbench Cross-clock disagreement benchmark (measures gaps-analysis G1)"
@echo " make fair Score open aging datasets for FAIR/reproducibility → docs/FAIR.md (G2)"
@echo " make check validate + audit + eval + clockbench + fair + build drift-gate (CI finish line)"
@echo " make repos Per-repo KG + agentic-tooling generation recipes → docs/REPOS.md (hub)"
@echo " make check validate + audit + eval + clockbench + fair + repos + build drift-gate (CI finish line)"

build:
python3 scripts/build.py
Expand Down Expand Up @@ -54,11 +55,17 @@ clockbench:
fair:
python3 scripts/fair.py

# Per-repo KG + agentic-tooling generation recipes → docs/REPOS.md (docs/HUB_ARCHITECTURE.md).
repos:
python3 scripts/repos.py

# Finish line: data well-formed, still an AI-native loop, executions honestly eval'd
# (no fake ✅), the clock-disagreement + FAIR metrics self-verify, and generated docs match.
# (no fake ✅), the clock-disagreement + FAIR + repo-recipe artifacts self-verify, and docs match.
check: validate audit eval
python3 scripts/clockbench.py --selftest
python3 scripts/fair.py --selftest
python3 scripts/fair.py --gate
python3 scripts/fair.py --check
python3 scripts/repos.py --selftest
python3 scripts/repos.py --check
python3 scripts/build.py --check
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ Sequenced bridge: convert the computational finding + your credential network (G

<h2 id="execution">✅ Roadmap Execution</h2>

**11/22 done (50%).** Each execution is a checkbox with a before→after eval; a box only ticks with a real result (`done` requires a non-pending before AND after — no evidence ⇒ not done, gated in CI).
**12/23 done (52%).** Each execution is a checkbox with a before→after eval; a box only ticks with a real result (`done` requires a non-pending before AND after — no evidence ⇒ not done, gated in CI).

- [x] **E1** (P1 · Foundation) — Stand up the AI-native loop (repo + self-audit + gated `make check`)
- [x] **E2** (P1 · Foundation) — Map the field into a verified knowledge base
Expand All @@ -181,6 +181,7 @@ Sequenced bridge: convert the computational finding + your credential network (G
- [x] **E20** (P1 · Foundation) — Learn from mims-harvard/ATHENA; integrate the frontier signal into the field map
- [x] **E21** (P1 · Foundation) — Problem map: top aging problems as a relationship graph + the one target + low-hanging fruit
- [x] **E22** (P1 · Foundation) — Publish the tracker.json data contract for the interactive portfolio instance (data-first slice)
- [x] **E23** (P1 · Foundation) — Hub architecture (knowledge·tooling·experts per cited repo) + the per-repo generation-recipe layer

### Eval reports — before → after

Expand Down Expand Up @@ -208,6 +209,7 @@ Sequenced bridge: convert the computational finding + your credential network (G
| ✅ E20 | external frontier work studied → integrated (cited) into the loop | ATHENA / ToolUniverse / Zitnik not tracked; no study on file | research/athena-study.md (cited analysis) + Zitnik (people), ATHENA-R1 (frontier), ToolUniverse (stack) added; finding: ATHENA's citation-allow-list = our existing no-evidence-no-claim; RAG/multi-agent/ToolUniverse overkill for a solo code-only loop |
| ✅ E21 | a clear, visual answer to 'what are we shooting, and what's the quick win' | gaps ranked (gaps-analysis.md) but relationships + the single target + quick wins were implicit | docs/PROBLEMS.md — Mermaid relationship graph (G1 measurement = root bottleneck we target) + top-problems table + how/when/why + 3 low-hanging fruit with done-when; README pointer |
| ✅ E22 | a stable, derived JSON contract the agentic-portfolio instance can consume (no drift) | no machine-readable mission/progress feed for an external interactive surface | site/tracker.json — mission + 48% progress + 21 milestones + roadmap + low-hanging + frontier + 12 hubs + links, generated by build_site.py, deployed on Pages |
| ✅ E23 | a fresh, cheap, future-proof way to offer a KG + agentic tooling for each cited repo | cited repos listed in stack.yml but no KG/tooling per repo; no design for how to do it well | docs/HUB_ARCHITECTURE.md (index+JIT design; reuse graphify/understand/reverse; MCP; SHA-cached) + scripts/repos.py → docs/REPOS.md (22 repos, per-repo KG + reverse recipes), gated in make check |

---

Expand All @@ -228,6 +230,12 @@ Sequenced bridge: convert the computational finding + your credential network (G

---

<h2 id="hub">🧩 Knowledge · Tooling · Experts hub</h2>

This repo is a **hub**: for each cited repo it offers a knowledge graph (knowledge) + agentic tooling (skills/plugins/workflows), and links them to the experts who build them. Design (high-quality · fast · cheap · fresh · future-proof): **[docs/HUB_ARCHITECTURE.md](docs/HUB_ARCHITECTURE.md)** · per-repo generation recipes: **[docs/REPOS.md](docs/REPOS.md)**. Fresh index, just-in-time depth — deep artifacts are generated on demand and cached by commit SHA, never committed stale.

---

<h2 id="target">🎯 What we're shooting</h2>

The top aging problems, how they relate, the **one root bottleneck we target** (G1 — measuring aging), and the **low-hanging fruit** for quick wins — as a graph: **[docs/PROBLEMS.md](docs/PROBLEMS.md)**. Root bottleneck = our lane; everything downstream is gated on being able to measure aging in a shared, validated way.
Expand Down
9 changes: 9 additions & 0 deletions data/executions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,12 @@
before: "no machine-readable mission/progress feed for an external interactive surface"
after: "site/tracker.json — mission + 48% progress + 21 milestones + roadmap + low-hanging + frontier + 12 hubs + links, generated by build_site.py, deployed on Pages"
proof: "scripts/build_site.py · site/tracker.json · research/portfolio-extension-study.md"
- id: E23
phase: "P1 · Foundation"
item: "Hub architecture (knowledge·tooling·experts per cited repo) + the per-repo generation-recipe layer"
done: true
eval:
metric: "a fresh, cheap, future-proof way to offer a KG + agentic tooling for each cited repo"
before: "cited repos listed in stack.yml but no KG/tooling per repo; no design for how to do it well"
after: "docs/HUB_ARCHITECTURE.md (index+JIT design; reuse graphify/understand/reverse; MCP; SHA-cached) + scripts/repos.py → docs/REPOS.md (22 repos, per-repo KG + reverse recipes), gated in make check"
proof: "docs/HUB_ARCHITECTURE.md · scripts/repos.py · docs/REPOS.md · make check"
106 changes: 106 additions & 0 deletions docs/HUB_ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# The hub architecture — knowledge · tooling · experts, per cited repo

_How this repo becomes the **hub** that offers, for each top cited repo, a **knowledge graph**
(knowledge) and **agentic tooling** (skills / plugins / workflows) — engineered to be high-quality,
fast, cheap, always up-to-date, and future-proof._

---

## The one decision that determines everything: INDEX + JIT, not a warehouse of frozen artifacts

The tempting move — eagerly generate and commit a KG + a skill + a plugin + a workflow for all ~27
cited repos — is the wrong one, and it fails **every** quality bar at once:

| Bar | Why the "generate-all-now" approach fails it |
|---|---|
| High quality | Bulk auto-generation is slop — 27 repos × 4 artifacts, none verified against the real repo. |
| Fast | Serial LLM generation over 100+ artifacts is slow to produce and slow to review. |
| Cheap | You pay LLM cost for every repo × every artifact, including repos no one ever uses. |
| Up-to-date | Repos push weekly; a committed KG/skill is **stale the day after** you generate it. |
| Future-proof | A frozen pile of bespoke artifacts is a maintenance graveyard, not an asset. |

So the architecture is three layers, each optimized for a different bar:

```
① INDEX (committed, cheap, always fresh) ── the hub's front door
registry (data/stack.yml) + live GitHub metadata → a ranked, queryable repo index
│ (cheap: API only, no LLM · fresh: weekly cron · deterministic)
② RECIPE (committed, deterministic) ── how to make the deep artifacts, per repo
per-repo generation commands: KG via graphify/understand-anything · tooling via anyagent reverse
│ (this file's PoC: scripts/repos.py → docs/REPOS.md)
③ JIT DEEP (generated on demand, cached by commit SHA, NOT committed) ── the actual KG + skills
run the recipe only when you need a repo, cache it, regenerate only when the repo's SHA changes
│ (quality: verified/gated · cheap: never for an unused repo · fresh: SHA-keyed)
④ INTEROP (MCP + portable formats) ── future-proof
KGs as JSON · skills as SKILL.md · tools via `anyagent mcp` → any agent client can consume the hub
```

**The thesis in one line:** keep a *fresh, cheap index* committed; generate the *expensive deep
artifacts just-in-time* by reusing existing engines, keyed to the source commit so they're never
stale — you never pay for a repo you don't use, and you never ship one that's out of date.

---

## Reuse the engines — don't rebuild them (backbone: *call it, don't absorb it*)

Every deep artifact maps to a tool that already exists. The hub **orchestrates**; it does not
reimplement graph extraction or skill synthesis.

| Deep artifact | Engine to reuse | Input |
|---|---|---|
| Knowledge graph (codebase) | `understand-anything` (code → architecture KG) | a cloned repo |
| Knowledge graph (docs/model card/paper) | `graphify` (any input → clustered KG + HTML/JSON) | README / model card / paper |
| Agentic tooling blueprint | `anyagent reverse <repo> --markdown` | repo URL |
| Skill / workflow from that blueprint | `anyagent build` / `refine` (gated to a score) | the blueprint |
| Serve it to any client | `anyagent mcp` (MCP) / a `SKILL.md` | the generated tool |
| Live repo metadata | GitHub API (already in `scripts/track.py`) | the repo slug |

If a better KG engine appears, you swap the **recipe**, not the data — that's the future-proofing.

---

## How each quality bar is met

- **High quality** — *no evidence ⇒ no claim* applied to generation: each artifact records the source
repo + **commit SHA** it was built from, and a skill only ships if `anyagent refine` clears its
score gate (does it build? does it run?). Provenance = reproducible + drift-detectable.
- **Fast** — the index is API-only (seconds); deep generation fans out **per-repo in parallel**
(subagents / a workflow), and only for repos you asked for or that changed.
- **Cheap** — zero LLM in the index. LLM only in ③, only JIT, only when the SHA moved. An unchanged
repo is never regenerated; an unused repo is never generated at all.
- **Up-to-date** — the weekly cron (extend `track.yml`) refreshes ① and stamps each deep artifact with
its source SHA; when the repo moves, the artifact is flagged **stale**, not silently wrong.
- **Future-proof** — portable standard formats (JSON KG, `SKILL.md`, MCP), registry-driven from the
single source of truth (`data/stack.yml`), swappable engines. The hub is MCP-queryable, so any
future client (Claude Desktop/Code, another agent) consumes it without bespoke glue.

## The third pillar — experts — is already wired

`people.yml` + `frontier.yml` + the bi-temporal field graph (`build_graph.py`) already link
experts ↔ orgs ↔ topics ↔ recent works. The hub adds the missing edge: **repo → maintainer/lab →
expert**, so "knowledge (KG) · tooling (skills) · experts (people)" become one graph you can traverse
(e.g. *pyaging → rsinghlab → the clocks experts to reach*).

---

## Phased rollout (decompose — never one-shot)

- **Phase 0 (this PR):** the RECIPE layer — `scripts/repos.py` turns `data/stack.yml` into a per-repo
generation index (`docs/REPOS.md`): KG command + tooling command + provenance, deterministic and
gated. Proves the model with **zero** wasteful generation.
- **Phase 1:** enrich ① with live GitHub metadata (stars, last-push SHA, license, health) on the
weekly cron — extend `track.py`; write a generated `data/_repos.yml` (like `_frontier.yml`).
- **Phase 2:** JIT deep generation for the **top-N** repos only (start with the North-Star lane:
Biolearn, pyaging, Geneformer, scGPT), via `graphify` + `anyagent reverse`, cached by SHA, exposed
via MCP. Verified/gated before publish.
- **Phase 3:** surface the hub in the agentic-portfolio instance (reads the same `tracker.json` +
`_repos.yml`), so the KG/tooling/experts hub is browsable and agent-answerable.

**Anti-pattern to avoid at every phase:** committing a deep artifact without its source SHA, or
generating for a repo before someone needs it. Fresh index, lazy depth.

_Not medical advice. All artifacts are computational metadata about open-source repos; no wet-lab or
therapeutic claim._
51 changes: 51 additions & 0 deletions docs/REPOS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Repo hub — generation recipes (RECIPE layer)

_Generated from `data/stack.yml` + `data/frontier.yml` by `scripts/repos.py`. See the design in [HUB_ARCHITECTURE.md](HUB_ARCHITECTURE.md)._

**22 cited repos** (20 on GitHub). This is the **recipe**, not the artifacts: run a repo's commands **on demand** and cache the result by the repo's commit SHA — deep knowledge graphs and skills are made just-in-time, never committed stale. You never pay for a repo you don't use.

## The recipe (per repo)

```bash
# 1. knowledge graph
understand-anything <clone-dir> # GitHub codebases (architecture KG)
graphify <url> # model cards / docs / papers (clustered KG)
# 2. agentic tooling (skill / plugin / workflow)
anyagent reverse <url> --markdown # → blueprint, then:
anyagent build/refine "<blueprint>" --target-score 80 # → a gated skill/workflow
# 3. serve to any client
anyagent mcp # expose the generated tools over MCP
```

> Quality gate: stamp every generated artifact with the source repo + commit SHA (reproducible + staleness-detectable); a skill ships only after `refine` clears its score.

## The repos

| Repo | Kind | Host | KG engine | Tooling |
|---|---|---|---|---|
| [BixBench](https://github.com/Future-House/BixBench) | benchmark | github | `understand-anything` | `anyagent reverse Future-House/BixBench` |
| [LAB-Bench](https://github.com/Future-House/LAB-Bench) | benchmark | github | `understand-anything` | `anyagent reverse Future-House/LAB-Bench` |
| [Open Problems in Single-Cell](https://github.com/openproblems-bio/openproblems) | benchmark | github | `understand-anything` | `anyagent reverse openproblems-bio/openproblems` |
| [DunedinPACE](https://github.com/danbelsky/DunedinPACE) | clock | github | `understand-anything` | `anyagent reverse danbelsky/DunedinPACE` |
| [scAge](https://github.com/alex-trapp/scAge) | clock | github | `understand-anything` | `anyagent reverse alex-trapp/scAge` |
| [Marinka Zitnik](https://github.com/mims-harvard/ATHENA) | frontier-work | github | `understand-anything` | `anyagent reverse mims-harvard/ATHENA` |
| [Morgan Levine](https://github.com/HigginsChenLab/methylCIPHER) | frontier-work | github | `understand-anything` | `anyagent reverse HigginsChenLab/methylCIPHER` |
| [cognee](https://github.com/topoteretes/cognee) | graph | github | `understand-anything` | `anyagent reverse topoteretes/cognee` |
| [Graphiti](https://github.com/getzep/graphiti) | graph | github | `understand-anything` | `anyagent reverse getzep/graphiti` |
| [PyTorch Geometric Temporal](https://github.com/benedekrozemberczki/pytorch_geometric_temporal) | graph | github | `understand-anything` | `anyagent reverse benedekrozemberczki/pytorch_geometric_temporal` |
| [TGB (Temporal Graph Benchmark)](https://github.com/shenyangHuang/TGB) | graph | github | `understand-anything` | `anyagent reverse shenyangHuang/TGB` |
| [AltumAge](https://github.com/rsinghlab/AltumAge) | model | github | `understand-anything` | `anyagent reverse rsinghlab/AltumAge` |
| [ESM-2](https://github.com/facebookresearch/esm) | model | github | `understand-anything` | `anyagent reverse facebookresearch/esm` |
| [Geneformer](https://huggingface.co/ctheodoris/Geneformer) | model | huggingface | `graphify` | `anyagent reverse <url>` |
| [scGPT](https://github.com/bowang-lab/scGPT) | model | github | `understand-anything` | `anyagent reverse bowang-lab/scGPT` |
| [TxGNN](https://github.com/mims-harvard/TxGNN) | model | github | `understand-anything` | `anyagent reverse mims-harvard/TxGNN` |
| [AlphaFold](https://github.com/google-deepmind/alphafold) | tool | github | `understand-anything` | `anyagent reverse google-deepmind/alphafold` |
| [Biolearn](https://bio-learn.github.io/) | tool | web | `graphify` | `anyagent reverse <url>` |
| [CZ CELLxGENE Census](https://github.com/chanzuckerberg/cellxgene-census) | tool | github | `understand-anything` | `anyagent reverse chanzuckerberg/cellxgene-census` |
| [pyaging](https://github.com/rsinghlab/pyaging) | tool | github | `understand-anything` | `anyagent reverse rsinghlab/pyaging` |
| [scanpy](https://github.com/scverse/scanpy) | tool | github | `understand-anything` | `anyagent reverse scverse/scanpy` |
| [ToolUniverse](https://github.com/mims-harvard/ToolUniverse) | tool | github | `understand-anything` | `anyagent reverse mims-harvard/ToolUniverse` |

---

KG engine split: `understand-anything` for the 20 GitHub codebases, `graphify` for model cards / docs. PR a repo into `data/stack.yml` and it joins the hub automatically.
9 changes: 9 additions & 0 deletions scripts/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,15 @@ def render_readme() -> str:
"carries a datasheet + an honest self-assessment (unknown ⇒ no). Full scorecard with "
"load recipes: **[docs/FAIR.md](docs/FAIR.md)**.", "", "---", ""]

# The hub (static pointer to the repo-recipe index + architecture)
L += ['<h2 id="hub">🧩 Knowledge · Tooling · Experts hub</h2>', "",
"This repo is a **hub**: for each cited repo it offers a knowledge graph (knowledge) + "
"agentic tooling (skills/plugins/workflows), and links them to the experts who build them. "
"Design (high-quality · fast · cheap · fresh · future-proof): "
"**[docs/HUB_ARCHITECTURE.md](docs/HUB_ARCHITECTURE.md)** · per-repo generation recipes: "
"**[docs/REPOS.md](docs/REPOS.md)**. Fresh index, just-in-time depth — deep artifacts are "
"generated on demand and cached by commit SHA, never committed stale.", "", "---", ""]

# What we're shooting (static pointer to the problem map)
L += ['<h2 id="target">🎯 What we\'re shooting</h2>', "",
"The top aging problems, how they relate, the **one root bottleneck we target** "
Expand Down
Loading
Loading