Own the entire modern LLM lifecycle on the hardware you have. Two tracks: (T1) train a small (10–50M-param) transformer from scratch, and (T2) MLX-QLoRA fine-tune + DPO-align a ≤1.5B open model — then quantize, serve with a from-scratch micro-batcher, and govern with an eval + safety layer. One reproducible pipeline on an Apple M1 (8 GB). No rented GPU required.
Status: 🔨 M0–M3 of 8 done (2026-07-27). Domain committed (clinical/medical Q&A),
held-out benchmark frozen and stratified (300 MedMCQA items, sha 41716514e227…),
toolchain verified on the M1 (make check 4/4), and a clean, decontaminated 145,838-row
corpus whose cleanliness is shown by a positive control, not asserted: a leakage probe
scores 99.3% on a deliberately contaminated corpus and 0% on the shipped one. M2
trains a 10.79M-param char-level transformer from scratch in MLX — held-out perplexity
16.9 → 4.24, coherent medical MCQ text from random init, and a headline finding that
overturns the M1 prediction: the over-parameterized model did not memorize (gap −0.07,
0.96 novelty) because a sub-epoch budget plus dropout regularizes it. M3 QLoRA-SFTs a
Qwen2.5-0.5B-4bit base and reports an honest negative: SFT does not beat the base
(letter-only 31.0% vs base 31.3%; the base already sits at the 31.7% majority floor via a
degenerate always-A policy that SFT fixes without adding accuracy), and reasoning-completion
SFT regresses to 7.0% by teaching the small model verbose, non-committal generation (72%
never emit an answer). The 0.5B is capacity-bound; beating it needs a 1.5B+ base, which
8 GB can't train. The SFT pipeline (letter-logprob + CoT-generation evaluator, both
completion styles, fuse) is complete and correct. 82 tests pass. Build from
docs/02-build-plan.md. Next: M4 (hand-rolled DPO).
Capstone #2 of 3. Siblings: self-improving-agent-platform,
realtime-decision-intelligence. Effort: 6–12 months solo.
Interactive/exportable version: docs/assets/architecture.html.
Most engineers only ever call models. This project proves you understand them from the inside — data, training dynamics, alignment, quantization, inference optimization, and safety — by carrying models through the full lifecycle on a laptop, without pretending to run infrastructure you don't have.
An 8 GB M1 physically cannot do billion-scale continued-pretraining or multi-GPU FSDP. So instead of faking that on cloud rentals, we own two lifecycles completely on hardware we have:
- T1 — from-scratch tiny LM (10–50M params). The purest "I built the model, not just the app" artifact. Trains on the M1 GPU in hours. The mid-2026 fine-tuning caveat (~80% of jobs don't ask for it) does not apply — this is understanding, not a hire-checkbox.
- T2 — adapt an open model (≤1.5B). The practical skill: MLX-QLoRA SFT + DPO alignment at a size the M1 actually runs. Proves you can bend a real model to a domain.
Extends the existing llm-quantization-bench (already proven on this exact M1) into the
full stack above it.
- A from-scratch tiny LM that goes from noise → coherent domain text (loss curve + generation samples).
- A QLoRA-adapted 1.5B model that beats its base on a domain benchmark.
- A full quantization Pareto chart (size × speed × accuracy) for both.
- A served endpoint hitting stated p50/p99 latency SLOs via a from-scratch micro-batcher.
- An auto-generated model card + safety report (red-team before/after).
- One command replays raw-data → governed endpoint on the M1.
| Task | Tool | Notes |
|---|---|---|
| T1 from-scratch training | nanoGPT-in-MLX | 10–50M params, fits in memory, trains in hours |
| T2 SFT + DPO | mlx-lm (QLoRA, 4-bit) | ≤1.5B base, --batch-size 1 |
| Quantize | llama.cpp (GGUF Q4/Q8, Metal) | reuse quantization-bench harness |
| Serve | from-scratch asyncio micro-batcher + FastAPI | vLLM won't fit — hand-roll it |
| Optional large run (7B QLoRA / distributed) | rented A100 (Lambda/RunPod/Colab) | off critical path, documented only |
PyTorch / DeepSpeed / vLLM / FSDP appear only in the optional cloud-burst script, never on the laptop.
raw domain corpus
│ dedup · quality-filter · decontaminate · synth-gen
▼
┌─────────────┐ ┌──────────────────────────┐
│ T1: from │ │ T2: adapt ≤1.5B open base │
│ scratch │ │ QLoRA SFT → DPO align │
│ 10–50M MLX │ │ (measure alignment tax) │
└──────┬──────┘ └────────────┬─────────────┘
└──────────┬─────────────────┘
▼
quantize (GGUF Q4/Q8) → Pareto: size × speed × accuracy
▼
serve (from-scratch micro-batcher + KV-cache, p50/p99 SLO)
▼
govern: eval @ every stage · model card · red-team · drift
Detail: docs/01-architecture.md.
ondevice-model-lifecycle/
├── README.md · .gitignore · requirements.txt
├── docs/ 00-what-it-is · 01-architecture · 02-build-plan · 03-setup
├── data_engine/ # collect, dedup, filter, decontaminate, synth
├── t1_scratch/ # nanoGPT-in-MLX tiny LM
├── t2_adapt/ # QLoRA SFT + DPO
├── quantize/ # GGUF conversion + Pareto bench
├── serve/ # micro-batcher + FastAPI endpoint
├── govern/ # eval suite, model card gen, red-team
└── models/ # gitignored artifacts
source ~/miniconda3/etc/profile.d/conda.sh && conda activate personal
make install # pinned deps: mlx, mlx-lm, llama-cpp-python (Metal), ...
make check # verify MLX/Metal, mlx-lm, llama.cpp and the frozen benchmark
make test # fast offline suiteMilestone targets (m1-data, m2-t1, m3-sft, …) land as each milestone does; see the
stub list at the bottom of the Makefile. make help lists what currently works.
Python 3.12 · Apple MLX / mlx-lm · nanoGPT-in-MLX · HF datasets · llama.cpp (GGUF, Metal) · from-scratch asyncio micro-batcher · FastAPI streaming · MLflow/W&B · DVC · matplotlib. Cloud (optional burst only): PyTorch · DeepSpeed/FSDP · vLLM · AWQ/GPTQ.
Private. All rights reserved.
