| title | Ani |
|---|---|
| emoji | 🌱 |
| colorFrom | green |
| colorTo | yellow |
| sdk | docker |
| app_port | 7860 |
| pinned | true |
| license | mit |
One photo → an AI grade + spoilage-urgency score → a ranked feed of live Metro Manila buyers.
A fine-tuned Gemma-3-27B grader, served privately on a single AMD Instinct MI300X.
Ani Demo
▶ Live demo · Judges start here · Measured results · Business case
AMD Developer Hackathon: ACT II — Track 3 (Unicorn) · entry for “Best AMD-Hosted Gemma Project.”
Ani's grader isn't an API wrapper — it's a Gemma-3-27B model LoRA fine-tuned and served on a
single MI300X, and every number below traces to a committed receipt in
training/receipts/. Full analysis: METRICS.md.
| What | Measured | Receipt |
|---|---|---|
| Grading accuracy | base 86.7% → tuned 100% (+13.3 pts) on a balanced held-out set | b7_metrics.json |
| Grade latency (end-to-end) | p50 1.40 s, p95 1.40 s — image → grade → buyer match | latency_bench.json |
| Throughput | ~43 grades/min sequential, source: mi300x verified 20/20 |
latency_bench.json |
| VRAM in use | 153.9 GiB during image inference (51.66 GiB weights + 99.42 GiB KV) | rocm-smi-loaded.log |
| Concurrency | 22.69× @ 8,192-token context | vllm_serve.log |
| Card | AMD Instinct MI300X (192 GB HBM3), gfx942, ROCm + vLLM |
vllm_serve.log |
Read the accuracy honestly. The tuned 100% is a point estimate on a small held-out set
(n = 30, single seed) — its Wilson 95% confidence interval is [88.6%, 100%], i.e. the true
accuracy is most consistent with ≥ ~89%, a solid lift over the base model's 86.7%. We report
the interval (and ship a multi-seed variance harness, run_variance_eval.sh) rather than headline
a bare 100%. The honest limitations are logged in ROCM_NOTES.md.
Why AMD is non-negotiable, not decorative: the grader reserves ~151 GiB and measures 153.9 GiB in use — that co-hosts on one MI300X (192 GB) with headroom, but does not fit a single 80 GB H100. A farmer co-op can therefore run the whole grader privately, on-site — pricing, contracts, and farmer photos never leave the cooperative.
- Open the live demo (the Hugging Face Space runs the same app).
- Add a harvest photo → the grader reads it and fills the crop + volume.
- Press ⚡ Grade & match → watch the pipeline: grade (fine-tuned Gemma vision) → demand match (embeddings over live NCR prices) → dispatch (most-perishable-first).
The web tier degrades gracefully to a built-in stub if the backend is unreachable, so the demo
never hard-fails in front of a judge — but with INFERENCE_BASE_URL set, it runs live on the MI300X
(responses are tagged source: mi300x). See JUDGES_START_HERE.md.
| Tier | What | Where it runs |
|---|---|---|
| 1 — Web / UI | Next.js app (web/) — capture, agent trace, results, ROI calculator |
HF Space / Vercel · Docker :7860 |
| 2 — Inference / orchestration | FastAPI (inference/) — /analyze /grade /match /process, LangGraph multi-agent |
MI300X host (stub / Fireworks / mi300x backends) |
| 3 — Models | Fine-tuned Gemma-3-27B grader + embeddings via vLLM on ROCm (training/) |
AMD MI300X (192 GB) |
Next.js (Tier 1) ──INFERENCE_BASE_URL──▶ FastAPI (Tier 2) ──ANI_BASE_URL──▶ vLLM/ROCm on MI300X (Tier 3)
Swapping the backend from stub → Fireworks → MI300X is a single environment variable — no code changes. That contract is the whole point: the demo is portable, the compute is AMD.
Gemma-3-27B, LoRA (r=16, α=32) on a Benguet-relevant produce set (240 train / 60 balanced eval),
one epoch on the MI300X. Scripts in training/:
bash run_finetune.sh # fine-tune + eval on the card
SEEDS="42 7 123" bash run_variance_eval.sh # multi-seed variance (full 60-ex eval)
python3 aggregate_metrics.py # -> receipts/variance/summary.md (mean±std + Wilson CI)
bash serve_vllm.sh & FIREWORKS_API_KEY=… bash capture_receipts.sh # loaded rocm-smi snapshot
ANI_PROCESS_URL=http://127.0.0.1:8000 python3 bench_latency.py --n 20 # grade p50/p95 + grades/minBenguet grows ~80% of Metro Manila's highland vegetables, yet 42–50% is lost across harvest →
transport → market — a coordination-and-timing problem software can fix this season, not a growing
problem. Ani grades a harvest, scores its spoilage urgency, and matches it to live NCR demand so the
most perishable loads move first. Paying customers are farmer co-ops, traders/consolidators, and
institutional buyers; the beachhead is one Benguet co-op + the La Trinidad trading post, expanding
across Luzon then SEA cold chains. Full sizing, revenue model, and unit economics in
BUSINESS.md — and the interactive ROI calculator is live in the demo's Market
section (edit volume / price / spoilage, see value recovered).
web/ Tier 1 — Next.js app (UI, ROI calculator, API proxy routes)
inference/ Tier 2 — FastAPI + swappable backends (stub · fireworks · langgraph/mi300x)
training/ Tier 3 — LoRA fine-tune, vLLM serving, benchmarks, and committed receipts/
BUSINESS.md market, revenue model, unit economics
JUDGES_START_HERE.md 60-second judge path
cd web && npm install && npm run dev # http://localhost:3000git add . && git commit -m "update" && git push # HF builds the root Dockerfile → port 7860Built by Angelo Sebaxtian Vasquez, Jan Añonuevo, Jakey Manahan, and JC Diamante for the AMD Developer Hackathon: ACT II (Track 3 — Unicorn).
Gemma-3-27B (Google DeepMind) · served with vLLM on AMD ROCm · MI300X via AMD Developer Cloud. Licensed MIT.