A transit-domain function-calling model: a fine-tune of Cactus-Compute/needle
(a 26M-parameter Simple Attention Network) that turns a
natural-language transit query into a single tool call — or refuses when none applies. Ships with a
finetuning toolkit so you can adapt it to your own tools.
Open weights:
| Model | Format | Use |
|---|---|---|
rasaboun/needle-transit |
JAX/Flax .pkl |
Training / Python inference |
rasaboun/needle-transit-onnx |
ONNX | In-browser / on-device (onnxruntime-web) |
This repo is a fork of cactus-compute/needle; the base architecture, pretraining, and runtime are theirs. What's added here is the transit finetune and the finetuning enhancements below.
- Tunable finetuning hyperparameters —
--lr,--muon-lr,--warmup-ratio,--decay-ratioexposed onneedle train/needle finetune(previously hard-coded). - Per-field loss weighting —
--w-name,--w-value,--w-keyweight the loss on tool-name, argument-value, and argument-key tokens independently. Defaults2.0 / 4.0 / 1.5bias the model toward getting argument values right. - Metrics logging — set
NEEDLE_METRICS_LOG=path.jsonlto stream per-step{step, epoch, text_loss, grad_norm, adam_lr, muon_lr, val_ppl}for offline plotting. - Local-checkpoint reuse — finetuning uses a local checkpoint if present instead of always re-downloading from HF.
26M-parameter encoder-decoder Simple Attention Network (d_model=512, enc×12, dec×8, GQA 8H/4KV). An extractor: it lifts slot values verbatim from the query; canonical resolution (station disambiguation, routing) is the backend's job. Handles French, English, and FR/EN code-switching, plus typos and SMS-style compression.
[
{"name": "search_itinerary", "parameters": {
"origin": {"type": "string", "required": true},
"destination": {"type": "string", "required": true},
"time_human": {"type": "string", "required": false},
"time_mode": {"type": "string", "description": "depart_at | arrive_by", "required": false}}},
{"name": "get_next_arrivals", "parameters": {
"station": {"type": "string", "required": true},
"line": {"type": "string", "required": false}}}
]| Query | Output |
|---|---|
Itinéraire de Bastille à Nation |
[{"name":"search_itinerary","arguments":{"origin":"Bastille","destination":"Nation"}}] |
De Issy à Charles-de-Gaulle, départ 14h |
[{"name":"search_itinerary","arguments":{"origin":"Issy","destination":"Charles-de-Gaulle","time_human":"départ 14h","time_mode":"depart_at"}}] |
How do I get from Gare du Nord to La Défense? |
[{"name":"search_itinerary","arguments":{"origin":"Gare du Nord","destination":"La Défense"}}] |
Prochain métro à Bastille ligne 1 ? |
[{"name":"get_next_arrivals","arguments":{"station":"Bastille","line":"1"}}] |
cmt aller a chatelet depuis nation |
[{"name":"search_itinerary","arguments":{"origin":"nation","destination":"chatelet"}}] |
Quel temps fait-il ? |
[] |
git clone https://github.com/Rasaboun/needle-transit.git
cd needle-transit && source ./setup
needle playgroundWeb UI at http://127.0.0.1:7860 to test and finetune on your own tools. Weights auto-download.
from needle import SimpleAttentionNetwork, load_checkpoint, generate, get_tokenizer
params, config = load_checkpoint("checkpoints/needle-transit.pkl")
model = SimpleAttentionNetwork(config)
result = generate(
model, params, get_tokenizer(),
query="Prochain métro à Bastille ligne 1 ?",
tools='[{"name":"get_next_arrivals","description":"Next departures at a stop.","parameters":{"station":{"type":"string","required":true},"line":{"type":"string","required":false}}}]',
stream=False,
)
print(result)
# [{"name":"get_next_arrivals","arguments":{"station":"Bastille","line":"1"}}]Adapt the model to your own tools/domain:
# Auto-downloads base weights (or reuses a local --checkpoint), trains, evaluates
needle finetune data.jsonl
# Tune the run
needle finetune data.jsonl \
--lr 3e-5 --muon-lr 0.02 --warmup-ratio 0.05 --decay-ratio 0.05 \
--w-name 2.0 --w-value 4.0 --w-key 1.5
# Stream training metrics to a JSONL file
NEEDLE_METRICS_LOG=logs/run.jsonl needle finetune data.jsonlBest checkpoint is saved as checkpoints/needle_finetuned_<id>_best.pkl.
--w-name / --w-value / --w-key scale the loss on different parts of the tool call. Raise --w-value
when argument values are the hard part (the default 4.0 already does); raise --w-name if the model
picks the wrong tool. Weights are applied at train time, so tokenized data is weight-agnostic.
Each JSONL line has query, tools, answers (tools/answers are JSON-encoded strings):
{"query": "What's the weather in Paris?", "tools": "[{\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"location\":{\"type\":\"string\",\"required\":true}}}]", "answers": "[{\"name\":\"get_weather\",\"arguments\":{\"location\":\"Paris\"}}]"}Provide ≥120 examples per tool (100 train / 10 val / 10 test). Vary phrasing and include multi-tool examples; too few will overfit.
The rasaboun/needle-transit-onnx artifacts were produced with the
onnx-community/needle-onnx port-and-copy
pipeline (JAX → PyTorch → ONNX), parametric on the checkpoint. See that repo's PORTING.md.
needle playground Test and finetune via web UI
needle finetune <data.jsonl> Finetune on your own data (tunable LR / loss weights)
needle run --query "..." --tools Single inference
needle train Full training run
needle eval --checkpoint <path> Evaluate a checkpoint
needle tpu <action> TPU management (see docs/tpu.md)
MIT. Fine-tune of cactus-compute/needle (© Cactus Compute) — see LICENSE and the base model card for architecture and pretraining details.
@misc{ndubuaku2026needle,
title={Needle},
author={Henry Ndubuaku and Jakub Mroz and Karen Mosoyan and Roman Shemet and Parkirat Sandhu and Satyajit Kumar and Noah Cylich and Justin H. Lee},
year={2026},
url={https://github.com/cactus-compute/needle}
}