Skip to content

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

Repository files navigation

Needle Transit

A transit-domain function-calling model: a fine-tune of Cactus-Compute/needle (a 26M-parameter Simple Attention Network) that turns a natural-language transit query into a single tool call — or refuses when none applies. Ships with a finetuning toolkit so you can adapt it to your own tools.

Open weights:

Model Format Use
rasaboun/needle-transit JAX/Flax .pkl Training / Python inference
rasaboun/needle-transit-onnx ONNX In-browser / on-device (onnxruntime-web)

This repo is a fork of cactus-compute/needle; the base architecture, pretraining, and runtime are theirs. What's added here is the transit finetune and the finetuning enhancements below.

What this fork adds

  • Tunable finetuning hyperparameters — --lr, --muon-lr, --warmup-ratio, --decay-ratio exposed on needle train / needle finetune (previously hard-coded).
  • Per-field loss weighting — --w-name, --w-value, --w-key weight the loss on tool-name, argument-value, and argument-key tokens independently. Defaults 2.0 / 4.0 / 1.5 bias the model toward getting argument values right.
  • Metrics logging — set NEEDLE_METRICS_LOG=path.jsonl to stream per-step {step, epoch, text_loss, grad_norm, adam_lr, muon_lr, val_ppl} for offline plotting.
  • Local-checkpoint reuse — finetuning uses a local checkpoint if present instead of always re-downloading from HF.

The model

26M-parameter encoder-decoder Simple Attention Network (d_model=512, enc×12, dec×8, GQA 8H/4KV). An extractor: it lifts slot values verbatim from the query; canonical resolution (station disambiguation, routing) is the backend's job. Handles French, English, and FR/EN code-switching, plus typos and SMS-style compression.

Tools

[
  {"name": "search_itinerary", "parameters": {
      "origin": {"type": "string", "required": true},
      "destination": {"type": "string", "required": true},
      "time_human": {"type": "string", "required": false},
      "time_mode": {"type": "string", "description": "depart_at | arrive_by", "required": false}}},
  {"name": "get_next_arrivals", "parameters": {
      "station": {"type": "string", "required": true},
      "line": {"type": "string", "required": false}}}
]

Examples (verified)

Query Output
Itinéraire de Bastille à Nation [{"name":"search_itinerary","arguments":{"origin":"Bastille","destination":"Nation"}}]
De Issy à Charles-de-Gaulle, départ 14h [{"name":"search_itinerary","arguments":{"origin":"Issy","destination":"Charles-de-Gaulle","time_human":"départ 14h","time_mode":"depart_at"}}]
How do I get from Gare du Nord to La Défense? [{"name":"search_itinerary","arguments":{"origin":"Gare du Nord","destination":"La Défense"}}]
Prochain métro à Bastille ligne 1 ? [{"name":"get_next_arrivals","arguments":{"station":"Bastille","line":"1"}}]
cmt aller a chatelet depuis nation [{"name":"search_itinerary","arguments":{"origin":"nation","destination":"chatelet"}}]
Quel temps fait-il ? []

Quickstart

git clone https://github.com/Rasaboun/needle-transit.git
cd needle-transit && source ./setup
needle playground

Web UI at http://127.0.0.1:7860 to test and finetune on your own tools. Weights auto-download.

Inference (Python)

from needle import SimpleAttentionNetwork, load_checkpoint, generate, get_tokenizer

params, config = load_checkpoint("checkpoints/needle-transit.pkl")
model = SimpleAttentionNetwork(config)

result = generate(
    model, params, get_tokenizer(),
    query="Prochain métro à Bastille ligne 1 ?",
    tools='[{"name":"get_next_arrivals","description":"Next departures at a stop.","parameters":{"station":{"type":"string","required":true},"line":{"type":"string","required":false}}}]',
    stream=False,
)
print(result)
# [{"name":"get_next_arrivals","arguments":{"station":"Bastille","line":"1"}}]

Finetuning

Adapt the model to your own tools/domain:

# Auto-downloads base weights (or reuses a local --checkpoint), trains, evaluates
needle finetune data.jsonl

# Tune the run
needle finetune data.jsonl \
  --lr 3e-5 --muon-lr 0.02 --warmup-ratio 0.05 --decay-ratio 0.05 \
  --w-name 2.0 --w-value 4.0 --w-key 1.5

# Stream training metrics to a JSONL file
NEEDLE_METRICS_LOG=logs/run.jsonl needle finetune data.jsonl

Best checkpoint is saved as checkpoints/needle_finetuned_<id>_best.pkl.

Loss weighting

--w-name / --w-value / --w-key scale the loss on different parts of the tool call. Raise --w-value when argument values are the hard part (the default 4.0 already does); raise --w-name if the model picks the wrong tool. Weights are applied at train time, so tokenized data is weight-agnostic.

Data format

Each JSONL line has query, tools, answers (tools/answers are JSON-encoded strings):

{"query": "What's the weather in Paris?", "tools": "[{\"name\":\"get_weather\",\"description\":\"Get current weather for a city.\",\"parameters\":{\"location\":{\"type\":\"string\",\"required\":true}}}]", "answers": "[{\"name\":\"get_weather\",\"arguments\":{\"location\":\"Paris\"}}]"}

Provide ≥120 examples per tool (100 train / 10 val / 10 test). Vary phrasing and include multi-tool examples; too few will overfit.

Exporting to ONNX

The rasaboun/needle-transit-onnx artifacts were produced with the onnx-community/needle-onnx port-and-copy pipeline (JAX → PyTorch → ONNX), parametric on the checkpoint. See that repo's PORTING.md.

CLI

needle playground                  Test and finetune via web UI
needle finetune <data.jsonl>       Finetune on your own data (tunable LR / loss weights)
needle run --query "..." --tools   Single inference
needle train                       Full training run
needle eval --checkpoint <path>    Evaluate a checkpoint
needle tpu <action>                TPU management (see docs/tpu.md)

Credits & license

MIT. Fine-tune of cactus-compute/needle (© Cactus Compute) — see LICENSE and the base model card for architecture and pretraining details.

@misc{ndubuaku2026needle,
  title={Needle},
  author={Henry Ndubuaku and Jakub Mroz and Karen Mosoyan and Roman Shemet and Parkirat Sandhu and Satyajit Kumar and Noah Cylich and Justin H. Lee},
  year={2026},
  url={https://github.com/cactus-compute/needle}
}

About

No description, website, or topics provided.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages