This is a fork of ollama/ollama

Everything below this banner is upstream's README, describing upstream's work. Ollama is theirs; this fork only adds to it, and tries to add as little as possible.

It tracks upstream releases and carries a narrow set of changes:

• Vision-model correctness. Image token budgets and native-aspect dynamic resolution for gemma4 and nemotron, and an fp32-accumulation gate for qwen2.5-vl that closes an fp16 overflow in the vision tower — a few elements in millions reach inf on CUDA and the caption collapses into a repeated glyph. Offered upstream as ollama#18070.
• A vision regression suite. A preflight harness with recorded per-model expectations, generated (public, no-download) trigger images, and env-gated node-level instrumentation, run against every build before it deploys.
• Fixes carried until upstream takes them, each tracked against an upstream issue or PR, and deleted from here when it lands there — the retirement register lists every carried item, what retires it, and the test that gates its deletion.
• An experimental MLX runtime for Apple Silicon and CUDA — see the caveats below before using it for anything that matters.

Current fold: v0.40.2-dynres (#460): upstream v0.40.2, llama.cpp b11351 with compat patches 001, 002, upstream's 002-clef, 004, 005, 801, 802, 805, 903 and 908 (805 and 903 re-cut for b11351), MLX a59cc231. Its point tag v0.40.2-dynres.1 (39602449f) adds the fixes since: 903 becomes llama.cpp#29953 plus the y-tile guard (#462), the muse-glimmer fixes (#464–#468), the JSON grammar's whitespace (#470), an explicit num_ctx on MLX (#476) and nemotron3's converted vision budget (#483). Its one native change, 903, is in ggml-cuda, so on Metal it moves Go only. main moves ahead of the tags between folds; a tag is the fixed point to build and roll back to.

Deployed (versions read from each server's /api/version on 2026-10-13):

host	build	since	record
Apple Silicon, mlx-metal (:11435)	0.40.2-dynres.1-17-g75c6d35, the same build as the CUDA host's: main 75c6d350, a Go-only build on the point tag's payload	2026-10-14 10:13	#486, this record
CUDA host, ai-server (:11497)	0.40.2-dynres.1-17-g75c6d35: the v0.40.3 fold (#488) on the point tag plus #487, a Go-only build of main 75c6d350 (image main-75c6d350)	2026-10-14 07:53	#487, #488, #492
AMD gfx1151 (:11434)	0.40.2-dynres.1-6-g9f3baaf: the v0.40.3 fold (#488), a build of 9f3baafce. Its Go equals main's, and it pauses v0.40.2's store migration. The point tag's build served 12:47–13:51. Preflight passes rocm7-0-40-2 on production	2026-10-14 13:51	#485, the 2026-10-14 decision
H100 (CUDA)	0.35.0-dynres-15-ga657392, a build of main past the v0.35.0 tag	2026-10-03	v0.35.0 record

The CUDA host serves #487, merged after the point tag: without it, an MLX runner that can see the host's second GPU fails requests with cudaMemcpy … invalid argument. The hosts deployed since v0.34.4 run the two-pass think+format flow (OLLAMA_FORMAT_TWO_PASS=1, ADR 0045) and set OLLAMA_KV_CACHE_TYPE=f16 explicitly (#387). The ai-server CUDA container and the Apple Silicon launchd agent also keep OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0 (ADR 0033). The Apple Silicon agent still sets all three on 0.40.2-dynres.1-17-g75c6d35, checked after the swap. The H100's settings are in its own record.

On Apple Silicon, 0.40.2-dynres.1-17-g75c6d35 replaced the point tag's build on 2026-10-14, binary only: its Go is identical to #488's head, which passed the MLX tests and both Metal profiles, and it passed both profiles and the deploy smoke again on the stage and on production. Before that, the point tag replaced 0.40.2-dynres-0-g5623192. Before the swap, its build passed preflight, the MLX tests with vision golden parity, and the two-pass, converted-GGUF and vision smoke, on a :11437 stage serving an APFS clone of production's store; preflight passed again on production after it. The v0.40.2 swap before it also checked every GGUF model in the store, converted where v0.40.2 converts it, and a rollback to the outgoing build (fold record). Every build's binary and payload are archived for rollback (BINARIES.md).

Fork builds are stamped <upstream-version>-dynres-<n>-g<sha>; dynres names the change that started the fork, not the company that runs it. Fork-specific documentation, ADRs and measurements live in docs/maxusai/.

What tested green for this fold

surface	Build identity	MLX build pin	Toolchain pin	M5 tensor path	Image size ladder	Aspect ladder	Pinned image budget	thinking on/off	Output quality	fp16 overflow canary	Runner isolation	measured on
cuda	green	green	skipped	n/a	green	skipped	green	green	not run	skipped	green	0.40.2-dynres.1-17-g75c6d35
mlx-cuda	skipped	green	skipped	n/a	green	skipped	skipped	skipped	not run	skipped	green	0.40.2-dynres.1-17-g75c6d35
mlx-metal	skipped	green	skipped	green	green	skipped	skipped	green	skipped	skipped	green	0.40.2-dynres.1-17-g75c6d35
apple-silicon-mlx	not run	not run	not run	not run	not run	not run	not run	not run	not run	not run	not run	—
rocm7	green	skipped	green	n/a	green	green	green	green	green	green	green	0.40.2-dynres.1-6-g9f3baaf
rocm10	not run	not run	not run	not run	not run	not run	not run	not run	not run	not run	not run	—
cpu	not run	not run	not run	not run	not run	not run	not run	not run	not run	not run	not run	—

Generated by release_matrix.py from recorded preflight runs. A surface with no run for this release reads not run — absence is shown, never assumed green. A group is reported at its weakest check, so one skipped probe does not read as a pass.

cuda, mlx-metal and rocm7 here are the deployed serving surfaces, each measured on production itself. apple-silicon-mlx is the deprecated alias for mlx-metal and will disappear from the generator with it.

What differs from upstream, concretely

Measured against upstream ollama v0.40.2 at llama.cpp b11351. Every row is a capability the fork has and upstream does not; the record column is where the decision and its measurements live (docs/maxusai/). The retirement register tracks when each one can go.

Vision correctness on the llama.cpp path — the deployed engine

	upstream ollama	this fork	record
nemotron-3 vision	fixed 512×512 canvas — 256 tokens per image, whatever the aspect ratio. Upstream's MLX nemotron_h has been native-aspect since v0.34.3	native-aspect dynamic resolution, 256–3,328 tokens, position embeddings interpolated to the patch grid in-graph	patch 002, ADR 0001
gemma4 image budget	default limits 70–1,120 tokens (40–280 before b10864); an under-budget image keeps its natural rounded grid and is letterbox-padded	every image scaled to fill the requested budget and snapped to gemma4's supported ladder (70/140/280/560/1120), never padded — off-ladder grids measurably break box_2d vertical grounding. The budget is a per-request option (image_min_tokens/image_max_tokens, defaults 70/1120) and the scheduler reloads when the resolved flags change. Upstream has since adopted the same default limits; the fill is still fork-only	patch 004, ADR 0003/0008/0016
qwen2.5-vl on CUDA	f16 vision matmuls accumulate in fp16; on some ordinary images a few elements of millions reach inf at v.blk.31.ffn_down and the caption collapses into one repeated glyph	fp32 accumulation forced for every Qwen-VL runner (qwen2vl and qwen25vl, one family under two converter spellings), keyed on the GGUF architecture. Offered upstream as ollama#18070, still open	llm/llama_server.go
MoE + MMQ on CUDA	MMQ reads its buffers in whole tiles, but at b11351 pads them from the batch (get_J_max(ne11)), which rounds down while the launched tile rounds up; a broadcast ne11 == 1 gets no padding at all. Upstream's fix, llama.cpp#29953, merged on 2026-10-08, after b11351, so it is not in v0.40.2	#29953 itself, backported to b11351, plus a compile-time y-tile guard over every config table (#462, verified on CUDA and ROCm). The backport retires at the first pin that includes #29953; the guard stays after that	patch 903, the #29953 record
gemma4 flash attention on CUDA	b11081 retunes the MMA configs and tile sizes for head dims 256/512 (ce8caa6e6). Its accuracy is unchanged in llama.cpp's test-backend-ops, but the rounding moves. With ollama's Q4_K_M, gemma4:26b leaves 6 of 27 think-on cases in loops that never end, against 1 without it; with ggml-org's Q4_0, a case loops the other way	the device half of ce8caa6e6 is reverted to b10969's tiling, which keeps the numerics production was gated on; the host half, the decode selection, stays upstream's. The revert shows no speed cost: in the paired runs on CUDA, gemma4:31b decoded 57 tok/s with either tiling, and gemma4:26b 176 tok/s against the new tiling's 137. On gfx1151 the patch changes no kernel	patch 908, the v0.34.4 fold record

Structured output and generation control

	upstream ollama	this fork	record
think + format in one request	since v0.34.4, one pass: the grammar applies from the first token, after free thinking that ends at the parser's closing strings	the same by default. OLLAMA_FORMAT_TWO_PASS=1 keeps the fork's two-pass flow: pass one thinks without the grammar and stops at the think-close marker, and pass two answers under it. Production runs it, because on MLX it lets the thinking draft	ADR 0045 (0002/0004/0010)
format on muse-glimmer	the grammar binds from the first token, before the model has written its message header. With thinking off, GGUF answers `{…}<	eot	>` and MLX can write the header into the JSON; ollama#18687, open, fixes the parser half
whitespace in structured output (MLX)	the JSON grammar allows an unlimited whitespace run between tokens, so a stalled decode can fill num_predict with indentation and never close the answer	runs bounded at 22 characters per separator (max_whitespace_cnt on the json_schema element), the length of llama.cpp's longest run for a JSON schema but not its shape (any 22 whitespace characters); GGUF's format: "json" grammar uses llama.cpp's own rule, one per gap	ADR 0035 (proposed; amended 2026-10-12)
drafting under a grammar (MLX)	always on	on by default to match upstream; OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0 restores the gate, and production sets it	ADR 0033
stop sequences (MLX)	not honoured by the MLX runner	honoured, with a possible stop prefix held back until it matches or the stream ends	mlxrunner/stopper.go
KV cache type	one global OLLAMA_KV_CACHE_TYPE	per model, with K/V pair syntax; production runs f16 on every platform, and a quantized cache is per model or per request	ADR 0005, ADR 0043
llama-server's prompt cache	no --cache-ram, so llama.cpp's 8192 MiB host-RAM cache is on for every GGUF model	the same by default. prompt_cache_ram, per model or per request, sets --cache-ram, and 0 turns it off: up to a fifth less time per image request on gemma4:31b, with the same answers. The vision harness sends 0	ADR 0047

What production's choices cost, against stock v0.34.4

• Two-pass think+format (OLLAMA_FORMAT_TWO_PASS=1, which production sets):
  • A request prefills twice: pass two re-submits pass one's tokens.
  • num_predict bounds each pass, so the total can exceed it (8,290 against 8,192, measured).
  • It also handles a tool call after the thinking, and an end of turn inside it, differently. ADR 0045 tabulates both flows.
  • On MLX the thinking runs as fast as a single pass that drafts under the grammar, as stock's does (level within the spread, measured on the fork's build with drafting on). Production turns drafting under a grammar off (OLLAMA_MLX_DRAFT_UNDER_GRAMMAR=0), because it retains memory on the qwen3.5 family, and drafts in pass one, which has no grammar.
  • On GGUF neither flow drafts. The two flows leave the same cases unfinished, on all five models measured on gfx1151.
• The MLX runtime is slower than the llama.cpp engine on CUDA; see below.
• Patch 908 costs no speed (the table above), and an f16 KV cache is stock's default too.

Serving and scheduling

	upstream ollama	this fork	record
MLX admission	weights against free device memory (and, since v0.34.1, a system-memory bound on integrated GPUs)	weights + KV priced at the requested num_ctx + a per-architecture headroom; an explicit rung that does not fit is refused, an automatic one is clamped. A config.json the estimator cannot parse is named in the warning, where muse-glimmer's was once admitted on weights alone	ADR 0034, #466
MLX memory ceiling	none	OLLAMA_MLX_MEMORY_LIMIT and a cache limit, set per runner from the admitted budget	runner knobs
nvfp4 global scales	stored in MLX's m × 2688 form and divided back out wherever a wrapper applies the scale itself (globalScaleFactor, ollama#18550). That is not the identity in float32: 17 of gemma4:31b's 191 vision scales move one ulp	stored as the checkpoint's own multiplier m, with upstream's helpers defined in those terms, so every call site is exact	ADR 0039
model identity in a record	a tag, which since v0.40.0 can hold one manifest per runner, chosen by host: mlx first on Apple Silicon, llamacpp first elsewhere	the manifest digest: the library re-published gemma4:*-nvfp4 with bf16 vision towers under unchanged tags and config blobs, and one tag can now resolve to different weights on different hosts	ADR 0038 (proposed)
gemma4 image chunk vs. generation batch (GGUF)	the batch follows num_ctx (1024 above 4096). Since llama.cpp#29773, in b11351, llama-server caps a non-causal model's image_max_tokens to the ubatch, so a top-rung gemma4 image (1120 tokens) is shrunk to 1024 rather than split	a gemma4 vision runner starts from the batch rung that holds its image ceiling (2048 at 1120) and steps down only when it does not fit, so the image keeps its full budget and the cap never applies	ADR 0036
gemma4 on MLX	upstream's own vision and audio tower. Since v0.34.4 it picks each image's budget from the 70/140/280/560/1120 ladder, closest to the input resolution, with no API parameter (ollama#18603); v0.40.0 builds embeddinggemma-2 on the same tower	vision through the fork's pipeline on upstream's MediaModel, with the GGUF path's per-request budget (image_min_tokens/image_max_tokens, defaults 70/1120), filled and snapped to the ladder; audio not shipped, and embeddinggemma-2 not taken (below)	ADR 0021 (0003/0008)
transparent images (gemma4 on MLX)	alpha dropped by RGB conversion before the resize, so the colour stored under a transparent pixel shows	composited over white before the resize, as mlx-vlm's convert_to_rgb does	ADR 0015
media prompts on MLX	chunk boundaries extended around non-causal spans (extendChunk, since v0.34.1)	prefill chunks span-aligned around image blocks, the opening chunk included, with bidirectional expansions matched all-or-nothing; a late image is refused	ADR 0014
scheduler	—	log sites never drop fields under contention; head-of-line and evict-all-wait fixes; attached media charged against capabilities before the load; capability advertising corrected for MLX architectures	server/sched.go, images.go
panic hygiene (MLX)	—	a cleanup that fails while a request is unwinding never replaces the panic that caused it	mlxrunner/unwind.go

Not taken from upstream

• embeddinggemma-2, upstream's multimodal embedding model on MLX (v0.40.0, ollama#18820). It is built on upstream's gemma4 image and audio code, which the fork replaced with its own pipeline. It is deferred with the question of that pipeline (fold record). On the fork, the model fails to load as an unknown architecture.
• gemma4 audio on MLX, not shipped since the fork took over gemma4's media path (ADR 0021).

Measurement — nothing comparable upstream

	this fork	record
vision regression suite	preflight with versioned per-model expectations, generated (public, no-download) trigger images, an env-gated node-level meter, and a generated release matrix — run before every deploy	ADR 0011/0012, patch 801
campaigns	five report templates rendered only by generators; per-request memory and drafting analysis; bbox conformance scoped to image geometry	ADR 0012/0028/0030
bounding-box protocol	requests pin norm-1000 and carry a self-calibrating anchor, so the model's internal resize cannot contaminate coordinates: 111 of 112 cells convert cleanly across 14 geometries × 4 models × 2 think modes. A protocol and its measurements, not a runtime change	ADR 0027/0030
fork identity	builds stamped <upstream>-dynres-<n>-g<sha>, a tag per fold, release notes carrying the generated matrix	ADR 0032

Request options, and what each does to the runner

Set per request (options on /api/chat and /api/generate) or per model (a Modelfile PARAMETER, or the params a tag ships); the request wins. On the llama.cpp runner each is a launch flag, so a value that changes how llama-server would start reloads the model. The MLX scheduler reloads only on num_ctx.

option	from	llama.cpp (GGUF)	MLX	reloads
num_ctx	upstream	-c	the KV the admission prices	yes, both
image_min_tokens, image_max_tokens	fork	--image-min-tokens / --image-max-tokens for gemma4 (filled and snapped to its 70–1120 ladder) and nemotron_h_omni; the qwen VL family always gets a minimum of 1024; every other arch ignores them, muse-glimmer included (its projector caps an image at 4096)	per request: gemma4, qwen3.5 and nemotron_h honour both; muse-glimmer honours the maximum only (unset: 4096, its projector's cap). An explicit 70 or 1120 is honoured like any other value (ADR 0048)	GGUF: when the resolved flags change
kv_cache_type	fork	--cache-type-k / --cache-type-v; a K/V pair sets them apart	not read	GGUF: yes
prompt_cache_ram	fork	--cache-ram in MiB; 0 turns llama.cpp's 8192 MiB host-RAM prompt cache off	not read	GGUF: yes
draft_num_predict	upstream	--spec-draft-n-max when the model has a drafter (a separate draft defaults to 4; muse-glimmer's -dflash tags ship 3); 0 turns drafting off	per request, without a reload: 0 turns drafting off and N caps the depth the runner adapts; unset drafts whenever the model carries a drafter	GGUF: yes

Server-wide, with no per-request form: OLLAMA_FORMAT_TWO_PASS, OLLAMA_MLX_DRAFT_UNDER_GRAMMAR, OLLAMA_KV_CACHE_TYPE (the default kv_cache_type) and OLLAMA_MLX_MEMORY_LIMIT. Production's values are in the banner above.

MLX runtime — experimental, and slower on CUDA

It works. Models load, stay resident and generate correct output on both Metal and CUDA. But it is not the path to reach for by default:

• On CUDA it runs at 34–75% of the cuda path's decode throughput, median 46%. Measured across four matched model pairs on one host, one server process — full report. "Roughly half" is a fair central estimate and a poor description of any single case: the spread is 2.2× and it is not architectural (the two dense pairs sit at 75% and 53%, the two MoE at 34% and 39%). Two of the four mlx-cuda arms had no stable throughput to quote at all. The report is dated: it measured 0.33.2-dynres-5-g2b95b4a on 2026-08-30. The MLX pin has moved since then, to a59cc231 in v0.40.2, and the matched comparison has not been re-run.

• Its bigger cost is variance, not speed.

  • Throughput. mlx-cuda's per-request spread is ~5× the cuda path's and reaches 46% within a single arm — same host, same prompt, back to back — while every cuda arm held inside ±1.5% first time. It is per-model: gemma4:31b-nvfp4 reproduced to 1.1% across four independent measurements.
  • Output. At temperature 0 a drafted request takes its own path every time. Even undrafted, each cold load takes its own greedy trajectory. The same gemma4:26b request finished 1 to 3 times in 5 cold loads, depending on the prompt. On the cuda path, the same request repeats byte for byte (v0.34.4 fold record: the drafting probe, and open item 8).

  For anything that sets a timeout or compares two builds, this matters more than the ratio. Compare MLX builds with repeats, never single runs.

• Structured output with thinking depends on drafting. Such a request thinks 1.5–1.7× faster when its thinking drafts. Production runs the two-pass flow, which lets it draft (ADR 0045).

• Two traps, both measured:

  • Pin num_ctx. Unset, the default is derived from total VRAM: 262144 on the CUDA host's 105.5 GiB. That collapses decode 25×, to 1.48 tok/s against 37.97 at 8192 (record).
  • Persist the kernel cache. Without MLX_PTX_CACHE_DIR, every fresh container recompiles MLX's JIT kernels, so its first request takes 10–15 minutes. Keep one cache directory per GPU architecture (vision-suite README).

• On Metal it is the other way round. A matched campaign measured MLX ~2.4× faster than llama-server (gemma4 12b: 121 vs 50 tok/s decode; 2026-08-08, single samples, directional only). So the CUDA gap is CUDA-specific, not an MLX property — do not generalise either number to the other platform.

• Engine and quantization move together in every figure above — nvfp4 on MLX against q4_K_M on GGUF, because those are the artefacts that exist. So the throughput numbers describe the two stacks as shipped, not the engine in isolation, and the same confound makes MLX-vs-GGUF quality an uncontrolled comparison: a quality difference cannot be attributed to the engine either. Nobody has separated them; until someone does, treat "which is better" as open.

• It is converging with upstream's own MLX work, and is expected to be superseded by it.

  • The fork has retired its constrained-sampling layer in favour of upstream's engine (ADR 0033).
  • Since v0.34.4 it follows upstream's single-pass structured outputs by default (ADR 0045).
  • What remains fork-only on MLX is in the tables above.

Use GGML/llama-server for anything where throughput or comparability matters.

Every row in the tables above is a delta we would rather not have. Each is offered upstream where it is upstream's to take, and deleted from here once it lands there. The Qwen-VL accumulation gate (ollama#18070) is filed and pending. Upstream closed the MMQ padding fix (llama.cpp#27044) for its narrower #29941, which still reads past the buffer below 128 tokens. 903 now carries upstream's later fix, #29953, backported to b11351 with a y-tile guard (#462). The gemma4 tiling revert (908) is not upstream's to take: ce8caa6e6 loses no precision in llama.cpp's test-backend-ops, so the revert is the fork's own choice to keep production's gated numerics.

---

Ollama

Start building with open models.

Download

macOS

curl -fsSL https://ollama.com/install.sh | sh

or download manually

Windows

irm https://ollama.com/install.ps1 | iex

or download manually

Linux

curl -fsSL https://ollama.com/install.sh | sh

Manual install instructions

Docker

The official Ollama Docker image ollama/ollama is available on Docker Hub.

Libraries

• ollama-python
• ollama-js

Community

• Discord
• 𝕏 (Twitter)
• Reddit

Get started

ollama

You'll be prompted to run a model or connect Ollama to your existing agents or applications such as Claude Code, OpenClaw, OpenCode , Codex, Copilot, and more.

Coding

To launch a specific integration:

ollama launch claude

Supported integrations include Claude Code, Codex, Copilot CLI, DeepSeek Harness, Droid, and OpenCode.

AI assistant

Use OpenClaw to turn Ollama into a personal AI assistant across WhatsApp, Telegram, Slack, Discord, and more:

ollama launch openclaw

Chat with a model

Run and chat with Gemma 4:

ollama run gemma4

See ollama.com/library for the full list.

See the quickstart guide for more details.

REST API

Ollama has a REST API for running and managing models.

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'

See the API documentation for all endpoints.

Python

pip install ollama

from ollama import chat

response = chat(model='gemma4', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response.message.content)

JavaScript

npm i ollama

import ollama from "ollama";

const response = await ollama.chat({
  model: "gemma4",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);

Supported backends

• llama.cpp project founded by Georgi Gerganov.

Documentation

• CLI reference
• REST API reference
• Importing models
• Modelfile reference
• Building from source

Community Integrations

Want to add your project? Open a pull request.

Chat Interfaces

Web

• Open WebUI - Extensible, self-hosted AI interface
• Onyx - Connected AI workspace
• LibreChat - Enhanced ChatGPT clone with multi-provider support
• Lobe Chat - Modern chat framework with plugin ecosystem (docs)
• NextChat - Cross-platform ChatGPT UI (docs)
• Perplexica - AI-powered search engine, open-source Perplexity alternative
• big-AGI - AI suite for professionals
• Lollms WebUI - Multi-model web interface
• ChatOllama - Chatbot with knowledge bases
• Bionic GPT - On-premise AI platform
• Chatbot UI - ChatGPT-style web interface
• Hollama - Minimal web interface
• Chatbox - Desktop and web AI client
• chat - Chat web app for teams
• Ollama RAG Chatbot - Chat with multiple PDFs using RAG
• Tkinter-based client - Python desktop client

Desktop

• Dify.AI - LLM app development platform
• AnythingLLM - All-in-one AI app for Mac, Windows, and Linux
• Maid - Cross-platform mobile and desktop client
• Witsy - AI desktop app for Mac, Windows, and Linux
• Cherry Studio - Multi-provider desktop client
• Ollama App - Multi-platform client for desktop and mobile
• PyGPT - AI desktop assistant for Linux, Windows, and Mac
• Alpaca - GTK4 client for Linux and macOS
• SwiftChat - Cross-platform including iOS, Android, and Apple Vision Pro
• Enchanted - Native macOS and iOS client
• RWKV-Runner - Multi-model desktop runner
• Ollama Grid Search - Evaluate and compare models
• macai - macOS client for Ollama and ChatGPT
• AI Studio - Multi-provider desktop IDE
• Reins - Parameter tuning and reasoning model support
• ConfiChat - Privacy-focused with optional encryption
• LLocal.in - Electron desktop client
• MindMac - AI chat client for Mac
• Msty - Multi-model desktop client
• BoltAI for Mac - AI chat client for Mac
• IntelliBar - AI-powered assistant for macOS
• Kerlig AI - AI writing assistant for macOS
• Hillnote - Markdown-first AI workspace
• Perfect Memory AI - Productivity AI personalized by screen and meeting history

Mobile

• Ollama Android Chat - One-click Ollama on Android

SwiftChat, Enchanted, Maid, Ollama App, Reins, and ConfiChat listed above also support mobile platforms.

Code Editors & Development

• Cline - VS Code extension for multi-file/whole-repo coding
• Continue - Open-source AI code assistant for any IDE
• Void - Open source AI code editor, Cursor alternative
• Copilot for Obsidian - AI assistant for Obsidian
• twinny - Copilot and Copilot chat alternative
• gptel Emacs client - LLM client for Emacs
• Ollama Copilot - Use Ollama as GitHub Copilot
• Obsidian Local GPT - Local AI for Obsidian
• Ellama Emacs client - LLM tool for Emacs
• orbiton - Config-free text editor with Ollama tab completion
• AI ST Completion - Sublime Text 4 AI assistant
• VT Code - Rust-based terminal coding agent with Tree-sitter
• QodeAssist - AI coding assistant for Qt Creator
• AI Toolkit for VS Code - Microsoft-official VS Code extension
• Open Interpreter - Natural language interface for computers
• oxi - Native Rust desktop coding agent with editor, Git panel and terminal

Libraries & SDKs

• LiteLLM - Unified API for 100+ LLM providers
• Semantic Kernel - Microsoft AI orchestration SDK
• LangChain4j - Java LangChain (example)
• LangChainGo - Go LangChain (example)
• Spring AI - Spring framework AI support (docs)
• LangChain and LangChain.js with example
• Ollama for Ruby - Ruby LLM library
• any-llm - Unified LLM interface by Mozilla
• OllamaSharp for .NET - .NET SDK
• LangChainRust - Rust LangChain (example)
• Agents-Flex for Java - Java agent framework (example)
• Elixir LangChain - Elixir LangChain
• Ollama-rs for Rust - Rust SDK
• LangChain for .NET - .NET LangChain (example)
• chromem-go - Go vector database with Ollama embeddings (example)
• LangChainDart - Dart LangChain
• LlmTornado - Unified C# interface for multiple inference APIs
• Ollama4j for Java - Java SDK
• Ollama for Laravel - Laravel integration
• Ollama for Swift - Swift SDK
• LlamaIndex and LlamaIndexTS - Data framework for LLM apps
• Haystack - AI pipeline framework
• Firebase Genkit - Google AI framework
• Ollama-hpp for C++ - C++ SDK
• PromptingTools.jl - Julia LLM toolkit (example)
• Ollama for R - rollama - R SDK
• Portkey - AI gateway
• Testcontainers - Container-based testing
• LLPhant - PHP AI framework

Frameworks & Agents

• AutoGPT - Autonomous AI agent platform
• crewAI - Multi-agent orchestration framework
• Strands Agents - Model-driven agent building by AWS
• Cheshire Cat - AI assistant framework
• any-agent - Unified agent framework interface by Mozilla
• Stakpak - Open source DevOps agent
• Hexabot - Conversational AI builder
• Neuro SAN - Multi-agent orchestration (docs)

RAG & Knowledge Bases

• RAGFlow - RAG engine based on deep document understanding
• R2R - Open-source RAG engine
• MaxKB - Ready-to-use RAG chatbot
• Minima - On-premises or fully local RAG
• Chipper - AI interface with Haystack RAG
• ARGO - RAG and deep research on Mac/Windows/Linux
• Archyve - RAG-enabling document library
• Casibase - AI knowledge base with RAG and SSO
• BrainSoup - Native client with RAG and multi-agent automation

Bots & Messaging

• LangBot - Multi-platform messaging bots with agents and RAG
• AstrBot - Multi-platform chatbot with RAG and plugins
• Discord-Ollama Chat Bot - TypeScript Discord bot
• Ollama Telegram Bot - Telegram bot
• LLM Telegram Bot - Telegram bot for roleplay

Terminal & CLI

• aichat - All-in-one LLM CLI with Shell Assistant, RAG, and AI tools
• oterm - Terminal client for Ollama
• gollama - Go-based model manager for Ollama
• tlm - Local shell copilot
• tenere - TUI for LLMs
• ParLlama - TUI for Ollama
• llm-ollama - Plugin for Datasette's LLM CLI
• ShellOracle - Shell command suggestions
• LLM-X - Progressive web app for LLMs
• cmdh - Natural language to shell commands
• VT - Minimal multimodal AI chat app

Productivity & Apps

• AppFlowy - AI collaborative workspace, self-hostable Notion alternative
• Screenpipe - 24/7 screen and mic recording with AI-powered search
• Vibe - Transcribe and analyze meetings
• Page Assist - Chrome extension for AI-powered browsing
• NativeMind - Private, on-device browser AI assistant
• Ollama Fortress - Security proxy for Ollama
• 1Panel - Web-based Linux server management
• Writeopia - Text editor with Ollama integration
• QA-Pilot - GitHub code repository understanding
• Raycast extension - Ollama in Raycast
• Painting Droid - Painting app with AI integrations
• Serene Pub - AI roleplaying app
• Mayan EDMS - Document management with Ollama workflows
• TagSpaces - File management with AI tagging

Observability & Monitoring

• Opik - Debug, evaluate, and monitor LLM applications
• OpenLIT - OpenTelemetry-native monitoring for Ollama and GPUs
• Lunary - LLM observability with analytics and PII masking
• Langfuse - Open source LLM observability
• HoneyHive - AI observability and evaluation for agents
• MLflow Tracing - Open source LLM observability

Database & Embeddings

• pgai - PostgreSQL as a vector database (guide)
• MindsDB - Connect Ollama with 200+ data platforms
• chromem-go - Embeddable vector database for Go (example)
• Kangaroo - AI-powered SQL client

Infrastructure & Deployment

Cloud

• Google Cloud
• Fly.io
• Koyeb
• Harbor - Containerized LLM toolkit with Ollama as default backend

Package Managers

• Pacman
• Homebrew
• Nix package
• Helm Chart
• Gentoo
• Flox
• Guix channel