2.1.0-GA merge back to main

CHANGELOG.md

@@ -24,6 +24,82 @@ recompiling. A breaking change waits for 2.0. (Before RC.1, `1.0.0-beta.N` and e
 beta.4 → RC.1 changes that can break a build are listed under *Changed — breaking (pre-GA)* in the
 RC.1 entry, and each entry's own "Beta caveats" apply to the betas.)
 
+## 2.1.0 — GA — 2026-10-15
+
+Fixes and a newer MLX runtime; no API change (the public surface is identical to 2.0.0's).
+
+### Fixed
+
+- **An app embedding `LocalLMLabSDKInference` was rejected by App Store Connect** (error 90277: not a
+  valid bundle identifier). The framework's bundle ID was `mlx-swift_Cmlx`, and bundle IDs can't contain
+  underscores. It is now `lab.locallm.sdk.inference`, like Core's `lab.locallm.sdk.core`. MLX still finds
+  its Metal library: it loads `Resources/default.metallib` next to the framework's own binary, in a
+  `swift build` executable and in an app Xcode embeds it in alike. An app that copied
+  `mlx-swift_Cmlx.bundle` into its own Resources to work around this can stop doing so.
+- **Downloading a model whose Hugging Face repo has a subfolder failed** within seconds with
+  "Could not fetch metadata for 'optiq'": the downloader treated the folder itself as a file. This hit
+  every mlx-community OptiQ conversion (e.g. `gemma-4-12B-it-OptiQ-4bit`). Folders are now skipped and
+  the files inside them are downloaded, so the snapshot matches the repo.
+- **More MLX models can use tools.** Before each render, the SDK now fits the conversation to what
+  the model's chat template can actually show. It decides from the template and the vocabulary, not
+  from the model's name.
+  - **GLM-4-0414** (`GLM-4-9B-0414`, `GLM-4-32B-0414`) never saw a tool's result. Its template shows
+    results only under the `observation` role. The model kept repeating the same call (`echo` 3–4
+    times, `hub_repo_details` 4–5 times) and then answered without the result. Results now reach it,
+    and it answers from them.
+  - **Llama 3.1** failed a turn that made more than one tool call, with `Jinja.TemplateException`.
+    Its template allows only one call per assistant message. Such a turn is now replayed one call at a
+    time, each followed by its result. Llama 3.2 had the same problem.
+  - **Mistral-Small-3.2-24B-Instruct-2506** (mlx-community 4-bit and 8-bit) never called a tool. Its
+    conversions ship no chat template that renders tools, so when tools are requested the SDK now
+    uses a built-in one in Mistral's own format. The 4-bit passes all four of our tool tests; the
+    8-bit passes three and partly answers the fourth.
+  - When a model ships two chat templates and tools are requested, the one that renders tools is
+    used. Before, `chat_template.jinja` always won, even when only `tokenizer_config.json`'s template
+    supported tools.
+- **Adding a model could hang in its capability check.** `MLXModelProvider.capabilityProbe` now caps
+  each check's output length and stops any check that runs over 3 minutes. A check that times out is
+  listed in the report's `notes` and doesn't set `blocked`, so the model still gets added. Before,
+  adding `Mistral-Small-3.2-24B-Instruct-2506-8bit` was still generating after 30 minutes.
+
+### Changed
+
+- **`LocalLMLabSDKInference`'s MLX runtime moves to current `mlx-swift-lm`.** The pin goes from
+  `45693f6` (2026-07-27) to `30d755c`: upstream `8ec6a2c` (2026-10-15) plus one fix of ours, below.
+  `mlx-swift` goes from 0.31.6 to 0.32.3. It's still served from our mirror fork,
+  `ancientcomputing/mlx-swift-lm`. The update fixes several tool-call problems:
+  - **Gemma 4** keeps array and object arguments intact. Before, a call such as `hub_repo_details`
+    with a `repo_ids` list reached the tool garbled with `<|"|>` tokens.
+  - **gpt-oss** tool calls work. The runtime now parses OpenAI's Harmony format; before, the call
+    came back as raw text.
+  - **Muse Glimmer** (`muse_glimmer`, with its ATEM tool-call format) loads and calls tools. Checked
+    with `mlx-community/Muse-Glimmer-30B-4bit`.
+  - **Older Mistral checkpoints** (`model_type` `mistral`) now get Mistral's tool-call parser, chosen
+    from their chat template. Mistral 7B v0.3 still can't call tools: it writes the older
+    `[TOOL_CALLS] [{…}]` list form, and the parser rejects it.
+  - **Qwen3 calls that open a `<think>` block run.** Qwen3 sometimes opens `<think>` and goes straight
+    to the call without closing the block. Upstream treats everything in a reasoning block as inert,
+    so the call came back as text and never ran. A `<think>` block whose first content is a tool call
+    and that never closes now dispatches the call. A call after any reasoning text, or inside a closed
+    block, stays inert.
+- **A malformed tool call now fails the turn instead of showing up as text.** This covers an
+  unparseable payload, a call to a tool the session doesn't declare, and arguments that break the
+  tool's schema. The error is `LocalLMLabError.generation(underlying:)`, and its message names the reason (for
+  example `malformed_syntax`). The raw model output is never included. Before, the raw call markup came back as the model's answer, and nothing ran.
+  Generation finishes before the error is thrown, so nothing keeps running on the GPU after the turn
+  fails (a command-line tool that exited on the error could otherwise crash).
+
+### Checksums (SHA-256)
+
+```
+LocalLMLabSDKCore-2.1.0-GA.xcframework.zip       8d541e06f81d1586c959d6159a259c759853c88b2c68385ab28de493af6c5776
+LocalLMLabSDKClaude-2.1.0-GA.xcframework.zip     f54472c25063f7c6d6d5a49a91992beb1300d9b6d986ee83d41e682ac2a2c172
+LocalLMLabSDKInference-2.1.0-GA.xcframework.zip  77d3d1a1bae9df6de5955dfa945242c177cb129b4896c95befb81db6c923656f
+LocalLMLabSDKRemote-2.1.0-GA.xcframework.zip     347257172a6da7104b7081602e9dbfc853ed5d038b5e30831aad8dfa2753ae92
+```
+
+---
+
 ## 2.0.0 — GA — 2026-10-07 (binaries re-published 2026-10-08 and 2026-10-09)
 
 **A one-time breaking release.** From `2.0.0`, every 2.x release is source compatible with it

Components/Package.swift

@@ -24,17 +24,17 @@ struct SDKRelease {
 // or edit `defaultSDKVersion`. For a release not listed, add its entry (URL follows the pattern
 // below; checksum is the `.sha256` next to the zip on that GitHub release). Keep this in step
 // with `examples/model-switch/Package.swift`, which consumes Core through this package.
-let defaultSDKVersion = "2.0.0-GA"
+let defaultSDKVersion = "2.1.0-GA"
 
 let knownSDKReleases: [String: SDKRelease] = [
-    "1.0.0-GA": SDKRelease(
-        url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip",
-        checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e"
-    ),
     "2.0.0-GA": SDKRelease(
         url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip",
         checksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b"
     ),
+    "2.1.0-GA": SDKRelease(
+        url: "https://github.com/ancientcomputing/locallm/releases/download/v2.1.0-GA/LocalLMLabSDKCore-2.1.0-GA.xcframework.zip",
+        checksum: "8d541e06f81d1586c959d6159a259c759853c88b2c68385ab28de493af6c5776"
+    ),
 ]
 
 func failManifest(_ message: String) -> Never {

README.md

@@ -30,12 +30,13 @@ It's a single Dock app with five panels:
 It began local-only — hence the name — and 1.0 added the hosted providers behind
 the same interface, so "local" is now the default rather than the only option.
 On macOS 26 only Apple's on-device model runs; open-weight (MLX) models and the
-hosted providers need macOS 27. **Private Cloud Compute** works too: an app that
-uses it needs Apple's Private Cloud Compute entitlement.
+hosted providers need macOS 27. **Private Cloud Compute** will be in the Mac App
+Store version: it needs Apple's Private Cloud Compute entitlement, which the DMG
+doesn't carry.
 
 **Product page:** [thisbrain.ai/locallm](https://thisbrain.ai/locallm) — or
 get the app directly:
-[LocalLM Lab 2.0.0-GA (arm64 DMG)](https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLM.Lab-2.0.0-GA-arm64.dmg).
+[LocalLM Lab 2.1.0-GA (arm64 DMG)](https://github.com/ancientcomputing/locallm/releases/download/v2.1.0-GA/LocalLM.Lab-2.1.0-GA-arm64.dmg).
 
 ## LocalLM Lab SDK
 

docs/api-surface.md

@@ -2,7 +2,7 @@
 
 > Machine-generated by `scripts/dump-public-api.sh` from the built `.swiftinterface`s.
 > Do not hand-edit. `sdk-guide.md` §12 is the *narrative* reference; this is the check.
-> SDK version: `"2.0.0"` (the published `v2.0.0-GA` xcframeworks, re-published 2026-10-09) · generated 2026-10-09T14:46Z
+> SDK version: `"2.1.0"` (the published `v2.1.0-GA` xcframeworks, re-published 2026-10-17 from 8e5d287) · generated 2026-10-17T13:38Z
 
 ## LocalLMLabSDKCore
 

docs/sdk-guide.md

@@ -9,8 +9,8 @@ below for the working reference apps this guide is drawn from.
 
 Requires macOS 26+ on Apple Silicon, Swift 6 tools.
 
-**Status note**: this guide describes the API as of `2.0.0`, a one-time breaking release; every 2.x
-release after it is source compatible with it (see [§9](#9-scope-and-boundaries-what-core-doesnt-do)
+**Status note**: this guide describes the API as of `2.1.0`. `2.0.0` was a one-time breaking release;
+every 2.x release after it is source compatible with it (see [§9](#9-scope-and-boundaries-what-core-doesnt-do)
 for the compatibility policy). Coming from 1.x: [`migrating-to-2.0.md`](migrating-to-2.0.md); from
 `0.8.x`: [`migrating-to-1.0.md`](migrating-to-1.0.md) first. New in 2.0: per-server trust and tool
 approval ([§7c](#7c-tool-authorization-two-levers--which-tools-and-whether-they-ask-first)), MCP
@@ -1096,7 +1096,7 @@ relaunch.
 | Provider | Ships in | `scheme` | macOS | For |
 |---|---|---|---|---|
 | `SystemModelProvider` | Core | `system` | 26+ | Apple's on-device model — always there on an Apple-Intelligence Mac |
-| `PCCModelProvider` | Core | `pcc` | 27 | Apple's Private Cloud Compute model (Apple's own models, not your weights). Needs the PCC entitlement + App Store Small Business Program — there is no paid tier. Since `1.0.0-beta.3` the provider maps availability, quota, and typed errors cleanly, and adds `probe(timeout:)` — an async liveness check (`availability(for:)` alone can say `.available` while turns still throw on a build without the entitlement). |
+| `PCCModelProvider` | Core | `pcc` | 27 | Apple's Private Cloud Compute model (Apple's own models, not your weights). Needs Apple's managed PCC entitlement, which works only in your app's main executable, signed with a provisioning profile that grants it ([§10e](#10e-shipping-the-model-layer-through-app-store-connect)) — there is no paid tier. Since `1.0.0-beta.3` the provider maps availability, quota, and typed errors cleanly, and adds `probe(timeout:)` — an async liveness check (`availability(for:)` can say `.available` while turns still fail). Neither is a way to find out whether you have the entitlement: without it FoundationModels traps on the first PCC call, these included, so check your own entitlement first (§10e). |
 | `ClaudeModelProvider(auth:)` | **`LocalLMLabSDKClaude`** | `claude` | 27 | Claude, via a host-supplied API key or App Attest client id — the SDK stores neither |
 | `MLXModelProvider` | **Inference** | `mlx` | 27 | Locally-run open-weight models (Qwen, Llama, …) via MLX |
 | `try RemoteModelProvider(_:)` | **Remote** | *host-chosen* — set in `RemoteProviderConfig.scheme`, one instance per provider you configure | 27 | Hosted APIs — OpenAI, Anthropic's Messages API, OpenRouter, any OpenAI-compatible server. The one provider here that's data-driven rather than a fixed named type; details in [§6b](#6b-online-providers--gpt-claude-online-openrouter-locallmlabsdkremote) |
@@ -1190,6 +1190,19 @@ mlx.storageUsed      // total bytes of weights
 try mlx.remove(id)   // delete weights
 ```
 
+**Tool calls on a local model.** The MLX runtime parses the model's tool-call markup into calls to
+your tools. Since 2.1, a call it can't accept fails the turn instead of reaching the user: an
+unparseable payload, a tool the session doesn't declare, or arguments that break the tool's schema
+throw `LocalLMLabError.generation(underlying:)`, whose message names the reason (for example
+`malformed_syntax`) and never includes the raw model output. Before 2.1 the call markup came back as
+the model's answer and nothing ran, so if you showed that text, or retried on it, catch the error
+instead; running the turn again is often enough for a small model. A call written inside the model's
+reasoning (`<think>…</think>`) stays inert, since a model rehearsing a call hasn't decided to make
+it. The exception is a `<think>` block that opens directly on the call and never closes, which
+Qwen3 sometimes writes: that call runs. 2.1's runtime also fixes Gemma 4's array and object
+arguments and adds gpt-oss (Harmony) and Muse Glimmer tool calls; the
+[CHANGELOG](../CHANGELOG.md) has the details.
+
 **Where the weights land** — a standard Hugging Face cache
 (`models--<org>--<name>/snapshots/<sha>/`, deduped `blobs/` underneath):
 
@@ -2744,8 +2757,10 @@ The following are deliberate boundaries, not gaps waiting on a future release:
 > [§10d](#10d-one-time-apple-developer-portal-setup-for-mas-signing) below for which is which.
 
 Core has been tested under App Sandbox — confirmed via a real sandboxed, Developer-ID-signed,
-notarized build (not just code review), not through the actual MAS submission pipeline itself yet.
-Here's exactly what's needed and what was actually verified.
+notarized build (not just code review). Since 2.1 two apps built on the SDK, LocalLM Lab (with
+Private Cloud Compute and local MLX models) and JevDK, have also gone through App Store Connect's
+upload validation into TestFlight; [§10e](#10e-shipping-the-model-layer-through-app-store-connect)
+has what that turned up. Here's exactly what's needed and what was actually verified.
 
 > **Related, for model downloads:** the sandbox is one half of the security story for an app that
 > fetches models at runtime. The other half is what it fetches: the trust policy, preflight, hash
@@ -2843,7 +2858,8 @@ submission of your own app needs your own bundle ID from the start.
 - Leave every Capability unchecked unless something you're using genuinely needs an App
   ID-level capability (iCloud, Push Notifications, etc.) — App Sandbox, network client, and the
   Calendar/Reminders/Contacts/Location entitlements are all declared directly in the entitlements
-  file (10a), not toggled here.
+  file (10a), not toggled here. The exception is Private Cloud Compute: once Apple grants it to
+  your account, turn it on here ([§10e](#10e-shipping-the-model-layer-through-app-store-connect)).
 
 **3. A Mac App Store distribution provisioning profile**, tying the App ID to the certificate.
 
@@ -2870,6 +2886,57 @@ Apple-Distribution-signed, sandboxed `.pkg` build this setup enables — Develop
 notarization is the other supported path (`build-and-sign.sh`), for distributing outside the Mac
 App Store.
 
+### 10e. Shipping the model layer through App Store Connect
+
+What uploading LocalLM Lab and JevDK to App Store Connect turned up. None of it needed an SDK API
+change.
+
+**Embedding `LocalLMLabSDKInference` (local MLX models).** Use SDK 2.1 or later. In 2.0 the
+framework's bundle ID was `mlx-swift_Cmlx`, and App Store Connect rejects an app containing a bundle
+ID with an underscore (error 90277: not a valid bundle identifier). Since 2.1 it is
+`lab.locallm.sdk.inference`, and the xcframework embeds as is: no renaming, and no copying
+`mlx-swift_Cmlx.bundle` into your app's Resources. If you did that for 2.0, remove it when you
+upgrade: the copy and the framework's own copy share the bundle ID `mlx-swift.Cmlx.resources`, and a
+duplicate bundle ID can fail validation too (90685). MLX loads its Metal library from the framework's
+own `Versions/A/Resources/default.metallib`, so don't strip the framework's Resources. Before an
+upload, check that every bundle ID in the app is only letters, digits, hyphens and dots:
+
+```bash
+find "MyApp.app" -name Info.plist -exec /usr/libexec/PlistBuddy -c 'Print :CFBundleIdentifier' {} \; 2>/dev/null \
+  | grep -vE '^[A-Za-z0-9.-]+$' || echo "all bundle IDs valid"
+```
+
+**Private Cloud Compute.** `com.apple.developer.private-cloud-compute` is a managed capability:
+
+- Apple grants it to your developer account ("Access to models on Private Cloud Compute"). Then turn
+  it on for your App ID ([§10d](#10d-one-time-apple-developer-portal-setup-for-mas-signing)) and
+  regenerate your provisioning profiles.
+- Sign the entitlement only when the embedded profile grants it. An entitlement the profile doesn't
+  cover stops the app from launching.
+- Run PCC in the profile-covered executable: your app's main executable. A helper tool your app
+  launches has no App ID or profile of its own, so the entitlement can't work there, and a sandboxed
+  helper signed with `com.apple.security.inherit` may carry no other entitlement. Register
+  `PCCModelProvider` in the app's own `LocalLMLab`, not in a spawned process.
+- Check the entitlement before touching PCC. Without it FoundationModels traps on the first PCC call,
+  `availability(for:)` and `probe(timeout:)` included, so read the running process's own entitlement
+  first and register the provider only when it's there:
+
+```swift
+import Security
+
+let pccEntitled: Bool = {
+    guard let task = SecTaskCreateFromSelf(nil),
+          let value = SecTaskCopyValueForEntitlement(
+              task, "com.apple.developer.private-cloud-compute" as CFString, nil) else { return false }
+    return (value as? Bool) ?? true
+}()
+if pccEntitled { providers.append(PCCModelProvider()) }
+```
+
+LocalLM Lab's own Developer ID build (the DMG) doesn't carry the PCC entitlement, so it has no PCC;
+its Mac App Store build does. Whether a Developer ID provisioning profile can grant it hasn't been
+tested.
+
 ## 11. Components: prebuilt SwiftUI (MCP servers + the model layer)
 
 > **Reach for this when** you want a working settings surface — for **MCP servers** (add /

docs/tested-models.md

@@ -1,5 +1,10 @@
 # Tested open-weight (MLX) models
 
+> **Current list:** the maintained list of MLX models — newer Qwen, Gemma, Mistral, Granite and
+> other families, sized for a 24 GB MacBook Air and a 64 GB Mac Studio, with a test queue — is
+> [thisbrain.ai/locallm/mlx-models.html](https://thisbrain.ai/locallm/mlx-models.html). This page
+> stays as the SDK-side record (tool-calling run, deciders, capacity notes).
+
 > **Point-in-time, not maintained.** This is a snapshot of what we saw on our hardware,
 > our SDK build, and one specific tool-calling prompt, as of **2026-08-27** (SDK
 > `1.0.0-beta`, requires macOS 27). Model conversions, chat templates, and the MLX stack all