Repository navigation
✨ Add Apple MLX local inference and first-class ClankeR onboarding - #15842
Conversation
Signed-off-by: Andy Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
Signed-off-by: Andy Anderson <andy@clubanderson.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
Expanded first-class MLX acceptance deliveredCommit 057f74e adds a dedicated Apple MLX engine choice/tile on the chosen hive's Canonical interface from the source checkout: # Native Apple Silicon Mac, already-downloaded tool-capable model, separate terminal:
HIVE_MLX_PYTHON="$PWD/.venv-mlx/bin/python" just contribute-mlx-serve /Users/YOU/models/tool-coder
# Explicit opt-in to two bounded streamed tool-call/result verification turns:
just contribute-mlx-setup /Users/YOU/models/tool-coder --verify-tools
export AGENT_MODEL=mlx//Users/YOU/models/tool-coder
export HIVE_HUB=https://YOUR-CHOSEN-HIVE
just contribute-check pi
just contribute-setup pi
just contribute-mlxThe serve recipe is foreground, loopback-only and forces Hugging Face offline. No packages/weights are installed/downloaded or models started automatically. Docker Desktop 4.34+ host networking must be enabled explicitly; MLX-LM stays on the Mac and Pi/tools stay inside the relay container. Podman machine and Kubernetes are refused for this route. Explicit MLX model selection survives saved hosted-provider profiles without changing generic Pi precedence (review finding fixed with a focused shell-seam regression). The public homepage now has a distinct Local AI inference / Apple MLX integration group, linking to the real hub Regression additions exercise actual page command-generation logic, unsupported OS/mode/runtime/model inputs, shell quoting, real recipe delegation/config precedence, mocked offline server argv, rendered onboarding and homepage grouping. Repository rules prohibit local test/build suites; CI remains the sole verdict. Live Apple GPU inference and Docker Desktop host connectivity remain unverified. Nothing was deployed, merged, downloaded or launched on this laptop. |
Signed-off-by: Andy Anderson <andy@clubanderson.com>
|
stage 080-awaiting-merge-admission → 050-review-pending: required-check-pending:Sign-off survives the squash |
c9a108c
ℹ️ Sentinel notice — informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @clubanderson, head
Hive did not add the |
|
@clubanderson this PR is held at Failing checks:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
There was a problem hiding this comment.
Reviewed (correctness) — read at head c9a108c; applied the diff to merge-base 79e0ba6 and ran node --test bin/mlx-backend.test.js (14 pass), node bin/contributor-relay.test.js (736 pass), node --test src/pkg/dashboard/contribute_mlx.test.js (3 pass). Cross-checked the probe against MLX-LM v0.32.2 server.py (indexed streaming tool_calls, finish_reason: "tool_calls", {mlx_lm}-{mlx}-… fingerprint) and the extension against pi-ai 1.4.0 compat (openAICompletionsApi, PI_AI_VERSION exported). No blocking findings.
Low — three tests are accidentally nested inside another test (bin/mlx-backend.test.js:239-291). 'explicit serve recipe …', 'MLX just launcher …' and 'MLX launcher preserves explicit model …' are declared between http.createServer(...) and await new Promise(resolve => server.listen(...)) inside 'Pi inference transport rejects redirects after successful setup'. node:test registers them as subtests of the redirect test; they pass today only because the parent happens to await long enough (node:test cancels unfinished subtests when the parent finishes). Move the three test(...) blocks to top level (after the closing }); of the redirect test) so they are independent and can't be cancelled by the parent's timing.
Not verified here (author also states it): live Pi 1.4.0 behaviour with a models.json provider whose api is the custom hive-mlx id before mlx-pi-extension.js registers the override.
— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Style review (head c9a108c) — one maintainability finding, non-blocking.
Three unrelated tests are nested inside the body of another test — bin/mlx-backend.test.js:228-293. The test 'Pi inference transport rejects redirects after successful setup' creates its HTTP server at L229, then L234-287 define three top-level-looking test(...) calls (explicit serve recipe…, MLX just launcher…, MLX launcher preserves explicit model…) before the parent resumes at L288 with server.listen and the actual redirect assertion. Node's runner does run them (as subtests of the redirect test — verified with node --test, 14 pass), but the report groups serve/Justfile coverage under a transport test, the parent's setup is split across 60 lines, and anyone trimming the redirect test will delete the Justfile seam with it. Every other test in the file is flat; this looks like a misplaced paste.
Fix: move L234-287 out of the parent, e.g. to just before L228 (no code inside the three blocks needs to change — they only use their own t):
test('Pi inference transport rejects redirects after successful setup', async t => {
const server = http.createServer((req, res) => {
res.writeHead(307, { Location: 'https://api.openai.com/v1/chat/completions' });
res.end();
});
-
- test('explicit serve recipe uses native Python, loopback and offline weights without changing RAM settings', t => {
- ...
- });
-
- test('MLX just launcher remains a Pi wrapper with explicit model/runtime refusal', () => {
- ...
- });
-
- test('MLX launcher preserves explicit model across both saved-profile loads without changing generic Pi defaults', t => {
- ...
- });
await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
t.after(() => new Promise(resolve => server.close(resolve)));
const endpoint = `http://127.0.0.1:${server.address().port}/v1`;
const guarded = extension.localFetch(endpoint);
await assert.rejects(guarded(`${endpoint}/chat/completions`, { method: 'POST', body: 'private task' }));
});
+
+test('explicit serve recipe uses native Python, loopback and offline weights without changing RAM settings', t => { /* unchanged body */ });
+test('MLX just launcher remains a Pi wrapper with explicit model/runtime refusal', () => { /* unchanged body */ });
+test('MLX launcher preserves explicit model across both saved-profile loads without changing generic Pi defaults', t => { /* unchanged body */ });Otherwise the new code follows repo idioms: marker-block extraction in src/pkg/dashboard/contribute_mlx.test.js:9-12 matches fleet_cost_rotor.test.js:19; SETUP_BACKEND flavor mapping (contribute_landing.go:1791) reuses the litellm pattern; the .test.js is picked up by the existing node --test src/pkg/dashboard/*.test.js CI step.
— hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Reviewed (intent-alignment, head c9a108c) — the diff satisfies #15839: Apple Silicon/macOS 14+/native Python ≥3.10 gating (bin/mlx-backend.js:37-54), opt-in streaming tool-call verification that gates readiness via a digest-checked receipt (bin/mlx-backend.js:148-173, :220-231), loopback-only endpoint with no hosted fallback (:23-33, bin/mlx-pi-extension.js:6-15), reuse of Pi custom models and the existing ~/.pi staging (bin/pi-backend.js:160-180 keeps providers.mlx; receipt copied with the dir), and Mac-host vs container errors (Justfile contribute-hive pi) case, bin/contributor-agent.sh:621-623). Both new Node suites are wired into CI (.github/workflows/v2-ci.yml:886 glob, :908).
One advisory, non-blocking:
- Public homepage expansion is outside #15839's scope —
src/pkg/hub/static/index.html:813-819adds a new "Local AI inference" integration group (and restructures the band to 2 columns,:319) plussrc/docs/integrations.md. The issue asks for contributor-relay inference; marketing-site placement is a product/branding call that could land separately if maintainers prefer. Not a defect; fine to keep if intended.
— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
Signed-off-by: Andy Anderson <andy@clubanderson.com>
CI follow-through: concrete failure fixed, awaiting new-head verdictHead
The new fix replaces that inline declaration with a tokenized The public Apple MLX integration card and |
|
hive/awaiting-ci (scanner): failing on c9a108c: 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
Actual repaired-head CI verdictExact head: All 12 required checks now report SUCCESS, including the underlying regression shard, aggregate tests, measured coverage, build-and-test, lint, security checks, changelog/citation guard and DCO:
The failure on the previous head was our added inline styling, fixed with tokenized CSS rather than weakened baselines. No blanket retries, base refresh, trust/sentinel changes, labels, or concurrent workflow edits were used. One non-required ARM image build remains in progress: build (linux/arm64). I am not describing that unfinished check as green. The public Apple MLX integration card and |
|
stage 080-awaiting-merge-admission → 050-review-pending: check-pending |
|
stage 080-awaiting-merge-admission → 050-review-pending: review-gate:no-review-bundle |
There was a problem hiding this comment.
Reviewed (docs-currency) — one low, non-blocking note.
Two workflow summaries skip the required just contribute-setup pi step. bin/README.md:91 ("Use just contribute-mlx-serve, just contribute-mlx-setup <model> --verify-tools, then just contribute-mlx") and docs/inference-backends.md:15-17 list the three MLX recipes as the whole workflow, but contribute-mlx delegates to contribute-hive, which exits with "Not set up yet. Run: just contribute-setup " when contributor.env is absent (Justfile:975-977). The canonical sequence in src/docs/contributor-relay.md, docs/backend-setup.md:15, the hub landing (src/pkg/hub/server.go:4838) and the generated UI commands (src/pkg/dashboard/contribute_landing.go:1818) all include it; only these two summaries drifted.
| `mlx-backend.js` | Optional local inference | Apple Silicon runtime/config checks, explicit tool-call verification and offline loopback server launch for Pi's `mlx` provider. Use `just contribute-mlx-serve`, `just contribute-mlx-setup <model> --verify-tools`, `just contribute-setup pi`, then `just contribute-mlx`; see [MLX setup](../src/docs/contributor-relay.md#apple-mlx-local-inference). No automatic installation or model downloads. |
Verified against head 9a598f3: Pi pin 1.4.0 (src/Dockerfile.contributor:487), 4‑GiB container cap (Justfile:33), HIVE_MLX_* variables documented in src/docs/env-vars.md:501-504, and the /v1/models resolved-path / Hub-ID listing and <mlx-lm>-<mlx>- fingerprint claims match MLX‑LM v0.32.2 server.py (handle_models_request, get_system_fingerprint). CI runs both new Node suites (v2-ci.yml:886,908).
— hive: agent=adjudicator-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Style review (head 9a598f3) — approve; advisory items only, none blocking #15839.
src/pkg/dashboard/contribute_mlx_test.go:29— the inline-style assertion sits inside thefor _, want := rangeloop, so a regression prints the samet.Error14 times. Hoist it out:
} {
if !strings.Contains(body, want) {
t.Errorf("Apple MLX onboarding missing %q", want)
}
- if strings.Contains(body, `<div id="pi-mlx-note" style=`) {
- t.Error("MLX help must use tokenized CSS rather than add inline styling debt")
- }
}
+ if strings.Contains(body, `<div id="pi-mlx-note" style=`) {
+ t.Error("MLX help must use tokenized CSS rather than add inline styling debt")
+ }
}bin/mlx-backend.js:179-185—agentDir/readJSONduplicatepiAgentDir/readJSONinbin/pi-backend.js:76-80(with a subtle drift:env.HOMEvsenv.HOME || ''). Export the Pi helpers and import them soPI_CODING_AGENT_DIRresolution has one owner:
-const { parsePiModelSelection } = require('./pi-backend.js');
+const { parsePiModelSelection, piAgentDir: agentDir, readJSON } = require('./pi-backend.js');(and add piAgentDir, readJSON to module.exports in pi-backend.js:197.)
-
Pinned version literals are copy-pasted —
pi-coding-agent@1.4.0appears atcontribute_landing.go:1621and:1803,mlx-lm==0.32.2at:1805, and themacOS 14 / Python 3.10 / Node 22.19 / Pi 1.4.0 / Docker Desktop 4.34prerequisite sentence is restated atcontribute_landing.go:1689,:1800,src/pkg/hub/server.go:4828,static/index.html, and three docs.src/Dockerfile.contributor:487already ownsPI_CODING_AGENT_VERSION=1.4.0. At minimum hoistMLX_PI_VERSION/MLX_LM_VERSIONconstants abovemlxCommands(contribute_landing.go:1796) and build the<option data-host-install>from them, so the next pin bump cannot leave the picker and generator disagreeing. -
bin/contributor-relay.js:3228, :3927, :9913—checkMlxRuntime(a synchronousexecFileSyncchild with a 20 s timeout,mlx-backend.js:282) is invoked at three sites. The startup gate at:9913already fails closed; the two inner calls re-run the subprocess on everybuildLaunchCommand/buildHeadlessArgv. If re-checking per task is intentional, a singleensureMlxReady()helper with a one-line comment stating that intent would make the triple call read as design rather than duplication. -
src/pkg/hub/contribute_landing_test.go:43—strings.SplitN(body, "...Sign-in</p>", 2)[1]panics with an index error if the label is renamed, unlike thestart < 0 → t.Fatalguard used four lines earlier for the inference group. Use the same guard.
— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Summary
Fixes #15839.
Adds optional, first-class Apple MLX local inference for ClankeR, using the existing Pi coding agent, not a pretend MLX coding CLI or a hosted Hive backend.
/contributeselector, branded tile, anchored requirements/help and OS/mode/runtime-aware copyable command generator. UI enginemlxmaps to actual executorpiand modelmlx/<exact-model-id-or-path>.just contribute-mlx-serve <already-downloaded-model>explicitly starts foreground, offline MLX-LM on Mac loopback;just contribute-mlx-setup <model> --verify-toolsexplicitly opts into two bounded streamed tool-call/result checks;just contribute-mlxlaunches the actual Pi contributor workflow. Explicit MLX selections survive saved hosted-provider profiles without changing generic Pi precedence.hive-mlxAPI fails closed if the extension is missing./contribute#apple-mlxsetup/usage instructions, not Apple sign-in or endorsement. Integration catalog, backend/inference/environment guides, helper index, RAM/model/troubleshooting docs and changelog are aligned.Canonical workflow
Full installation instructions are in
src/docs/contributor-relay.md#apple-mlx-local-inferenceand generated by the selected hive's/contributepage. Not every MLX model supports structured tools; passing the probe is not a coding-quality benchmark. Host weights/KV cache consume shared RAM outside the relay container's memory cap.Version assumptions and sources
Protocol/source checked against MLX-LM v0.32.2 and published Pi 1.4.0 APIs; contributor image pins Pi 1.4.0 and this route requires at least that guarded transport API. Other MLX-LM versions must pass the same explicit tool round trip.
Validation and limits
gofmtand Git diff checks perAGENTS.md; no local Go/Node repository suites, builds or citation guards were run.No merge, approval or deployment requested by this PR task.