Skip to content

✨ Add optional Apple MLX inference to the ClankeR contributor relay #15839

Closed

Description

@clubanderson

Goal

Let contributors on supported Apple Silicon Macs run ClankeR coding tasks against a user-managed local MLX-LM server, using the existing tool-capable Pi runner rather than treating an inference server as an agent.

Compatibility and boundaries

  • Apple Silicon, native Python >=3.10, macOS >=14; not Intel Macs or Rosetta Python.
  • Optional setup only; no automatic dependencies, model downloads, heavy workloads, public listening address, hosted fallback, or changes to running relays.
  • Preserve contributor admission/trust rules, model identity, task lifecycle, sandbox requirements, and existing backend defaults.
  • Verify the selected model/server can return structured streaming tool calls before advertising readiness. MLX-LM alone does not edit files or open PRs; Pi supplies that workflow.
  • Reuse Pi custom models and existing container staging/network boundary, with actionable Mac-host vs Linux-container errors.

References

Verification

Mock/process/HTTP regression tests without GPU/model weights; CI is the repository test verdict. Actual Apple Silicon inference must remain explicitly unverified unless separately exercised on a consenting contributor machine.

Activity

  1. clubanderson commented on Oct 30, 2026

    @clubanderson
    MemberAuthor

    Implementation architecture: reuse the existing Pi coding runner with a custom mlx provider, not a standalone text-generation backend. Fresh worktree feat/clanker-mlx; optional native Mac preflight, explicit streaming tool-call/result verification, localhost-only configuration and staged Pi config. MLX-LM server protocol checked against released v0.32.2; contributor image pins Pi 1.4.0. No models installed/downloaded/started and no live relay settings changed. Review caught a transport-boundary issue (Pi/OpenAI SDK follows inference redirects by default); addressing it before PR so localhost-only is enforced on real coding requests as well as setup probes.

  2. clubanderson commented on Oct 30, 2026

    @clubanderson
    MemberAuthor

    Implementation PR: #15842 (base v5), expanded through commit 057f74e.

    The owner's first-class acceptance surfaces are included: dedicated Apple MLX engine selection/tile and command generator on each hive's /contribute, canonical just contribute-mlx-serve, just contribute-mlx-setup <model> --verify-tools, and just contribute-mlx recipes, a separate Local AI inference / Apple MLX integration group on the public homepage, a real hub /contribute#apple-mlx installation/usage section, and aligned integration/backend/inference/helper documentation. Actual execution remains the existing tool-capable Pi runner; MLX-LM provides native Mac inference only.

    Apple Silicon/macOS 14+/native Python 3.10+ and Pi 1.4.0+ are explicit. Host inference is loopback-only and user-managed/offline; Docker Desktop host networking connects the Pi container. Unsupported Intel/Rosetta/Linux-host/Podman-machine/Kubernetes routes do not silently become hosted-provider usage. No Apple sign-in or endorsement implication.

    Code review found and fixed explicit-model precedence against saved hosted-provider profiles, with a shell-seam regression. Mock/process/HTTP/UI tests are committed; CI is queued and has not produced a verdict. Live MLX GPU inference and Docker Desktop connectivity are explicitly unverified. No package install, model download/start, memory adjustment or deployment was performed on the owner's Mac.

  3. added
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    on Oct 30, 2026
  4. github-actions commented on Oct 30, 2026

    @github-actions
    Contributor

    @clubanderson this issue was closed by @hivecommons-hive[bot] as completed.

    If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it — check your running version against the fix before concluding the fix failed.

    If you are still seeing the problem, comment /reopen and it will be reopened for you — you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions