SKILL.md

name

localforgellm

description

Choose local LLM models and quantizations for the user's real hardware and target tokens per second; compare CPU/RAM/GPU placement. Build, tune and repair dense or MoE stacks with APEX, REAP, Bonsai ternary kernels and MTP; connect Pi, OpenShell, llama UI, Hermes and optional Jev API/browser-use tools. Use for model selection, memory/speed feasibility, installation, context tuning and derived weights. Generative image, video and audio workflows are a planned branch.

LocalForgeLLM

Turn the user's request into a working, reproducible local AI stack. Select the model, engine, weight format and interface together; measure the result on the intended workload. Use existing implementations and their native configuration before writing a custom integration.

This skill belongs to the complete LocalForgeLLM repository. Resolve the links below relative to this file. Keep docs/ with the skill; copying this file alone loses its operational references. Project AGENTS.md directs agents here. For discovery outside this checkout, follow native skill registration.

Choose the work

Request	Route	Result
Recommend a model/quant or assess a target tok/s	Candidate comparison → Level 2	Hardware/resource budget and compared placements; measured speed or an explicit estimate with a validation step
Install a model or build a stack for a task	Level 1	Compatible dependencies, weights, server and working client
Change context, memory, speed or model behavior	Level 2	Measured configuration or a separately identified model modification
Select, build or switch CUDA/Vulkan	Backend workflow	Compatible runtime/device, preserved model and rollback; backend, RAM and MTP effects compared separately
Download MTP weights; enable, disable or diagnose speculation	MTP operations	Verified compatible head when needed, preserved main placement, effective mode and a working rollback
Create a custom model from full weights	Level 3	Derived artifact with source revision, recipe and quality results
Prune experts, rebuild APEX or cache individual experts	REAP and expert-cache procedure	Separate topology, precision and placement experiments with retained quality failures
Fit or accelerate Ternary Bonsai 2	Bonsai runtime and cache recipe	Verified packing/kernel pair, calibrated cache, actual context and measured memory
Add Jev decisions, browser-use, evaluate Cua or reuse Hermes tools in llama UI	Jev API and upstream implementations → Native MCP and skills	Host-side API/tool integration; local text model and harness ownership preserved; Cua mapping is source-checked, not locally deployed
Update, resume, repair or replace a component	Operations	Working stack with a tested rollback path
Connect an agent or customize its shell	Interfaces	Working client/harness connection and the requested extension
Image, video or audio generation	Generative branch	Explain planned status; do not claim an implemented workflow

Levels describe the nature of the task, not mandatory steps. Installing a published quant does not require building one. A tuning task may move into level 3 when it needs different weights; explain the additional compute, storage and artifact changes before that work.

Execute

1. Read architecture and agent workflow. Extract the task, capabilities, existing choices, hardware limits and intended interface. Ask only for missing information that changes the result; inspect discoverable machine state yourself.
2. Find an existing stack record and verify its processes, paths, versions and endpoint, including stacks outside this checkout. Identify whether process/device inspection sees the host or only a sandbox. Preserve the working baseline and inventory actual available resources before selecting an implementation.
3. Read the relevant level guide, engine selection, and the selected model/runtime's own documentation. Verify exact architecture, checkpoint format, backend, context, template and tool-call support. The catalog is a starting point, not a closed compatibility list. For a model/quant or speed decision, compare concrete CPU/GPU placements and compatible candidates against the existing measurements. Budget weights, KV/recurrent state, buffers and application headroom in both RAM and VRAM. A GGUF larger than VRAM is a placement problem to investigate, not a reason by itself to reject it or select the smallest quant. Keep the user's selected model/version fixed unless they agree to a replacement.
4. Perform the requested work. Save concrete commands, dependency versions and effective settings in the stack's local record. Use isolated environments and a candidate output/profile when replacing something that already works.
5. Follow validation: check the endpoint, requested capabilities, actual client tool cycle, speed, memory and the target context. A plain chat response does not prove a coding agent works.
6. Promote the candidate after it meets the user's criteria. Record the result and rollback procedure. Give the user the exact start/stop/connect instructions and measured limits.

Keep the distinctions

• The coding agent orchestrates; the inference engine executes the model; the harness owns tools and history; OpenShell owns its execution environment and access rules.
• APEX selects precision for weights. It is optional, is not an agent harness and is not a runtime optimizer. Read APEX only when using that method.
• REAP prunes experts using calibration statistics; hot-expert caching places retained weights. Neither automatically lowers top-k. Our rented-server experiment reduced storage without a pruning speedup; caching had a separate measured gain and RAM cost.
• Bonsai PTQ1_0 and PQ2_0 have different sizes and require compatible ternary kernels. Reuse the measured profiles as evidence with their actual hardware, cache, thinking mode and input/output lengths; do not transfer the subscriber's 5070 speed to the 4060.
• Jev is a hosted typed-decision API, not the local chat model. Keep keys host-side and model-generated decisions within the harness's existing permissions. Typed choices do not prove task correctness; sharing MCP tools and skill text does not transplant Hermes or Pi lifecycle hooks.
• CUDA and Vulkan are runtime backends. Follow the backend workflow to check support, build/select a candidate and compare it on the actual hardware. Reuse compatible weights; switching backends does not itself convert or requantize the model. Keep placement, RAM cap and MTP fixed for the first comparison, then rebalance them separately. Never attribute a combined tuning result to CUDA alone.
• MTP drafts tokens for target verification. Follow MTP operations to discover/download only a missing compatible head and switch the saved profile on or off. Preserve the main model's layers, CPU expert placement and context for an MTP-only request. A loaded head or high acceptance does not establish useful output or a speed gain; keep manual quality checks pending until the user's feedback. See the Gemma case study for measured results and their limits.
• CPU/GPU placement does not change the number of experts selected by the learned router. Changing routing, pruning tensors and changing tensor precision have different compatibility and quality implications.
• Advertised model context, server context per slot, client context and output budget must agree. Cumulative session tokens are not the current context size.
• “Build from scratch” in level 3 means constructing a derived deployment artifact from existing full weights. Training a new base model requires a separate explicit training task, data and compute plan.
• Completion means a working result for the requested workload. Report source-inspected, estimated and measured facts separately. Never borrow benchmark speed or memory from another machine as a local result.
• An advice-only request can end with a comparative assessment and clearly identified unknowns. A required speed is verified only by local measurements at a stated context and workload; file size, an accepted flag or another model's result cannot establish it.

Extend and maintain

For an unfamiliar implementation, use Learn an implementation. For custom interfaces, use the code maps in Pi, OpenShell, llama UI and Hermes. Read the actual checkout before changing it; versions can change both configuration and architecture.

Keep model files, raw sessions and credentials local. A task to configure a stack does not itself authorize publishing weights, renting GPUs, changing unrelated global providers, deleting sandbox volumes or weakening access policy. Continue all already-authorized work and request only the additional decision actually needed.