Skip to content

feat(workers): Local LLM First-Class Tool Calling Adapter for Ollama / vLLM #10

Description

@sdageltc

Summary & Motivation

letitloop already features a zero-dependency REST client (orchestrator/llm.py) that connects to any OpenAI-compatible endpoint (such as Ollama or vLLM). However, advanced coding models (like Qwen 2.5 Coder 32B, Llama 3.3 70B, and DeepSeek Coder) provide structured function/tool calling that can execute multi-turn file edits and bash commands directly.

We want to build a dedicated Local Tool-Calling Worker Adapter that empowers users to run complete, self-contained coding loops on consumer GPUs (or Apple Silicon Macs) without third-party agent CLIs or cloud API keys.


Proposed Solution

  1. LocalToolWorkerAdapter (orchestrator/worker_adapters.py):
    • Connects directly to local Ollama / vLLM /v1/chat/completions API.
    • Defines a standard local tool schema: read_file, write_file, replace_lines, execute_command.
    • Executes multi-turn tool loops within the contract's allowed workspace scope until the objective is achieved or max iterations are reached.
  2. Deterministic Output & Evidence Recording:
    • Records every executed tool call and terminal output into the contract's task execution journal.

Acceptance Criteria

  • Works seamlessly with local Ollama (ollama run qwen2.5-coder) and vLLM instances.
  • Restricts local tool execution strictly to contract.workspace_scope.
  • Handles tool call parse errors and JSON decoding failures gracefully with automatic prompt repair.
  • Unit tests with mocked local LLM tool-calling responses.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is neededintegrationsWorker adapters, LLM providers, and external tools

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions