Skip to content

feat(reasoning): opt-in remote endpoints for distillation LLM and embedder (3.9.0) - #202

Merged
acidkill merged 1 commit into
mainfrom
feature/remote-reasoning-endpoints
Sep 3, 2026
Merged

feat(reasoning): opt-in remote endpoints for distillation LLM and embedder (3.9.0)#202
acidkill merged 1 commit into
mainfrom
feature/remote-reasoning-endpoints

Conversation

@acidkill

@acidkill acidkill commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds reasoning_training.allow_remote_endpoints (default false, env SURREAL_MEMORY_REASONING_ALLOW_REMOTE): an explicit opt-in that lets the distillation LLM endpoint (pattern naming), the distiller's embedding provider, and the stop hook's semantic dedup use a configured remote http(s) gateway (e.g. LiteLLM) instead of loopback-only servers.
  • Adds Bearer authentication for the naming request: SURREAL_MEMORY_LLM_API_KEY is sent as Authorization: Bearer when set; loopback servers keep going keyless with no header; a remote endpoint allowed by the opt-in without a key warns at namer build time instead of failing.
  • Config plumbing: new field round-trips through to_dict/from_dict/TOML save-load and is overridable via env; emitted in _reasoning_toml_lines after distill_llm_endpoint; docs/reference/config.md regenerated.
  • Version bump 3.8.1 → 3.9.0 everywhere (16 files verified by scripts/pre_ship.py --only versions), CHANGELOG entry.

Why

The reasoning pipeline's loopback-only invariant exists so raw model thinking never leaves the machine by accident. That invariant also made it impossible to run the pipeline against a remote OpenAI-compatible gateway — the smem stack is moving off Titan onto k3s (run 010), where embeddings/rerank/LLM are served by LiteLLM over the network. This PR keeps the default byte-for-byte identical (opt-in off ⇒ no remote endpoint is ever used, exactly as before) while letting an operator widen the gate deliberately. The opt-in widens where traces may be sent, never what counts as a URL: non-http(s) schemes are refused either way, and the unconfigured probe fallback stays loopback-only regardless of the flag.

Test plan

  • pytest tests/ -m "not stress" -n auto — 7334 passed, 4 skipped, 1 xfailed (new tests in test_reasoning_naming.py, test_reasoning_distiller.py, test_stop_hook.py, test_unified_config.py; TDD RED→GREEN)
  • ruff check src/ tests/ clean; ruff format clean
  • mypy src/ --ignore-missing-imports clean (354 files)
  • python scripts/pre_ship.py --only versions — every version-bearing file at 3.9.0
  • python scripts/gen_config_docs.py --check — reference up-to-date
  • Manual: smem 3.9.0 installed via pipx on the production host; smem doctor green against the existing stack (default config ⇒ behavior unchanged)

Verified by

@acidkill

…edder

reasoning_training gains allow_remote_endpoints (default false, env
SURREAL_MEMORY_REASONING_ALLOW_REMOTE). With the flag off nothing changes:
non-loopback endpoints still resolve to no namer and no embedder, so
reasoning traces never leave the machine. With the flag on, the naming LLM
endpoint, the distiller's embedding provider and the stop hook's semantic
dedup accept a configured remote http(s) gateway instead of loopback-only
servers; non-http(s) schemes stay refused either way and the unconfigured
probe fallback stays strict.

The naming request now authenticates: SURREAL_MEMORY_LLM_API_KEY is sent as
a Bearer header when set, loopback servers keep going keyless with no
Authorization header, and a remote endpoint without a key warns at namer
build time instead of failing.

Bump 3.9.0.
@acidkill
acidkill marked this pull request as ready for review September 3, 2026 06:30
@acidkill
acidkill merged commit 0f88f77 into main Sep 3, 2026
9 checks passed
@acidkill
acidkill deleted the feature/remote-reasoning-endpoints branch September 3, 2026 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant