Skip to content

About

Experiments on feedback, working memory, and reusable skills for LLM agents.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

61 Commits

Folders and files

Repository files navigation

Sparse Signal Loop

Does feedback about one violated criterion help an LLM revise its answer, and how does that depend on conversation history, working memory, and carried skills? This harness compares chat agents with recursive language model (RLM) agents using a Python workspace on LongBench-Pro and Mini SWE Agent Plus. The recorded comparisons do not establish a consistent advantage for the RLM harness under single-criterion feedback.

Experiments and findings · Recorded runs · Run guide

Preview a run

With Python 3.11 or later and uv:

git clone https://github.com/latentpaths/sparse-signal-loop.git
cd sparse-signal-loop
uv sync --locked
./scripts/smoke.sh --dry-run

The preview prints the three LongBench-Pro phase configurations without inference calls. It writes local logs under outputs/smoke_logs/. The root installation includes all four environments and experiment CLIs.

LongBench-Pro · RLM LongBench-Pro · Mini-SWE · RLM Mini-SWE

What changes between conditions

Phase Comparison
0 Total-score versus single-criterion feedback, each with chat and RLM harnesses
1 Conversation memory versus files in the Python workspace
2 Carried SKILL.md files and chat reinjection controls

Judge acceptance, environment reward, and task scores are separate measurements. The research account defines them, links the underlying reports, and reports recorded rollout counts, errors, and incomplete conditions. Turn budgets and model settings differ across some conditions and phases.

Evaluate and inspect

Real evaluations consume inference and sandbox credits. Set PRIME_API_KEY in your shell or local .env, then run ./scripts/smoke.sh. Use --dry-run first. Keep new runs separate from retained outputs/ evidence.

See the run guide for individual phases, configuration, report regeneration, and remote compute. The research account explains the recorded comparisons and incomplete conditions.

Development

uv sync --locked --group dev
uv run --locked python -m unittest discover -s tests -v
uv run --locked ruff check src tests
uv run --locked ruff format --check src tests
uv build --wheel

Citation

@misc{bodadearticlesparsesignalloop,
  author = {Bodade, Ayush},
  title = {{Sparse Signal Loop}},
  year = {2026},
  month = {04},
  day = {11},
  date = {2026-04-11},
  howpublished = {Latent Paths},
  url = {https://www.latentpaths.com/writings/sparse-signal-loop},
  note = {Blog post}
}

About

Experiments on feedback, working memory, and reusable skills for LLM agents.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages