Skip to content

Use explicit Transfer for tail Resume handlers#357

Open
proboscis wants to merge 2 commits into
mainfrom
fix/vm-tail-resume-transfer
Open

Use explicit Transfer for tail Resume handlers#357
proboscis wants to merge 2 commits into
mainfrom
fix/vm-tail-resume-transfer

Conversation

@proboscis
Copy link
Copy Markdown
Owner

@proboscis proboscis commented Mar 20, 2026

Summary

  • revert the do.py bytecode-based tail-Resume rewrite entirely
  • switch the memory-leak regression handlers to explicit Transfer in tail position
  • add DOEFF032 to the linter to flag return (yield Resume(k, ...)) and suggest yield Transfer(k, ...)
  • add a public API regression that makes Resume semantics explicit: handlers unwind only after the remainder continuation completes

Why

The new weakref probe looked like finished handler generators were being retained by the VM, but stepping the runtime showed that is not what is happening.

Resume is non-terminal: the handler does not finish per iteration. It suspends on the remainder continuation and only receives a value back when that continuation completes. Existing public API behavior already depends on this (resume_value = yield Resume(...) post-processing).

That means looped tail code such as return (yield Resume(k, payload)) will intentionally stack suspended handler frames until the whole run completes. The safe fix is to use explicit Transfer for true tail-position handlers rather than trying to rewrite VM ownership or add bytecode-dependent magic.

Evidence

Acceptance checks for vm-memory-leak were re-run on this branch:

  • tests/test_vm_memory_leak.py with explicit Transfer
    • 4 passed in 4.01s
  • tests/test_try_finally_in_do.py
    • 8 passed in 0.05s
  • tests/public_api/test_types_001_handler_protocol.py::TestHP03PostProcess
    • 2 passed in 0.04s
  • cargo test in packages/doeff-linter
    • 300 passed

Runtime probe demonstrating actual Resume semantics:

handler before resume 1
program saw 1
handler before resume 2
program saw 2
handler after resume 2 -> done
handler after resume 1 -> done

This shows the earlier handler frames stay suspended because the continuation has not completed yet, not because finished generators are leaked by stale VM references.

Tests

  • /home/kento/.orch/worktrees/vm-memory-leak/23039e_codex_20260321-015536/.venv/bin/python -m pytest tests/test_vm_memory_leak.py -q -s
  • /home/kento/.orch/worktrees/vm-memory-leak/23039e_codex_20260321-015536/.venv/bin/python -m pytest tests/test_try_finally_in_do.py -q
  • /home/kento/.orch/worktrees/vm-memory-leak/23039e_codex_20260321-015536/.venv/bin/python -m pytest tests/public_api/test_types_001_handler_protocol.py::TestHP03PostProcess -q
  • cargo test in packages/doeff-linter

Refs: vm-memory-leak

@proboscis proboscis changed the title Release tail Resume handlers promptly Use explicit Transfer for tail Resume handlers Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant