Skip to content

stop() neither cancels nor waits for an in-flight MCPL inference/request #418

Closed

Description

@Anarchid

What's wrong

InferenceRouter doesn't pass an AbortSignal to membrane, and it keeps its stream in a local variable (main e6a0317). The framework's inference-request handler in src/framework.ts awaits the router without registering the operation anywhere stop() looks. stop() cancels and awaits agent streams only, then closes the MCPL servers.

Why it matters

After shutdown, the provider call keeps running, and keeps billing, until it finishes or times out. Its answer is then dropped on a closed connection. This is the cost of restarting a host while a server-initiated inference is running, and bounding it will matter more once #378's progress deadline lands.

How it was found

Codex flagged it while reviewing #380, and the reviewers confirmed it in source. #380 lists it as out of scope. It relates to #373 and #335 but is not covered by either.

Suggested fix

Track in-flight router inferences with an abort controller, and have stop() abort and await them before it closes the servers.

Found during our review of #380: #380 (review)

Posted by the maintainer's desk assistant.

Activity

  1. added 2 commits that reference this issue on Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions