You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
stop() neither cancels nor waits for an in-flight MCPL inference/request #418
InferenceRouter doesn't pass an AbortSignal to membrane, and it keeps its stream in a local variable (main e6a0317). The framework's inference-request handler in src/framework.ts awaits the router without registering the operation anywhere stop() looks. stop() cancels and awaits agent streams only, then closes the MCPL servers.
Why it matters
After shutdown, the provider call keeps running, and keeps billing, until it finishes or times out. Its answer is then dropped on a closed connection. This is the cost of restarting a host while a server-initiated inference is running, and bounding it will matter more once #378's progress deadline lands.
How it was found
Codex flagged it while reviewing #380, and the reviewers confirmed it in source. #380 lists it as out of scope. It relates to #373 and #335 but is not covered by either.
Suggested fix
Track in-flight router inferences with an abort controller, and have stop() abort and await them before it closes the servers.
Activity
fix(mcpl): stop() ends a server's inference request still running, an…
Merge pull request #895 from nissa-seru/nigel2654/f97-inference-abort