Repository navigation
feat(sdk)!: pause takes force and returns whether the sandbox was paused and whether forcefully - #1983
feat(sdk)!: pause takes force and returns whether the sandbox was paused and whether forcefully#1983tomassrnka wants to merge 3 commits into
Conversation
|
API counterpart: https://github.com/e2b-dev/belt/pull/3536 (the pause contract: refusal when the guest agent is unresponsive with the sandbox left running, the force override, the 200 result). |
🦋 Changeset detectedLatest commit: 891dec7 The changes in this PR will be included in the next version bump. This PR includes changesets to release 7 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
PR SummaryHigh Risk Overview Reviewed by Cursor Bugbot for commit 891dec7. Bugbot is set up for automated code reviews on this repo. Configure here. |
Package ArtifactsBuilt from dbda15a. Download artifacts from this workflow run. JS SDK ( npm install ./e2b-dockerfile-utils-0.1.2-feat-pause-force.0.tgz ./e2b-2.56.1-feat-pause-force.0.tgzCLI ( npm install ./e2b-cli-2.22.1-feat-pause-force.0.tgzCode Interpreter JS SDK ( npm install ./e2b-code-interpreter-2.9.1-feat-pause-force.0.tgzDesktop JS SDK ( npm install ./e2b-desktop-2.5.1-feat-pause-force.0.tgzPython SDK ( pip install ./e2b_dockerfile_utils-0.1.1+feat.pause.force-py3-none-any.whl ./e2b-2.56.0+feat.pause.force-py3-none-any.whlCode Interpreter Python SDK ( pip install ./e2b_code_interpreter-2.11.0+feat.pause.force-py3-none-any.whlDesktop Python SDK ( pip install ./e2b_desktop-2.7.0+feat.pause.force-py3-none-any.whl |
ed2b77a to
fa5a195
Compare
|
On the red checks at ed2b77a: Generated files is red by design until spec/runtime-ref carries the new contract. The four Staging SDK jobs fail on the template build timeout and the run-command-as-missing-user message in both SDKs, which the three most recent completed PR runs on this repository (38414259158, 38411904011, 38391379064) fail identically, so they predate this branch. The one difference from baseline is Production Python's paginate-paused-sandboxes test (one paused sandbox listed instead of two right after two pauses); its async twin passed in the same job and this branch does not touch listing or the production pause contract, which still answers 204 and is mapped to paused true; I have re-run the failed jobs to discriminate a listing flake from a cause. |
|
Re-run at fa5a195: the Production legs are green, including the auto-resume and file-watch tests that failed once, so those were flakes; what stays red is Generated files (by design until the runtime pin moves) and the four Staging legs failing on the pre-existing template-build and run-command tests. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa5a19561d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| force: | ||
| type: boolean | ||
| default: false |
There was a problem hiding this comment.
Advance the source pin before changing the synced spec
Because spec/runtime-ref is unchanged, the Generated files workflow will run make codegen, re-fetch this file from the old pinned runtime commit, and report the resulting spec/client changes as uncommitted, so every PR containing this commit deterministically fails that required check. Wait for the upstream runtime spec, bump the pin, and regenerate rather than applying the pause schema directly.
AGENTS.md reference: AGENTS.md:L4-L4
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed, and this is the known sequencing constraint rather than an oversight: the pause contract lands in belt 3536 first, mirrors to runtime, then this PR bumps spec/runtime-ref and runs make codegen, which replaces the hand-applied hunks with the generated ones and turns Generated files green. The hunks are kept in the meantime so reviewers can read the contract the SDK code is built against; the PR body states the sequence and the PR does not merge before it. Leaving this thread open as the tracker until the pin moves.
There was a problem hiding this comment.
Note
Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| force: | ||
| type: boolean | ||
| default: false | ||
| description: >- | ||
| Take the pause even when the sandbox guest agent is unresponsive. | ||
| Applies to memory pauses only. The memory snapshot may not resume; | ||
| a reboot resume recovers the disk state. Defaults to false. | ||
|
|
||
| SandboxPauseResult: | ||
| type: object | ||
| required: | ||
| - forced | ||
| properties: | ||
| forced: | ||
| type: boolean | ||
| description: >- | ||
| Whether the pause was taken with the force override, skipping the | ||
| guest agent health check. |
There was a problem hiding this comment.
🔴 The next make codegen at the current pins will silently undo the pause contract in both generated clients. CLAUDE.md forbids editing spec/ by hand. This hunk adds force, SandboxPauseResult and the 200/400 responses, but leaves spec/runtime-ref unchanged. Codegen re-fetches the spec at that pinned commit, which does not have these hunks. Fix: keep spec/openapi.yml and the generated files byte-equal to codegen at the pinned refs. Land this once runtime-ref carries the belt change, or bump the pin in this PR. [also at: spec/openapi.yml:3338 - Maintainers get a hand-edited API spec that the next sync will overwrite, which breaks the root CLAUDE.md rule against editing spec/ by hand.]
Why this was flagged
spec/runtime-ref still pins 90ad3bdbb1a4ad78161463dc90b66b4f8303897b. Per CLAUDE.md, make codegen re-fetches spec/openapi.yml at that commit before generating, and the PR description says that spec does not have the pause hunks. So the next contributor who runs codegen for an unrelated spec change drops SandboxPauseRequest.force, SandboxPauseResult and the pause 200 response from spec/openapi.yml, schema.gen.ts and post_sandboxes_sandbox_id_pause.py. In JS, res.data?.forced at packages/js-sdk/src/sandbox/sandboxApi.ts:1678 then no longer typechecks. In Python, the regenerated client goes back to parsing only 204, so a 200 pause loses its body and forced comes back as False. The PR author says the generated-files check stays red until the pin moves. That confirms the drift; it does not prevent it. The base has no such mismatch.
Verification: The diff hand-edits spec/openapi.yml, but spec/runtime-ref is not touched and still reads 90ad3bdbb1a4ad78161463dc90b66b4f8303897b. The Makefile defines codegen: fetch-specs, which overwrites spec/openapi.yml with the copy at the pinned commit. So the next make codegen at this pin drops force, SandboxPauseResult and the 200 response from spec/openapi.yml and packages/js-sdk/src/api/schema.gen.ts.
There was a problem hiding this comment.
Agreed, and this is the known sequencing constraint rather than an oversight: the pause contract lands in belt 3536 first, mirrors to runtime, then this PR bumps spec/runtime-ref and runs make codegen, which replaces the hand-applied hunks with the generated ones and turns Generated files green. The hunks are kept in the meantime so reviewers can read the contract the SDK code is built against; the PR body states the sequence and the PR does not merge before it. Leaving this thread open as the tracker until the pin moves.
fa5a195 to
10864ed
Compare
|
Pushed 10864ed after a full adversarial pass: the error message is the API's verbatim with the guidance moved to the class docs; Python tests now run the generated client's real response parsing for 200, 204 and the refusal; the CLI gains --force and reports a forced pause; the error-code constant is exported from both package roots; the Python transports read only 503 bodies and tolerate a non-dict body; the JS retry predicate bounds the body read by the retry budget. The spec hunks stay hand-applied for readability; the PR body states the sequence that turns Generated files green (belt 3536 merges, the runtime mirror lands, bump spec/runtime-ref, make codegen). |
10864ed to
1b4eff5
Compare
|
Pushed 1b4eff5: the JS retry predicate owns its reader, caps the peek at 8 KiB and bounds it by the retry delay; the Python transports read a 503 body only when small and inside the retry's error handling; the CLI prints a refused pause as one line; the changeset marks the four re-exporting dependents major. Both regressions from the last pass have tests that fail on the previous head. |
1b4eff5 to
d575ecb
Compare
|
Pushed d575ecb: the dripping-body test source is cancel-safe, so the JS unit run exits clean in every runtime again; the peek returns not-a-refusal before touching a consumed, locked or missing body, a non-JSON body or an already-aborted request; the untouched-body assertions sit on a response that was actually peeked; the peek waits through the injected sleep with a 50 ms floor; Python subtracts the body-read time from the following sleep and re-checks the deadline. |
d575ecb to
cdeda30
Compare
|
Pushed cdeda30 for the three findings on the retry peek: a bounded stream read in Python with no length-header dependency, an inspection budget independent of the backoff in JS, and a case-insensitive media-type check in both. |
cdeda30 to
6396fcb
Compare
|
Pushed 6396fcb: a 503 whose body read fails and whose budget is then exhausted is returned to the caller as the JS side does, with tests sync and async; the CLI passes force only when the flag is given, following the SDK's omit path. |
6396fcb to
504f271
Compare
|
Pushed 504f271. The last pass caught two Python regressions that the fakes had masked: the rebuilt refusal response read a request httpx attaches only later, and an empty-body 503 broke the generated parser. Both fixed; the pause tests now run through the SDK's own retry transport over fakes shaped like the production transport (no pre-attached request), and the previous head fails that test with the runtime error while this one passes. Also in: the wall-clock cap on the Python peek, the rebuilt response dropping transfer-encoding and the trailers extension, and the retry docs stating the inspection budget. |
504f271 to
9a7e507
Compare
|
Pushed 9a7e507: the Python lint format failure fixed, the exhausted-budget outcomes distinguished (a complete body is returned, an over-cap body is handed on whole through a resumed stream so the caller still reads the refusal, a budget cut-off raises the retry timeout, a read error re-raises), each tested sync, async and through Sandbox.pause; the peek closes its generator explicitly when it gives up. |
9a7e507 to
4dc4424
Compare
|
Pushed 4dc4424: a peek cut off by the wall-clock budget at an exhausted retry budget is handed on whole through the resumed stream, as the over-cap case and the JS side do; the leftover iterator is closed deterministically on retry and on stream close; one rebuild path; tests cover the live-iterator replay, the header stripping on the stream form, the async complete body and the over-cap refusal through Sandbox.pause. |
|
Pushed 6eba48d: the Python peek treats a RecursionError from a pathological JSON body as not-a-refusal, closes the inner response on any BaseException out of the peek (cancellation tested), and the resumed stream closes the inner response in a finally; over-cap fixtures prove the byte-cap branch on its own; one rebuild path. |
6eba48d to
7f66599
Compare
|
Pushed 7f66599: a non-string error_code is treated as not-a-refusal; a running sync generator's close is tolerated; buffered chunks that already form the refusal are returned without pulling the tail; the handed-on tail is bounded by the request deadline. Consequence stated in the body: a refusal body still trickling at the request deadline ends in the read timeout rather than being read past the deadline. |
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
|
Final adversarial pass at 7f66599 scores this 9/10, mergeable once the known spec sequence runs (belt 3536 merges, the runtime mirror lands, bump spec/runtime-ref, make codegen replaces the hand-applied hunks and turns Generated files green). Verified on a clean box: lint, format, type check, the non-live Python suite, the full JS unit project with zero unhandled errors, CLI build and tests, changeset plan (six majors, CLI minor), and the refusal, cut-off, over-cap, cancellation and cross-close paths against real httpx. Two notes for a later tidy, neither blocking: the resumed-stream classes are reachable only in a one-clock-read window now that the tail is deadline-bounded, so they could collapse to replay-then-timeout; and a trickling 503 at the deadline surfaces as httpx's ReadTimeout rather than the SDK's TimeoutException, which is the API client's pre-existing contract. |
…esult
What changed: spec/openapi.yml takes the pause contract from belt pull
request 3536 (head b65468a1a7, oss/runtime/spec/openapi.yml):
SandboxPauseRequest gains an optional boolean `force` (default false), a
successful pause answers 200 with the SandboxPauseResult object
`{"forced": boolean}` instead of 204, the endpoint documents 400 (force
on a filesystem-only pause), and the sandbox_pause_refused clause is
inserted into the existing Error.error_code list with belt's sentence
verbatim; the other codes belt's spec has gained are not pulled in. The
JavaScript schema and the Python client are regenerated from it with the
repo's generators (redocly 2.54.3 + openapi-typescript,
e2b-openapi-python-client 0.26.2 + ruff), at the pins in
codegen.Dockerfile and the python-sdk codegen group.
Why: the node may refuse a pause (503 with error_code
sandbox_pause_refused) and keep the sandbox running, and lifts the
guest-agent health check only when the caller passes force; the SDKs
need the field, the result type and the code to expose that.
Only the pause hunks are taken: belt's spec is ahead of the runtime pin
in spec/runtime-ref by unrelated changes (egress proxy TLS, pauseMode,
admin sandbox lookup, idempotency keys). spec/openapi.yml is normally
copybara-synced from e2b-dev/runtime at that pin, so the generated-files
check stays red until runtime-ref moves to a runtime commit that carries
this contract; bump the pin and re-run `make codegen` then.
Verification: the regenerated diff is limited to the pause endpoint,
SandboxPauseRequest.force, the new SandboxPauseResult model and the
Error.error_code clause; `pnpm build` (tsc --noEmit) and `ruff check` /
`ruff format --check` / `ty check` pass.
…ndboxPauseRefusedError for a refused pause
What changed: `SandboxPauseOpts.force` is sent in the pause request body
when set (omitted otherwise, like `memory`). `SandboxApi.pause()`,
`Sandbox.pause()` and the deprecated `betaPause()` return
`Promise<SandboxPauseResult>` (an interface `{ paused, forced }`,
exported) instead of a boolean: 200 answers
`{ paused: true, forced: body.forced }`, 409 (already paused)
`{ paused: false, forced: false }`, and a 204 from an older API
`{ paused: true, forced: false }`. The CLI's `sandbox pause` reads
`paused` from the result, takes `-f, --force` (passed only when given,
so the SDK omits it from the body otherwise), and says when the result
reports a forced pause.
A 503 whose body carries `error_code: "sandbox_pause_refused"` is raised
as the new `SandboxPauseRefusedError` (exported; `statusCode` 503,
`errorCode`, subclass of `ServiceBusyError` so existing handlers keep
working). The mapping lives in `apiErrorFromCode` / `handleApiError`,
keyed on the body's `error_code` and passed through for errors embedded
in fork results too, so any endpoint answering the code gets the typed
error; the pause call site keeps only the 409 → `paused: false` case.
The message is the API's verbatim: belt's messages already name the
reason and what to do for it (retry, retry without force, or pause with
force and the reboot caveat), so the SDK appends nothing and the
general guidance lives in the class docs. A 503 without the code stays
a `ServiceBusyError`, 404 stays `SandboxNotFoundError`, and 400 (force
on a filesystem-only pause) is the generic `SandboxError` carrying the
status. The code string is one constant,
`SANDBOX_PAUSE_REFUSED_ERROR_CODE`, exported from the package and read
by the error class, the mapping and the retry predicate.
The transport no longer replays a 503 carrying that code: `withRetry`
peeks at a 503 body and returns the response when its `error_code` is
non-retryable, so a refusal surfaces after one request instead of
re-running the node's health probe and restore cycle on each of the
retries. The peek runs only for a body whose media type is JSON
(lower-cased, parameters dropped), unconsumed and unlocked, on a
request not already aborted (anything else is "not a refusal" and
retried as before, instead of a synchronous TypeError failing the
request); it reads a clone through a reader it owns, capped at 8 KiB
and bounded by its own inspection budget of max(retry delay, 1 s)
within the remaining request budget — so a slow refusal body is still
caught when the first backoff is 50 ms, and a trickling non-refusal
body costs at most a second per attempt — and by the request signal,
waited through the injected sleep so RetryDependencies.sleep governs
all waiting (clamped so a huge budget cannot overflow setTimeout); when it stops early the reader is
cancelled so the abandoned read stops consuming the stream, the time
spent counts towards the delay, the deadline is re-checked, and the
503 is retried as before, so a stalled or trickling body cannot eat
the retry budget. The caller's body stays untouched. Other 503s are
retried as before. The CLI prints a refused
pause's API message (no stack) before exiting 1.
The docs on the option, the methods and the error class describe the
refusal as the node refusing the pause with the sandbox kept running and
the reason in the message (belt emits the code for an unresponsive guest
agent, a busy node and a forced pause it cannot pin), and present force
as the answer for the unresponsive-guest reason only, with the reboot
caveat.
Why: the API now refuses a memory pause when the sandbox guest agent is
unresponsive and keeps the sandbox running; callers need to tell that
apart from a busy node (retry) and need the override to snapshot
anyway, accepting that the memory may not resume (a resume with
`onResume: 'reboot'` recovers the disk state). The result object is a
breaking change, decided by the operator: `forced` is informative, not
an echo of the option, since the API reports `forced: false` when a
forced request joined a normal pause already in flight, and a boolean
cannot carry it. Migration: read `paused` where the boolean was read.
Verification: unit tests (msw-mocked API) cover force omitted / true /
false / with mode in the body; 200 with forced false and true; forced
reported false for a forced request that joined a normal pause; 204
compatibility; 409; the instance method; the coded refusal mapped to
SandboxPauseRefusedError (still instanceof ServiceBusyError) with the
API message verbatim, after exactly one request; the busy-node
refusal; a 503 with no error_code staying ServiceBusyError; 400
carrying the status; 404. handleApiError tests cover the central
mapping on the code regardless of endpoint and a 503 with another code
staying ServiceBusyError. The transport tests assert a coded 503 is
returned after one request with its body intact, a 503 with another
code is still retried, a stalled body is retried after each attempt's
inspection budget and delay (sleeps 1000/50, 1000/100, 1000/200 ms on
the stub clock, four requests), a refusal arriving within the delay and
one arriving after the first backoff but within the inspection budget
both short-circuit, a mixed-case Content-Type is matched by media type, a dripping body (cancel-safe source) is
retried with the peek's reader cancelled (the retried sources
cancelled, no further pulls past the tee's high-water mark; the
untouched-body assertions sit on the peeked-and-returned refusal), an
oversized body is a plain 503, and a consumed body, a locked body and
an already-aborted request are not peeked at. The whole js-sdk unit
project runs with zero unhandled errors; its only failures without an
API key are the live-sandbox suites. CLI tests cover the default, --force /
-f, a forced request the API did not force, a refused pause printing
the API message and exiting 1, and an already paused sandbox. `pnpm
build`, oxlint, prettier, the CLI typecheck and lint, and the unit
suites pass (249 js-sdk tests across the five touched files, 5 CLI
tests).
…; SandboxPauseRefusedException for a refused pause What changed: `Sandbox.pause()` and `AsyncSandbox.pause()` (instance and class forms, and the deprecated `beta_pause` forms, matching the JavaScript `betaPause`) take `force: Optional[bool] = None`, sent in the pause request body when set and omitted otherwise, like `mode`, and return the `SandboxPauseResult` dataclass (`paused`, `forced`), exported from `e2b` under its own section, instead of a bool: 200 gives `paused=True, forced=body.forced`, 409 (already paused) `paused=False, forced=False`, and a 204 from an older API `paused=True, forced=False`. A 503 whose body carries `error_code: "sandbox_pause_refused"` is raised as the new `SandboxPauseRefusedException` (exported; `status_code` 503, the `error_code` attribute, subclass of `ServiceBusyException` so existing handlers keep working). The mapping lives in `api_exception_from_code` / `handle_api_exception`, keyed on the body's `error_code` and passed through for errors embedded in fork results too, so any endpoint answering the code gets the typed exception; a non-object JSON body (a list) carries neither field and maps as before; the pause call site keeps only the 409 → `paused=False` case. The message is the API's verbatim: belt's messages already name the reason and what to do for it, so the SDK appends nothing and the general guidance lives in the class docstring. A 503 without the code stays a `ServiceBusyException`, 404 raises `SandboxNotFoundException`, and 400 (force on a filesystem-only pause) is the generic `SandboxException` carrying the status. The code string is one constant, `SANDBOX_PAUSE_REFUSED_ERROR_CODE`, exported from `e2b` and read by the exception class, the mapping and the retry predicate. `RetryableTransport` / `AsyncRetryableTransport`, which retry 503s the same way the JavaScript transport does, peek at the body only for a 503 whose media type is JSON (lower-cased, parameters dropped) and that is not declared larger than 8 KiB, streaming it through iter_bytes / aiter_bytes under the existing try with that cap and a wall-clock budget of min(max(delay, 1 s), remaining request budget) measured with the transport's monotonic clock across chunks (each chunk is also bounded by the request's read timeout as pyqwest applies it, none when the request timeout is disabled; a chunk whose wait began within the budget is kept, only waiting for more past it gives up), so a chunked or HTTP/2 refusal without Content-Length is read and a trickling body cannot hold the loop: a read that fails, times out, exceeds the cap or outlives the budget decides nothing and the 503 is closed and retried, where an earlier draft let the exception escape after one request; the time the read took is subtracted from the following sleep and the deadline re-checked, as the JavaScript side does; with no retry budget left, the 503 is the outcome as on the JavaScript side: a body read in full is returned as is, one stopped at the byte cap or cut off by the peek budget is handed on whole (the peeked chunks first, then the rest of the stream, through a resumed stream whose close releases the body iterator and the original response), and a failed read re-raises its error, never a fabricated body (the API client parses every 503 body as JSON). While budget remains, the iterator a stopped peek left is closed explicitly before the retry, so nothing is left to garbage collection; the rebuilt response is constructed once, with httpx taking the stream over the content when both are given. A body nested too deep for json.loads (RecursionError, not ValueError) is not a refusal; a BaseException out of the peek (cancellation, KeyboardInterrupt) closes the inner response before propagating, so the connection is not leaked; the resumed stream closes the inner response in a finally and tolerates a body generator a reader is still running (RuntimeError for an async generator, ValueError for a sync one; left to that reader). An ``error_code`` that is not a string (a list or object from a proxy's or a buggy server's body) is not a refusal rather than a TypeError out of the transport. A peek cut off by the budget on what turns out to be the last chunk holds one complete JSON document: it is treated as the body read in full (a refusal is returned at once; any other complete API error follows the normal path, retried while budget remains and returned whole when it is exhausted) and the tail is left unread; only a body that does not yet parse is handed on. Without this a busy 503 whose body completed in the cut-off chunk surfaced as ReadTimeout instead of ServiceBusyException. The handed-on stream's live tail is bounded by the request deadline: a chunk whose wait would begin past it is not waited for, the inner response is closed and ReadTimeout is raised, so the API client's read ends at the deadline instead of running on chunk by chunk past the request timeout. Because the peek's time counts against the retry delay, a budget exhausted after a peek means the clock is at the deadline, so in practice an over-cap or cut-off body handed on at that point yields what the peek buffered and then the timeout; a body read in full, or buffered chunks that already form the refusal, are returned as content and never reach the tail. The two readers' cross-close tests therefore build the resumed streams directly with the deadline ahead. A refusal is returned after one request as a response rebuilt from the bytes read: the transport's response carries no request yet (httpx attaches it after the transport returns; pyqwest's arrive without one), so the handled request is attached explicitly, Content-Encoding, Transfer-Encoding and Content-Length are dropped (the content is decoded and complete; httpx sets the length), and only plain extensions such as http_version are copied, never pyqwest's get_trailers bound to the closed native response. The earlier draft read response.request and raised RuntimeError on the production transport for every refused pause; the pause tests missed it because their fakes attached request= and bypassed the retry layer. `@e2b/code-interpreter`, `@e2b/desktop`, `@e2b/code-interpreter-python` and `@e2b/desktop-python` take the same major in the changeset: the JavaScript two re-export `e2b` and subclass its Sandbox, the Python two subclass the Python Sandbox, and `updateInternalDependencies: patch` would otherwise ship the breaking `pause()` in a patch of theirs. The two Python dependents' `e2b` bound is raised from `<3.0.0` to `<4.0.0` so they keep resolving against e2b 3.x (their locks are unchanged: they resolve the workspace e2b). `pnpm changeset status --verbose` lists exactly those six majors and the CLI minor. Docstrings on every overload and on the exception describe the refusal as the node refusing the pause with the sandbox kept running and the reason in the message, and present force as the answer for the unresponsive-guest reason only, with the reboot caveat. The two live tests that asserted the old bool read `.paused`. The changeset is major for both SDKs (minor for the CLI) with the migration note. Why: the API now refuses a memory pause when the sandbox guest agent is unresponsive and keeps the sandbox running; callers need to tell that apart from a busy node (retry) and need the override to snapshot anyway, accepting that the memory may not resume (a resume with `on_resume="reboot"` recovers the disk state). The result object is a breaking change, decided by the operator: `forced` is informative, not an echo of the option (the API reports `forced=False` when a forced request joined a normal pause already in flight). `force` is `Optional[bool]` rather than `bool = False` so an unset flag is omitted from the payload (#1749: no SDK-side defaults in API request bodies). The generated 200 model is imported as `SandboxPauseResultAPI` to keep the public dataclass name. Verification: unit tests with the generated client mocked cover force omitted / true / false / with mode in the body; 200 with forced False and True; forced reported False for a forced request that joined a normal pause; 204 compatibility; 409; the coded refusal raising SandboxPauseRefusedException (a ServiceBusyException) with the API message verbatim; the busy-node refusal; a 503 with no error_code staying ServiceBusyException; 400 carrying the status; 404, for sync and async. Tests over an httpx.MockTransport-backed ApiClient / AsyncApiClient now run through RetryableTransport / AsyncRetryableTransport with responses built as httpx and pyqwest hand them over (no request=): the refusal-through-real-parsing tests fail on the earlier draft with RuntimeError and pass here; they also cover 200 with forced True and False, the 204 older-API path (parsed is None) and, through Sandbox.pause, a refusal body read that times out with no budget left surfacing as the timeout. The transport fakes lost their request= too. The transport tests assert a coded 503 is returned after one request with its body intact, a chunked refusal without Content-Length is returned after one request with its body readable (sync and async), an oversized chunked body and a non-JSON body are retried, a trickling body is retried once the peek budget is spent (sync and async), a complete but slow body read with no budget left is returned with its body (sync and async), an over-cap body with no budget left surfaces the 503 whole with its third chunk read only by the resumed stream and no length or transfer-encoding header (sync and async, and through Sandbox.pause as SandboxPauseRefusedException), a read cut off by the budget with no budget left is handed on whole too (sync and async, and through Sandbox.pause as the typed refusal), a 3000-deep JSON body is retried (sync and async), cancelling the peek while it awaits a chunk closes the inner response exactly once, and closing the handed-on response while a reader awaits its live tail closes the inner exactly once (an asyncio task, and a thread inside iter_bytes), a list-valued error_code is retried (sync and async), and a cut-off peek whose buffered chunks form the refusal returns it without retrying and without pulling the tail (sync and async, and through Sandbox.pause and AsyncSandbox.pause), a busy 503 whose complete body lands in the cut-off chunk is retried while budget remains and surfaces as ServiceBusyException through Sandbox.pause and AsyncSandbox.pause when none is left, the over-cap deadline cut is covered through AsyncSandbox.pause too, a handed-on tail is cut at the deadline with ReadTimeout and the inner closed once (transport sync and async, and through Sandbox.pause for both the cut-off and the over-cap case), a failed read with no budget left re-raises (sync and async), a mixed-case Content-Type is matched by media type, the read time counts towards the delay, the rebuilt refusal drops the transfer headers and native extensions and carries the request, a 503 with another code is still retried, sync and async. `ruff check`, `ruff format --check`, `ty check` and the non-live pytest suite pass (670 tests); `make lint` exits 0 and a second `ruff format` leaves every file unchanged.
7f66599 to
891dec7
Compare
|
Pushed 891dec7 for the last point: a complete API error body that lands in the cut-off chunk is surfaced as that error rather than a read timeout; async twins of the over-cap and cut-off pause tests added. On the two production legs that went red at 7f66599: the JS node leg failed an env-var test with a bare fetch failure and the code-interpreter Python leg a context removal with a connection reset, both live-sandbox network errors on paths this change does not touch. |
|
Confirming pass at 891dec7: the 9/10 holds. The complete-JSON shortcut is prefix-safe for object bodies (the API error schema is always an object), the 8 KiB bound precedes parsing, and the busy 503 takes the ordinary retry or return-whole path. Three docstring sites still describe the earlier hand-on-whole wording and the shortcut could require a dict rather than any JSON value; both are tidy-ups for a later pass, not blockers. |
Summary
pause()takes a newforceoption and returns a result object instead of a boolean, in both SDKs, following the API's new pause contract: the node may refuse a pause (503,error_code: "sandbox_pause_refused") and the sandbox keeps running, with the reason in the message. When the reason is an unresponsive sandbox guest agent,forcetakes the snapshot without the health check. A successful pause now answers200 {"forced": boolean}instead of204.Depends on the API change in belt pull request 3536; the generated-files check stays red until the runtime pin carries the new spec.
What changed
Generated clients (
spec/openapi.yml, JS schema, Python client)spec/openapi.ymlis never hand-edited in this repo: it is copybara-synced from e2b-dev/runtime atspec/runtime-ref, and the requiredGenerated filescheck regenerates from that pin. This PR nevertheless carries the pause hunks of belt'soss/runtime/spec/openapi.yml(pull request 3536 head) hand-applied to the tracked spec so reviewers can read the contract next to the SDK code:SandboxPauseRequest.force(default false), theSandboxPauseResultschema, the endpoint's200and400responses, and thesandbox_pause_refusedclause inserted into the existingError.error_codelist (belt's wording verbatim; the other codes belt's spec has gained are not pulled in). Both clients are regenerated from it with the repo's generators at their pinned versions; the regenerated diff is limited to those items.The
Generated filescheck stays red until the pin carries the contract. Sequence to green: (1) belt 3536 merges; (2) the belt→runtime mirror lands the spec in e2b-dev/runtime; (3) bumpspec/runtime-refto that runtime commit; (4) runmake codegen, which re-fetches the spec at the pin — replacing the hand-applied hunks with the synced file — and regenerates both clients; commit the result. If the synced spec differs from the hand-applied hunks only by the unrelated changes runtime has gained meanwhile, the generated clients still change only where those land.JavaScript SDK (
e2b)SandboxPauseOpts.force?: boolean, sent in the request body when set and omitted otherwise.SandboxApi.pause()/Sandbox.pause()(and the deprecatedbetaPause) returnPromise<SandboxPauseResult>withpausedandforced(type exported).200→{ paused: true, forced: body.forced },409→{ paused: false, forced: false },204from an older API →{ paused: true, forced: false }.SandboxPauseRefusedError(subclass ofServiceBusyError,statusCode503,errorCode) for the refused pause, mapped centrally inapiErrorFromCode/handleApiErroron the body'serror_code(so any endpoint answeringsandbox_pause_refusedgets it, including errors embedded in fork results); its message is the API's verbatim, which already names the reason and what to do for it (the SDK appends nothing; the general guidance lives in the class docs). A 503 with noerror_codestaysServiceBusyError; 409 is still handled at the pause call site; 404 staysSandboxNotFoundError; 400 (force on a filesystem-only pause) isSandboxErrorwithstatusCode400. The code string is one constant,SANDBOX_PAUSE_REFUSED_ERROR_CODE, exported from the package and read by the error class, the mapping and the retry predicate;SandboxPauseResultis an interface.error_code: "sandbox_pause_refused"is not replayed bywithRetry(other 503s are retried up toretriestimes), so a refusal surfaces after one request instead of re-running the node's health probe four times. The body is peeked at only when its media type is JSON (case-insensitive, parameters ignored), unconsumed and unlocked and the request is not already aborted, through a reader the predicate owns, capped at 8 KiB and bounded by its own inspection budget of 1 s (at least the retry delay, within the remaining request budget, so a slow refusal body is still caught and a trickling non-refusal body costs at most a second per attempt) and the request signal, waited through the injectedsleepso one clock governs all waiting; a stalled or oversized body is not a refusal, the reader is cancelled so the abandoned read stops consuming the stream, the time spent peeking counts towards the delay, and the 503 is retried as before. The original body stays untouched for the caller.e2b sandbox pausetakes-f, --force, passes it only when given (so the SDK omits it from the request body otherwise), says when the result reports a forced pause, and prints a refused pause's API message (no stack) before exiting 1.Python SDK (
e2b)pause(..., force: Optional[bool] = None)onSandboxandAsyncSandbox(instance and class forms); unset is omitted from the payload.SandboxPauseResultdataclass (paused,forced), exported frome2b, with the same mapping as the JavaScript SDK.SandboxPauseRefusedException(subclass ofServiceBusyException,status_code503,error_code), mapped centrally inapi_exception_from_code/handle_api_exceptionon the body'serror_code, with the API's message verbatim; a non-object JSON body is treated as carrying neither field. The code string is one constant,SANDBOX_PAUSE_REFUSED_ERROR_CODE, exported frome2b. The other mappings are unchanged. The deprecatedbeta_pauseforms takeforcetoo, matching the JavaScriptbetaPause.RetryableTransport/AsyncRetryableTransport(which retry 503s like the JavaScript SDK) peek at a 503 body only when its media type is JSON (case-insensitive, parameters ignored) and it is not declared larger than 8 KiB, streaming it under the existingtrywith an 8 KiB cap and a wall-clock budget of min(max(delay, 1 s), remaining request budget) measured with the transport's monotonic clock across chunks (each chunk is also bounded by the request's read timeout as pyqwest applies it; none when the request timeout is disabled) — a chunk whose wait began within the budget is kept, only waiting for more past it gives up; a failed, timed-out, over-cap or over-budget read decides nothing and the 503 is closed and retried; with no retry budget left the 503 is the outcome, as on the JavaScript side: a body read in full is returned as is, one stopped at the byte cap or cut off by the wall-clock budget is handed on whole (peeked chunks first, then the rest of the stream — as before this PR), and a failed read re-raises its error — never a fabricated body, since the API client parses every 503 body as JSON. While budget remains, the iterator a stopped peek left is closed explicitly before the retry. A body nested too deep to parse is not a refusal; an interruption mid-peek (cancellation,KeyboardInterrupt) closes the inner response before propagating; the handed-on stream closes the inner response in afinallyand tolerates a body generator a reader is still running; a non-stringerror_codeis not a refusal rather than aTypeError; a peek cut off by the budget on the last chunk holds one complete JSON document and is treated as the body read in full (a refusal is returned at once, any other complete API error such as a busy 503 follows the normal path), the tail left unread; only a body that does not yet parse is handed on. A handed-on stream's live tail is bounded by the request deadline (a chunk whose wait would begin past it is not waited for; the inner response is closed andReadTimeoutraised), so the API client's read ends at the deadline instead of running on chunk by chunk — and since a retry budget can only be exhausted after a peek when the clock is already at the deadline, a hand-on in practice delivers what the peek buffered and then the timeout, while a body read in full or buffered chunks that already form the refusal come back as content. A refusal is returned as a response rebuilt from the bytes read (the transport's response carries no request yet, so the handled request is attached explicitly;Content-Encoding,Transfer-EncodingandContent-Lengthdropped; only plain extensions such ashttp_versioncopied, never pyqwest'sget_trailersbound to the closed native response), so the caller reads it as the API sent it. The pause tests overhttpx.MockTransportnow run through these transports with responses built the way httpx and pyqwest hand them over (norequest=), which is what exposed the missing-request failure.forcedreports what the API did rather than echoing the option: the API answersforced: falsewhen a forced request joined a normal pause already in flight. The refusal code covers more than an unresponsive guest agent (belt also emits it for a busy node and for a forced pause it cannot pin); the message carries the reason, and the docs on every method, overload and error class say so, with the reboot caveat for a forced snapshot (its memory may not resume;onResume: 'reboot'/on_resume="reboot"recovers the disk state).Migration
pause()returnedtruewhen the sandbox got paused andfalsewhen it was already paused. Readpausedfrom the result where that boolean was read:A truthiness check on the result object is always true, so every call site that branched on the old boolean must be updated. Changeset: major for
e2b(JavaScript) and@e2b/python-sdk, and — because@e2b/code-interpreterand@e2b/desktopexport * from 'e2b'and subclass itsSandbox, ande2b-code-interpreter/e2b-desktopsubclass the PythonSandbox— major for those four as well (.changeset/config.jsonhasupdateInternalDependencies: patch, which would otherwise ship the breakingpause()in a patch of theirs);@e2b/climinor. The two Python dependents'e2bbound is raised from<3.0.0to<4.0.0in theirpyproject.toml(locks regenerated withuv lock; they still resolve the workspacee2b) so they keep resolving againste2b3.x. Verified withpnpm changeset status --verbose.Verification
pnpm build(tsc --noEmit + tsdown), oxlint, prettier; vitest unit project ontests/sandbox/pause.test.ts,tests/sandbox/apiDefaults.test.ts,tests/sandbox/lifecycleRequest.test.ts,tests/api/handleApiError.test.ts,tests/retry.test.ts(msw-mocked API: force omitted/true/false/with mode in the body; 200 with forced true and false; forced reported false for a forced request that joined a normal pause; 204 compatibility; 409; coded refusal →SandboxPauseRefusedError, stillinstanceof ServiceBusyError, after exactly one request; the busy-node refusal; 503 with noerror_code→ServiceBusyError; 400 with the status; 404; at the transport, a coded 503 answered after one request with its body intact, a 503 with another code still retried, a 503 whose body stalls or drips retried after the delay with the peek's reader cancelled, a refusal arriving within the delay still short-circuiting, and an oversized body treated as a plain 503).ruff check,ruff format --check,ty check; pytest ontests/shared/sandbox/test_pause.py,tests/shared/sandbox/test_api_defaults.py,tests/test_api_exception.py,tests/test_rate_limit_retry_transport.py, same cases for sync and async, plus tests that go through the generated client's real response parsing over anhttpx.MockTransport(200 withforcedtrue and false, the 204 older-API path, a coded refusal), and a JSON-list 503 body still mapping toServiceBusyException.pnpm typecheck, oxlint, prettier; vitest ontests/commands/sandbox/pause.test.ts(default,--force/-f, forced-but-not-forced, already paused).