backend: agents/service/workflows hardening, plus the hosting seam the routes now ask - #152
Draft
kai-openswarm wants to merge 7 commits into
Draft
backend: agents/service/workflows hardening, plus the hosting seam the routes now ask#152kai-openswarm wants to merge 7 commits into
kai-openswarm wants to merge 7 commits into
Conversation
… a window, so reducer tests run under node:test Both modules touched window at import time (port/host derivation, the fetch interceptor install, the debug handle), so any node:test file that imports a reducer importing API_BASE died with 'window is not defined' before its first assertion; fetchSessionsStrip.test.ts has been red that way since the resilience work landed, unnoticed because nothing runs these tests in CI. In a renderer (window present) nothing changes: same port/host, same interceptor, same handle. Without one the module answers with the defaults and installs nothing.
Nothing ran any of them in CI: the 235-file backend pytest suite, the 22 renderer node:test files and the edge suite were run by hand, one file at a time, so a regression only surfaced when someone happened to run the right one. Three small workflows, hosted ubuntu, path-filtered, read-only token: - backend-tests: pytest on Python 3.13 from the locked requirements, plus a completion assertion (junit testcase count == collect-only count) so a test process that dies mid-run can never read as green - frontend-tests: tsc --noEmit + node:test via tsx over src/**/*.test.ts(x), through frontend/scripts/run-tests.mjs (the runner the tests already name) - edge-tests: pytest for openswarm-edge All three are green on the current tree: 2951 backend tests, 143 frontend tests across 22 files, 14 edge tests.
Two of five hosted runs of the backend suite stalled at 99% until the job cap with no summary and no junit: one test blocked forever on a bare ws.receive_json() (fixed on its own in a separate change). A CI lane should never depend on every test being unable to hang, so add pytest-timeout to the dev requirements and run the suite with --timeout=300. On Linux the default signal method fails just the offending test and the run continues, so the report and the "every collected test ran" assertion stay meaningful.
test_ws_endpoint_streams_a_full_turn_end_to_end read the socket with a bare
ws.receive_json() in a 40-iteration loop and broke only on the assistant reply.
When the loop ends early for any reason (fewer than 40 events, no reply), the
next receive blocks forever and the whole pytest run stalls at 99% until the
job cap. On hosted runners it does exactly that intermittently, on Linux and
Windows alike: the turn path's configure_provider_env decides whether 9Router
needs reviving from provider evidence earlier tests may leave behind, and that
revival spawns/installs the router behind a module-level asyncio.Lock; the
background turn-label aux call reaches the same machinery. Neither is part of
this test's contract ("SDK and WS auth mocked, everything else real").
Pin both out with monkeypatch, bound every receive at 5s (a regression now
fails this test instead of hanging the runner), and wait for the turn's
completed status before asserting on session.messages so the assertion cannot
race the loop's tail.
Same class as the config/backendConnection change: safeMode.ts read `window` at import, and dashboardLayoutSlice imports it, so any reducer test that imports the slice died under node:test before it ran. Guard the read; in a renderer nothing changes.
…e routes now ask
The agent manager, its routes, and the service/workflows/settings apps, with the
hardening a hosted deployment forced us to build but that stands on its own:
- Containment (R0): a session's browser commands, tool results and approvals are
bound to their session and owner (ws_manager: send/resolve carry owner and
claimant, connections carry an optional identity with expiry and revocation);
the sessions list, history and prompt prediction never read across owners.
- Redaction (F1): settings leave the process redacted (a {configured, last4}
marker instead of the secret) on the analytics sync and the /settings routes'
consumers; the redaction helper is shared.
- Launch stamps the session at birth (see also openswarm-ai#145) and the launch route runs
the first turn only after admission says so; concurrent root turns are capped
(TurnAdmission, OSW_MAX_CONCURRENT_TURNS) so firing thirty agents does not
spawn thirty CLIs at once.
- SSRF guard: v4-mapped and transition encodings are judged as network targets;
loopback is allowed on the desktop (previews) and only a hosting policy can
turn it into a blocked target.
- Service: the shutdown fuse disarms last on a clean shutdown; the runtime port
and settings gateway are injected boundaries; the analytics client redacts.
- Workflows: the scheduler claim gate is off-means-off; a deterministic
at-least-once scheduler contract with its reference model; lifecycle events.
- Settings store: the in-memory mirror is keyed by path+stat, and the atomic
writer stages its temp file beside the settings file.
- Agent manager: session store, event sink (a Null sink by default) and provider
runtime are injected, which is what the delivered tests build on.
The hosting seam (backend/apps/hosting/policy.py): routes take
`scope: RequestScope = REQUEST_SCOPE` and managers ask `hosting_policy()`; both
answer with the desktop defaults here (nobody owns anything, everything is
allowed, no per-owner workspace roots, no tool denials, loopback allowed). A build
that hosts supplies its own provider; nothing else in the app knows which build
it is. REQUEST_SCOPE is a FastAPI dependency that is also the desktop scope, so
routes called directly (as several tests do) keep working.
Not in this change: the outputs app (separate change), dashboards (mixed with a
card-family feature; separate change), anything hosted or multi-tenant.
Proof: this tree with Python 3.13 from requirements.lock — 3074 passed / 15
skipped (upstream's suite plus the delivered tests); upstream's linter reports
no new findings beyond pre-existing debt (two grandfather entries added, both
noted in linter/config/config.json).
kai-openswarm
added a commit
to kai-openswarm/openswarm
that referenced
this pull request
Aug 18, 2026
…runs it; the warm cache is keyed by that interpreter; importing the backend starts nothing The previous commit moves the embedded interpreter. Every generated app's backend runs on it: run.sh receives it as OPENSWARM_PYTHON and creates the workspace's .venv with it, and the shared warm venv (copied into new workspaces by backend_init.sh) is built by it. Neither noticed the interpreter changing: - run.sh reused any existing .venv (its fast path never looked inside), so after the update every app's venv still pointed at the previous interpreter and its backend never came up again; - the warm cache was keyed by the template's dependencies alone, so new apps kept receiving a venv built by the previous interpreter. Now a .venv is reused only when it is ours and its interpreter is the one that will run it; otherwise it is rebuilt — and only when it is ours: a .venv the template did not create, or a symlink/junction standing in for one, is preserved untouched with a message, and nothing is ever `rm -rf`ed. The rules live in one stdlib-only helper the template ships (backend/config/python_runtime_guard.py): ownership markers, identity (version, implementation, ABI tag, platform, machine) proven by asking the venv's interpreter, `pyvenv.cfg` checked before that interpreter is ever run, reparse points refused. Venvs the previous template wrote (their `.openswarm_installed` sentinel is present) are adopted, then judged like any other. Nothing pins a version: the comparison is against the running interpreter, so the next runtime bump needs no code change here. The warm cache (backend/apps/outputs/python_venv/) is keyed by the template's dependencies and the interpreter's identity, built in a staging directory and published atomically, owned by markers, and only reported complete after its interpreter proves it is ours. It installs exactly what webapp_template/backend/pyproject.toml declares (parsed, not a duplicate list). backend_init.sh copies only an owned, complete cache, through a staging directory; run.sh re-verifies the copy with the interpreter that will run it. backend_init.sh runs from the agent's shell, which has no OPENSWARM_PYTHON, so it uses whatever Python 3 that shell has (macOS's own 3.9 included) for its two stdlib-only jobs — the structural cache check and picking a free port (which no longer hard-codes `python3`, absent on Windows). Importing backend.main is now side-effect free: the App Builder's warmup (template node_modules + backend venv) is registered with the health route and runs as a background task after the first successful /api/health/check, so a venv build and a pip install cannot compete with the server's own start-up on a cold machine, overlapping health probes cannot start two warmers, and importing the module in tests or an import probe starts nothing. OPENSWARM_BACKEND_IMPORT_ONLY=1 (the shell's import probe) and OPENSWARM_DISABLE_TEMPLATE_WARM_CACHE=1 skip the registration. The health.py hook is the same one openswarm-ai#152 adds, so either PR can merge first. Fixed in passing, both found while proving this: values written into a workspace's .env are shell-quoted, so an install path with a space (or backslashes) no longer breaks `source .env` in the template's scripts (and a re-seed no longer runs a Windows path through re.sub's escape processing); a `python -m venv` that fails half-way (a runtime without ensurepip, a full disk) now cleans up what it created instead of leaving a .venv the next start must not touch. Tests: python_venv (identity/ownership/atomic publish/lock), the guard's CLI on real venvs and through run.sh (unowned preserved, other-interpreter rebuilt, failed creation cleaned up, system-python cache check), the .env round trip through bash and read_env_value, and the import/health behaviour in a subprocess. Full backend suite: 2994 passed / 16 skipped on 3.14 and on 3.13.
This was referenced Aug 18, 2026
… at boot, pass through the hosting policy Two more no-op-on-the-desktop hooks on HostingPolicy, in the same spirit as the rest of the seam: hydrate_settings(settings, save) runs once at boot on the loaded settings (the desktop returns them untouched; a build that lends a provider key pins it here), and present_settings(payload) is applied to every settings payload a route returns (the desktop shows everything; a build that lends a key masks it). No behaviour changes on the desktop; nothing else in the app knows which build it is.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ws_manager: send/resolve carryowner/claimant; connections carry an optional identity with expiry and revocation). The sessions list, history and prompt prediction never read across owners.{configured, last4}marker instead of the secret) on the analytics sync; the helper is shared (settings/redaction.py).TurnAdmission,OSW_MAX_CONCURRENT_TURNS, default 8) so firing thirty agents does not spawn thirty CLIs at once.Nullsink by default) and provider runtime are injected — the delivered tests build on that.backend/apps/hosting/policy.py: routes takescope: RequestScope = REQUEST_SCOPEand managers askhosting_policy(). On the desktop both answer with the defaults (nobody owns anything, everything is allowed, no per-owner workspace roots, no tool denials, loopback allowed). A build that hosts supplies its own provider; nothing else in the app knows which build it is.REQUEST_SCOPEis a FastAPI dependency that is also the desktop scope, so routes called directly (several tests do) keep working.Not in this PR: the outputs app (next PR), dashboards (mixed with a card-family feature; separate), anything hosted.
Linter: no new findings beyond pre-existing debt; two grandfather entries added and noted in
linter/config/config.json(manager/runfolder count; theowner_account_idand internal event ids indangling-refs).Proof: this tree with Python 3.13 from
requirements.lock— 3074 passed / 15 skipped (upstream's suite plus the delivered tests); packaged macOS app with this backend: upstream smoke 5/5 + characterization suite (#146) 19/19.Follow-up commit (
settings: the settings payload a client sees, and the settings loaded at boot, pass through the hosting policy): two more no-op-on-the-desktop hooks onHostingPolicy—hydrate_settings(settings, save)at boot andpresent_settings(payload)on every settings payload a route returns — so a build that lends a provider key has a place to pin and mask it; nothing changes on the desktop. Settings/seam tests 89/89; linter unchanged.