Repository navigation
[sentry-diagnostic:7851254696] FOUNDRY_LOCAL_HOST_ERROR on atomic-chat-core@0.23.4 (dist c21d7796d3b2) is raised only after the hidden foundry-host child exits before the "ready":true line. `hostEr - #119
Merged
Conversation
Ooooze
marked this pull request as ready for review
November 18, 2026 11:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Diagnostic instrumentation only. The original issue is NOT fixed or closed.
FOUNDRY_LOCAL_HOST_ERROR on atomic-chat-core@0.23.4 (dist c21d779) is raised only after the hidden
foundry-hostchild exits before the"ready":trueline.hostErrorin src/runtime/foundry-local/runtime.ts collapses every such outcome into one of two AtomicCoreError messages, andloadFailureReportrecords neither the host exit code nor its stderr, so the event carries failure_reason=unknown, cause_code=FOUNDRY_LOCAL_HOST_ERROR and a generic exception valueruntime load: unknown. The competing explanations (exit 3: the native Foundry Local library cannot load on this Windows machine; exit 4: the library loaded but a call failed; a signal death during load; exit 2: a core-side argv bug) are indistinguishable from the current telemetry. Release 0.23.4 and base a66aad0 differ only in src/diffusion/args*, src/cli/owner.test.ts, src/runtime/fastflowlm/runtime.test.ts and src/version.ts, so the foundry-local path is byte-identical to base and there is no existing fix. This phase implemented the planned bounded diagnostic candidate: the controller-generated observer now carries the Foundry Local sibling pair of the existing GenieX codes, and the regression test src/telemetry/sentry-agent-diagnostics.test.ts asserts both codes through the existing diagnosticTags/classifyFailure boundary, plus privacy, cyclic-cause and unchanged-reporting cases; it passes on the generated observer and fails on the original base where the rules are absent. No TypeScript defect is claimed and no repair is proposed; the original failure remains unexplained until released events carry the codes.Evidence:
Event ATOMIC-CHAT-CORE-CM (ced119d9c8434e5e9a41fa0e30013202): type FOUNDRY_LOCAL_HOST_ERROR, value
runtime load: unknown, facts provider=foundry-local, component=runtime, operation=load, source=model_load, failure_category=unknown, failure_reason=unknown, diagnostic_version=core-v1, active_backend=win-cuda-13.3-x64, gpu_model=NVIDIA GeForce RTX 5060 Ti, context_length=49152, gpu_layers=100, release atomic-chat-core@0.23.4 dist c21d779.Stack: start (src/runtime/foundry-local/runtime.ts:303) → startHost (:364) → spawnAndAwaitReady (src/runtime/shared/process.ts:305) → hostError (src/runtime/foundry-local/runtime.ts:135) — the innermost frame is hostError, i.e. the child exited before readiness.
src/runtime/foundry-local/runtime.ts
hostError(exitCode, stderr): exit 5 → FOUNDRY_LOCAL_ACCELERATOR_NOT_PREPARED; exit 3 → FOUNDRY_LOCAL_HOST_ERROR with message 'The Foundry Local library does not load on this computer.'; every other code → FOUNDRY_LOCAL_HOST_ERROR with message 'The Foundry Local host stopped before the model was ready.'; details = last 5 stderr lines.src/runtime/foundry-local/host/run.ts documents the host exit codes: 0 done, 2 usage (a core bug), 3 the library does not load or speaks another C API, 4 a library call failed, 5 accelerator not prepared. Only 3 and 5 are distinguishable to a reader of the message.
src/runtime/shared/process.ts
spawnAndAwaitReadyroutes an early exit, a clean exit without the ready marker, and a signal death toclassifyExit; a signal death passescode: null, so a native crash of the host also becomes FOUNDRY_LOCAL_HOST_ERROR with no signal recorded.src/telemetry/reports.ts
loadFailureReporttags only provider, error_code, model_id, quant, backend, context_length, gpu_layers, is_embedding and oom_subtype; the host exit code and stderr never reach the report, matching the observed absence of exit_code/signal tags.src/telemetry/diagnostics.ts
classifyFailurehas no FOUNDRY_LOCAL_HOST_ERROR entry in EXPECTED or OPERATIONAL, so the category and reason stay 'unknown'; the event's exception valueruntime load: unknownis that component/operation/reason triple.Release diff for 0.23.4#c21d7796d3b2 → base a66aad0 touches only src/cli/owner.test.ts, src/diffusion/args.ts, src/diffusion/args.test.ts, src/diffusion/compat.ts, src/diffusion/compat.test.ts, src/diffusion/server-process.test.ts, src/runtime/fastflowlm/runtime.test.ts and src/version.ts; src/runtime/foundry-local/ is unchanged, so the defect is present in base and is not an existing fix.
src/telemetry/sentry-agent-diagnostics.ts (generated, base + this candidate) now holds 14 rules: the 12 pre-existing sd.cpp / GenieX / mlx-server rules unchanged, plus 'The Foundry Local library does not load on this computer.' → FOUNDRY_LOCAL_LIBRARY_NOT_LOADED and 'The Foundry Local host stopped before the model was ready.' → FOUNDRY_LOCAL_HOST_STOPPED_EARLY, both sourced from src/runtime/foundry-local/runtime.ts, with include_cause_codes=true.
src/telemetry/sentry-agent-diagnostics.ts reads own-property
messageandcodeup to five causes, and AtomicCoreError declares both as own properties (src/contracts/errors.ts:318), so exact-message rules match without exposing details or paths.Regression test src/telemetry/sentry-agent-diagnostics.test.ts imports the existing boundary (classifyFailure, diagnosticTags, failureObserver from ./diagnostics.js), asserts both Foundry Local codes, asserts the untagged FOUNDRY_LOCAL_ACCELERATOR_NOT_PREPARED sibling, cyclic causes and unchanged AtomicCoreError.toJSON()/failureObserver capture. check run: 9/9 tests passed on the generated observer; on the original base the 'tags Foundry Local host load failures with finite diagnostic codes' case fails because diagnostic_code stays undefined.
diagnostic_policy (this session, base a66aad0): status available, existing_rules 12, remaining_rule_capacity 52, cause_codes_enabled true; the proposed two-rule plan returned status accepted.
Question: When a Foundry Local model load fails with FOUNDRY_LOCAL_HOST_ERROR, was the hidden host unable to load Microsoft's native library (exit 3) or did the library or host fail after loading (exit 4/2) or exit without readiness?
Expected observation: Existing FOUNDRY_LOCAL_HOST_ERROR events from atomic-chat-core@0.23.4 gain diagnostic_code=FOUNDRY_LOCAL_LIBRARY_NOT_LOADED for the exit-3 message and diagnostic_code=FOUNDRY_LOCAL_HOST_STOPPED_EARLY for the other exit paths, so the two explanations separate on the same events without any new report, severity or exit-code change.
Removal condition: Remove the two rules once the library-not-loaded versus stopped-early split is established from released events, or once the host exit code and signal are carried in the report tags directly instead of being reduced to a fixed message.
Bounds: two finite tags on an existing event, five Error causes; no new events, free-form messages, user content or reporting-policy changes.
Regression tests: src/telemetry/sentry-agent-diagnostics.test.ts
Full CI and human review are required.
Report: http://127.0.0.1:8080/tenants/707d0855-80ab-4e1f-a156-f1c4546cbf52/runs/85d5e810-05b5-45d5-b39b-74dc5d8036b4
Sentry: https://atomic-chat.sentry.io/issues/7851254696/