From 028e84f1f55df96dd1b74492189fb2dc5e2e2c7f Mon Sep 17 00:00:00 2001
From: "helpcode.ai" <340500827+helpcodeai@users.noreply.github.com>
Date: Tue, 13 Oct 2026 14:50:39 +0200
Subject: [PATCH] Write a Node diagnostic report when the backend dies of a
 fatal V8 error

On 13 Oct the Cloud backend died with "Zone Allocation failed - process out of memory" with the heap at a third of its cap, and the log had no stack. start.sh now passes --report-on-fatalerror so the next such crash leaves a report (native stack, heap spaces, resource usage, JS stack when available) in HEAP_SNAPSHOT_DIR, which is a volume on Cloud. Environment variables are excluded; on a Node without --report-exclude-env the report stays off rather than writing secrets to disk or failing to start.
---
 docs/operations/observability.md |  4 ++++
 start.sh                         | 16 +++++++++++++++-
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/docs/operations/observability.md b/docs/operations/observability.md
index fbfb44c91..8d87b4746 100644
--- a/docs/operations/observability.md
+++ b/docs/operations/observability.md
@@ -176,6 +176,10 @@ Use the *raw* request body, not a re-serialized JSON object, since re-serializin
 
 The webhook URL is validated with the same guard (`SSRF_GUARD`, private/loopback/metadata-address blocking) used for every other outbound call this app makes on a user's behalf, both when it's saved and again on every dispatch, and redirects are not followed. On AnythingMCP Cloud the URL must use `https`, and it ignores both the env (`SSRF_ALLOWED_HOSTS`) and the admin-editable SSRF allowlist, which name hosts only the platform may reach. On a self-hosted instance the operator's allowlist applies, so an internal Mattermost or Slack proxy can receive alerts.
 
+## Fatal crash reports (always on in the Docker image)
+
+A fatal V8 error, such as an out-of-memory abort, ends the backend before any JavaScript handler runs, so the log has no stack. `start.sh` therefore starts the backend with `--report-on-fatalerror`: on such a crash Node writes a diagnostic report (`report.<date>.<time>.<pid>...json`) with the native stack, heap spaces and resource usage, plus the JavaScript stack when V8 can still produce one. It goes to `HEAP_SNAPSHOT_DIR` (`/app/diagnostics` in the image, a volume on AnythingMCP Cloud), next to the heap snapshots. Environment variables are left out of the report (`--report-exclude-env`); a Node older than 22.13 cannot do that, so on such a build the report stays off. Self-hosted and Cloud behave the same; to keep the reports across container recreation on a self-hosted install, mount a volume at `/app/diagnostics`.
+
 ## Correlating across pipelines
 
 The same `req.id` UUID appears in:
diff --git a/start.sh b/start.sh
index 3a7468b1e..ee84f5977 100644
--- a/start.sh
+++ b/start.sh
@@ -46,7 +46,21 @@ run_migrations() {
 # suits a ~4GB host. The backend's own heap guard (see
 # packages/backend/src/common/process-vitals.ts) exits gracefully at a
 # percentage of whatever this is, before V8 hits the wall.
-BACKEND_CMD="node --max-old-space-size=${NODE_MAX_OLD_SPACE_MB:-2048} dist/src/main.js"
+#
+# A fatal V8 error (e.g. "Zone Allocation failed" on 13 Oct, with the heap at
+# a third of its cap) kills the process before any JS handler runs, so the
+# log shows no stack. --report-on-fatalerror writes a Node diagnostic report
+# (native stack, heap spaces, resource usage, and the JS stack when V8 can
+# still produce one) next to the heap snapshots; on Cloud that directory is a
+# volume that outlives the restart. The report would include every
+# environment variable, secrets too, so it is only switched on where Node can
+# leave them out (--report-exclude-env, Node >= 22.13); an older Node, which
+# would refuse to start on the unknown flag, runs without it.
+REPORT_FLAGS=""
+if node --report-exclude-env -e 0 >/dev/null 2>&1; then
+  REPORT_FLAGS="--report-on-fatalerror --report-exclude-env --report-compact --report-directory=${HEAP_SNAPSHOT_DIR:-/tmp}"
+fi
+BACKEND_CMD="node --max-old-space-size=${NODE_MAX_OLD_SPACE_MB:-2048} ${REPORT_FLAGS} dist/src/main.js"
 
 case "$MODE" in
   backend)