Skip to content

Fix for V8 OOM crash issue#4862

Closed
jdetter wants to merge 1 commit intomasterfrom
jdetter/fix-v8-oom-crash-issue
Closed

Fix for V8 OOM crash issue#4862
jdetter wants to merge 1 commit intomasterfrom
jdetter/fix-v8-oom-crash-issue

Conversation

@jdetter
Copy link
Copy Markdown
Collaborator

@jdetter jdetter commented Apr 21, 2026

Description of Changes

Note: the changes here were 100% generated by claude code but the PR description was written by me.

  • This fixes the issue where when a V8 module uses a ton of RAM (more than 100GB) it causes an OOM crash which crashes the actual SpacetimeDB process. Here is an example:
<--- Last few GCs --->

[1:0x7fcd4c001000]  1114272 ms: Mark-Compact 1388.8 (1401.3) -> 1388.8 (1401.3) MB, pooled: 3.0 MB, 2419.09 / 0.00 ms (average mu = 0.080, current mu = 0.000) allocation failure; GC in old space requested
[1:0x7fcd4c001000]  1116691 ms: Mark-Compact (reduce) 1388.8 (1401.3) -> 1388.8 (1401.3) MB, pooled: 0.0 MB, 2418.83 / 0.00 ms (average mu = 0.042, current mu = 0.000) last resort; GC in old space requested



#
# Fatal JavaScript out of memory: Ineffective mark-compacts near heap limit
#
==== C stack trace ===============================

    spacetimedb-cloud(+0x60b0773) [0x556765742773]
    spacetimedb-cloud(+0x60af5db) [0x5567657415db]
    spacetimedb-cloud(+0x60b19c8) [0x5567657439c8]
    spacetimedb-cloud(+0x6108c14) [0x55676579ac14]
    spacetimedb-cloud(+0x62ce1f7) [0x5567659601f7]
    spacetimedb-cloud(+0x62cc587) [0x55676595e587]
    spacetimedb-cloud(+0x62ccd42) [0x55676595ed42]
    spacetimedb-cloud(+0x62c062c) [0x55676595262c]
    spacetimedb-cloud(+0x62bff66) [0x556765951f66]
    spacetimedb-cloud(+0x62a08cf) [0x5567659328cf]
    spacetimedb-cloud(+0x62b1d3f) [0x556765943d3f]
    spacetimedb-cloud(+0x611311a) [0x5567657a511a]
    spacetimedb-cloud(+0x3b86d9a) [0x556763218d9a]
    spacetimedb-cloud(+0x7792c21) [0x556766e24c21]

This fixes the issue by registering a callback for when the V8 runtime is almost out of heap space.

API and ABI breaking changes

None

Expected complexity level and risk

1 - This just registers a callback for the V8 runtime

Testing

  • I loaded up the large database we've been debugging (over 50GB snapshot) and it didn't load on master but it did load after this patch.

@joshua-spacetime
Copy link
Copy Markdown
Collaborator

I think we want to prioritize #4777 over this patch. Ultimately it should have the same effect which is to avoid crashing the server, but it will also terminate the request instead of increasing the heap limit unconditionally.

@jdetter
Copy link
Copy Markdown
Collaborator Author

jdetter commented Apr 22, 2026

Ok sounds good 👍

@jdetter jdetter closed this Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants