intlayer watch has memory leak (OOM at ~4GB) and orphans child processes on exit #398
Description
Description
Running intlayer watch --with 'exec next dev --turbopack' causes two critical issues:
-
Memory leak — The Node.js process steadily grows to ~4GB heap and crashes with
Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memoryafter ~10 minutes of normal development. -
Orphaned child processes — When the watcher is terminated (Ctrl+C or SIGINT), the
--withchild process (e.g., Next.js/Turbopack) is not killed and continues running in the background, requiring manualpkillcleanup.
Environment
intlayer:8.4.5next-intlayer:8.4.5next:16.2.0- Node.js:
v25.8.1 - Bun:
1.3.10 - OS: Ubuntu 24.04.4 LTS (kernel 6.17.0-19-generic)
- Dev command:
intlayer watch --with 'exec next dev --turbopack --inspect'
OOM Stack Trace
<--- Last few GCs --->
[44372:0x3c5b7000] 606603 ms: Scavenge (during sweeping) 4078.5 (4090.0) -> 4073.8 (4091.5) MB, pooled: 0.0 MB, 8.71 / 0.00 ms (average mu = 0.307, current mu = 0.273) allocation failure;
[44372:0x3c5b7000] 607489 ms: Mark-Compact (reduce) 4081.9 (4093.5) -> 4076.1 (4081.2) MB, pooled: 0.0 MB, 68.51 / 0.00 ms
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
Part 1: Memory Leak
Multiple unbounded data structures grow on every file change event and are never cleaned up:
1. Global cache Maps never cleared on content file changes
@intlayer/config/src/utils/cacheDisk.ts:89 and cacheMemory.ts:306 both have module-level Map<string, any> instances. clearAllCache() is only called when the configuration file changes (watcher.ts:160), but regular .content.ts file changes — the common case during development — go through handleContentDeclarationFileChange() which never clears either cache.
Every loadContentDeclaration() call writes to these maps via cacheDisk().set() (which also hydrates cacheMap in cacheDisk.ts:175). Over time both maps accumulate entries that are never evicted.
2. esbuild build() called repeatedly without service cleanup
Every .content.ts file change calls loadExternalFile() → transpileTSToCJS() → esbuild.build(). Each build() invocation is a standalone one-shot build that allocates internal esbuild state. No esbuild.stop() or shared build context is used. With rapid HMR-driven file saves, hundreds of esbuild builds accumulate internal state.
3. vm.runInNewContext() creates sandbox contexts that retain references
parseFileContent.ts:130 runs transpiled code in runInNewContext() with a sandbox that copies all globalThis properties (parseFileContent.ts:114-118). Each sandbox context holds references to process, require, console, React, and the full globalThis — creating heavyweight closures on every file load.
4. Promise chain in processingQueue grows unboundedly
watcher.ts:27-36 chains every file event as .then() on a single Promise:
let processingQueue = Promise.resolve();
const processEvent = (task: () => Promise<void>) => {
processingQueue = processingQueue.then(async () => {
await task();
});
};Each .then() creates a new Promise node holding a closure over the task, configuration, and file path. After thousands of file events this becomes a significant linked list in memory.
Part 2: Orphaned Child Processes
watch.ts:58 tries to kill the parallel process during shutdown:
if (parallelProcess && 'child' in parallelProcess) {
parallelProcess.child?.kill('SIGTERM');
}But ParallelHandle (defined in runParallel/index.ts:5-9) does not expose a child property — it only has { kill, result, commandText }. So 'child' in parallelProcess is always false and the child is never killed. The existing parallelProcess.kill() method that would work is never called.
Additionally, there are competing signal handlers:
runParallel/index.ts:129-131registers SIGINT/SIGTERM handlers that callchild.kill('SIGTERM')on the shell processwatch.ts:70-71registers its own SIGINT/SIGTERM handlers
The shell (/bin/sh -c '...') receives SIGTERM but doesn't propagate it to its children (Next.js/Turbopack). And spawnPosix.ts:24 uses an async pidtree lookup to find descendant PIDs, but returns true immediately — by the time the async callback fires, the parent has already called process.exit(0).
Result: After Ctrl+C, the intlayer process and shell exit, but Next.js/Turbopack continues running as an orphan.
Related: #386 (chokidar watchers not cleaned up in vite-intlayer)
Current Workaround
{
"scripts": {
"dev:kill": "pkill -9 -f next-server; pkill -9 -f 'next dev'; pkill -9 -f 'intlayer watch'; pkill -9 -f postcss; rm -f .next/dev/lock"
}
}