Conversation
This was referenced Aug 13, 2026
Two related delegate-lifecycle fixes in createModel: 1. Skip null delegates: delegate factories can legitimately return nullptr (e.g. TfLiteCoreMlDelegateCreate on devices without a Neural Engine when enabled_devices is ANE-only). Registering that nullptr with TfLiteInterpreterOptionsAddDelegate crashes/corrupts the interpreter. Now a null delegate is skipped so the model falls back to CPU, and getDelegates() reports only the delegates that were actually registered. 2. Free delegates: TFLite's C API does not transfer delegate ownership to the interpreter - the caller must delete delegates itself after the interpreter is destroyed. They were never freed, so every model destruction leaked the delegate's compiled kernels / driver contexts (GPU: TfLiteGpuDelegateV2Delete, NNAPI: TfLiteNnapiDelegateDelete, CoreML: TfLiteCoreMlDelegateDelete). Each delegate is now held in a unique_ptr with its own delete function; ownership moves into the interpreter's shared_ptr deleter, so delegates are freed right after TfLiteInterpreterDelete (they must outlive the interpreter) and on every failure path in createModel via normal unwinding.
HybridTfliteModel inherits Nitro's default no-op dispose(), so JS calling
model.dispose() frees nothing - the interpreter, delegates and model buffer
only go away when GC drops the last reference. Worklet runtimes (frame
processors) may not GC for a long time, especially while the app is
backgrounded, so multi-hundred-MB models and their GPU contexts stay
resident with no way to release them deterministically.
This implements a real dispose():
- Resets the interpreter shared_ptr immediately. The model is its only
owner, so the interpreter deleter runs right away: TfLiteInterpreterDelete,
then the delegates. Our references to the model bytes and the cached
output buffers are dropped too.
- Thread-safe via a lifecycle mutex: dispose() blocks until an in-flight
inference on another thread completes - freeing the interpreter under a
running TfLiteInterpreterInvoke would be a native crash. The mutex is
uncontended in normal operation (~ns per lock vs ~ms per inference).
- run() re-checks disposal on the async thread, since dispose() may land
between the caller-thread input copy and the async invoke.
- All post-dispose calls (runSync/run/getInputs/getOutputs) throw a
catchable JS error ('TFLite: Model was disposed!') instead of crashing.
- Idempotent; the destructor stays defaulted since a disposed model holds
nothing.
jslok
force-pushed
the
feat/deterministic-dispose
branch
from
September 14, 2026 05:46
b1ca7cc to
bc2c9d4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #197 (its commit is the first one here); review the second commit, or merge #197 first and this rebases cleanly.
HybridTfliteModelinherits Nitro's no-opdispose(), somodel.dispose()from JS frees nothing until GC drops the last reference. Worklet runtimes may not GC for a long time, especially while backgrounded, so large models and their GPU delegate contexts stay resident with no way to release them. In our app, releasing scanner models on background through a workingdispose()frees ~86 MB.dispose()now resets the interpretershared_ptrunder a lifecycle mutex. The model is the interpreter's only owner, so that runsTfLiteInterpreterDeleteand then frees the delegates immediately; the model-bytes and cached output-buffer references are dropped too. The mutex serializes it against an in-flight inference on another thread (freeing the interpreter mid-TfLiteInterpreterInvokeis a native crash),run()re-checks on the async thread, and post-dispose calls throwTFLite: Model was disposed!as a catchable JS error. Idempotent; the destructor stays defaulted.We run the pre-#205 form of this as a patch-package fix in production (models disposed on background with an active frame processor racing it, reloaded on foreground) without crashes. The rebased version is compile-checked against the NDK; iOS not rebuilt yet.