Skip to content

Fix CUDA context leak during NVENC probing - #424

Merged
gtong-nv merged 1 commit into
mainfrom
dev/gtong/fix-nvencoder
Aug 7, 2026
Merged

Fix CUDA context leak during NVENC probing#424
gtong-nv merged 1 commit into
mainfrom
dev/gtong/fix-nvencoder

Conversation

@gtong-nv

@gtong-nv gtong-nv commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Restore PyTorch's CUDA device context after probing PyNvVideoCodec for NVENC support.

GetEncoderCaps can replace the calling thread's active CUDA context when the probe fails. Lingbot then falls back to aiortc, but subsequent Triton kernels run with the wrong context and fail with invalid resource handle. Retrying the partially advanced rollout produces the secondary AR step out of order error.

The encoder-selection boundary now restores the original PyTorch device context in a finally block for every hardware-probe outcome.

@copy-pr-bot

copy-pr-bot Bot commented Aug 6, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Restores the initialized PyTorch CUDA device after every hardware-encoder probe outcome, preventing PyNvVideoCodec probing from contaminating subsequent model execution.

  • Extracts hardware selection into _select_hardware_encoder.
  • Wraps hardware selection in device capture and finally-based restoration.
  • Adds coverage for restoration after an automatic NVENC capability-probe fallback.

Confidence Score: 5/5

The PR appears safe to merge, with CUDA-device restoration consistently applied across all hardware-probe outcomes.

The new wrapper preserves existing encoder-selection behavior while ensuring its cleanup runs through a finally block, and the regression test covers the reported automatic-fallback path.

Important Files Changed

Filename Overview
flashdreams/flashdreams/serving/webrtc/encoders.py Hardware encoder selection now restores the previously active PyTorch CUDA device on success, fallback, and error paths.
flashdreams/tests/test_encoders.py Adds a focused regression test confirming restoration after a failed NVENC capability probe falls back to the default encoder.

Reviews (1): Last reviewed commit: "Fix CUDA context leak during NVENC probi..." | Re-trigger Greptile

@jarcherNV jarcherNV left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gtong-nv

gtong-nv commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test d4f3532

@gtong-nv
gtong-nv enabled auto-merge August 7, 2026 00:19
@gtong-nv
gtong-nv added this pull request to the merge queue Aug 7, 2026
Merged via the queue into main with commit 996417e Aug 7, 2026
7 checks passed
@gtong-nv
gtong-nv deleted the dev/gtong/fix-nvencoder branch August 7, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants