[Chao Wang (chaoworks)]

Problem and change

A transparent PNG is decoded differently during rollout and training. The recommended vLLM 0.20.2 backend composites RGBA images onto a white background; _load_pil_image() currently calls convert("RGB"), which discards alpha and exposes the hidden RGB channels. For example, (10, 20, 30, 0) becomes white for rollout but (10, 20, 30) for training. The generated response/logprobs are therefore paired with different vision inputs when constructing the training batch.

Composite RGBA onto white after decoding any supported URL scheme (data:, file://, HTTP(S)). Keep the existing conversion for other image modes. Document how to pre-composite images when a serving deployment uses a custom background color.

This is a source-audited bug with a deterministic reproduction. The reference behavior is in vLLM 0.20.2's image conversion and ImageMediaIO; that backend version is recommended in this repository's installation guide.

Validation

• Six regression cases cover RGB/RGBA × data/file/HTTPS. The RGBA fixture includes fully transparent, partially transparent, and opaque pixels. On unmodified main bc3a9e8: 3 failed, 3 passed; with the fix: all six pass.
• Frozen Python 3.12 CPU environment: uv run --locked --no-sync pytest -q tests → 247 passed, 4 skipped.
• An additional offline experiment compared decoded images with the unmodified vLLM 0.20.2 conversion helper, then processed them with the real Qwen2VLImageProcessorPil. Before: maximum pixel_values difference 3.5766143798828125. After: image bytes, pixel_values (shape [12, 1176]), and image_grid_thw match exactly.
• All-file pre-commit, Ruff lint/format, header checks, and Pyright pass (0 errors/warnings).
• mkdocs build --strict and uv build --no-sources pass.

No GPU model rollout or RL training was run; the reproduction exercises image decoding and actual vision preprocessing on CPU. Custom serving background colors are not inferred from rollout events; pre-compositing to RGB keeps those deployments consistent, as documented.

AI assistance was used for implementation and validation. The diff and caller behavior were reviewed before submission.

[Copilot]

🟢 Approval recommended

No active review issues are reported for the change aligning training image decoding with rollout behavior.

No open issues 🎉

What changed in this PR

This PR aligns training image decoding with vLLM’s default rollout behavior by compositing RGBA images onto white, preventing mismatched vision inputs between rollout and training.

Changes:

• Applies shared image conversion across data, file, and HTTP(S) URLs while preserving conversion for other image modes.
• Adds six regression cases covering RGB and RGBA images across supported URL paths.
• Documents pre-compositing for servers using custom background colors.

File	Description
tests/​verl/​test_rollout_adapter.py	Adds pixel-level regression coverage for RGB and RGBA decoding.
docs/​80-example-multimodal-qa.md	Explains white compositing and custom-background handling.
agentlightning/​verl/​rollout_adapter.py	Unifies decoding and composites RGBA images onto white.

_{Review effort: Balanced}