[Zhengyang Liu (liuzhengyang699)]

Fixes #695.

Different images can produce identical prompt token IDs because the IDs represent image placeholders, not pixels. The triplet view currently treats those requests as retries and drops the earlier image-conditioned transition. The manager repeats the same token-only deduplication when aligning images.

Use a shared key containing prompt IDs and ordered image URLs. Deduplicate before trimming the raw events, then retain the existing triplet schema. The manager uses the same key to align images. Route compaction also preserves previous routed-expert payloads when the image inputs differ; otherwise the newly retained row would fail when R3 requires routes. True same-image retries still keep the last response.

A CPU reproduction with Qwen2-VL preprocessing produced identical 88-token prompts for red and blue images but different pixel tensors. Three requests (red, red retry, blue) previously produced one blue training row; they now produce one red and one blue row. Regression tests exercise the real event API, manager, PIL decoding, and adapter with and without required routed experts. They also cover image order, serialized content, and unchanged trimmed output without data URLs.

Validation

The development environment uses the locked dev and verl-cpu dependencies. PYTHONPATH was set to this checkout's root for the commands below, and imported module paths were verified explicitly.

• .venv/bin/pytest -v --durations=20 tests — 245 passed, 4 skipped.
• .venv/bin/pyright — 0 errors, 0 warnings.
• .venv/bin/pre-commit run --all-files --show-diff-on-failure — passed.
• .venv/bin/ruff check ., .venv/bin/ruff format --check ., and .venv/bin/python scripts/check_headers.py — passed.
• UV_PROJECT_ENVIRONMENT=/tmp/agent-lightning-image-docs uv run --locked --no-sync mkdocs build --strict, uv build --no-sources, and git diff --cached --check — passed.

The separate docs environment was installed with the locked docs group. Independent review also checked mixed response schemas and server/manager alignment.

This was found during a follow-up audit with AI assistance. AI assistance was also used for the fix, tests, and review. The Qwen check used preprocessing only; no model weights, GPU inference, or end-to-end training were run.

[Copilot]

🟡 Changes recommended

Same-image context extensions still delete routes from retained training rows, causing require_routed_experts=True to raise and requiring route preservation plus a multi-turn regression test.

1 open issue

• Preserve routes for same-image context extensions · New

What changed in this PR

Fixes #695 by distinguishing image-conditioned requests that share prompt token IDs, preserving distinct transitions through event processing and training-data alignment.

Changes:

• Share a deduplication key based on prompt token IDs and ordered image URLs.
• Align image recovery and route retention with image-aware identity while preserving the trimmed event schema.
• Add API and training-batch regression coverage and document retry behavior.

File	Description
tests/​verl/​test_agl_rollout_manager.py	Tests image alignment and server-to-training batches with optional required routes.
tests/​server/​test_endpoints.py	Tests image identity, ordering, serialized content, and unchanged triplet output.
docs/​80-example-multimodal-qa.md	Documents image-aware deduplication and last-response retry behavior.
agentlightning/​verl/​agl_rollout_manager.py	Uses shared prompt identity to align images with triplets.
agentlightning/​server/​routes/​events.py	Deduplicates raw events before trimming and preserves routes across different images.
agentlightning/​_prompt_identity.py	Adds shared image extraction and prompt-key helpers.

_{Review effort: Balanced}