Skip to content

Fix MTGS scene mapping and stale actor rendering - #160

Closed
hctian713 wants to merge 8 commits into
NVlabs:e2e_challengefrom
hctian713:e2e_challenge
Closed

hctian713 wants to merge 8 commits into
NVlabs:e2e_challengefrom
hctian713:e2e_challenge

Conversation

@hctian713

Copy link
Copy Markdown

Summary

  • Resolve NuPlan scene tokens to their shared MTGS asset folders, including configs without an explicit road_block_name.
  • Propagate the asset folder through newly generated trajdata caches.
  • Prevent inactive or out-of-range MTGS actors from leaving stale Gaussian data in later frames.
  • Add regression coverage for scene mapping, timestamp handling, and active-actor cache updates.

Root cause

Multiple NuPlan scene tokens can share one MTGS road-block asset folder, but the renderer previously fell back to the unique scene token when cache metadata was missing. Separately, actors outside their valid replay interval could reuse stale render state, producing visible ghosting.

Impact

The affected NuPlan scene now resolves the correct background asset and no longer shows stale actor trails during rollout.

Validation

  • MTGS plugin tests: 20 passed.
  • Pre-commit checks passed.
  • GPU smoke test completed for 2021.05.25.14.16.10_veh-35_03373_03550-80a4b14aef3f5a52.
  • Rendered output was inspected manually and no visible ghosting remained.

This is a draft so the target repository can run its own local GPU smoke test before review.

@WCJ-BERT

WCJ-BERT and others added 6 commits July 13, 2026 12:33
Multiple scene tokens sharing the same MTGS road block config were each
trying to resolve their own asset folder (named after their unique token),
which only exists for the first token (road_block_name).

Fix via two complementary layers:

1. Runtime resolver in main.py (no cache regen required): at server startup,
   read all MTGS config YAMLs from the configs/ directory and build a
   token → road_block_name mapping.  This is injected as asset_folder_resolver
   into TrajdataDataSource so any token resolves to its shared asset folder
   regardless of cache state.

2. Cache pipeline fix in scene_loader.py + nuplan_utils.py: propagate
   road_block_name as asset_folder through central_tokens_config → scene
   record → data_access_info so newly built caches also carry the correct
   mapping.

Also add nuplan_private → private to DATASET_NAME_MAPPING in the MTGS server.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
yaml.load() always returns a dict, so the isinstance/getattr branch is
redundant. Use cfg.get() directly, per code review suggestion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
# Conflicts:
#	plugins/mtgs/server/main.py
Fix MTGS scene mapping and stale actor rendering
@copy-pr-bot

copy-pr-bot Bot commented Aug 22, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@mwatson-nvidia

Copy link
Copy Markdown
Collaborator

Closing as a duplicate of #111. Before the review follow-up, both PR heads resolved to the identical tree hash c782b40ae798d3fdebf5fc0a6889443c7c5d72dd; #111 has now been merged, so no changes are lost.

mwatson-nvidia added a commit that referenced this pull request Aug 24, 2026
## Summary

- Add standalone SimScale NAVSIM/MTGS E2E submissions for:
  - Latent TransFuser
  - DiffusionDrive
  - GTRS-Dense
- Add container build, asset preparation, local launcher, probe, and
benchmark scripts.
- Add checkpoint hash validation, pinned runtime dependencies, and
hardened read-only container settings.
- Add unit tests covering preprocessing, trajectory generation, model
structure, checkpoint loading, driver service behavior, batching, and
container files.
- Add `NAVSIM_MODEL_ADAPTATION.md` documenting the NAVSIM-to-AlpaSim
inference contract and model adaptation workflow.

Existing starter-kit and VAVAM examples are preserved. This branch is
based directly on the current upstream `e2e_challenge` branch and is
independent from the MTGS fixes in #160.

## Validation

- `git diff --check` passed.
- The branch contains only the two NAVSIM submission commits on top of
the current upstream base.
- Full model/container tests were not run because they require
downloaded checkpoints, Docker images, and the corresponding runtime
environment.

---------

Co-authored-by: Michael Watson <mwatson@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants