[amtellezfernandez]

The zero speed/acceleration readings come from the runtime, not the driver.

controller_service.py's force-GT branch replaces the controller response with poses interpolated from the fallback (ground-truth) trajectory, but pairs them with a default-constructed DynamicState, i.e. all zeros:

# When force_gt, ignore the controller response and populate from the
# fallback (ground-truth) trajectory so downstream always sees GT poses.
if force_gt:
    ...
    dynamic_state=DynamicState(),
    dynamic_state_estimated=DynamicState(),

The skip-mode branch and the interpolated-intermediates backfill in the same file do the same thing. Those zeros then flow through ego_trajectory_estimate and interpolate_dynamics into the driver's egomotion observations, so for the whole force_gt_duration_us window the driver sees poses advancing at driving speed while the dynamic state says the vehicle is stopped.

Measured at the driver on main (1e801ca), with a video-model chunking preset (force_gt_duration_us = 2_033_313, control_timestep_us = 266_664):

frame	DynamicState speed	pose-derived speed
0	5.158	5.158
1–7	0.0	10.9 – 11.8
8	11.919	11.919

Frames 1 through 7 are exactly the control steps inside the force-GT window; frame 8 is the first one outside it, and the zeros never recur. With the default force_gt_duration_us = 500_000 the window is shorter, but it's present in every rollout. Since skip_driver_during_force_gt defaults to False, the driver is queried throughout that window by default, so a history-based policy builds its context on those contradictory observations during precisely the period meant to warm it up on realistic ones.

Fix

Derive the fallback dynamic state from the same trajectory the poses come from. Trajectory.interpolate_delta(t0, t1) returns pose(t0).inverse() @ pose(t1), which is the motion expressed in the rig frame at t0, and that's the frame dynamic states are reported in, so linear velocity and yaw rate fall out of a ±100 ms window around the query time. Accelerations stay zero, matching how context dynamics are seeded in event_loop.py; double-differencing recorded poses would mostly amplify noise.

All three zero sites (force-GT, skip mode, intermediates backfill) go through one helper. The driver's _get_speed_and_acceleration docstring promised a finite-difference fallback that was never implemented, so it's corrected to describe what the code actually does.

Tests

Three tests in src/runtime/tests/services/test_controller_api.py drive run_controller_and_vehicle through the skip, intermediates and force-GT paths using the existing fixture's 40 m/s fallback trajectory, and assert the returned dynamics are real.

On unpatched main all three fail in about 0.05 s, CPU only, with no GPU, renderer or scene data involved. With the change, tests/services/ is 22/22 and the full runtime suite is 346 passed, with one pre-existing failure (test_route_generator_map) that fails identically without this patch.

This replaces the driver-side fallback the PR originally proposed; the review comments above refer to that earlier diff.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[mwatson-nvidia]

I wonder what your thoughts are on the potential misinterpretation of the numbers, here. For instance, if the speed/acceleration is 0.0 directly after a stop, maybe this results in some weird hiccup?

Sorry to be a bit pedantic about this, but was there a case where you stumbled across this and it caused an issue or was this more of a "the comments don't match the code" sort of thing? The reason I ask is that, a while back, I thought (and, with code changes or bugs, this may no longer be the case) that I had rooted out this issue so we didn't need the fallback.

[missjetson]

Hi @mwatson-nvidia ! No worries :) Actually, fair point on post-braking transients, I double-checked with a synthetic test and confirmed the residual.

Running a 5 m/s → 0 stop through Trajectory.velocities("forward") still yields ~1.25 m/s at the exact frame the vehicle comes to a rest. A bare zero check would definitely introduce a hiccup there.

On whether this was a real incident: pulled this from a live rollout today (scene clipgt-90d1908c-9fdc-40ea-a5a1-351240aa323e): DynamicState reported speed_mps=0.0 for the first several frames of the session while ego_pose_history showed the car had actually moved 1.084m in 0.1s, ~10.84 m/s. Happens right at rollout start (frames 2-9ish), ~8–10% of frames overall hit this stale zero state right at startup. Maybe please check again.

To handle both stale states and stop transients cleanly, I propose maybe adding a minimum motion floor (MOTION_THRESHOLD_MPS = 0.5 (?) ). That way, real stops (and braking lag) stay at 0.0, while sustained movement triggers the fallback.

[mwatson-nvidia]

Sorry for the delay--I haven't had a chance to try and repro this yet. My thinking is that it might be better to try and ensure that the driver actually gets this data from the start rather than trying to add some more logic in the driver. Do you think it's possible to fix the issue that way, or is there something about the initialization and early messages from runtime that prevents this? For instance, there were recently some changes to initialization to support the video model, and maybe we just missed something at that time.

Thanks! and sorry again for the slow review.

[amtellezfernandez]

Sorry for the slow reply @mwatson-nvidia

I've tracked this down: the zero readings actually originate in the runtime, not the driver. The force-GT branch, skip-mode branch, and intermediates backfill in controller_service.py all substitute ground-truth poses but pair them with a default-constructed, all-zero DynamicState. These zeros propagate through ego_trajectory_estimate into the driver's egomotion for the entire force_gt_duration_us window (0.5 s by default, ~2 s with video-model chunking presets). That explains why your earlier cleanup didn't catch it—the zeros are unconditional in those branches, independent of initialization.

Per your suggestion that the fix belongs at the source, I’ve completely reworked the PR (see the updated description). All three fallback sites now derive linear velocity and yaw rate directly from the fallback trajectory using interpolate_delta over a 100 ms window (already expressed in the rig frame). Accelerations stay zero to match context seeding in event_loop.py, and the driver docstring has been corrected to match actual behavior.

Repro without hardware: Three new unit tests in `tests/services/test_controller_api.py`` fail on main in under a second (CPU-only).

Let me know if this source-level approach looks good to you, or if you'd prefer I split this into a fresh runtime PR keeping the original scope.

[sikai-sky-chen]

We ran into this independently, in case it helps. With a driver plugin that logs its inputs, control_timestep_us = 1_500_000 and force_gt_duration_us = 3_000_000 (main at affc2ea), the first drive call at the take-over got speed = 0 while the ego was moving at the recording's speed, and the next call got a normal value. Our plugin builds its path from that speed, so it returned a standstill path on that first call (that part is on our plugin, not AlpaSim). We now compute the speed from the last 0.5 s of poses instead.

We also checked at the default step (0.1 s), same scene and take-over, with the stock Alpamayo 1.5 and only input logging added. Same thing: the call at the take-over got speed = 0 while the poses gave the recording's speed, and the next call 0.1 s later got a normal value. That matches your diagnosis that the take-over instant is still inside the force-GT window.

As far as we can see AlpaSim's own models aren't affected (alpamayo_base.predict says command, speed and acceleration are unused, and the VAM model doesn't read speed). Thanks for the fix, and for fixing the docstring too.