Gap
The DDPG integration suite (tests/test_ddpg_integration.py, 5 tests) covers the
critic-loss trend (TICKET-054) and the in-loop Polyak soft-update, but the
actor side is only ever checked for finiteness.
test_closed_loop_training_step (line 51) calls update_actor(states) once
and asserts only math.isfinite(actor_loss) (line 71).
test_update_actor_gradient_ascent (test_actor_critic.py:390) computes
q_before / q_after around a single update_actor call, but the
assertions are only np.isfinite(...) — the docstring says "Q-values should
generally increase (gradient ascent)" yet the increase is never asserted.
There is no test that runs repeated update_actor steps and asserts the actor's
Q-value (the quantity the actor is trained to maximise) actually improves.
Impact
- False confidence on actor learning. A regression that silently breaks the
actor's gradient (sign flip, detached tape, zeroed actor LR) would still pass
every existing test.
- Asymmetric coverage. The critic-learning trend is asserted (TICKET-054);
the actor-learning trend is not.
Verified empirically (prototype, seed=42 fixture, 50 update_actor steps)
- Mean Q rises +0.12691 -> +0.13858 (delta +0.01166), identical across 5 runs.
- Robust across seeds 0/1/7/42 (delta +0.0017 to +0.008), all positive.
- actor_loss (= -mean Q) falls correspondingly.
Fix
Add one integration test to tests/test_ddpg_integration.py: run repeated
update_actor steps on a fixed batch and assert the actor's mean Q-value
strictly improves (later > earlier) and the actor loss decreases.
Gap
The DDPG integration suite (tests/test_ddpg_integration.py, 5 tests) covers the
critic-loss trend (TICKET-054) and the in-loop Polyak soft-update, but the
actor side is only ever checked for finiteness.
test_closed_loop_training_step(line 51) callsupdate_actor(states)onceand asserts only
math.isfinite(actor_loss)(line 71).test_update_actor_gradient_ascent(test_actor_critic.py:390) computesq_before/q_afteraround a singleupdate_actorcall, but theassertions are only
np.isfinite(...)— the docstring says "Q-values shouldgenerally increase (gradient ascent)" yet the increase is never asserted.
There is no test that runs repeated
update_actorsteps and asserts the actor'sQ-value (the quantity the actor is trained to maximise) actually improves.
Impact
actor's gradient (sign flip, detached tape, zeroed actor LR) would still pass
every existing test.
the actor-learning trend is not.
Verified empirically (prototype, seed=42 fixture, 50 update_actor steps)
Fix
Add one integration test to tests/test_ddpg_integration.py: run repeated
update_actorsteps on a fixed batch and assert the actor's mean Q-valuestrictly improves (later > earlier) and the actor loss decreases.