frost(sdpa): SM80 _compile_cached keys the THD compile on the packed token totals (per-step recompile)

[vedaanta]

Context

The SM80 (A100) FROST SDPA engines (#493) compile through _compile_cached in python/cudnn/sdpa/fwd/kernels/prefill_f16_sm80.py — an lru_cache whose key includes SQ and SKV. Under THD_VARLEN those are the packed token totals, read out of the seq-length tensors at execute time.

That is the same defect #543 fixed on SM100/SM120 (the recompile leg of #552): under continuous batching the packed totals change every step, so the cache never hits and each step pays a fresh multi-second cute.compile. AGENTS.md Hard Rule 4 (added in #543) now bans runtime-data compile keys and names this site as the known open cleanup.

Proposal

Migrate the SM80 THD compile to the same shape as the SM100/SM120 fix in #543:

• token extents as cute.sym_int symbols in the fake tensors (one symbol per ragged group), so one compiled artifact re-binds any packed total;
• runtime launch scalars (grid extents, max_s_q) as cutlass.Int32 call arguments, never compile parameters — note sdpa_fwd_wrapper_sm80 already takes max_s_q from the caller;
• derived key components audited for leaks (e.g. strides computed from totals), per Rule 4;
• compile moved to plan time, with a cache-miss regression test mirroring test_dsl_sm{100,120}_thd_compile_key_plan_time_only (two executes with different packed totals, zero new cute.compile calls, numerics checked per total).

Also worth sweeping while in there: the SM80 packed-THD backward does cu_seqlens_{q,k}.to(dtype=..., device="cpu") (sdpa/bwd/kernels/bprop_f16_sm80.py) — listed in Rule 3's known violations; it is the first thing to fix if the sdpa_bwd_sm80 spec ever gains THD.

Related

• frost(sdpa): bind THD host prep to the launch stream; plan-time-only THD compile keys (#552) #543 — the SM100/SM120 fix this mirrors (dynamic token extents, plan-time-only compile keys).
• frost(sdpa): THD execute host round-trip — D2H sync, per-total recompile, and CUDA-graph capture #552 — the umbrella THD execute host-round-trip issue (D2H sync / capture legs still open).
• SM80 (A100) SDPA: FROST engines + cudnn.sdpa adapters #493 — where the SM80 engines landed.