test: keep the fp8 tests runnable in a single pytest process - #2
Open
MT-z wants to merge 4 commits into
Open
Conversation
`test_store_kv_writes_the_slot_the_attend_kernel_will_read` reaches row 256 -- its comment asks for the 255/256 page boundary -- against a four-page, 256-slot pool, so `codes[out_loc]` gathers one row past the view. Five pages is what that row list needs. The gather raises a device-side assert, and the CUDA context does not recover from it, so in a single-process run everything scheduled afterwards is reported as failing as well -- not because those tests stop working, but because there is no longer a context to run them in. One file per process does not show that. Assisted-by: Claude Opus 5
`test_encoder_inverts_the_grid_through_the_scale_one_path` moves `rows` to the device and leaves the V tensor beside it on the host, so the kernel is handed a CPU pointer. Assisted-by: Claude Opus 5
…its ids `test_layer_ids_remap_applies_to_scales_too` backs `layer_ids=(1, 3)` on a pool built with `num_layers=LAYERS`, and LAYERS is 3, so id 3 is one past the end and the constructor raises before the assertion it is there to make. The helper now takes the depth. Assisted-by: Claude Opus 5
…the backend now reads `k_scale` / `v_scale` arrived with the fp8 store, and the backend reads them on every path, including a 16-bit pool -- which answers None. The two hand-rolled `FakeKVCache` classes in this file do not inherit the base pool, so they were left without them and raise `AttributeError` instead. Assisted-by: Claude Opus 5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nothing is wrong with the branch -- @gdevenyi has it in production and I have been running it
here since yesterday. This is about how I was running the tests.
I had been going one file per process. Running it the way
AGENTS.mdgives it --pytest tests/ -m "not slow", one process -- brought these four up, and they are the onesalready named in @gdevenyi's table on FlashML-org#354. Each is one line:
test_qsa_pool_fp8: the round-trip reaches row 256, so the pool gets the fifth pagetest_kv_fp8: the scale-one encoder's V tensor gets a devicetest_mha_pool_fp8: the remap pool is built deep enough forlayer_ids=(1, 3)test_triton_attention: the twoFakeKVCachedoubles answerk_scale/v_scalewith NoneWith them the suite reports the same either way, one process or one file per process. RTX 4090
(sm_89), i9-14900KF, torch 2.11.0+cu130, triton 3.6.0, no checkpoint; nothing outside
tests/is touched. The reasoning for each is in its commit message.
Second time at your door with a patch, and I am grateful you opened it the first.
Assisted-by: Claude Opus 5