Slice 6: video LTX-Video 2.3 22B t2v + i2v (#8)#18
Open
jmpijll wants to merge 1 commit into
Open
Conversation
Add two new `video` Modality manifests on the LTX-Video 2.3 22B distilled checkpoint, reusing slice 5's VideoPlugin verbatim: - workflows/ltxv_2_3_22b_t2v.json + manifest: text-to-video via CheckpointLoaderSimple -> LTXAVTextEncoderLoader(gemma) -> LoraLoaderModelOnly -> ModelSamplingLTXV -> CLIPTextEncode -> EmptyLTXVLatentVideo -> LTXVScheduler / CFGGuider / KSamplerSelect / RandomNoise -> SamplerCustomAdvanced -> VAEDecode -> VHS_VideoCombine (MP4). - workflows/ltxv_2_3_22b_i2v.json + manifest: same chain but with LoadImage feeding LTXVImgToVideo. `init_image` Slot at attachment_position 1 (role: init_image) so it chains from qwen_image_2512's output_image. - qwen_image_2512.yaml: add an `animate_ltx` Action targeting the new ltxv_2_3_22b_i2v Manifest (side-by-side with the existing `animate` -> wan22_i2v). Live discovery against `/object_info` at http://172.27.1.165:8188 revealed two server-side issues with the originally-suggested LTX nodes: - LTXVGemmaCLIPModelLoader needs a Gemma `tokenizer.model` file that isn't installed -> swapped for LTX 2.x's successor LTXAVTextEncoderLoader, which loads the same Gemma weights and exposes a standard CLIP socket. - LTXVQ8LoraModelLoader requires the `hadamard_transform` CUDA kernel which isn't installed -> swapped for plain LoraLoaderModelOnly (the LTX distilled LoRA is in its option list). Documented both in docs/v3/smoke/SLICE_6_RUNS.md so a follow-up slice can re-enable the LTX-specific loaders once the server gains the missing pieces. VideoPlugin needed no changes: its dual-sampler progress mapper sums per-node Progress streams and naturally handles LTX's single SamplerCustomAdvanced; the MP4 renderer and slot validation work unchanged. tests/test_video_plugin_ltx_compat.py adds a synthetic LTX event stream + slot validation coverage to lock in the contract. Live smokes (full evidence in docs/v3/smoke/SLICE_6_RUNS.md): - t2v prompt_id `5955945b-0162-4730-b886-99d82c1a42ca`, 384 KB MP4, 33 s wall-clock. - i2v prompt_id `c043322d-886f-44da-902c-ce978248eb67`, 459 KB MP4, 43 s wall-clock, source image from qwen_image_2512 (`402f9033-617c-4413-8d2d-b39128c974ec`). 369 / 370 tests pass; the surviving skip is unrelated (`@pytest.mark.skip` on a v2 integration helper). Ran from worktree `/Users/jamievanderpijll/discomfy-slice-6-ltx`. Closes #8 Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #8
Summary
Adds two new
video-Modality Manifests for LTX-Video 2.3 22B - text-to-video and image-to-video - and chains the i2v variant fromqwen_image_2512so a finished Qwen image can be animated end-to-end. Reuses the slice 5VideoPluginverbatim (no plugin changes).Acceptance criteria
workflows/ltxv_2_3_22b_t2v.json- text-to-video graphworkflows/ltxv_2_3_22b_i2v.json- image-to-video graphworkflows/manifests/ltxv_2_3_22b_t2v.yaml(slots: prompt, negative_prompt, width, height, frame_count, seed, lora, lora_strength)workflows/manifests/ltxv_2_3_22b_i2v.yaml(above +init_imageat attachment_position 1; chained fromqwen_image_2512'soutput_image)VideoPlugin- no plugin changes (the slice 5 mapper handles LTX's singleSamplerCustomAdvancedProgress stream as the degenerate case of its WAN 2.2 dual-stream design)http://172.27.1.165:8188- bothSMOKE OKqwen_image_2512firsttests/test_ltxv_manifest_loading.py+tests/test_video_plugin_ltx_compat.pypytestpasses locally (369 passed, 1 pre-existing skip)requires:resolved live against/object_infoLive smoke evidence
Full transcripts + reasoning in
docs/v3/smoke/SLICE_6_RUNS.md.ltxv_2_3_22b_t2v5955945b-0162-4730-b886-99d82c1a42caqwen_image_2512(source for i2v)402f9033-617c-4413-8d2d-b39128c974ecltxv_2_3_22b_i2vc043322d-886f-44da-902c-ce978248eb67Plugin change summary
No plugin changes -
VideoPluginhandled LTX-Video 2.3 22B as-is.Progressstreams; with one stream it degrades to a clean monotone 0->95% then bumps on post-sampleExecuting(VAEDecode, VHS_VideoCombine) and flips to 100% onExecutionComplete. Covered byTestSingleSamplerProgressMappingintests/test_video_plugin_ltx_compat.py.render_outputspath andvalidate_slot_valuescoercion both work as-is against the LTX manifests' slot shapes (width/height multiple_of=32, frame_count min=9 max=257, dynamic LoRA enum fromcomfyui.loras).Server-side notes for follow-up
During live discovery two nodes the slice 6 ticket suggested ("at minimum") turned out to be unusable on this server install (not a workflow bug):
LTXVGemmaCLIPModelLoaderneeds a Gemmatokenizer.modelsentencepiece file that isn't installed. UsedLTXAVTextEncoderLoader(LTX 2.x successor, loads the same Gemma weights) instead.LTXVQ8LoraModelLoaderimportshadamard_transformwhich isn't installed. Used plainLoraLoaderModelOnlyinstead - the LTX distilled LoRA is in its option list.LTXVGemmaEnhancePromptraisedmin() iterable argument is emptyagainst theLTXAVTextEncoderLoaderCLIP. Skipped; the user's prompt feeds straight intoCLIPTextEncode.These are server install gaps, not architectural choices - documented in
docs/v3/smoke/SLICE_6_RUNS.mdso a follow-up slice can re-enable the LTX-specific loaders once the missing files / Python deps land.Anchors
options_from: comfyui.lorasTest plan
pytest -q-> 369 passed, 1 skippedpytest tests/test_ltxv_manifest_loading.py tests/test_video_plugin_ltx_compat.py -q-> 43 passedSMOKE OKinit_imageuploaded via_upload_image_slots->SMOKE OKProcess
Ran from worktree
/Users/jamievanderpijll/discomfy-slice-6-ltx(per the slice 6 issue's "workspace isolation" rule); no files modified in the parent/Users/jamievanderpijll/discomfycheckout.Made with Cursor