Skip to content

Slice 6: video LTX-Video 2.3 22B t2v + i2v (#8)#18

Open
jmpijll wants to merge 1 commit into
v3.0from
slice/8-ltx-video
Open

Slice 6: video LTX-Video 2.3 22B t2v + i2v (#8)#18
jmpijll wants to merge 1 commit into
v3.0from
slice/8-ltx-video

Conversation

@jmpijll

@jmpijll jmpijll commented Jun 1, 2026

Copy link
Copy Markdown
Owner

Closes #8

Summary

Adds two new video-Modality Manifests for LTX-Video 2.3 22B - text-to-video and image-to-video - and chains the i2v variant from qwen_image_2512 so a finished Qwen image can be animated end-to-end. Reuses the slice 5 VideoPlugin verbatim (no plugin changes).

Acceptance criteria

  • workflows/ltxv_2_3_22b_t2v.json - text-to-video graph
  • workflows/ltxv_2_3_22b_i2v.json - image-to-video graph
  • workflows/manifests/ltxv_2_3_22b_t2v.yaml (slots: prompt, negative_prompt, width, height, frame_count, seed, lora, lora_strength)
  • workflows/manifests/ltxv_2_3_22b_i2v.yaml (above + init_image at attachment_position 1; chained from qwen_image_2512's output_image)
  • Reuse VideoPlugin - no plugin changes (the slice 5 mapper handles LTX's single SamplerCustomAdvanced Progress stream as the degenerate case of its WAN 2.2 dual-stream design)
  • Live smoke for both manifests against http://172.27.1.165:8188 - both SMOKE OK
  • i2v smoke source image generated by qwen_image_2512 first
  • Unit tests: tests/test_ltxv_manifest_loading.py + tests/test_video_plugin_ltx_compat.py
  • pytest passes locally (369 passed, 1 pre-existing skip)
  • All filenames in requires: resolved live against /object_info

Live smoke evidence

Full transcripts + reasoning in docs/v3/smoke/SLICE_6_RUNS.md.

Manifest prompt_id size latency
ltxv_2_3_22b_t2v 5955945b-0162-4730-b886-99d82c1a42ca 384 KB MP4 33.04 s
qwen_image_2512 (source for i2v) 402f9033-617c-4413-8d2d-b39128c974ec 3.13 MB PNG 27.71 s
ltxv_2_3_22b_i2v c043322d-886f-44da-902c-ce978248eb67 459 KB MP4 42.85 s

Plugin change summary

No plugin changes - VideoPlugin handled LTX-Video 2.3 22B as-is.

  • The dual-sampler progress mapper sums per-node Progress streams; with one stream it degrades to a clean monotone 0->95% then bumps on post-sample Executing (VAEDecode, VHS_VideoCombine) and flips to 100% on ExecutionComplete. Covered by TestSingleSamplerProgressMapping in tests/test_video_plugin_ltx_compat.py.
  • The MP4 render_outputs path and validate_slot_values coercion both work as-is against the LTX manifests' slot shapes (width/height multiple_of=32, frame_count min=9 max=257, dynamic LoRA enum from comfyui.loras).

Server-side notes for follow-up

During live discovery two nodes the slice 6 ticket suggested ("at minimum") turned out to be unusable on this server install (not a workflow bug):

  1. LTXVGemmaCLIPModelLoader needs a Gemma tokenizer.model sentencepiece file that isn't installed. Used LTXAVTextEncoderLoader (LTX 2.x successor, loads the same Gemma weights) instead.
  2. LTXVQ8LoraModelLoader imports hadamard_transform which isn't installed. Used plain LoraLoaderModelOnly instead - the LTX distilled LoRA is in its option list.
  3. LTXVGemmaEnhancePrompt raised min() iterable argument is empty against the LTXAVTextEncoderLoader CLIP. Skipped; the user's prompt feeds straight into CLIPTextEncode.

These are server install gaps, not architectural choices - documented in docs/v3/smoke/SLICE_6_RUNS.md so a follow-up slice can re-enable the LTX-specific loaders once the missing files / Python deps land.

Anchors

  • PRD: §3 Modality coverage (video), §5 Manifests as data
  • ADR-0001 (manifest format) - new manifests obey the schema verbatim
  • ADR-0002 (modality plugins) - VideoPlugin reused verbatim
  • ADR-0003 (slot UI hints) - dynamic LoRA enum via options_from: comfyui.loras

Test plan

  • pytest -q -> 369 passed, 1 skipped
  • pytest tests/test_ltxv_manifest_loading.py tests/test_video_plugin_ltx_compat.py -q -> 43 passed
  • Live t2v smoke against ComfyUI -> SMOKE OK
  • Live i2v smoke against ComfyUI with init_image uploaded via _upload_image_slots -> SMOKE OK

Process

Ran from worktree /Users/jamievanderpijll/discomfy-slice-6-ltx (per the slice 6 issue's "workspace isolation" rule); no files modified in the parent /Users/jamievanderpijll/discomfy checkout.

Made with Cursor

Add two new `video` Modality manifests on the LTX-Video 2.3 22B
distilled checkpoint, reusing slice 5's VideoPlugin verbatim:

- workflows/ltxv_2_3_22b_t2v.json + manifest: text-to-video via
  CheckpointLoaderSimple -> LTXAVTextEncoderLoader(gemma) ->
  LoraLoaderModelOnly -> ModelSamplingLTXV -> CLIPTextEncode ->
  EmptyLTXVLatentVideo -> LTXVScheduler / CFGGuider / KSamplerSelect
  / RandomNoise -> SamplerCustomAdvanced -> VAEDecode ->
  VHS_VideoCombine (MP4).
- workflows/ltxv_2_3_22b_i2v.json + manifest: same chain but with
  LoadImage feeding LTXVImgToVideo. `init_image` Slot at
  attachment_position 1 (role: init_image) so it chains from
  qwen_image_2512's output_image.
- qwen_image_2512.yaml: add an `animate_ltx` Action targeting the
  new ltxv_2_3_22b_i2v Manifest (side-by-side with the existing
  `animate` -> wan22_i2v).

Live discovery against `/object_info` at http://172.27.1.165:8188
revealed two server-side issues with the originally-suggested
LTX nodes:

- LTXVGemmaCLIPModelLoader needs a Gemma `tokenizer.model` file
  that isn't installed -> swapped for LTX 2.x's successor
  LTXAVTextEncoderLoader, which loads the same Gemma weights and
  exposes a standard CLIP socket.
- LTXVQ8LoraModelLoader requires the `hadamard_transform` CUDA
  kernel which isn't installed -> swapped for plain
  LoraLoaderModelOnly (the LTX distilled LoRA is in its option
  list).

Documented both in docs/v3/smoke/SLICE_6_RUNS.md so a follow-up
slice can re-enable the LTX-specific loaders once the server gains
the missing pieces.

VideoPlugin needed no changes: its dual-sampler progress mapper
sums per-node Progress streams and naturally handles LTX's single
SamplerCustomAdvanced; the MP4 renderer and slot validation work
unchanged. tests/test_video_plugin_ltx_compat.py adds a synthetic
LTX event stream + slot validation coverage to lock in the contract.

Live smokes (full evidence in docs/v3/smoke/SLICE_6_RUNS.md):

- t2v prompt_id `5955945b-0162-4730-b886-99d82c1a42ca`, 384 KB MP4,
  33 s wall-clock.
- i2v prompt_id `c043322d-886f-44da-902c-ce978248eb67`, 459 KB MP4,
  43 s wall-clock, source image from qwen_image_2512
  (`402f9033-617c-4413-8d2d-b39128c974ec`).

369 / 370 tests pass; the surviving skip is unrelated (`@pytest.mark.skip`
on a v2 integration helper). Ran from worktree
`/Users/jamievanderpijll/discomfy-slice-6-ltx`.

Closes #8

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant