Skip to content

feat(video): Add ComfyUI workflow presets for Wan 2.2, LTX-2.5, and HunyuanVideo #9

Description

@spelech

🎯 Objective & Overview

Integrate local Diffusion Transformer (DiT) video generation workflows into LocalLLMServerManager by orchestrating ComfyUI execution pipelines for Wan 2.2, LTX-2.5, and HunyuanVideo 1.5.


📂 Files & Components to Touch

  1. Workflow Presets (Workflows/Video/):
    • Workflows/Video/wan2.2_t2v.json (Wan 2.2 Text-to-Video API workflow)
    • Workflows/Video/wan2.2_i2v.json (Wan 2.2 Image-to-Video API workflow)
    • Workflows/Video/ltx2.5_t2v.json (LTX-2.5 Text-to-Video with synced audio)
    • Workflows/Video/hunyuanvideo1.5_t2v.json (HunyuanVideo 1.5 cinematic T2V)
  2. Endpoints (Endpoints/WorkflowEndpoints.cs):
    • Add GET /api/video/workflows listing available video workflow templates.
    • Add POST /api/video/generate queuing video generation on ComfyUI /prompt.
    • Add GET /api/video/files listing generated .mp4 / .webm outputs in wwwroot/output_video/ or configured video directory.
  3. Settings (LocalLLMServerManager.Shared/Models/AppSettings.cs):
    • Add string VideoModelsPath = "" and string VideoOutputPath = "" to AppSettings.
  4. VRAM Orchestration (Services/VramOrchestrator.cs):
    • Ensure EnsureVramForComfyUiAsync() unloads all active Ollama LLM models before video rendering to avoid CUDA OOM.

🔌 API Contract

POST /api/video/generate

{
  "workflowId": "wan2.2_t2v",
  "prompt": "Cinematic shot of a neon cyberpunk city at night, rain reflections, 4k",
  "negativePrompt": "blurry, low quality, distorted",
  "width": 832,
  "height": 480,
  "frames": 49,
  "fps": 16,
  "seed": -1
}

Response: 200 OK

{
  "promptId": "a1b2c3d4-e5f6-...",
  "status": "queued",
  "wsUrl": "ws://127.0.0.1:8188/ws"
}

🛠️ Step-by-Step Implementation Guide

  1. Export clean API-format ComfyUI workflow JSONs with prompt placeholder tokens ({{PROMPT}}, {{WIDTH}}, {{HEIGHT}}, {{FRAMES}}).
  2. Implement backend substitution and queuing logic in WorkflowEndpoints.cs.
  3. Track execution via ComfyUI WebSocket/polling and save output videos to output_video/.
  4. Add unit and integration tests in LocalLLMServerManager.Tests/WorkflowEndpointsTests.cs.

🧪 Verification

  • dotnet build
  • dotnet test --filter Category!=LiveExternal
  • npm run lint & npx tsc --noEmit

Activity

  1. google-labs-jules commented on Aug 23, 2026

    @google-labs-jules
    Contributor

    Jules is on it. When finished, you will see another comment and be able to review a PR.

  2. google-labs-jules commented on Aug 23, 2026

    @google-labs-jules
    Contributor

    Ready for a review! A PR has been created.

  3. spelech commented on Aug 24, 2026

    @spelech
    OwnerAuthor

    Resolved in v3.7.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions