Skip to content

feat(llm): enrich polish prompts with directing heuristics for higher image/video quality - #62

Open
Yuyyxz wants to merge 1 commit into
alibaba:mainfrom
Yuyyxz:feat/director-methodology
Open

feat(llm): enrich polish prompts with directing heuristics for higher image/video quality#62
Yuyyxz wants to merge 1 commit into
alibaba:mainfrom
Yuyyxz:feat/director-methodology

Conversation

@Yuyyxz

@Yuyyxz Yuyyxz commented Aug 18, 2026

Copy link
Copy Markdown

Summary

The three default polish prompts (DEFAULT_STORYBOARD_POLISH_PROMPT, DEFAULT_VIDEO_POLISH_PROMPT, DEFAULT_R2V_POLISH_PROMPT) currently give the LLM only high-level instructions ("describe motion + camera"). This leaves quality to the model's defaults, which often produces vague adjectives ("beautiful / stunning / masterpiece") and stacked actions that cause limb deformation.

This PR injects a set of directing heuristics into the prompts, so the LLM polishes every shot with concrete, model-friendly constraints.

What changed

All changes are in src/apps/comic_gen/llm.py (3 default prompt strings, +29/-1). No call-site logic changed — these prompts were already designed to be overridable.

1. DEFAULT_STORYBOARD_POLISH_PROMPT (image)

  • Ban filler adjectives; require concrete visual facts (light direction/quality/highlight/shadow)
  • Require lighting sculpting (source direction + quality + highlight + shadow placement)
  • Require material-level detail density (skin pores, fabric folds, dust in light)
  • One image one job; enforce style consistency

2. DEFAULT_VIDEO_POLISH_PROMPT (video)

  • One primary action per shot (stacking actions = 之前oss可以不用配置的,现在要强制使用了嘛? #1 cause of limb breakdown)
  • Motion-amplitude tiers (slight/slowly/uniform/obvious; default slight/slowly; ban violent/fast)
  • Body linkage (turn head → shoulder turn → hair follows)
  • Body-part-level detail (fingertips, jawline, Adam's apple, eyelashes, breath rhythm)
  • Order multi-person actions explicitly
  • Dialogue mouth-motion cue

3. DEFAULT_R2V_POLISH_PROMPT (reference-to-video)

  • Verbatim anchor restatement (appearance anchors must be copied verbatim, no drift)
  • Motion-amplitude tiers
  • Style consistency with reference assets

Verification

Tested against a deliberately bad draft ("a beautiful girl stands up, walks to the window, turns, sits, stands again, gorgeous, masterpiece, cinematic"). The polished output now includes motion-amplitude words (slowly/gently) and body linkage ("head rotates smoothly, causing her hair to sway; shoulders dip and rise") — heuristics the old prompt never produced.

Notes

  • These are default prompts; per-project overrides via PromptConfig still take precedence (existing 3-level fallback unchanged).
  • Content is original methodology, no copyright concerns.

Happy to adjust wording or split into separate PRs if preferred.

将 KPE + Master-director 的导演方法论注入三个 polish prompt:

- DEFAULT_STORYBOARD_POLISH_PROMPT(图像): 拒绝空话套话/光影塑形/细节密度/一图一职/风格一致性
- DEFAULT_VIDEO_POLISH_PROMPT(视频): 一镜一主动作/动作幅度分级/身体联动/细节到部位级/多人动作顺序/对白口型
- DEFAULT_R2V_POLISH_PROMPT(参考视频): 锚点逐字复述/动作幅度分级/风格一致

实测: 故意给违反铁律的草稿(堆5个动作+空话), LLM 按导演铁律输出(幅度分级slight/slowly+身体联动+光影塑形)。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant