feat: add text context CLI parameter#2
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 532d513866
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| truncate_text_context_tensor(&condition->c_crossattn, text_ctx, name); | ||
| for (size_t i = 0; i < condition->extra_c_crossattns.size(); ++i) { | ||
| std::string extra_name = std::string(name) + ".extra_c_crossattns[" + std::to_string(i) + "]"; | ||
| truncate_text_context_tensor(&condition->extra_c_crossattns[i], text_ctx, extra_name.c_str()); |
There was a problem hiding this comment.
This helper only shortens c_crossattn and extra_c_crossattns, but some models carry per-token data alongside the context. In particular, Chroma's T5CLIPEmbedder stores the DiT text mask in c_vector (conditioner.hpp:1370), and Flux pads that y tensor into txt_img_mask (flux.hpp:1004) for attention. With --text-ctx smaller than the original Chroma context and chroma_use_dit_mask enabled, the context length is reduced while the mask keeps the old length, so attention receives mismatched/incorrect masking. Please truncate the paired mask tensor when the text context is truncated, or reject this option for masked models.
Useful? React with 👍 / 👎.
| params.prompt = prompt.c_str(); | ||
| params.negative_prompt = negative_prompt.c_str(); | ||
| params.clip_skip = clip_skip; | ||
| params.text_ctx = text_ctx; |
There was a problem hiding this comment.
This copies the new field only into sd_img_gen_params_t. The same SDGenerationParams is used by the video CLI path (examples/cli/main.cpp:768 calls to_sd_vid_gen_params_t()), but sd_vid_gen_params_t and GenerationRequest(sd_vid_gen_params_t*) never carry text_ctx, so --text-ctx N in video generation is accepted and printed in the generation params but leaves request.text_ctx at 0 and never truncates in prepare_image_generation_embeds. Please add the field to the video params path or reject it for video mode.
Useful? React with 👍 / 👎.
1 participant
Summary
Adds a
--text-ctxCLI option and passes it through the stable-diffusion.cpp API/runtime configuration.This is needed by the 1024-resolution Flux/Z-Image reproduction commands that use a shorter text context.
Notes
This PR is stacked on #1 and does not change ggml itself.