Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .claude/agents/kie-model-onboarding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
name: kie-model-onboarding
description: Use this subagent when the user wants to add a new Kie.ai image or video model into kie-api using the repo's existing spec, validation, prompt preset, and artifact workflow.
tools: Bash, Read, Grep, Glob
---

You are the Kie.ai model onboarding specialist for this repo.

Use this workflow:
1. inspect the live Kie.ai model page and docs page
2. capture the exact request body shape and provider model string
3. decide how the model fits the existing task-shape system
4. update model specs and provenance
5. add prompt preset coverage for each supported request shape
6. add dry-run normalization, validation, and payload tests
7. run the cheapest practical live smoke path
8. verify artifacts and docs

Important rules:
- do not invent provider fields
- if docs and live behavior disagree, prefer live behavior and document the mismatch
- do not treat every new video model like Kling if its shape is different
- if the model supports mutually-exclusive input scenarios, validate them explicitly

Seedance 2.0 note:
- treat first-frame, first+last-frame, and multimodal reference as separate validated scenarios
- do not model it as Kling-style `multi_prompt` unless the API actually requires it

If the user asks for help, explain that this subagent is for safely bringing new models online in kie-api, not for running a single generation request.
34 changes: 34 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Tests

on:
pull_request:
push:
branches:
- main

jobs:
pytest:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.11"]

steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install package and test dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -e '.[dev]'

- name: Verify packaged specs are in sync
run: python scripts/sync_packaged_specs.py --check

- name: Run test suite
run: python -m pytest -q
24 changes: 22 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Kie.ai is a marketplace-style API platform for creative AI models. Instead of on
Examples of model families currently covered by this repo:
- Nano Banana 2
- Nano Banana Pro
- Seedance 2.0
- Kling 2.6 text-to-video
- Kling 2.6 image-to-video
- Kling 3.0 text-to-video
Expand Down Expand Up @@ -73,6 +74,12 @@ These descriptions are based on the current public Kie.ai model pages and live p
- What it does: creates short videos from text or images with native audio support and structured text-to-video or image-to-video flows
- Who it is by: Kling AI on Kie.ai

### Seedance 2.0

- Good at: multimodal guided video generation with first-frame, first+last-frame, and mixed image/video/audio references
- What it does: creates videos from plain prompts or reference-driven payloads, supports first and last frame guidance, multiple reference assets, optional audio generation, and reference-aware prompting
- Who it is by: ByteDance on Kie.ai

### Kling 3.0

- Good at: higher-end cinematic video generation, stronger consistency, multi-shot direction, and image-to-video workflows
Expand Down Expand Up @@ -148,6 +155,19 @@ Use one model’s output as another model’s input:
- feed that image into Kling 3.0 image-to-video
- store both runs and link them through artifacts

### 5. Multimodal reference video

Use `seedance-2.0` when you need richer video guidance:
- plain text-to-video with no media
- one first-frame image
- first+last-frame image guidance
- multimodal reference video with reference images, videos, and audio

Seedance uses role-aware media references in the runtime request model:
- `first_frame`
- `last_frame`
- `reference`

## First live proof path

The easiest way to prove your setup works is:
Expand Down Expand Up @@ -217,7 +237,7 @@ Start with:
- Your KIE key belongs in `KIE_API_KEY`.
- Do not hardcode it in Python.
- Do not commit `.env.live`.
- Use the tracked [`.env.example`](/Users/evilone/Documents/Development/Video-Image-APIs/kie-ai/kie_codex_bootstrap/.env.example) as the local template.
- Use the tracked [`.env.example`](.env.example) as the local template.
- The repo ignores local env files so your personal key stays out of Git.

## What it includes
Expand Down Expand Up @@ -479,4 +499,4 @@ Pricing note:

## License

This repo is licensed under the MIT License. See [LICENSE](/Users/evilone/Documents/Development/Video-Image-APIs/kie-ai/kie_codex_bootstrap/LICENSE).
This repo is licensed under the MIT License. See [LICENSE](LICENSE).
1 change: 1 addition & 0 deletions agent_skills/codex/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Included skills:
- `nano-banana`
- `kling-video`
- `chain-image-to-video`
- `model-onboarding`

To install them into your local Codex skills folder:

Expand Down
106 changes: 106 additions & 0 deletions agent_skills/codex/model-onboarding/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
---
name: kie-model-onboarding
description: Onboard a new Kie.ai image or video model into kie-api using the repo's spec-first workflow. Use when adding a new Kie.ai model or mode, verifying docs vs live behavior, creating prompt presets, adding tests, and preparing the model for wrapper or dashboard use.
---

# Kie Model Onboarding

Use this skill when bringing a new Kie.ai model online in `kie-api`.

This is not a generation skill. It is a spec-first implementation workflow.

## Primary goal

Bring a new model online in a way that matches the existing system:
- model spec
- validation
- prompt preset coverage
- payload construction
- pricing/preflight
- live smoke verification
- artifact compatibility

## Required workflow

1. Read the live Kie.ai market page and docs page.
2. Capture the provider model string and exact request body shape.
3. Decide whether the model should be:
- a new standalone model key
- a family alias branch
- or a new request shape on an existing endpoint
4. Add or update the model spec under `specs/models/`.
5. Record field-level provenance.
6. Add prompt presets for every supported request shape.
7. Add dry-run tests for:
- normalization
- validation
- payload building
- preset resolution
8. Run the cheapest practical live smoke path.
9. Verify outputs, download flow, and artifacts.
10. Update docs.

## Decision rules

- Do not invent provider fields that the docs or live surface do not expose.
- If docs and live behavior disagree, prefer live provider truth and document the mismatch.
- If a model supports multiple mutually-exclusive input scenarios, model that explicitly in validation instead of hiding it inside passthrough options.
- Reuse existing request shapes when they truly fit.
- Add a new input-pattern concept only when the current patterns are not expressive enough.

## Existing shape system

Common task shapes already used in this repo:
- `prompt_only`
- `single_image`
- `first_last_frames`
- `image_edit`
- `motion_control`

If a new model introduces a materially different shape, add it deliberately and document why.

## Seedance 2.0 guidance

For Seedance 2.0 specifically:
- provider model is `bytedance/seedance-2`
- it is one multimodal video endpoint with mutually-exclusive scenarios
- first-frame / first+last-frame / multimodal-reference should be treated as separate validated shapes
- do not model it as Kling-style `multi_prompt` unless the docs and live API explicitly require that
- multimodal reference support should account for:
- `reference_image_urls`
- `reference_video_urls`
- `reference_audio_urls`
- the current docs indicate:
- `resolution`: `480p | 720p`
- `aspect_ratio`: `16:9 | 4:3 | 1:1 | 3:4 | 9:16 | 21:9`
- `duration`: provider-controlled numeric seconds
- `return_last_frame`
- `generate_audio`
- `web_search`

## Required files to update

- `specs/models/*.yaml`
- packaged spec copy under `src/kie_api/resources/specs/models/`
- `specs/prompt_profiles/*` when new presets are needed
- packaged preset copy under `src/kie_api/resources/prompt_profiles/*`
- runtime request/validator/normalizer/payload code when shape support changes
- tests
- docs

## Validation gates

Before calling a model wrapper-ready, all of these should be true:
- dry-run normalization works
- validation catches impossible combinations
- payload tests pass
- prompt preset resolution is correct
- one live smoke path succeeds
- artifact output was inspected

## Help

If the user asks for help, explain:
- this skill is for adding or hardening models, not running generation
- the repo is spec-first
- the safest order is docs -> spec -> tests -> live smoke -> docs update
8 changes: 8 additions & 0 deletions docs/AGENT_SKILLS.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,15 @@ Codex:
- `kie-nano-banana`
- `kie-kling-video`
- `kie-chain-image-to-video`
- `kie-model-onboarding`

Claude Code:
- `.claude/agents/kie-find-latest-media.md`
- `.claude/agents/kie-check-credits.md`
- `.claude/agents/kie-nano-banana.md`
- `.claude/agents/kie-kling-video.md`
- `.claude/agents/kie-chain-image-to-video.md`
- `.claude/agents/kie-model-onboarding.md`

## What these skills do

Expand Down Expand Up @@ -78,6 +80,12 @@ Claude Code:
- reads the artifact index and manifest layer
- returns direct file paths so the user does not need to hunt

### `kie-model-onboarding`
- adds new Kie.ai image or video models to `kie-api`
- follows the repo's spec-first onboarding workflow
- covers request shape analysis, prompt presets, tests, and live smoke verification
- is the right workflow for models like Seedance 2.0

## Standard result contract

Generation skills should return the actual media, not just a success message.
Expand Down
68 changes: 68 additions & 0 deletions docs/LIBRARY_USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,74 @@ print(validation.normalized_request.debug["frame_guidance_mode"])

Use `examples/dry_run_kling_frame_guidance.py` for a local dry-run check, including a path that can reuse the latest successful Nano Banana 2 artifact as the first frame.

## Seedance 2.0 multimodal reference shape

Seedance 2.0 is the first model in `kie-api` that uses role-aware media references.

Rules:
- keep media grouped by type:
- images in `images`
- videos in `videos`
- audios in `audios`
- add a `role` to each media item:
- `first_frame`
- `last_frame`
- `reference`
- the valid Seedance scenarios are:
- text-to-video with no media
- one `first_frame` image
- one `first_frame` plus one `last_frame` image
- multimodal reference media using `reference` roles
- first/last-frame mode and multimodal-reference mode are mutually exclusive

Example:

```python
from kie_api import normalize_request, resolve_prompt_context, validate_request
from kie_api.models import RawUserRequest

request = RawUserRequest(
model_key="seedance-2.0",
prompt="Reference @image1 and @video1 for the fighter's design and motion language.",
images=[
{"url": "https://example.com/character.png", "role": "reference"},
{"url": "https://example.com/scene.png", "role": "reference"},
],
videos=[
{"url": "https://example.com/motion.mp4", "role": "reference", "duration_seconds": 12},
],
audios=[
{"url": "https://example.com/rhythm.mp3", "role": "reference"},
],
options={"duration": 8, "resolution": "720p", "generate_audio": True},
)

normalized = normalize_request(request)
context = resolve_prompt_context(normalized)
validation = validate_request(normalized)

print(context.input_pattern)
print(context.resolved_preset_key)
print(context.rendered_system_prompt)
print(validation.state)
```

Seedance prompt context adds reference-aware render variables:
- `first_frame_present`
- `last_frame_present`
- `reference_image_count`
- `reference_video_count`
- `reference_audio_count`
- `reference_asset_guide`

`reference_asset_guide` renders deterministic prompt tokens like:
- `@image1`
- `@image2`
- `@video1`
- `@audio1`

Those tokens are for wrapper-side prompt enhancement and future dashboard/editor UX. They do not change the upload-first requirement.

## Kling 3.0 multi-shot shape

Kling 3.0 now supports the docs-aligned `multi_prompt` request shape in the runtime models.
Expand Down
7 changes: 7 additions & 0 deletions docs/MODEL_ONBOARDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Typical request shapes:
- `first_last_frames`
- `image_edit`
- `motion_control`
- `multimodal_reference` when a provider supports mixed image/video/audio guidance that is not equivalent to first/last-frame or edit mode

A preset is ready when:
- the template exists
Expand Down Expand Up @@ -61,6 +62,12 @@ For advanced provider-specific shapes such as Kling 3.0 multi-shot mode:
- validate cross-field rules explicitly
- do not hide docs-only shape differences inside generic passthrough options

For multimodal video models such as Seedance 2.0:
- treat first-frame, first+last-frame, and multimodal-reference as mutually-exclusive validated scenarios if the provider documents them that way
- do not force them into a Kling-style multi-shot abstraction unless the provider request shape actually exposes shot arrays
- if multimodal references introduce mixed image/video/audio guidance, prefer a dedicated input-pattern binding over overloading existing `single_image` or `first_last_frames` logic
- use role-aware media references when the same endpoint needs to distinguish first-frame, last-frame, and general reference assets

Known current TODO:
- Kling 3.0 `kling_elements` / element-reference support is documented by Kie.ai, but is not yet modeled in the runtime request types or upload flow here

Expand Down
6 changes: 6 additions & 0 deletions docs/PRICING_AND_PREFLIGHT.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,12 @@ That code is:
- not authoritative
- not a replacement for a verified billing API contract

Current example:
- Seedance 2.0 pricing is now derived from the public site pricing API rows for:
- `480p` vs `720p`
- `with video input` vs `no video input`
- the runtime turns that into a dry-run `pricing_variant` internally based on request shape, so wrappers do not need to send pricing-only fields

Manual candidate refresh:

```bash
Expand Down
Loading
Loading