Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Most editors keep a project in a format built for their own app, not for you or
- **Colour**: brightness, contrast, saturation and temperature on any clip, logo or picture-in-picture, and one click to give every clip the same look, for flat log footage or two cameras that do not match. The preview draws it with the same maths the export renders.
- **Transitions**: put a dissolve, a fade through black or white, a wipe, a slide, a blur or a pixelate on any cut, from the mark between two clips on the timeline, or on every cut at once. A transition is centred on its cut and moves no clip, so titles and music stay where you put them, and the preview draws each one frame for frame as it exports.
- **Captions**: switch them on for the whole video, pick one of twelve languages and one style. The words come from each clip's transcript, so captions follow every trim, split and reorder.
- **AI, in one conversation**: add your clips and say what you want beside the cut, in your own words: it builds the cut from them ("a 30 second video from my clips") or changes the one you have. It remembers the conversation, so "shorter" or "a square one too" knows what it refers to, and it asks back instead of guessing when a request could mean two things. The model calls a fixed set of checked operations, so the edit is always a valid document. One request, one undo.
- **AI, in one conversation**: add your clips and say what you want beside the cut, in your own words: it builds the cut from them ("a 30 second video from my clips") or changes the one you have. It remembers the conversation, so "shorter" or "a square one too" knows what it refers to, and it asks back instead of guessing when a request could mean two things. A request that would watch a lot of footage says how much first and, on a yes, runs in the background: keep editing or close the tab, and it never lands on top of a change you made meanwhile. The model calls a fixed set of checked operations, so the edit is always a valid document. One request, one undo.
- **AI assist**: Auto-cut watches several clips together and assembles the strongest sequence for what the video is for, captions included; Clean up trims one clip down to its good parts.
- **Export to MP4** in draft, standard or high quality, on Clawnify's managed edit service.
- **Share by link**: one link per project plays an export to anyone you send it to, with no sign-in, and a Download button. The link stays on the export you shared until you move it to a newer one, so a draft never reaches viewers; turning it off ends it for everyone.
Expand Down
23 changes: 21 additions & 2 deletions agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,21 @@ more than two clips, it answers with `asked: true` and a question or plan in
`said`, and changes nothing. Answer by sending the reply as the next
`instruction`.

A request that would watch more than about three minutes of footage is not
answered inline: the reply has `asked: true` and a `job` with `status:
"proposed"`, and `said` says how much it would watch. Run it with
`POST …/ask/jobs/{job}/run`, or send `background: true` with the instruction
to skip the proposal when you already know you want it. Then read
`GET /api/projects/{id}/ask` until the job is no longer `queued` or `running`.
A job runs a few steps at a time; while it runs, `doing` says what it is doing
now and `applied` lists what it has done so far. It ends as:

- `applied`: the cut was changed (or the answer was a question, in the conversation)
- `stale`: someone changed the cut while it ran, so the result was kept aside,
never written over their edit. Apply it with `…/apply` only if the person wants it
- `failed` (with `error`, e.g. "stopped without finishing" when it has not moved
for 20 minutes) or `cancelled`

Google Drive limits how often a file shared with the link can be downloaded,
and on a heavy day it refuses the files of a whole folder for a while. It
refuses the whole file but still serves pieces of it, so a refused file of up
Expand Down Expand Up @@ -301,8 +316,12 @@ in line, and asks Drive again now for the clips waiting on it. `DELETE /api/proj
| GET | `/api/projects/{id}/highlights/export` | `format=csv\|xml`: a sheet, or a Premiere Pro / Resolve timeline of the picks |
| PUT | `/api/projects/{id}` | Update `{ name?, brief?, edl? }` — the EDL is validated on save |
| POST | `/api/projects/{id}/autocut` | `{ asset_ids, prompt? }` — AI assembles the main track from several clips |
| POST | `/api/projects/{id}/instruct` | `{ instruction }` — change the existing cut in words → `{ edl, said, applied, asked }`; `asked: true` means `said` is a question back and nothing was changed |
| GET | `/api/projects/{id}/ask` | The project's AI conversation, oldest first → `{ messages: [{ role, text, author, applied, asked, failed }] }` |
| POST | `/api/projects/{id}/instruct` | `{ instruction, background? }` — change the existing cut in words → `{ edl, said, applied, asked, job? }`; `asked: true` means `said` is a question back (or, with `job`, a proposal) and nothing was changed. `background: true` → `202` with a queued `job` |
| GET | `/api/projects/{id}/ask` | The project's AI conversation, oldest first, and its recent background jobs → `{ messages: [{ role, text, author, applied, asked, failed, job_id }], jobs }` |
| GET | `/api/projects/{id}/ask/jobs/{job}` | One background job with `base_edl` (the cut it started from), `draft_edl` (its cut so far, while it runs) and `result_edl` |
| POST | `/api/projects/{id}/ask/jobs/{job}/run` | Yes to a proposal: run it in the background → `202` |
| POST | `/api/projects/{id}/ask/jobs/{job}/cancel` | Drop a proposal, stop a queued or running job (its result is not applied), or let go of a kept one |
| POST | `/api/projects/{id}/ask/jobs/{job}/apply` | Apply a result kept aside because the cut changed while it ran |
| DELETE | `/api/projects/{id}/ask` | Start the conversation over; the cut is untouched |
| DELETE | `/api/projects/{id}` | Delete a project, its export history and its footage from Drive; `202 { remaining }` means call again |
| POST | `/api/projects/{id}/export` | Export `{ quality?, format? }` → returns the job, `status: "exporting"` |
Expand Down
3 changes: 2 additions & 1 deletion clawnify.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
{ "path": "/api/drive/imports/step", "methods": ["POST"] },
{ "path": "/s/**", "methods": ["GET", "HEAD"] },
{ "path": "/s/*/comments", "methods": ["POST"] },
{ "path": "/api/footage/step", "methods": ["POST"] }
{ "path": "/api/footage/step", "methods": ["POST"] },
{ "path": "/api/ask/step", "methods": ["POST"] }
],
"server_routes": ["/s/*"]
},
Expand Down
38 changes: 38 additions & 0 deletions schema.sql
Original file line number Diff line number Diff line change
Expand Up @@ -428,3 +428,41 @@ CREATE TABLE IF NOT EXISTS ask_messages (
);

CREATE INDEX IF NOT EXISTS idx_ask_messages_project ON ask_messages(project_id, created_at);

-- A request that watches more footage than a quick answer allows runs in the
-- background (src/server/ask-jobs.ts), a few steps per queue delivery. `proposed`: told what it would watch,
-- waiting for a yes. `queued` -> `running` -> `applied` (the cut was changed),
-- `stale` (the cut changed while it ran, so it was kept aside: `result_edl`,
-- applied only on request), `failed` or `cancelled`. `base_edl` is the cut it
-- started from, so it never overwrites an edit made meanwhile.
CREATE TABLE IF NOT EXISTS ask_jobs (
id TEXT PRIMARY KEY DEFAULT (lower(hex(randomblob(8)))),
project_id TEXT NOT NULL,
instruction TEXT NOT NULL,
status TEXT NOT NULL CHECK (status IN ('proposed', 'queued', 'running', 'applied', 'stale', 'failed', 'cancelled')),
clips INTEGER NOT NULL DEFAULT 0,
seconds REAL,
base_edl TEXT,
result_edl TEXT,
said TEXT,
applied TEXT,
error TEXT,
author TEXT,
-- The request in progress (src/server/instruct.ts InstructState), saved
-- after every step, and what it is doing now in words.
state TEXT,
doing TEXT,
-- A delivery holds the job while it runs a step; past this time another may.
leased_until TEXT,
-- When the next step was last booked on the queue.
step_booked_at TEXT,
started_at TEXT,
finished_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX IF NOT EXISTS idx_ask_jobs_project ON ask_jobs(project_id, created_at);

-- The background job an AI message belongs to, when it does.
ALTER TABLE ask_messages ADD COLUMN job_id TEXT;
Loading
Loading