Skip to content

Run long AI requests in the background, a few steps per delivery - #85

Merged
pallaoro merged 3 commits into
mainfrom
ai-stepped-jobs
Nov 10, 2026
Merged

pallaoro merged 3 commits into
mainfrom
ai-stepped-jobs

Conversation

@pallaoro

Copy link
Copy Markdown
Member

Replaces #83, reworked after a pressure test.

Why the rework

#83 ran a whole request inside one platform-queue delivery. The services queue delivers at most 4 jobs at a time for every app together, a consumer run is capped at 15 minutes wall time, and a delivery that died left the job running forever. Reframe and highlights in the chat (next) would pass 15 minutes.

What it does

  • Everything from Run long AI requests in the background #83: a request that would watch more than ~3 minutes of footage is proposed first (what it would watch, Run in background / Not now). It runs as a job applied only over the cut it started from (otherwise kept aside: Apply anyway / Discard), and the editor takes the result in as one undo step even with the panel closed. background: true lets agents skip the proposal.
  • Stepped. The request is plain data (InstructState in instruct.ts). instructStep does one model call or one operation, and /instruct runs the same steps inline to the end. A delivery takes a 10-minute lease on the job and runs steps until it has watched footage once, is about to make a third model call, or has spent 45 s. Then it saves where it got to and books the next delivery. A second delivery while one holds the job does nothing.
  • Never stuck. Reading the conversation books a job again when its next delivery was lost (3 min), and marks a job failed with a message when it has not moved for 20 minutes.
  • Visible. While it runs, the job says what it is doing ("Watching "talk 1.mp4" for the parts worth keeping") and what it has done. The chat shows both. GET …/ask/jobs/{job} returns draft_edl, its cut so far, for a live preview next.
  • Rebased on One AI conversation beside the cut #82 and Make a video from the clips you added, from the conversation #84: add_clip and assemble run inside jobs too (assemble counts as slow).

Verified

  • Unit (293): inline and stepped runs give the same result after a JSON round trip of the state at every step, watching footage is a slow step and names the clip, plus the watching-limit tests from Run long AI requests in the background #83.
  • E2E step 16 covers everything from Run long AI requests in the background #83, plus:
    • delivery 1 stops before a third model call and books the next; progress is readable mid-job and nothing reaches the cut yet
    • a delivery while another holds the job does nothing; the next carries on and finishes
    • a lost booking is booked again on read, and a stalled job is marked failed with its message
  • Built UI under a local Worker at 1280 and 390 px: a running job shows its current step with a spinner, the steps done with ticks, and Cancel; the AI button spins.

A request that would watch more than about three minutes of footage no
longer runs inline: it changes nothing and says how much it would watch.
On a yes it becomes a job the platform queue runs (POST /api/ask/step),
from the cut as it is then. The result is written only over the cut it
started from; if someone edited meanwhile it is kept aside and offered
(Apply anyway). The editor, not the panel, watches its jobs, so a result
that lands while the panel is closed still becomes one undo step.

Agents can skip the proposal with background: true and read the job from
GET /api/projects/{id}/ask.
…rent step

A background request no longer runs whole inside one queue delivery: the
platform queue delivers a few jobs at a time for every app, and a delivery
may not outlive its 15-minute run. The request is now plain data
(InstructState): a delivery takes a lease on the job, runs steps (one model
call, or one operation it asked for) until it has watched footage once, made
two model calls or spent 45 s, saves where it got to and books the next.
A job whose booking was lost is booked again when read, and one that has
not moved for 20 minutes is marked failed, so nothing shows working forever.

While it runs the job says what it is doing and what it has done; the chat
shows both. GET …/ask/jobs/{job} also returns its cut so far (draft_edl).
The queue starts a booked delivery only on its per-minute clock, so stopping
after every footage analysis added up to a minute of waiting per clip. A
delivery now runs steps for up to two minutes (or four model calls), still far
inside the queue's 15-minute limit.
@pallaoro
pallaoro merged commit 50ad3e5 into main Nov 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant