Repository navigation
Run long AI requests in the background, a few steps per delivery - #85
Merged
Merged
Conversation
A request that would watch more than about three minutes of footage no
longer runs inline: it changes nothing and says how much it would watch.
On a yes it becomes a job the platform queue runs (POST /api/ask/step),
from the cut as it is then. The result is written only over the cut it
started from; if someone edited meanwhile it is kept aside and offered
(Apply anyway). The editor, not the panel, watches its jobs, so a result
that lands while the panel is closed still becomes one undo step.
Agents can skip the proposal with background: true and read the job from
GET /api/projects/{id}/ask.
…rent step
A background request no longer runs whole inside one queue delivery: the
platform queue delivers a few jobs at a time for every app, and a delivery
may not outlive its 15-minute run. The request is now plain data
(InstructState): a delivery takes a lease on the job, runs steps (one model
call, or one operation it asked for) until it has watched footage once, made
two model calls or spent 45 s, saves where it got to and books the next.
A job whose booking was lost is booked again when read, and one that has
not moved for 20 minutes is marked failed, so nothing shows working forever.
While it runs the job says what it is doing and what it has done; the chat
shows both. GET …/ask/jobs/{job} also returns its cut so far (draft_edl).
The queue starts a booked delivery only on its per-minute clock, so stopping after every footage analysis added up to a minute of waiting per clip. A delivery now runs steps for up to two minutes (or four model calls), still far inside the queue's 15-minute limit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #83, reworked after a pressure test.
Why the rework
#83 ran a whole request inside one platform-queue delivery. The services queue delivers at most 4 jobs at a time for every app together, a consumer run is capped at 15 minutes wall time, and a delivery that died left the job
runningforever. Reframe and highlights in the chat (next) would pass 15 minutes.What it does
background: truelets agents skip the proposal.InstructStateininstruct.ts).instructStepdoes one model call or one operation, and/instructruns the same steps inline to the end. A delivery takes a 10-minute lease on the job and runs steps until it has watched footage once, is about to make a third model call, or has spent 45 s. Then it saves where it got to and books the next delivery. A second delivery while one holds the job does nothing.GET …/ask/jobs/{job}returnsdraft_edl, its cut so far, for a live preview next.add_clipandassemblerun inside jobs too (assemble counts as slow).Verified