Skip to content

Cancelable LLM calls#165

Open
saimouu wants to merge 21 commits intomainfrom
feature/cancelable-llm-calls
Open

Cancelable LLM calls#165
saimouu wants to merge 21 commits intomainfrom
feature/cancelable-llm-calls

Conversation

@saimouu
Copy link
Collaborator

@saimouu saimouu commented Mar 6, 2026

Adds the ability to cancel on going tasks: /job/{uuid}/cancel?delete_data={'false' OR 'true'}. Cancellation happens in the frontend through CancelJobModal.

If delete_data=true is passed the whole job will be deleted after cancellation.

The status of a Job is handled according to the status of JobTasks related to it. So when a job is cancelled and data is not deleted all of the not-finished JobTasks will get a status of CANCELLED. The state management is kind of complicated but having to somehow sync JobStatus and JobTaskStatus would probably be even more complicated. Now it is only calculated on demand.

Tasks are cancelled by revoking the celery task (SIGTERM), meaning open db transactions and other processes will instantly fail. This will show in the db logs as unexpected EOF on client connection with an open transaction. Most likely isn't an issue but still something to investigate.

completedCount={jobToCancel.completedCount}
totalCount={jobToCancel.totalCount}
onClose={() => setJobToCancel(null)}
onKeepData={async () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using pure async functions as React component props are not best practice, instead x.then().catch() pattern should be used. useCallback could memoize the functions to optimize rendering. Also, see useTransition (https://react.dev/reference/react/useTransition) if it could also be used.

completedCount: number;
totalCount: number;
onClose: () => void;
onKeepData: () => void;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type here should use Promise<void>, as async functions were passed to the prop. As a better alternative, the prop should still use type () => void but triggering the async prop done in a different way.

@saimouu saimouu requested a review from alehuo March 13, 2026 11:10
import { FileDropArea } from "../components/FileDropArea";
import { ExpandableToast } from "../components/ExpandableToast";
import { TruncatedFileNames } from "../components/TruncatedFileNames";
import { ConrimationModal } from "../components/ConfirmationModal";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conrimation -> Confirmation

async (actions, { jobUuid, projectUuid }, { injections }) => {
const { jobService } = injections;
await jobService.cancelJob(jobUuid);
await actions.fetchJobsForProject(projectUuid); // TODO: Maybe no fetch here
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. Optimally fetching jobs could be done outside of the thunk. Also, using abortcontroller to cancel requests if the page unmounts could be a good idea. Can be done later on

EventName.JOB_TASK_RUNNING,
{"job_task_id": job_task.id, "status": JobTaskStatus.RUNNING},
)
# await _publish_redis_event(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These can be removed from code if they are not needed anymore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants