fix(api/task): guard empty results list in process_results#1305
Draft
Luodian wants to merge 1 commit into
Draft
Conversation
For generate_until / generate_visual_cot tasks, when a sample's generation failed upstream (model raised, retries exhausted, etc.) the results list can be empty. The existing isinstance check then falls through to `results[0]` on the list-of-list branch and raises IndexError — which aborts the full postprocess loop for that task, not just the missing sample. Add a leading `results and` so an empty list falls through to the else branch that does `[res.strip() for res in results]` (empty output) — downstream task-level process_results then receives an empty list and can decide how to score the missing sample without taking down the whole task.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
For
generate_until/generate_visual_cottasks, when a sample's generation failed upstream (model raised, retries exhausted, returned[]), the results list handed toConfigurableTask.process_resultscan be empty. The existingisinstancecheck then falls through toresults[0]on the list-of-list branch and raisesIndexError— which aborts the whole postprocess loop for that task, not just the missing sample.Change
Add a leading
results andso an empty list routes to the else branch (which does[res.strip() for res in results]and produces[]). Downstream task-levelprocess_resultsthen receives an empty list and can decide how to score the missing sample without taking down the whole task.Test plan
[]— verify the task finishes and the missing sample is scored by downstreamprocess_results.