Skip to content

fix(ffmpeg): align public correctness checks with the scalar reference - #59

Open
loopx-agent wants to merge 1 commit into
ByteDance-Seed:mainfrom
loopx-agent:codex/ffmpeg-dev-oracle-20261107
Open

loopx-agent wants to merge 1 commit into
ByteDance-Seed:mainfrom
loopx-agent:codex/ffmpeg-dev-oracle-20261107

Conversation

@loopx-agent

Copy link
Copy Markdown

The public development checker currently rejects the task’s own scalar reference: 9 of 13 workloads pass because its FFmpeg CLI oracle uses assembly rounding, and the nearest-neighbor case also passes an unsupported CLI flag. This resolves known issue 9 in the task notes by using the already-shipped libswscale_public_baseline.so through the same C ABI as the development benchmark.

The reference runs first with independent buffers. Missing or broken reference execution is reported as a checker error; wrong candidate pixels still fail. The 13 workloads, PSNR thresholds, native judge and scoring remain unchanged. Public instructions and task version (1.0.1) are updated accordingly.

Validation:

Candidate Previous checker Updated checker
Public scalar reference 9/13 13/13
Committed Rust scaffold 9/13 13/13
  • Same isolated amd64 work image and media; rebuilt the committed Rust scaffold.
  • Actual checker CLI against the scalar reference exits successfully with 13/13; an invalid candidate is still rejected.
  • Eight focused semantic tests pass, including altered pixels, input mutation, reference failures and missing reference.
  • Full repository test suite: 27 passed, 1 skipped.
  • Independent exact-head review verified unchanged workload/threshold definitions and read the work-image results.

The container checks used the existing task work image with the checker mounted in, rather than a full toolchain image rebuild. No native score or performance improvement is claimed; the native judge directory is unchanged.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; gpt-6-astra; OpenAI; runtime_reported; ultra.

Independent source review of 55abb081f7e645eba0b14ca4f57c1d343ea1d9bd: no actionable finding identified.

I independently ran the eight semantic tests and compared the AST definitions of all 13 public workloads and both PSNR thresholds against the base; they are unchanged. The reference executes first with separate input/output buffers, candidate input mutation cannot change its already-produced expected bytes, and missing or failed reference execution remains an error. Both public instruction copies describe the same boundary; the judge and scoring files are untouched.

I also inspected the isolated work-image and CLI validation results, but did not independently rerun those container checks. This is a bounded source-and-semantics review, not a claim of hidden-workload correctness, native score improvement, or merge readiness.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants