Repository navigation
juliet_vulnerability_analyzer: accept the release as a CWE-416 source - #55
Draft
findalexli wants to merge 1 commit into
Draft
findalexli wants to merge 1 commit into
findalexli wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task finding: #50.
What and why
The README says
source_stmt"must identify the source/allocation/release witness", buttrace_validrejects a CWE-416 finding that starts at the release. For CWE-416 only, it now also accepts afreeordeleteof a value the sink uses, in the sink's function, between the labelled allocation and the sink, with[release, sink]as the required witness. The public scorer is the same file and gets the same change.Decision for maintainers
This PR follows the README and can't lower a score. Or change the README so CWE-416 names the allocation: no score moves, and the analyzer below stays at 85.66. We prefer this PR because the README is what solvers were told. Someone who knows the field should check that the release is an acceptable source for a use-after-free trace.
It extends a looseness the judge already has:
trace_validaccepts any existing statements between source and sink, so[release, statement, sink]passes as[allocation, statement, sink]does. Tightening that would be a separate change.Old vs new judge
tools/judge_local.py, f92857a (1.1.0) against this branch, three grades per judge per row on one worker, reversing order, no spread.For each of the 723 hidden labels, every statement in its case as
source_stmt, with trace[statement, sink], old against newtrace_valid: 118255 comparisons, 0 accepted traces rejected, 120 newly accepted, one release per CWE-416 label. Of 38 distinct Opus 5.5 analyzers, one uses the release and 37 don't, so their scores don't move. Census and public-label results: https://github.com/findalexli/EdgeBench/tree/da540564098c8bbc0ad5b9b4cfce0ce53045f8d6/juliet_vulnerability_analyzer.Checklist for task changes
versionintask.tomlis bumped (any change that can move scores).NOTES.md: the change and its score effect are under Changes. It isn't a listed Known issue.