[findalexli]

What and why

Task finding: #39.

score.py subtracts 100 points if any submitted .py or .sh file contains the word infer, in any case, even in a comment. The README bans calling the Infer tool, but it also says "Infer external inputs". An Opus 5.5 analyzer had this docstring line and scored 0 instead of 79.59:

2. Infer the "bad" (attacker reachable) region of the program from the

The merged change drops infer from the banned words (b9125bd, from the maintainers). This PR first narrowed the pattern to command-like text. That still flagged stats["infer"] and missed infer -j 4 run, and Infer is not in the image and grading is offline. The README's rule against calling it stays, without a text check. Scores can only rise, and only for submissions containing the word.

This fixes part of Known issue 3; other banned words in comments still zero a submission (last row).

Checklist for task changes

• version in task.toml is bumped (any change that can move scores).
• NOTES.md: the change and its score effect are under Changes; Known issue 3 stays, with its example changed.
• Old vs new judge on the same submissions, on the same host, results below.
• New blobs: none.

Old vs new judge

tools/judge_local.py. Old is the judge built from 1187e2c. New is the merged judge, built from f92857a, three grades per row on one worker. Scores had zero spread. The added call lines are never executed; they scored 0 only through the text check.

Submission	Old	New
Opus 5.5 analyzer with the line above	0	79.59
Best analyzer from another Opus 5.5 run, no banned word	90.70	90.70
That analyzer + # Infer external inputs from call facts.	0	90.70
That analyzer + # we infer the sink from the token facts	0	90.70
That analyzer + # infer run lengths from buffer sizes	0	90.70
That analyzer + subprocess.run(["infer", "run", "--", "gcc"])	0	90.70
That analyzer + subprocess.run(["/opt/infer/bin/infer", "run", "--", "gcc"])	0	90.70
That analyzer + scan.sh containing infer run -- gcc	0	90.70
Empty starter	10	10
Crashing analyzer	10 (invalid)	10 (invalid)
That analyzer + # does not use CodeQL	0	0

[Odysseusqs]

Thanks for this. We pushed b9125bd to your branch: it drops infer from FORBIDDEN_RE and updates Changes in NOTES.md. Infer isn't in the image and grading is offline, while the command-like pattern still missed infer -j 4 run and still flagged stats["infer"]. The README rule against calling it stays.

Old vs new on the same host (--cpus 4), using one archived analyzer that scores 94.04:

Submission	Old	New
Six archived submissions	0, 70, 84.88, 89.73, 91.39, 94.04	same
Empty starter	10	10
+ Infer the "bad" ... comment	0	94.04
+ # Infer external inputs ...	0	94.04
+ subprocess.run(["infer", "run", "--", "gcc"]) (not executed)	0	94.04
+ stats["infer"]	0	94.04
+ scan.sh with infer run -- gcc	0	94.04
+ # does not use CodeQL	0	0
+ scan.sh with curl	0	0

None of the 1185 archived submissions we scanned, from 14 runs, contains the word, so no archived score moves.

The tables in this PR's description and in #39 still describe the earlier pattern: the infer call rows now score the analyzer's normal value instead of 0. Could you update those two when you get a chance? We'll merge once CI is green.