Shared: Adjust semantics of optional inline expectation tags#21772
Shared: Adjust semantics of optional inline expectation tags#21772hvitved wants to merge 1 commit intogithub:mainfrom
Conversation
hvitved
force-pushed
the
inline-test-optional
branch
from
7429434 to
79baaad
Compare
Rerun has been triggered: 9 restarted 🚀 |
There was a problem hiding this comment.
Pull request overview
Adjusts the inline-expectations test harness so that “optional tag” annotations on expectations are no longer ignored, and are instead validated against actual results (while keeping the optionality asymmetric for results).
Changes:
- Treat expectation comments with optional tags as requiring a matching actual result.
- Introduce an
ignoreTaghook to allow tests to explicitly ignore expectations for certain tags.
Show a summary per file
| File | Description |
|---|---|
| shared/util/codeql/util/test/InlineExpectationsTest.qll | Updates optional-tag semantics, adds an ignore-tag hook, and updates related documentation/comments. |
Copilot's findings
- Files reviewed: 1/1 changed files
- Comments generated: 1
hvitved
force-pushed
the
inline-test-optional
branch
from
79baaad to
545d79f
Compare
hvitved
force-pushed
the
inline-test-optional
branch
from
545d79f to
572e712
Compare
|
Out of interest, what are optional inline expectation tags? Are they documented anywhere? I sometimes think we need better documentation of (post-processed) inline expectation tests since we are asking people to use them. I sometimes refer back to the description of this PR, but it is missing lots of information that would be useful. (I'm always looking for how to specify which query an alert comes from.) |
|
|
I see, so it's defined separately for each test, and it's used when you want to require that a particular query has certain results without having to specify all of the results. That sounds useful. |
Rerun has been triggered: 5 restarted 🚀 |
Before this PR, inline expectation comments with optional tags were simply ignored, which is most likely not what was intended.
With this PR, inline expectation comments with optional tags are instead checked, meaning there must be an accompanying actual result. The converse, however, is not checked (which is what makes the comment optional).