53 analyzer expand security coverage enforce global strict test runner stabilize expectation matching#54

Open

SizzleUnrlsd wants to merge 143 commits intomainfrom

53-analyzer-expand-security-coverage-enforce-global-strict-test-runner-stabilize-expectation-matching

Contributor

SizzleUnrlsd commented Mar 1, 2026

Summary

This PR consolidates a broad analyzer + test infrastructure effort:

expanded security analysis coverage,
hardened run_test.py with global strict validation,
fixed expectation matching regressions (both too brittle and too permissive cases),
added/fixed/reorganized security fixtures.

Main changes

1) Analyzer / pipeline

Added and integrated multiple security analyses (including null deref, integer overflow, TOCTOU, command injection, OOB read, type confusion).
Updated core components:
- src/analyzer/AnalysisPipeline.cpp
- src/analyzer/DiagnosticEmitter.cpp
- src/app/AnalyzerApp.cpp
- src/analysis/*
- include/analysis/*
Updated models:
- models/resource-lifetime/generic.txt
- models/buffer-overflow/generic.txt

2) Test runner (`run_test.py`)

Enabled strict warning/error count as a global policy for fixtures.
Enforced strict validation even when expectations are partial.
Improved matching for combined diagnostics within the same location block.
Fixed location tolerance behavior:
- resilient to realistic line drift,
- rejects obviously stale/far-off line anchors.
Relaxed brittle inter-TU checks that depended on exact line-number strings.

3) Test corpus

Backfilled missing expectations in many fixtures.
Added advanced regression cases (nested control flow, inter-TU, tricky paths).
Reorganized security fixtures into:
- test/security/<vulnerability-category>/...
Kept legacy compatibility through test/files shim documentation/Makefile.

Validation

Command:
- python3 run_test.py --jobs=8
Result:
- 821/821 passed

Behavior changes

Strict count is now a central quality gate:
- tests fail when warning/error comments do not match analyzer output count.
Stale line anchors are no longer silently accepted by over-tolerant matching.

Exclusions for this PR

Please explicitly exclude:

tmp/
.cache/
local artifacts (.DS_Store, __pycache__, local analysis outputs)

Checklist

Analyzer integrated and buildable
Strict runner behavior stabilized
Regression suite green
Security corpus reorganized
Local artifacts excluded

SizzleUnrlsd added 30 commits

March 1, 2026 23:13


          fix(test-runner): harden strict expectation matching and location tol…

eedf272

…erance


          test(fixtures): update expectations in recursive-controlled-alloca.c

b6506c6


          test(fixtures): update expectations in recursive-infinite-alloca.c

250efb3


          test(fixtures): update expectations in bound-storage-if-statement.c

ce76163


          test(fixtures): update expectations in bound-storage.c

102f717


          test(fixtures): update expectations in indirection-profonde-aliasing.c

675b4d4


          test(fixtures): update expectations in bound_index.cc


          test(fixtures): update expectations in dead_code.cc

0ed2266


          test(fixtures): update expectations in format_problem.c

180baa3


          test(fixtures): update expectations in null_pointer.c

7a0e586


          test(fixtures): update expectations in bad-usage-memcpy.c

842324b


          test(fixtures): update expectations in bad-usage-memset.c

6d1d1a4


          test(fixtures): update expectations in unbounded-strcpy-model.c

e7a770e


          test(fixtures): update expectations in indirect-callback-unknown-targ…

0b0cad1

…et-with-safe-candidate.c


          test(fixtures): update expectations in return-stack-address-through-p…

2c7c88a

…ointer-slot.c


          test(fixtures): update expectations in stack_escape.c

334cf7c


          test(fixtures): update expectations in virtual-strategy-local-no-esca…

97d4ecc

…pe.cpp


          test(fixtures): update expectations in unique_ptr_state.cpp

07685d6


          test(fixtures): update expectations in stb-like-next-code-uninitializ…

f48cf07

…ed.c


          test(fixtures): update expectations in external-wrapper-known-no-acqu…

a1cec04

…ire-double-release.c


          test(fixtures): update expectations in external-wrapper-unknown-out-n…

e83e7da

…o-double-release.c


          test(fixtures): update expectations in external-wrapper-unknown-ref-o…

0913d9f

…ut-no-double-release.cpp


          test(fixtures): update expectations in new-double-delete.cpp

701e857


          test(fixtures): update expectations in uninitialized-local-pointer-re…

8250b8b

…direct.c


          chore(test-compat): keep legacy test/files shim in sync (README.md)

384da9c


          chore(test-compat): keep legacy test/files shim in sync (Makefile)

f8c736f


          test(inter-tu): add cross-tu regression fixture cross-tu-tricky-def.c

da170c8


          test(inter-tu): add cross-tu regression fixture cross-tu-tricky-use.c

d6e52a8


          test(inter-tu): add cross-tu regression fixture cross-tu-uaf-def.c

1b4f90e


          test(inter-tu): add cross-tu regression fixture cross-tu-uaf-use.c

e084f05

SizzleUnrlsd added 13 commits

March 1, 2026 23:16


          test(fixtures): add or update memcpy-name-false-positive.c

160ff37


          test(fixtures): add or update custom-wrapper-buffer-model.txt

d3dc8bc


          test(fixtures): add or update cxa-demangle-balanced-no-incomplete.cpp

74188bc


          test(fixtures): add or update local-non-escaping-no-incomplete.cpp

3cb39b7


          test(fixtures): add or update nocapture-local-handle-no-incomplete.cpp

37a0cc1


          test(fixtures): add or update summary-release-aggregate-field-no-inco…

e2e9eb2

…mplete.cpp


          test(fixtures): add or update summary-release-aggregate-local-no-inco…

f049b0b

…mplete.cpp


          test(fixtures): add or update uninitialized-local-cpp-aggregate-ctor-…

3281c8a

…copy.cpp


          test(fixtures): add or update uninitialized-local-cpp-ctor-forgets-fi…

9c932c6

…eld-warns.cpp


          test(fixtures): add or update uninitialized-local-cpp-nested-aggregat…

d1ce7ca

…e-copy.cpp


          test(fixtures): add or update uninitialized-local-cpp-parameterized-c…

3fd7efa

…tor-copy.cpp


          test(fixtures): add or update uninitialized-local-cpp-sret-complex-re…

3d9f5dd

…turn.cpp


          chore(docs): remove obsolete PATCH.md

244b4ff

SizzleUnrlsd self-assigned this

SizzleUnrlsd linked an issue

that may be closed by this pull request

[Analyzer] Expand security coverage + enforce global strict test runner + stabilize expectation matching #53

Open

SizzleUnrlsd added 14 commits

March 2, 2026 01:55


          ci(github-actions): optimize Linux/macOS pipeline with Ninja, ccache,…

3cbb31d

… and lld


          test(runner): optimize analyzer test runner with precompiled regex an…

9601b43

…d parallel check orchestration


          fix(ci): stabilize Linux analyzer test diagnostics across toolchains

07db5da


          chore(style): format code with clang-format

fb5d874


          fix(test-runner): enforce trailing --compile-arg overrides for determ…

608bb51

…inistic compile flag precedence


          ci(tests): remove Linux-specific extra analyzer args from test run

09356a5


          build(docker): split image into base, dev, builder, and runtime stage…

0d39e45

…s, docs(docker): document dev, builder and runtime targets with copy-paste commands


          docs(readme): align Docker usage guide with multi-target Dockerfile a…

1945fce

…nd runtime entrypoint defaults


          fix(diagnostics): classify implicit gets declaration as deprecated un…

bd09a58

…safe function


          fix(integer-overflow): recognize LLVM mem intrinsics as size sinks

18982a5


          fix(uninitialized): improve memtransfer initialization modeling and C…

7e82209

…-string write handling


          test(const-correctness): include stdint header for fixed-width intege…

6d2bb03

…r types


          test(unsafe-functions): use clang pragma to downgrade implicit gets d…

29ec7ce

…eclaration diagnostic


          chore(style): format code with clang-format

d891388

github-advanced-security bot found potential problems

View reviewed changes

src/analysis/IntegerOverflowAnalysis.cpp

+                                                         llvm::SmallPtrSetImpl<const llvm::Value*>& visited,
+                                                         unsigned depth)
+                      {
+                          if (!value || depth > 32)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/IntegerOverflowAnalysis.cpp

+                      static bool hasKnownNonNegativeRange(const llvm::Value* value,
+                                                           const std::map<const llvm::Value*, IntRange>& ranges)
+                      {
+                          if (!value)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/IntegerOverflowAnalysis.cpp

+                          const llvm::Value* value, const std::map<const llvm::Value*, IntRange>& ranges,
+                          llvm::SmallPtrSetImpl<const llvm::Value*>& visited, unsigned depth)
+                      {
+                          if (!value || depth > 32)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/IntegerOverflowAnalysis.cpp

+                      resolveConstIntRecursive(const llvm::Value* value,
+                                               llvm::SmallPtrSetImpl<const llvm::Value*>& visited, unsigned depth)
+                      {
+                          if (!value || depth > 32)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/IntegerOverflowAnalysis.cpp

+                                                         llvm::SmallPtrSetImpl<const llvm::Value*>& visited,
+                                                         unsigned depth)
+                      {
+                          if (!value || depth > 32)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/OOBReadAnalysis.cpp

+                                                         llvm::SmallPtrSetImpl<const llvm::Value*>& visited,
+                                                         unsigned depth)
+                      {
+                          if (!value || depth > 32)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/OOBReadAnalysis.cpp

+                      static std::optional<IntRange>
+                      lookupRange(const llvm::Value* value, const std::map<const llvm::Value*, IntRange>& ranges)
+                      {
+                          if (!value)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/ResourceLifetimeAnalysis.cpp

+                                                      llvm::SmallPtrSet<const llvm::Value*, 32>& visited,
+                                                      bool& sawReleaseUse, unsigned depth = 0)
+                      {
+                          if (!value || depth > 10)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/StackBufferAnalysis.cpp

+                      static bool isArrayBackedType(const llvm::Type* type)
+                      {
+                          using namespace llvm;
+                          if (!type)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

src/analysis/TypeConfusionAnalysis.cpp

+                                                              std::unordered_set<const llvm::Type*>& visiting,
+                                                              unsigned depth = 0)
+                      {
+                          if (!haystack || !needle || depth > 24)

Check notice

Code scanning / coretrace-stack-analyzer

None Note

[ !Info! ] recursive or mutually recursive function detected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet