Test Infrastructure

Test Infrastructure

The project uses a comprehensive testing approach with regression tests, unit tests, and CI integration tests.

---

Regression Tests

Test Runner

The main test runner is run_test.py (repository root), a Python script that:

• Discovers test fixture files
• Runs the analyzer on each fixture
• Validates output against expected diagnostics from source comments
• Supports parallel execution and caching

Running Tests

# Run all tests
python3 run_test.py --analyzer ./build/stack_usage_analyzer

# Run a specific test directory
python3 run_test.py --analyzer ./build/stack_usage_analyzer test/bound-storage/

# Run a single test file
python3 run_test.py --analyzer ./build/stack_usage_analyzer test/bound-storage/bound-storage.c

# Parallel execution
python3 run_test.py --analyzer ./build/stack_usage_analyzer --jobs 4

Test Fixture Format

Test fixtures are C/C++ source files with expected diagnostic annotations in comments:

// at line 12, column 14
// [ !!Warn ] potential stack buffer overflow on variable 'test' (size 10)
// ... alias path: test
// ... constant index 11 is out of bounds (0..9)
// ... (this is a write access)
int main(void)
{
    char test[10];
    test[11] = 'a';  // line 12
    return 0;
}

Annotation Syntax

Warning-level diagnostic:

// [ !!Warn ] <message prefix>

Error-level diagnostic:

// [!!Error] <message prefix>

Info-level diagnostic:

// [!Info] <message prefix>

Location annotation (must precede the diagnostic):

// at line <N>, column <N>

Per-file model selection:

// resource-model: models/resource-lifetime/generic.txt
// escape-model: models/stack-escape/generic.txt

---

Test Directories

Directory	What it tests
test/alloca/	Dynamic alloca, VLA, user-controlled sizes, oversized
test/bound-storage/	Buffer overflow, alias chains, struct arrays, reachability
test/cpy-buffer/	memcpy/memset overflow detection
test/diagnostics/	Duplicate if-else conditions
test/escape-stack/	Stack pointer escape (store, callback, return)
test/false-positive-repro/	Regression tests for known false positives
test/local-storage/	Stack frame size, callee propagation
test/multiple-storage/	Multiple stores to same buffer
test/no-error/	Files that should produce no diagnostics
test/offset_of-container_of/	container_of macro, pointer arithmetic
test/pointer_reference-const_correctness/	Const parameter detection
test/resource-lifetime/	Resource lifecycle (missing/double release, cross-TU)
test/size-arg/	Size-minus-one patterns
test/uninitialized-variable/	Uninitialized local variable reads
test/vla/	Variable-length array detection

---

Unit Tests

Fine-grained unit tests for core analyzer modules.

Building

cmake -S . -B build -DBUILD_ANALYZER_UNIT_TESTS=ON ...
cmake --build build

Running

# Via CTest
cd build && ctest -R analyzer_module_unit_tests

# Direct execution (pass source dir as argument)
./build/stack_usage_analyzer_unit_tests /path/to/coretrace-stack-analyzer

Covered Modules

• LocationResolver -- debug info to source location conversion
• Reachability -- unreachable code filtering heuristics
• ModulePreparationService -- stack size computation, call graph building

Location

test/unit/analyzer_module_unit_tests.cpp

The test runner also invokes unit tests automatically via check_analyzer_module_unit_tests().

---

CI Integration Tests

Fixtures

fixtures/ contains CI integration test fixtures:

• CMake projects with compile_commands.json generation
• Tests the full pipeline: build + analyze + validate

CI Workflows

Workflow	Purpose
.github/workflows/ci.yml	Main CI: build + test on Ubuntu
.github/workflows/clang-format.yml	Code format compliance check
.github/workflows/commit-check.yml	Commit message validation
.github/workflows/publish-docker.yml	Docker image build and push
.github/workflows/test-ci-integration.yml	End-to-end CI integration tests

---

Writing Effective Tests

For a new analysis check:

1. Create a directory under test/ matching your check name
2. Add one or more .c/.cpp files with annotated expectations
3. Include both positive (triggering) and negative (clean) cases
4. Add a false-positive regression test if needed under test/false-positive-repro/

Test naming conventions:

• basic-case.c -- simple triggering case
• no-error.c or *-ok.c -- clean code, no diagnostics expected
• *-interproc*.c -- cross-function or cross-TU tests
• *-nested*.c -- nested construct variations

Cross-TU tests:

For tests that require multiple files (resource or uninitialized cross-TU), use matching prefixes:

test/resource-lifetime/cross-tu-wrapper-def.c   # defines the wrapper
test/resource-lifetime/cross-tu-wrapper-use.c   # uses the wrapper
test/uninitialized-variable/cross-tu-uninitialized-wrapper-def.c
test/uninitialized-variable/cross-tu-uninitialized-wrapper-use.c

---

Test Caching

run_test.py supports caching to avoid re-running unchanged tests. Cache is invalidated when:

• The analyzer binary changes
• The test file changes
• The expected annotations change