Migrating to LLVM18 - #48
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…iterate_phdr LLVM 18's opaque pointers removed DFSan's trampoline mechanism for custom functions taking function-pointer args. Port the runtime custom wrappers (dl_iterate_phdr, pthread_create, write) to the no-trampoline ABI and add dfsan_clear_thread_local_state() to zero the args/retval TLS before invoking instrumented callbacks directly. LLVM 16+ libunwind prefers glibc _dl_find_object over dl_iterate_phdr, but _dl_find_object cannot resolve unwind info for symsan's fixed-address (taint.ld) binaries and silently breaks exception handling. Force the DFSan-intercepted dl_iterate_phdr path via an idempotent patch applied by rebuild.sh / rebuild_native.sh. dfsan.cpp/dfsan.h carry only dfsan_clear_thread_local_state, required by the dfsan_custom.cpp wrappers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port correctness fixes from upstream DFSan (LLVM 14->18) plus a latent bug: - Wrap glibc 2.38 __isoc23_strtol/strtoll/strtoul/strtoull. On glibc 2.38+ <stdlib.h> redirects the strto* family to these __isoc23_* variants at compile time under -std=c23 or (very commonly) -D_GNU_SOURCE, so on Ubuntu 24.04 user calls to strtol&co were hitting uninstrumented symbols and losing symbolic tracking. Add custom abilist entries + thin __dfsw___isoc23_* forwarders to the existing symbolic taint_strtol wrappers. - sprintf: pass INT32_MAX instead of ~0ul as the buffer size. On glibc >=2.37 / musl, snprintf computes `str + n` which wraps for an unbounded size and drops the last character (glibc PR30441). - dlopen(NULL, ...): add a `filename &&` guard so we don't clear the shadow of the main executable's already-live globals for PIE mains (the l_addr check only covered non-PIE mains). - Fix __dfsw_strtoull: its `int base` and `dfsan_label nptr_label` params were swapped vs. the other strto* wrappers, so `base` received a label value. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Programs that parse tainted input with sscanf previously lost all taint (the sscanf family was uninstrumented), making downstream branches on parsed values unsolvable. Add a custom sscanf implementation that connects each conversion specifier to SymSan's existing symbolic ops, so no solver algorithm changes are needed. Runtime (runtime/dfsan): - dfsan_custom.cpp: extend Formatter with scan support (build_format_string(with_n) + scan()/scan(arg); per-directive consumed length is measured via an appended %n), add scan_buffer(), and __dfsw_sscanf / __dfsw___isoc99_sscanf / __dfsw___isoc23_sscanf wrappers. Numeric specifiers (%d %i %u %x %X %o) attach a fatoi (string-to-int) label with the right base (10/16/8; %i uses 10, the only bases the solver renders); %s/%c copy the matched input bytes' labels to the destination buffer; floats/%p are cleared; %* is skipped. - taint_strtol gains add_null (default true for atoi/strtol; sscanf passes false). - dfsan.h: FATOI_NO_NULL / FATOI_BASE_MASK, packed into a fatoi label's op1. - done_abilist.txt: sscanf / __isoc99_sscanf / __isoc23_sscanf = custom. Solvers: - z3-ts.cpp, rgd-parser.cpp: the fatoi handler now accepts either a Load label or a single raw input byte, so single-digit fields (len == 1) solve instead of hitting the Load-only assertion. atoi/strtol are unchanged. - fatoi carries a "no trailing NUL" flag (FATOI_NO_NULL) threaded through the Z3 atoi variable name and the RGD atoi_info; the NUL write is gated in both solution generators (z3-ts.cpp, jit-solver.cpp) so an embedded sscanf field does not clobber its separator/following bytes. Tests: sscanf_int.c (all numeric bases + single-digit) and sscanf_str.c (%s). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nteeType koGetPointeeType() always returns null on LLVM >= 15 (opaque pointers carry no pointee type), which silently zeroed the resign size in visitIndirectCallBase and broke KO_RESIGN_PTRARGS for indirect calls (test/indirect.c regressed: flags 201/202 never triggered). Replace all koGetPointeeType() call sites and delete it: - getUnderlyingObjectType(): trace back to the alloca/global a pointer originates from. Used where the pointer is handed to opaque/external code (resign, inline asm, wrapped calls). - findPointeeTypeFromUses(): walk forward through uses (incl. PHIs) for a consuming GEP, which still carries an explicit element type. Used for pointer-typed loads (e.g. linked-list `next`) and entry-function args, so lazily-created objects size correctly on first touch and unittest-style seed gen sees real struct types. New getOrCreatePointerTypeID() keys these by pointee (not by the now-opaque pointer Type, which collapses everything to one shared "ptr" entry). Also fixes two makeArrayRef deprecation warnings and bumps CMAKE_CXX_STANDARD 14 -> 17 (needed for std::optional from AllocaInst::getAllocationSize). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
LLVM 17 changed ExecutionSession::lookup to return Expected<ExecutorSymbolDef> instead of Expected<JITEvaluatedSymbol>, and getAddress() to return an ExecutorAddr wrapper rather than a raw uint64_t JITTargetAddress. - rgdJit.h: let GradJit::lookup's return type be deduced (auto) so it compiles across LLVM versions. - jit.cc: version-gate getAddress() -> getAddress().getValue() for LLVM >= 17. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ubuntu 24.04 ships LLVM 18 as its default toolchain, and SymSan's design now lives on the post-opaque-pointer (LLVM 15+) side: the runtime custom wrappers use the no-trampoline ABI unconditionally, so a pre-15 build would have a pass<->runtime ABI mismatch. Make LLVM 18 the minimum and remove the legacy version-guarded code. - CMakeLists.txt: fatal-error if LLVM_VERSION_MAJOR < 18. (find_package(LLVM 18) can't be used: LLVM's version config requires an exact major.minor match.) - version.h: drop all sub-18 version-guided branches, the unused legacy macros (LLVM_ATTRIBUTE_LIST/LLVM_NEW_ALLOCINST/LLVM_REMOVE_ATTRIBUTE/SCL_INSECTION/ LLVM_ADD_PARAM_ATTR), KO_ENDSWITH and koGetPointeeType. The single-def compat macros are inlined at their call sites, so the passes no longer include it. - TaintPass.cpp: remove the <15 trampoline machinery and the <17 PassManagerBuilder include; inline the former macros (PointerType::getUnqual, std::nullopt, getABITypeAlign().value(), the MaybeAlign scale, starts_with) and include <optional>/AttributeMask.h/TargetParser/Triple.h directly. - UCSanPass.cpp: same macro inlining + direct includes. - LoopOutlinePass.cpp: LLVM 18 build fixes (Dwarf.h, CodeExtractor Suffix arg) and inline starts_with; no longer needs version.h. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prebuilt archives regenerated with clang-18 (via rebuild.sh / rebuild_native.sh against llvm_project-18): - build_taint: taint-instrumented libc++ / libc++abi / libunwind - build_native: plain libc++abi / libunwind (EH runtime for ucsan-only C++) The libunwind archives are built with the dl_iterate_phdr force (see the rebuild scripts) so C++ exception handling works for symsan's fixed-address binaries under LLVM 18. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- ko_clang.c: gate out -fexperimental-new-pass-manager (removed in clang 16; the new PM is the default). Keep both -fplugin and -fpass-plugin: on clang 18 -fpass-plugin alone does not register the pass's -mllvm cl::opt options, only -fplugin does (verified). - CMakeLists.txt: pass LLVM_VERSION_MAJOR/MINOR to KOClang so the driver can version-gate the flags it forwards. - ucsan_opt: use opt-18/llc-18/clang-18; drop the redundant -load (on LLVM 18 -load-pass-plugin alone registers both the pipeline and the cl::opt options). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
clang-18 -O3 merges the skip-loop byte checks differently than clang-14, so the "hello" solution now lands at id-0-0-1 instead of id-0-0-3. Both still generate the input that reaches the strcmp match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The vendored sanitizer_common is pristine LLVM 14.0.6. Most 14->18 changes are irrelevant to SymSan (it bypasses the allocator/TLS/interceptor subsystems), but a few portable fixes are worth taking: - Symbolizer::SymbolizeData/SymbolizeFrame now return false when no tool symbolized the address (upstream 19357b45df7d) instead of always true. - Use canonical Linux syscalls on x86_64 too (openat/newfstatat/dup3/...), matching upstream 34b676eb60ca; more robust under seccomp filters that only allow the canonical set. Done by enabling SANITIZER_USES_CANONICAL_LINUX_SYSCALLS for all SANITIZER_LINUX. - Add ReportMunmapFailureAndDie with a process-map dump on munmap failure (upstream 0458405a6bf0) instead of a bare CHECK. - Symbolizer output no longer truncated: replace the fixed 16 KiB buffer with a growable InternalMmapVector<char> (upstream acfeb1a6c244), adapted to the 14.0.6 API. Also fixes an upstream -Wshadow/logic bug where an inner `ret` shadowed the outer and dropped the read-failure return. Note: this makes sanitizer_common no longer byte-identical to 14.0.6. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update the GitHub Actions workflow to install llvm-18/clang-18 and the libc++-18/libc++abi-18/libunwind-18 dev packages, and configure/build/install with clang-18/clang++-18, matching the LLVM 18 minimum. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…sion The runner has multiple LLVM versions installed; find_package(LLVM) was resolving to llvm-17 and tripping the >= 18 requirement. Point LLVM_DIR at /usr/lib/llvm-18/lib/cmake/llvm explicitly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrating to LLVM18