From 15049bdf0a33bde6b6c698b9365ffb871395f148 Mon Sep 17 00:00:00 2001 From: mahaloz Date: Mon, 26 Oct 2026 02:01:00 +0000 Subject: [PATCH] [AUTOMATED] Recover Go string headers as bounded literals (the fatstr record pass's Go arm) Go packs string data as one unterminated .rodata blob, so the NUL scan recovers almost nothing: 164 functions across the 12-binary Go corpus carry a string literal today. The headers are there - the measured non-PIE go1.22 toolchain binaries keep (ptr, len) pairs as absolute words, 480 of the hello witness's 545 in .data and the rest in .rodata - and they are the same record shape the fatstr pass already walks for rustc. The Go arm registers the existing StrRecordPass for Go-detected images with the direct arm's SLOT regions widened to .data/.noptrdata; the TARGET test is unchanged (text must sit in genuinely read-only data), which keeps a writable slot's mutability irrelevant. Same-start records dedup longest-first (Go shares literal bytes between headers), and a record starting inside an earlier one is retired by the commit arm's occupied-address guard. Same option, same commit gate, no new settable; the fatstr row's registration prose now names both languages (fixture/options regenerated, no counts moved). Measured (12 Go corpus rows, decompile-all --max-fn-seconds 10): functions containing a string literal 164 -> 7,046; aggregate wall 839.7 s -> 849.4 s (+1.2%, budget 5%); the `kuna strings` INVENTORY count is unchanged by design (that surface scans with Termination::Any, independent of committed facts). E2E: the fatstr-records family gains a Go arm - a stripped go build of the same runtime-indexed table commits the literal's exact 21 bytes under auto and nothing states that extent under off. Unit tests pin the slot/target split and the longest-first dedup. Gates: make test PARITY OK, make test-stages PARITY OK (no baseline change), make rust-test green (one documented decompile_budget_cli wall-clock flake, idle-green), check-spec OK, test-cli 306/306, lint clean, catalog --check OK. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01WoTv7T2Ep5hmbZcyYSMk1g --- .../src/analyzers/strings/kuna_strrecords.rs | 120 +++++++++++++++--- decompiler/crates/kuna-analysis/src/passes.rs | 27 ++-- .../tests/verify_fatstr_records.rs | 89 +++++++++++++ decompiler/crates/kuna-decomp/phases.toml | 2 +- .../tests/fixtures/phase_catalog.json | 2 +- docs/options.md | 2 +- 6 files changed, 211 insertions(+), 31 deletions(-) diff --git a/decompiler/crates/kuna-analysis/src/analyzers/strings/kuna_strrecords.rs b/decompiler/crates/kuna-analysis/src/analyzers/strings/kuna_strrecords.rs index fbc7f2bfe..58fbc57e2 100644 --- a/decompiler/crates/kuna-analysis/src/analyzers/strings/kuna_strrecords.rs +++ b/decompiler/crates/kuna-analysis/src/analyzers/strings/kuna_strrecords.rs @@ -36,12 +36,27 @@ //! //! # Gating //! -//! One option, two recognizers: this pass is registered only for a -//! rustc-detected image (`passes_for`), and its commit is gated on the same +//! One option, two recognizers: this pass is registered only for rustc- and +//! Go-detected images (`passes_for`), and its commit is gated on the same //! `Architecture::fat_str` mode the engine rule reads (`off` drops the facts -//! at the read-symbols boundary, exactly like every analysis gate). Go string -//! headers share the record shape but belong to the Go pack's blob -//! segmentation item, deliberately not here. +//! at the read-symbols boundary, exactly like every analysis gate). +//! +//! # The Go arm +//! +//! Go string headers are the same `(ptr, len)` record over the same +//! unterminated `.rodata` blob, with one placement difference: the measured +//! go1.22 toolchain binaries are non-PIE and keep most headers in the +//! WRITABLE data image (`.data`: 480 of 545 records on the hello witness; +//! `.noptrdata` holds the pointer-free statics), with the remaining 65 in +//! `.rodata` itself. A Go registration therefore widens the SLOT regions to +//! those two sections by name ([`StrRecordPass::go`]); the TARGET test is +//! unchanged — the text must sit in genuinely read-only data — which is what +//! keeps a writable slot's mutability irrelevant (the record may move; the +//! literal it currently states cannot). Go also shares literal bytes between +//! headers (substring records): records are deduped longest-first per start +//! address, and a record starting inside an earlier one is retired by the +//! commit arm's occupied-address guard, so prefix/suffix sharing renders +//! through the containing literal. use object::read::{Object, ObjectSection}; use object::RelocationFlags; @@ -89,6 +104,15 @@ fn ro_regions<'a>(file: &'a object::File) -> Vec> { .collect() } +/// The additional SLOT regions a Go image keeps headers in: the writable data +/// image. Slots only — [`record_at`] still demands the target bytes in RO. +fn go_slot_regions<'a>(file: &'a object::File) -> Vec> { + file.sections() + .filter(|s| s.name().is_ok_and(|n| n == ".data" || n == ".noptrdata")) + .filter_map(|s| Some(RoRegion { vma: s.address(), data: s.data().ok()? })) + .collect() +} + /// Is this record text? Valid UTF-8 whose chars are >=90% printable or /// whitespace — the engine rule's text-evidence test, applied to exact bytes. fn is_text(bytes: &[u8]) -> bool { @@ -163,18 +187,19 @@ fn scan_reloc_records(file: &object::File, regions: &[RoRegion], width: u64) -> out } -/// The direct arm: absolute `(ptr, len)` word pairs in read-only data (the -/// non-PIE layout; on a PIE the unrelocated slots read zero and match nothing). -fn scan_direct_records(regions: &[RoRegion], width: u64) -> Vec { +/// The direct arm: absolute `(ptr, len)` word pairs whose slots sit in +/// `slots` and whose targets sit in `targets` (the non-PIE layout; on a PIE +/// the unrelocated slots read zero and match nothing). +fn scan_direct_records(slots: &[RoRegion], targets: &[RoRegion], width: u64) -> Vec { let mut out = Vec::new(); let w = width as usize; - for region in regions { + for region in slots { let data = region.data; let mut i = 0usize; while i + 2 * w <= data.len() { let ptr = read_word(&data[i..i + w]); let len = read_word(&data[i + w..i + 2 * w]); - if let Some(fact) = record_at(regions, ptr, len, DIRECT_MIN_LEN) { + if let Some(fact) = record_at(targets, ptr, len, DIRECT_MIN_LEN) { out.push(fact); if out.len() >= MAX_RECORDS { return out; @@ -186,9 +211,24 @@ fn scan_direct_records(regions: &[RoRegion], width: u64) -> Vec { out } -/// (kuna `fatstr`) The `&str`-record string pass. Registered for rustc-detected -/// images; committed under the `fatstr` mode gate. -pub struct StrRecordPass; +/// (kuna `fatstr`) The string-record pass. Registered for rustc- and +/// Go-detected images; committed under the `fatstr` mode gate. +pub struct StrRecordPass { + /// Widen the direct arm's SLOT regions to the Go data image + /// ([`go_slot_regions`]); targets stay read-only either way. + go_slots: bool, +} + +impl StrRecordPass { + /// The rustc registration: slots in read-only data only. + pub fn rust() -> StrRecordPass { + StrRecordPass { go_slots: false } + } + /// The Go registration: slots also in `.data`/`.noptrdata`. + pub fn go() -> StrRecordPass { + StrRecordPass { go_slots: true } + } +} impl AnalysisPass for StrRecordPass { fn phase(&self) -> Phase { @@ -204,11 +244,17 @@ impl AnalysisPass for StrRecordPass { } let width: u64 = if ctx.file.is_64() { 8 } else { 4 }; let mut facts = scan_reloc_records(ctx.file, ®ions, width); - facts.extend(scan_direct_records(®ions, width)); - // One fact per start address; the first (reloc-vouched) wins, and a - // longest-first order inside equal starts is irrelevant because records - // state exact lengths. - facts.sort_by_key(|f| f.addr); + if self.go_slots { + let slots = go_slot_regions(ctx.file); + facts.extend(scan_direct_records(&slots, ®ions, width)); + } + facts.extend(scan_direct_records(®ions, ®ions, width)); + // One fact per start address, LONGEST first: Go headers share bytes + // (prefix records at one start), and the longer record renders the + // shorter as a prefix read while the reverse would truncate. A record + // starting inside an earlier one is left to the commit arm's + // occupied-address guard. + facts.sort_by_key(|f| (f.addr, core::cmp::Reverse(f.len))); facts.dedup_by_key(|f| f.addr); AnalysisOutput { bounded_strings: facts, ..Default::default() } } @@ -264,7 +310,7 @@ mod tests { } data[0x100..0x100 + recs.len()].copy_from_slice(&recs); let regions = vec![region(0x2000, &data)]; - let found = scan_direct_records(®ions, 8); + let found = scan_direct_records(®ions, ®ions, 8); // The 17-byte record is found (once per slot it appears in); the 1-byte // record fails the direct-arm minimum. Dedup collapses the repeat. let mut facts = found; @@ -273,4 +319,40 @@ mod tests { assert_eq!(facts.len(), 1); assert_eq!((facts[0].addr, facts[0].len), (0x2000, 17)); } + + /// The Go slot split: records whose SLOTS sit outside the target regions + /// (the `.data` header area) are found when passed as the slot set, and + /// the targets are still held to the read-only set. + #[test] + fn go_slots_scan_separately_from_targets() { + let mut ro = vec![0u8; 0x40]; + ro[..17].copy_from_slice(b"hello multilang, "); + let targets = vec![region(0x2000, &ro)]; + // Header area at 0x5000 ("data"): one record to the RO text, one + // record whose target points back into the header area itself. + let mut hdr = vec![0u8; 0x40]; + hdr[..8].copy_from_slice(&0x2000u64.to_le_bytes()); + hdr[8..16].copy_from_slice(&17u64.to_le_bytes()); + hdr[16..24].copy_from_slice(&0x5000u64.to_le_bytes()); + hdr[24..32].copy_from_slice(&17u64.to_le_bytes()); + let slots = vec![region(0x5000, &hdr)]; + let found = scan_direct_records(&slots, &targets, 8); + assert_eq!(found.len(), 1, "only the RO-backed target is a record"); + assert_eq!((found[0].addr, found[0].len), (0x2000, 17)); + } + + /// Same-start records keep the LONGEST (Go prefix sharing): the shorter + /// renders as a prefix read of the longer. + #[test] + fn same_start_dedup_keeps_the_longest() { + let mut facts = vec![ + StringFact { addr: 0x2000, len: 5 }, + StringFact { addr: 0x2000, len: 17 }, + StringFact { addr: 0x2020, len: 4 }, + ]; + facts.sort_by_key(|f| (f.addr, core::cmp::Reverse(f.len))); + facts.dedup_by_key(|f| f.addr); + assert_eq!(facts.len(), 2); + assert_eq!((facts[0].addr, facts[0].len), (0x2000, 17)); + } } diff --git a/decompiler/crates/kuna-analysis/src/passes.rs b/decompiler/crates/kuna-analysis/src/passes.rs index f0c2149a1..ef84e2796 100644 --- a/decompiler/crates/kuna-analysis/src/passes.rs +++ b/decompiler/crates/kuna-analysis/src/passes.rs @@ -394,6 +394,12 @@ pub fn passes_for(compiler: Compiler, format: object::BinaryFormat) -> Vec` functions take the recovered name). if compiler.is_golang() { + // (kuna `fatstr`) The Go arm of the string-record pass: the same + // (ptr, len) record shape rustc emits, with the slots widened to the + // Go data image (.data/.noptrdata). FIRST in the Go block so + // `gopclntab` keeps its appended-last contract. Committed under the + // same `fatstr` mode gate as the rustc arm. + passes.push(Box::new(crate::strings::kuna_strrecords::StrRecordPass::go())); // (kuna `gonoreturn`) BEFORE `gopclntab`, preserving that pass's // appended-last contract: apply the vendored Go no-return list to the // pclntab's own function list, so a STRIPPED Go binary (no .symtab for @@ -409,7 +415,7 @@ pub fn passes_for(compiler: Compiler, format: object::BinaryFormat) -> Vec bool { .unwrap_or(false) } +/// The Go twin of [`MAIN_RS`]: the same runtime-indexed table, whose headers +/// the Go linker emits as absolute `(ptr, len)` pairs in the data image. +const MAIN_GO: &str = r#"package main + +import ( + "fmt" + "os" +) + +var table = [3]string{"kuna fatstr witness, ", "kuna second piece", "kuna third piece"} + +func main() { + i := len(os.Args) % 3 + fmt.Println(table[i] + "!") +} +"#; + +/// `true` if a runnable `go` toolchain is on PATH. +fn go_available() -> bool { + Command::new("go") + .arg("version") + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .map(|s| s.success()) + .unwrap_or(false) +} + +/// Build [`MAIN_GO`] stripped (`-ldflags=-s -w`) for the host, hermetically. +fn build_go(dir: &std::path::Path, out_bin: &std::path::Path) -> bool { + if std::fs::create_dir_all(dir).is_err() { + return false; + } + if std::fs::write(dir.join("main.go"), MAIN_GO).is_err() { + return false; + } + Command::new("go") + .args(["build", "-ldflags=-s -w", "-o"]) + .arg(out_bin) + .arg("main.go") + .current_dir(dir) + .env("GOCACHE", dir.join(".gocache")) + .env("GOPATH", dir.join(".gopath")) + .env("CGO_ENABLED", "0") + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .map(|s| s.success()) + .unwrap_or(false) +} + /// The virtual address of [`WITNESS`] inside the built binary's `.rodata`. fn witness_vma(bin: &std::path::Path) -> Option { let bytes = std::fs::read(bin).ok()?; @@ -160,3 +211,41 @@ fn rust_str_records_segment_the_glued_nul_run() { let _ = std::fs::remove_dir_all(&dir); } + +/// The Go arm: the same record shape, slots in the Go data image, under the +/// same option. A stripped Go build's table headers state the literal's exact +/// 21 bytes; `off` leaves the blob to the NUL scan, which cannot. +#[test] +fn go_string_records_bound_the_blob_literal() { + if !go_available() { + eprintln!("verify_fatstr_records: skipping Go arm (`go` not on PATH)"); + return; + } + let dir = std::env::temp_dir().join(format!("kuna_fatstr_go_{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + let out_bin = dir.join("go_fatstr"); + if !build_go(&dir, &out_bin) { + let _ = std::fs::remove_dir_all(&dir); + panic!("installed Go compiler failed to build the fixture"); + } + let bin = out_bin.to_str().expect("utf8 path"); + let vma = witness_vma(&out_bin).expect("the witness literal is in .rodata"); + + let bounded = committed_extent(bin, vma, None) + .expect("fatstr auto commits a bounded symbol at the literal"); + assert_eq!( + bounded, + WITNESS.len() as i32, + "the Go header-stated extent is the literal's own 21 bytes" + ); + + match committed_extent(bin, vma, Some("off")) { + Some(glued) => assert!( + glued != WITNESS.len() as i32, + "with fatstr off nothing states the literal's exact extent" + ), + None => {} + } + + let _ = std::fs::remove_dir_all(&dir); +} diff --git a/decompiler/crates/kuna-decomp/phases.toml b/decompiler/crates/kuna-decomp/phases.toml index fbff4f740..a3ef2a837 100644 --- a/decompiler/crates/kuna-decomp/phases.toml +++ b/decompiler/crates/kuna-decomp/phases.toml @@ -7385,7 +7385,7 @@ subphase = "const-pointer" strength = "HARD" rewind = "P5" issue = "kuna-multilang-fatstr" -summary = "Render a length-carried (ptr, len) constant call-argument pair as a bounded string literal. Rust (&str, &[u8]) and Go (string) carry a string's length beside its pointer and never NUL-terminate the bytes, so the classic literal path - which reads the image until a terminator - refuses them and every string argument renders as a raw rodata address beside a magic length: takes_str((char *)&DAT_00102000, 15). The length is right there in the adjacent argument, so the rule reads exactly that many bytes and routes them through the existing bounded literal mechanism (Funcdata::get_internal_string -> the BUILTIN_STRINGDATA user-op, the constseq path, which needs no terminator): takes_str(\"hello multilang\", 15). A pair is rewritten only when both arguments are p-code constants adjacent in the call's inputs, 2 <= len <= 4096, the pointed-at bytes are READ-ONLY in the image (query_global_properties, the msvcstrappend literal test), and the bytes decode as UTF-8 text with no control characters beyond newline/tab/CR and at least one alphanumeric - or the pointer's facing type carries a text language facet (the multilang W3 side table on the type factory), a type provider's statement that outranks the byte heuristic. The length argument is kept: the call really does pass it. At auto (default) the rule acts only when the image's source language is Rust or Go (the sourcelang fact), a structural no-op elsewhere; always drops the language test (stage-test bytechunks); off keeps the raw pointer. The option also gates the ANALYSIS-side recognizer (pass id fatstrrecords, registered for a rustc-detected image): rustc packs unterminated literals back to back in .rodata, so the NUL-run string scan glues adjacent literals into one wrong row; the static (ptr, len) records that carry every Rust literal's length (a PIE's R_*_RELATIVE slots in .data.rel.ro, absolute pairs on a non-PIE) are re-read as length-BOUNDED string facts, committed ahead of the NUL stream so the glued rows retire and the data symbol at each literal spans exactly the literal. off drops both recognizers; the record facts are dropped at the read-symbols boundary like every analysis gate." +summary = "Render a length-carried (ptr, len) constant call-argument pair as a bounded string literal. Rust (&str, &[u8]) and Go (string) carry a string's length beside its pointer and never NUL-terminate the bytes, so the classic literal path - which reads the image until a terminator - refuses them and every string argument renders as a raw rodata address beside a magic length: takes_str((char *)&DAT_00102000, 15). The length is right there in the adjacent argument, so the rule reads exactly that many bytes and routes them through the existing bounded literal mechanism (Funcdata::get_internal_string -> the BUILTIN_STRINGDATA user-op, the constseq path, which needs no terminator): takes_str(\"hello multilang\", 15). A pair is rewritten only when both arguments are p-code constants adjacent in the call's inputs, 2 <= len <= 4096, the pointed-at bytes are READ-ONLY in the image (query_global_properties, the msvcstrappend literal test), and the bytes decode as UTF-8 text with no control characters beyond newline/tab/CR and at least one alphanumeric - or the pointer's facing type carries a text language facet (the multilang W3 side table on the type factory), a type provider's statement that outranks the byte heuristic. The length argument is kept: the call really does pass it. At auto (default) the rule acts only when the image's source language is Rust or Go (the sourcelang fact), a structural no-op elsewhere; always drops the language test (stage-test bytechunks); off keeps the raw pointer. The option also gates the ANALYSIS-side recognizer (pass id fatstrrecords, registered for a rustc- or Go-detected image): both languages pack unterminated literals back to back in .rodata, so the NUL-run string scan glues adjacent literals into one wrong row; the static (ptr, len) records that carry every Rust literal's length (a PIE's R_*_RELATIVE slots in .data.rel.ro, absolute pairs on a non-PIE) are re-read as length-BOUNDED string facts, committed ahead of the NUL stream so the glued rows retire and the data symbol at each literal spans exactly the literal. off drops both recognizers; the record facts are dropped at the read-symbols boundary like every analysis gate." use_when = "Leave auto: on a Rust or Go binary every constant &str/string argument renders as its literal text instead of a raw &DAT_ address beside a magic length. Use always on a carved bytechunk or raw image with no language signal. Flip off when auditing exact rodata addresses, or if a read-only buffer of coincidentally printable bytes is being misread as text at a call site." example = "option fatstr always" source_decompiler = "kuna" diff --git a/decompiler/crates/kuna-decomp/tests/fixtures/phase_catalog.json b/decompiler/crates/kuna-decomp/tests/fixtures/phase_catalog.json index 562170f08..3a203ca8f 100644 --- a/decompiler/crates/kuna-decomp/tests/fixtures/phase_catalog.json +++ b/decompiler/crates/kuna-decomp/tests/fixtures/phase_catalog.json @@ -293,7 +293,7 @@ {"option": "entrythumbflow", "values": ["on", "off"], "default": "on", "destructive_as_default": false, "phase": "P1", "subphase": "code-data-partition", "strength": "HARD", "rewind": "P1", "issue": "kuna-analysis-entrythumbflow", "summary": "Carry the Thumb decode mode a container entry proves along the flow reachable from that entry, and nowhere else. The PE-family loaders share one ARM mode table: ARMNT declares a wholly Thumb stream on every container, ARM may interwork on every container, and machine 0x1c2 is read as its container family names it (THUMB on a PE, painted wholly Thumb; ARMTHUMB_MIXED on a UEFI TE, interworking). Where that table leaves the mode to the entry bit, the container makes no whole-image claim and the one fact it states is the entry's low bit. Ghidra's disassembler propagates that context along the flow it follows, but kuna's decompiler reads context per address and follows nothing, so this pass runs the same walk once at the analysis commit: decode from the even entry as Thumb through fall-through, direct branches, and direct BL calls (a BLX interworks and its target keeps the mode its own encoding selects), and paint TMode=1 over exactly the instruction bytes decoded, never a whole section, so A32 code elsewhere in the same section keeps its mode. The walk is bounded at 4096 instructions; reaching the bound keeps the ranges walked so far, says so once on stderr, and leaves the unreached code at the language default, which is what an even entry would have produced. Byte overlays are applied first, so the walk follows the effective instruction stream. Inert on every image without a Thumb-bit entry and on every non-ARM language.", "use_when": "On (default): a mixed ARM TE whose entry has the Thumb bit decompiles its entry function and the callees it reaches as Thumb while A32 code elsewhere stays A32. Flip off to decode the entry with the language default mode, or use --isa thumb when the whole image is Thumb and the 4096-instruction bound would otherwise leave code unpainted.", "example": "option entrythumbflow off", "source_decompiler": "kuna", "inspiration": "Ghidra's flow-following disassembler carrying TMode from an odd entry; EDK2 ARMTHUMB_MIXED TE modules whose entry is Thumb-2 while some assembly stays A32", "change_kind": "analysis-enablement", "tier": "analysis", "symptoms": ["a uefi te entry function decompiles as garbage a32 although its entry address is odd", "a callee reached by bl from a thumb entry decodes as a32", "a32 code in the same section as a thumb entry decodes as thumb", "loading a te reports the thumb context walk stopped"]}, {"option": "gopclntab", "values": ["on", "off"], "default": "on", "destructive_as_default": false, "phase": "P1", "subphase": "code-data-partition", "strength": "HARD", "rewind": "P1", "issue": "kuna-analysis-gopclntab", "summary": "Run the Go pclntab function-name recovery pass: when the binary is Go (detect_compiler == Go), locate and parse the embedded pclntab (PC->line/name table) and emit a function symbol per Go function, so main.main / runtime.* / package functions render NAMED instead of sub_ (the kuna analog of Ghidra's GolangSymbolAnalyzer, name-recovery half). Handles the go1.2/go1.16/go1.18/go1.20 header magics. Registered ONLY for a Go binary, so it is a structural no-op on every non-Go target. The existing symbol commit arm installs the names idempotently (a real .symtab name still wins; only a stripped Go binary's sub_ functions take the recovered name).", "use_when": "On (default) names Go functions from the pclntab (main.main, runtime.gopanic, ...); off leaves a stripped Go binary's functions as sub_.", "example": "option gopclntab off", "source_decompiler": "kuna", "inspiration": "Ghidra GolangSymbolAnalyzer (pclntab / GoPcHeader / GoModuledata / GoFuncData)", "change_kind": "analysis-enablement", "tier": "analysis", "symptoms": ["stripped go binary renders sub_ instead of main.main and runtime.* names", "go package function names missing"]}, {"option": "gonoreturn", "values": ["on", "off"], "default": "on", "destructive_as_default": false, "phase": "P1", "subphase": "code-data-partition", "strength": "HARD", "rewind": "P1", "issue": "kuna-multilang-gonoreturn", "summary": "Apply the vendored Go no-return list to the pclntab-recovered function list. The known-no-return pass (noreturn_known) matches GolangFunctionsThatDoNotReturn against the object's symbol tables, but a production Go binary is stripped (-ldflags=-s -w): both tables are gone, the pass emits zero facts, and every entry of the vendored list - runtime.gopanic, runtime.panicshift, runtime.panicunsafestringlen, all of them - goes unused on exactly the binaries it was vendored for. The fallout is structural: a call to runtime.panicshift keeps its fall-through edge, the panic path renders as if the helper returned, and any function whose stack-growth arm physically follows such a call keeps its morestack loop (the gomorestack fold refuses the broken shape, correctly). The names are still in the image - the pclntab names every function - so this pass walks the same table the gopclntab pass recovers names from (same executable-range admission) and matches each (addr, name) against the same vendored list with the same reader, emitting ordinary address-keyed no-return facts. The commit installs pclntab names before the no-return arm resolves, so the facts land whether or not a .symtab exists; on an unstripped binary they are idempotent duplicates of noreturn_known's. Registered ONLY for a Go-detected binary (beside gopclntab), so it is a structural no-op on every non-Go target; `option noreturn_known` continues to gate only the symbol-table pass, keeping the two sources separately flippable.", "use_when": "On (default) truncates flow at runtime panic/exit helpers on stripped Go binaries, exactly as unstripped ones already behave; off restores the fall-through reading of every runtime.panic*/throw/goexit call site (auditing, or a hand-patched runtime whose panic helpers really return).", "example": "option gonoreturn off", "source_decompiler": "kuna", "inspiration": "Ghidra noReturnFunctionConstraints.xml compiler id=golang arm (the list), applied through the pclntab name source its symbol-table reader cannot reach on a stripped binary", "change_kind": "analysis-enablement", "tier": "analysis", "symptoms": ["stripped go binary renders code after a runtime.panicshift or runtime.gopanic call as if the helper returned", "a go panic arm falls through into the next block and the output interleaves panic and normal paths", "a go function keeps its morestack entry loop because the grow arm follows a panic call kuna treats as returning"]}, - {"option": "fatstr", "values": ["off", "auto", "always"], "default": "auto", "destructive_as_default": false, "phase": "P5", "subphase": "const-pointer", "strength": "HARD", "rewind": "P5", "issue": "kuna-multilang-fatstr", "summary": "Render a length-carried (ptr, len) constant call-argument pair as a bounded string literal. Rust (&str, &[u8]) and Go (string) carry a string's length beside its pointer and never NUL-terminate the bytes, so the classic literal path - which reads the image until a terminator - refuses them and every string argument renders as a raw rodata address beside a magic length: takes_str((char *)&DAT_00102000, 15). The length is right there in the adjacent argument, so the rule reads exactly that many bytes and routes them through the existing bounded literal mechanism (Funcdata::get_internal_string -> the BUILTIN_STRINGDATA user-op, the constseq path, which needs no terminator): takes_str(\"hello multilang\", 15). A pair is rewritten only when both arguments are p-code constants adjacent in the call's inputs, 2 <= len <= 4096, the pointed-at bytes are READ-ONLY in the image (query_global_properties, the msvcstrappend literal test), and the bytes decode as UTF-8 text with no control characters beyond newline/tab/CR and at least one alphanumeric - or the pointer's facing type carries a text language facet (the multilang W3 side table on the type factory), a type provider's statement that outranks the byte heuristic. The length argument is kept: the call really does pass it. At auto (default) the rule acts only when the image's source language is Rust or Go (the sourcelang fact), a structural no-op elsewhere; always drops the language test (stage-test bytechunks); off keeps the raw pointer. The option also gates the ANALYSIS-side recognizer (pass id fatstrrecords, registered for a rustc-detected image): rustc packs unterminated literals back to back in .rodata, so the NUL-run string scan glues adjacent literals into one wrong row; the static (ptr, len) records that carry every Rust literal's length (a PIE's R_*_RELATIVE slots in .data.rel.ro, absolute pairs on a non-PIE) are re-read as length-BOUNDED string facts, committed ahead of the NUL stream so the glued rows retire and the data symbol at each literal spans exactly the literal. off drops both recognizers; the record facts are dropped at the read-symbols boundary like every analysis gate.", "use_when": "Leave auto: on a Rust or Go binary every constant &str/string argument renders as its literal text instead of a raw &DAT_ address beside a magic length. Use always on a carved bytechunk or raw image with no language signal. Flip off when auditing exact rodata addresses, or if a read-only buffer of coincidentally printable bytes is being misread as text at a call site.", "example": "option fatstr always", "source_decompiler": "kuna", "inspiration": "Rust &str / Go string fat-pointer call ABI measured against the NUL-terminated literal path (PrintC::pushPtrCharConstant requires a terminator); the rewrite reuses constseq's BUILTIN_STRINGDATA bounded-literal mechanism", "change_kind": "type-inference", "tier": "transform", "symptoms": ["rust or go string argument renders as a raw rodata address beside its length instead of the literal text", "takes_str(&DAT_00102000, 15) where the 15 bytes at that address are plainly the string", "string literals recovered on C binaries but never on rust or go ones"]}, + {"option": "fatstr", "values": ["off", "auto", "always"], "default": "auto", "destructive_as_default": false, "phase": "P5", "subphase": "const-pointer", "strength": "HARD", "rewind": "P5", "issue": "kuna-multilang-fatstr", "summary": "Render a length-carried (ptr, len) constant call-argument pair as a bounded string literal. Rust (&str, &[u8]) and Go (string) carry a string's length beside its pointer and never NUL-terminate the bytes, so the classic literal path - which reads the image until a terminator - refuses them and every string argument renders as a raw rodata address beside a magic length: takes_str((char *)&DAT_00102000, 15). The length is right there in the adjacent argument, so the rule reads exactly that many bytes and routes them through the existing bounded literal mechanism (Funcdata::get_internal_string -> the BUILTIN_STRINGDATA user-op, the constseq path, which needs no terminator): takes_str(\"hello multilang\", 15). A pair is rewritten only when both arguments are p-code constants adjacent in the call's inputs, 2 <= len <= 4096, the pointed-at bytes are READ-ONLY in the image (query_global_properties, the msvcstrappend literal test), and the bytes decode as UTF-8 text with no control characters beyond newline/tab/CR and at least one alphanumeric - or the pointer's facing type carries a text language facet (the multilang W3 side table on the type factory), a type provider's statement that outranks the byte heuristic. The length argument is kept: the call really does pass it. At auto (default) the rule acts only when the image's source language is Rust or Go (the sourcelang fact), a structural no-op elsewhere; always drops the language test (stage-test bytechunks); off keeps the raw pointer. The option also gates the ANALYSIS-side recognizer (pass id fatstrrecords, registered for a rustc- or Go-detected image): both languages pack unterminated literals back to back in .rodata, so the NUL-run string scan glues adjacent literals into one wrong row; the static (ptr, len) records that carry every Rust literal's length (a PIE's R_*_RELATIVE slots in .data.rel.ro, absolute pairs on a non-PIE) are re-read as length-BOUNDED string facts, committed ahead of the NUL stream so the glued rows retire and the data symbol at each literal spans exactly the literal. off drops both recognizers; the record facts are dropped at the read-symbols boundary like every analysis gate.", "use_when": "Leave auto: on a Rust or Go binary every constant &str/string argument renders as its literal text instead of a raw &DAT_ address beside a magic length. Use always on a carved bytechunk or raw image with no language signal. Flip off when auditing exact rodata addresses, or if a read-only buffer of coincidentally printable bytes is being misread as text at a call site.", "example": "option fatstr always", "source_decompiler": "kuna", "inspiration": "Rust &str / Go string fat-pointer call ABI measured against the NUL-terminated literal path (PrintC::pushPtrCharConstant requires a terminator); the rewrite reuses constseq's BUILTIN_STRINGDATA bounded-literal mechanism", "change_kind": "type-inference", "tier": "transform", "symptoms": ["rust or go string argument renders as a raw rodata address beside its length instead of the literal text", "takes_str(&DAT_00102000, 15) where the 15 bytes at that address are plainly the string", "string literals recovered on C binaries but never on rust or go ones"]}, {"option": "gomorestack", "values": ["off", "auto", "always"], "default": "auto", "destructive_as_default": true, "phase": "P2", "subphase": "flow-classification", "strength": "HARD", "rewind": "P2", "issue": "kuna-multilang-gomorestack", "summary": "Fold the Go stack-growth prologue. Every non-leaf function the Go compiler emits opens with a guard - compare SP against the goroutine's stackguard, and on failure call runtime.morestack(_noctxt) and LOOP BACK TO THE FUNCTION ENTRY to re-run the prologue on the grown stack. The check is pure runtime machinery (the function's own semantics are the fall-through path), but the back-edge makes the entry a loop head, so every live argument register joins its incoming value with its after-growth value and parameter recovery sees phi-fed arguments on every non-leaf Go function. The fold matches the tight guard/grow two-block cycle sitting on the inserted entry block (the shared entry-guard-loop shape, infra/idiom.rs) with the cold arm calling a runtime.morestack* routine - the name is only accepted IN the shape, so a coincidental name in straight-line flow cuts nothing - and removes the guard edge pre-SSA (the cleanupcode window), so the entry phi never forms and the guard's compare and stackguard load die to ordinary dead-code elimination. At auto (default) the fold acts only when the image's source language is Go (the sourcelang fact), making it a structural no-op on every non-Go binary; always drops the language test for corpora with no detection signal (a stage-test bytechunk); off keeps the guard for auditing.", "use_when": "Leave auto: on a Go binary the fold turns every function's entry-loop + runtime.morestack_noctxt() noise into straight-line flow and un-breaks parameter recovery; off only to audit the real prologue instructions (e.g. verifying a hand-patched stackguard). Use always on a Go function carved into a bytechunk or a raw image where detection has no buildinfo section to find.", "example": "option gomorestack off", "source_decompiler": "kuna", "inspiration": "The Go compiler's stack-growth prologue (cmd/compile, runtime.morestack_noctxt); the removal primitives are the securitycheck/stackguard pair (removeBranch + removeUnreachableBlocks) moved to the pre-SSA window", "change_kind": "structure-recovery", "tier": "transform", "symptoms": ["every function of a Go binary opens with if (condition) { runtime_morestack_noctxt(); } inside a loop to the entry", "go function parameters recovered as phi-fed locals or missing because the entry is a loop head", "do-while wrapping an entire go function body with a morestack call in its latch"]}, {"option": "goabi", "values": ["auto", "off"], "default": "auto", "destructive_as_default": false, "phase": "P1", "subphase": "environment-binding", "strength": "HARD", "rewind": "P1", "issue": "kuna-multilang-goabi", "summary": "Select the vendored golang compiler spec for a Go image. Go does not use the platform C ABI: since 1.17 its register ABI (abi-internal) passes arguments in RAX,RBX,RCX,RDI,RSI,R8-R11 on x86-64 and returns multiple values in those same registers, so under the default gcc model a Go function's parameters are invisible (RAX/RBX are scratch to SysV), a second return value is dead-coded away, and the statements computing it are DELETED - `func pair(a,b int) (int,int)` decompiles as `long pair(void){ return v1 + v2; }` with both locals unassigned and the product gone. Ghidra vendors golang cspecs (abi-internal default, abi0 the stack ABI, the write-barrier and duff helper models) for x86-64, x86-32 and AArch64; kuna ships them compiled and nothing selected them. At `auto` (default) an image whose effective source language is Go (the sourcelang fact, KUNA_SOURCELANG force included) composes the `:golang` compiler field in its SLEIGH language id on those three little-endian architectures; every other architecture and every non-Go image is untouched, and the engine's default-model retry covers a spec that fails to resolve. The id is composed inside `load file`, so the option travels as the KUNA_GOABI environment variable (the rexthunk/sourcelang loadtime pattern). The golang cspec also registers the `abi0` stack-ABI model, and a function or callee whose name ends `.abi0` (Go's dual-ABI stack alias - an unstripped go1.22 hello-world carries 115 such .symtab names; the pclntab carries none, so a stripped image keeps abi-internal everywhere) takes that model at the model-selection seams; the binding self-gates on the model being registered, so an explicit `--target ...:golang` gets it even with goabi off.", "use_when": "Leave auto: on a Go binary the golang cspec is the difference between invisible parameters with deleted statements and recovered register arguments with the multi-return join; flip off to read a Go image under the platform C model (comparing against another tool, or triaging a suspected cspec defect). Non-Go binaries never see the option either way.", "example": "option goabi off", "source_decompiler": "kuna", "inspiration": "Ghidra's vendored x86-64-golang/x86-32-golang/AARCH64_golang cspecs (abi-internal, abi0) and its Golang language-id selection; the loadtime env bridge is the rexthunk/sourcelang pattern", "change_kind": "analysis-enablement", "tier": "analysis", "symptoms": ["go function parameters missing or void while the body reads rax/rbx locals that are never assigned", "a go function returning two values loses the second and the statements computing it", "go output full of unassigned register-commented locals", "multi-return go call sites read registers the call never defines"]}, {"option": "gowritebarrier", "values": ["off", "auto", "always"], "default": "auto", "destructive_as_default": true, "phase": "P2", "subphase": "flow-classification", "strength": "HARD", "rewind": "P2", "issue": "kuna-multilang-gowritebarrier", "summary": "Fold the Go GC write-barrier diamond. Every pointer store the garbage collector must see compiles to a flag test: when runtime.writeBarrier is set, call a runtime.gcWriteBarrier thunk (which hands back the GC pointer buffer, R11 on amd64) and record the old and new pointer values in it; both paths then perform the program's own stores at the join. The cold arm is GC bookkeeping with no bearing on what the function computes, and it actively corrupts the output today: the thunk's buffer register is not a modeled return, so the recording stores read an undefined local and the value split can mangle even the real store's address expression. The fold matches the shared guarded-cold-call shape (infra/idiom.rs): a one-block cold arm holding exactly one call, whose guard rejoins the hot path in one hop - the callee name (runtime.gcWriteBarrier, gcWriteBarrier1..8, the bare assembly thunk gcWriteBarrier; never wbBufFlush, which user code cannot call inline) is only accepted INSIDE that shape, so a coincidental name in ordinary flow cuts nothing. The guard edge is removed pre-SSA (the cleanupcode/gomorestack window, registered directly after gomorestack), so the buffer-feeding loads die to ordinary dead-code elimination and the program's stores at the join are untouched. At auto (default) the fold acts only when the image's source language is Go; always drops the language test (stage-test bytechunks); off keeps the barrier for auditing GC behavior.", "use_when": "Leave auto: on a Go binary every guarded pointer store sheds its if (runtime_writeBarrier) { gcWriteBarrierN(); buffer writes } wrapper and renders as the plain store; off only to audit the GC's barrier placement. Use always on a Go function carved into a bytechunk where detection has no buildinfo section to find.", "example": "option gowritebarrier off", "source_decompiler": "kuna", "inspiration": "The Go compiler's GC write barrier (cmd/compile wbcall lowering, runtime.gcWriteBarrier1-8 buffer protocol); the removal primitives are the securitycheck/stackguard pair moved to the pre-SSA window", "change_kind": "structure-recovery", "tier": "transform", "symptoms": ["every pointer store in a go function wrapped in if (writeBarrier flag) with a gcWriteBarrier call", "stores through an undefined local after a gcWriteBarrier call because the thunk's buffer register is not modeled", "go output doubled in size by GC bookkeeping diamonds"]}, diff --git a/docs/options.md b/docs/options.md index fa675b9b7..223bc27fe 100644 --- a/docs/options.md +++ b/docs/options.md @@ -2262,7 +2262,7 @@ The control surface: each of these can make output worse on the wrong source sha ### `fatstr` -- off | auto | always, default `auto` - **Symptoms:** rust or go string argument renders as a raw rodata address beside its length instead of the literal text; takes_str(&DAT_00102000, 15) where the 15 bytes at that address are plainly the string; string literals recovered on C binaries but never on rust or go ones. -- **What it does:** Render a length-carried (ptr, len) constant call-argument pair as a bounded string literal. Rust (&str, &[u8]) and Go (string) carry a string's length beside its pointer and never NUL-terminate the bytes, so the classic literal path - which reads the image until a terminator - refuses them and every string argument renders as a raw rodata address beside a magic length: takes_str((char *)&DAT_00102000, 15). The length is right there in the adjacent argument, so the rule reads exactly that many bytes and routes them through the existing bounded literal mechanism (Funcdata::get_internal_string -> the BUILTIN_STRINGDATA user-op, the constseq path, which needs no terminator): takes_str("hello multilang", 15). A pair is rewritten only when both arguments are p-code constants adjacent in the call's inputs, 2 <= len <= 4096, the pointed-at bytes are READ-ONLY in the image (query_global_properties, the msvcstrappend literal test), and the bytes decode as UTF-8 text with no control characters beyond newline/tab/CR and at least one alphanumeric - or the pointer's facing type carries a text language facet (the multilang W3 side table on the type factory), a type provider's statement that outranks the byte heuristic. The length argument is kept: the call really does pass it. At auto (default) the rule acts only when the image's source language is Rust or Go (the sourcelang fact), a structural no-op elsewhere; always drops the language test (stage-test bytechunks); off keeps the raw pointer. The option also gates the ANALYSIS-side recognizer (pass id fatstrrecords, registered for a rustc-detected image): rustc packs unterminated literals back to back in .rodata, so the NUL-run string scan glues adjacent literals into one wrong row; the static (ptr, len) records that carry every Rust literal's length (a PIE's R_*_RELATIVE slots in .data.rel.ro, absolute pairs on a non-PIE) are re-read as length-BOUNDED string facts, committed ahead of the NUL stream so the glued rows retire and the data symbol at each literal spans exactly the literal. off drops both recognizers; the record facts are dropped at the read-symbols boundary like every analysis gate. +- **What it does:** Render a length-carried (ptr, len) constant call-argument pair as a bounded string literal. Rust (&str, &[u8]) and Go (string) carry a string's length beside its pointer and never NUL-terminate the bytes, so the classic literal path - which reads the image until a terminator - refuses them and every string argument renders as a raw rodata address beside a magic length: takes_str((char *)&DAT_00102000, 15). The length is right there in the adjacent argument, so the rule reads exactly that many bytes and routes them through the existing bounded literal mechanism (Funcdata::get_internal_string -> the BUILTIN_STRINGDATA user-op, the constseq path, which needs no terminator): takes_str("hello multilang", 15). A pair is rewritten only when both arguments are p-code constants adjacent in the call's inputs, 2 <= len <= 4096, the pointed-at bytes are READ-ONLY in the image (query_global_properties, the msvcstrappend literal test), and the bytes decode as UTF-8 text with no control characters beyond newline/tab/CR and at least one alphanumeric - or the pointer's facing type carries a text language facet (the multilang W3 side table on the type factory), a type provider's statement that outranks the byte heuristic. The length argument is kept: the call really does pass it. At auto (default) the rule acts only when the image's source language is Rust or Go (the sourcelang fact), a structural no-op elsewhere; always drops the language test (stage-test bytechunks); off keeps the raw pointer. The option also gates the ANALYSIS-side recognizer (pass id fatstrrecords, registered for a rustc- or Go-detected image): both languages pack unterminated literals back to back in .rodata, so the NUL-run string scan glues adjacent literals into one wrong row; the static (ptr, len) records that carry every Rust literal's length (a PIE's R_*_RELATIVE slots in .data.rel.ro, absolute pairs on a non-PIE) are re-read as length-BOUNDED string facts, committed ahead of the NUL stream so the glued rows retire and the data symbol at each literal spans exactly the literal. off drops both recognizers; the record facts are dropped at the read-symbols boundary like every analysis gate. - **When to flip:** Leave auto: on a Rust or Go binary every constant &str/string argument renders as its literal text instead of a raw &DAT_ address beside a magic length. Use always on a carved bytechunk or raw image with no language signal. Flip off when auditing exact rodata addresses, or if a read-only buffer of coincidentally printable bytes is being misread as text at a call site. - **Where / provenance:** P5/const-pointer · kuna · type-inference · kuna-multilang-fatstr - **Example:** `option fatstr always`