sanitize: clear recycled-block poison in the heap, not per handler

[sidick]

Fixes #95. Builds on #96 (@codewiz's commit is the first of the two here, unchanged) and finishes the job.

#96's diagnosis and its AllocVec shadow-marking fix are right. Two things were left over.

1. The clear reached 8 of ~48 allocation sites

AllocMem/AllocVec/AllocPooled are not the only things that carve guest structures out of GuestHeap -- dosfile, doslock, dosargs, dosseg, dosanchor, bsdsocket, intuition, graphics, locale, dosdevlist, exectask, execlib and dispatch all do, ~48 sites in total, and any of them can be handed a block that FreeMem/FreeVec/FreePooled poisoned.

A guest that frees a block larger than the 64 KiB quarantine budget (so it is released immediately rather than held back) and then opens a file reproduces #95's symptom exactly, against the FileHandle/MsgPort that dos Open carves out. With AllocVec(100000) -> FreeVec -> Open("T:x", MODE_NEWFILE) -> Write, on #96's branch and on main alike:

sanitizer: 1 site(s), 46 violation(s):
  PC 0x000007e2: 46 invalid 1-byte writes (freed block), addresses 0x00002c18-0x00002c43

Those are the Open handler's own initializing writes, attributed to the guest's PC. Worse than the reports themselves, the block stays Unaddressable while live, so later guest reads of that FileHandle report too.

Patching the other 40 sites by hand would work until the next one is added. So the clear moves to GuestHeap::alloc/alloc_with_requested -- the one point where a block leaves the heap's control -- and they now take &mut dyn AddressSpace, which turns "remember to clear the shadow map" from a convention into a compile error. The whole block is cleared, redzones included (nothing about the previous tenant applies to these addresses any more); callers wanting redzone/alignment-slack protection re-poison via poison_allocation_edges afterwards, which they already had to do after their own initializing writes.

The per-handler clear_fresh_block calls from #96 are gone, as is the need for handlers to know about any of this. GuestHeap's own placement/coalescing/quarantine tests keep a module-private alloc_bare shorthand, since they have no address space in play.

2. The wall-clock half of #95 was not the dedup hash

#95 guessed the slowdown was the (pc, addr, kind) hash past the MAX_VIOLATIONS cap. It measures as noise. The real cost is in check_read/check_write, which detected "did that byte report?" by comparing violation_count() before and after each byte -- and violation_count sums hits across the whole log. Once the log filled, every byte of every violating access walked 1000 entries twice. The byte checkers now return that answer directly.

On a 400,000-byte use-after-free loop, output byte-identical:

	before	after
wall clock	1.33 s	0.03 s

(I did implement the Bloom-filter pre-screen #95 suggests, measured it at 1.31 s -> 1.32 s, and dropped it rather than carry dead complexity.)

3. Smaller

• cargo fmt --all --check rejected sanitize: clear the freed-block poison when a block is reallocated #96's new test, which would have failed CI (ci.yml:29). Fixed.
• A clippy::collapsible_if in the reworked AllocVec block, likewise a -D warnings CI failure. Fixed.

Tests

• guestmem::tests::alloc_clears_a_recycled_blocks_freed_poison -- the fix where it lives, against a real FlatMemory.
• execmem::tests::a_genuine_use_after_free_is_still_reported -- the guard rail. alloc now clears whole blocks, so it matters that a block which has been freed and not reallocated stays poisoned, or --sanitize would have quietly stopped detecting use-after-free.
• sanitize: clear the freed-block poison when a block is reallocated #96's a_recycled_alloc_vec_block_is_no_longer_a_freed_block kept (verified it still fails with the fix stubbed out).

Verification

cargo test --release: 1010 passed, 0 failed. cargo clippy --all-targets -- -D warnings and cargo fmt --all --check clean.

Sanitizer output byte-identical to main on real workloads -- PhxAss under --sanitize and --sanitize-uninit, with and without --cpu 68020, both usage output and a real assemble; a real SAS/C sc hello.c; and all 12 fixtures under --sanitize.

Clean where they were not before: both #95 repro cases, and the Open-after-free case above.

Unrelated bug found while testing

Not touched here, reproduces on main with the repo's own fixtures: --ram above 16 MiB breaks any program that reaches an exec.library continuation.

$ volamos --ram 17M fixtures/exectest
volamos: fixtures/exectest: continuation stub trapped at 0x000000c4 with no pending
continuation (guest jumped to the trampoline stub directly, or ContinuationStack
bookkeeping is out of sync)
$ volamos --ram 16M fixtures/exectest
exec ok

fixtures/memtest does the same. fixtures/hello (dos only, no continuation) is fine at 64M. Worth its own issue.

🤖 Generated with Claude Code