Skip to content

refactor: name the failing arena explicitly in get_stack_trace - #1623

Merged
Wodann merged 1 commit into
mainfrom
refactor/explicit-failing-arena
Aug 31, 2026
Merged

refactor: name the failing arena explicitly in get_stack_trace#1623
Wodann merged 1 commit into
mainfrom
refactor/explicit-failing-arena

Conversation

@Wodann

@Wodann Wodann commented Aug 15, 2026

Copy link
Copy Markdown
Member

get_stack_trace used to take one flat iterator of trace arenas and convert whichever came last, relying on a doc comment — "assumes last trace is the error one" — to carry the whole contract. Every arena before the last was only ever walked to collect the creation/runtime bytecode of deployed contracts, needed to decode the failing trace.

This PR makes that split explicit in the signature:

  • failing_trace — the arena that is converted and analysed;
  • code_sources — arenas walked only for their CREATE nodes, to map contract addresses to code;
  • deployed_code — pre-computed address→code mappings, seeded before the arenas are walked. This replaces the previous third parameter, which could only seed runtime code; the provider now passes DeployedCode { runtime: Some(..) } with unchanged behaviour.

The test runner's collect_stack_trace helper requires the failing arena the same way. Its callers split it off execution_traces themselves, so the reasoning about whether a failing arena was recorded now lives at each call site, where it can actually be justified: the unit, table and fuzz sites run right after the failing call's arena was recorded under an active tracer; the one site where no arena may exist — an invariant campaign that failed before recording any EVM call — handles that case explicitly.

Why

Two reasons:

  1. Correctness by construction. The "last arena is the failing one" assumption was easy to break silently: any caller appending an arena after the failing one would change which trace gets analysed, with no compiler help. Now the failing arena is named at every call site.
  2. It unlocks the memory work. This establishes the invariant that only the failing arena's recorded EVM steps are ever read — every other arena contributes just its CREATE nodes. The follow-up PRs (fixing the OOM at Hardhat verbosity -vvv+) rely on exactly that to strip steps from every retained arena.

Behaviour change (one, deliberate)

An invariant campaign that fails without recording any EVM call (e.g. an ABI error, or too many vm.assume rejects) now reports no stack trace. Previously the "last arena" fallback silently pointed the stack-trace heuristics at the last setup arena — a call that necessarily succeeded, since a failed setUp() never reaches invariant execution — producing a heuristic failure at best. No arena describes such a failure; the test result's reason string already explains it ("failed to set up invariant testing environment: …").

Everything else is a pure refactor: same arenas walked in the same order, same seeding precedence (pre-computed code → code sources → failing trace, later wins).

Testing

  • cargo test -p edr_solidity_tests --test it (123 passing) — including always_mode_produces_stack_trace_for_failing_test, which exercises stack-trace generation through the unit, table, fuzz, invariant and invariant-campaign-failure paths.
  • Benchmarked runtime-neutral against the memory benchmark from bench: add a memory benchmark for the Solidity test runner #1621 (that harness was built to measure this PR's follow-ups; this refactor sits within run-to-run noise on every cell).

@changeset-bot

changeset-bot Bot commented Aug 15, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: af03cd9

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@Wodann
Wodann temporarily deployed to github-action-benchmark August 15, 2026 15:43 — with GitHub Actions Inactive
@codecov

codecov Bot commented Aug 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.80000% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.94%. Comparing base (a0a3ed8) to head (af03cd9).

Files with missing lines Patch % Lines
crates/edr_solidity_tests/src/runner.rs 92.53% 3 Missing and 2 partials ⚠️
crates/edr_solidity/src/solidity_stack_trace.rs 82.60% 1 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1623      +/-   ##
==========================================
+ Coverage   79.91%   79.94%   +0.02%     
==========================================
  Files         461      461              
  Lines       80116    80138      +22     
  Branches    80116    80138      +22     
==========================================
+ Hits        64023    64064      +41     
+ Misses      13883    13865      -18     
+ Partials     2210     2209       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Wodann
Wodann temporarily deployed to github-action-benchmark August 15, 2026 15:50 — with GitHub Actions Inactive
@Wodann
Wodann temporarily deployed to github-action-benchmark August 15, 2026 15:50 — with GitHub Actions Inactive
@Wodann Wodann added the no changeset needed This PR doesn't require a changeset label Aug 17, 2026
@Wodann Wodann self-assigned this Aug 17, 2026
@Wodann

Wodann commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

/bench hardhat-ref=update-oz-scenario-pin

@github-actions

Copy link
Copy Markdown
Contributor

⏳ EDR CI for this commit hasn't passed yet, so the regression benchmark was not started. Comment /bench again once CI is green.

@Wodann

Wodann commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

/bench

@github-actions

Copy link
Copy Markdown
Contributor

⏳ EDR CI for this commit hasn't passed yet, so the regression benchmark was not started. Comment /bench again once CI is green.

@Wodann
Wodann force-pushed the refactor/explicit-failing-arena branch from 731e708 to b446a04 Compare August 18, 2026 14:30
@Wodann
Wodann temporarily deployed to github-action-benchmark August 18, 2026 14:30 — with GitHub Actions Inactive
@Wodann
Wodann temporarily deployed to github-action-benchmark August 18, 2026 15:26 — with GitHub Actions Inactive
@Wodann
Wodann temporarily deployed to github-action-benchmark August 18, 2026 15:26 — with GitHub Actions Inactive
@Wodann
Wodann requested review from a team and a balanced review from Copilot August 24, 2026 12:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors stack-trace generation to explicitly identify the failing trace while treating preceding traces solely as deployed-code sources.

Changes:

  • Adds DeployedCode and updates get_stack_trace inputs and return type.
  • Updates provider, test runner, and invariant replay call sites.
  • Handles invariant failures with no recorded EVM trace explicitly.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
crates/edr_solidity/src/solidity_stack_trace.rs Introduces the explicit stack-trace API.
crates/edr_solidity_tests/src/runner.rs Selects failing traces at test-runner call sites.
crates/foundry/evm/evm/src/executors/invariant/replay.rs Selects failing traces during invariant replay.
crates/edr_provider/src/error.rs Adapts transaction failure handling to the new API.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@anaPerezGhiglia anaPerezGhiglia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Wodann
Wodann force-pushed the refactor/explicit-failing-arena branch from b446a04 to 2d909cc Compare August 28, 2026 10:34
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 10:34 — with GitHub Actions Inactive
@Wodann
Wodann had a problem deploying to github-action-benchmark August 28, 2026 10:37 — with GitHub Actions Failure
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 10:37 — with GitHub Actions Inactive
@Wodann
Wodann force-pushed the refactor/explicit-failing-arena branch from 2d909cc to e22b52d Compare August 28, 2026 11:58
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 11:59 — with GitHub Actions Inactive
@Wodann
Wodann had a problem deploying to github-action-benchmark August 28, 2026 12:08 — with GitHub Actions Failure
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 12:08 — with GitHub Actions Inactive
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 13:15 — with GitHub Actions Inactive
@Wodann
Wodann had a problem deploying to github-action-benchmark August 28, 2026 13:19 — with GitHub Actions Error
@Wodann
Wodann had a problem deploying to github-action-benchmark August 28, 2026 13:19 — with GitHub Actions Error
@Wodann
Wodann force-pushed the refactor/explicit-failing-arena branch from 0e52072 to c868758 Compare August 28, 2026 13:39
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 13:40 — with GitHub Actions Inactive
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 13:42 — with GitHub Actions Inactive
@Wodann
Wodann had a problem deploying to github-action-benchmark August 28, 2026 13:42 — with GitHub Actions Failure
@Wodann
Wodann force-pushed the refactor/explicit-failing-arena branch from c868758 to 97ca607 Compare August 28, 2026 21:31
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 21:31 — with GitHub Actions Inactive
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 21:34 — with GitHub Actions Inactive
@Wodann
Wodann had a problem deploying to github-action-benchmark August 28, 2026 21:34 — with GitHub Actions Error
@Wodann
Wodann force-pushed the refactor/explicit-failing-arena branch from 97ca607 to ff06e01 Compare August 28, 2026 22:16
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 22:16 — with GitHub Actions Inactive
@Wodann
Wodann temporarily deployed to github-action-benchmark August 28, 2026 22:19 — with GitHub Actions Inactive
@Wodann
Wodann had a problem deploying to github-action-benchmark August 28, 2026 22:19 — with GitHub Actions Failure
Base automatically changed from bench/solidity-tests-memory to main August 31, 2026 08:33
Previously the failing arena was implicit: `get_stack_trace` folded over
an iterator and converted whichever arena came last, with a doc comment
('assumes last trace is the error one') carrying the whole contract.
Every other arena was walked only to seed the creation/runtime code maps.

Make that split explicit in the signature:
- `failing_trace`: the arena that is converted and analysed,
- `code_sources`: arenas walked only for their CREATE nodes, which
  therefore never need recorded EVM steps,
- `deployed_code`: pre-computed creation/runtime code mappings, seeded
  before the arenas are walked (replaces the runtime-code-only third
  parameter; the provider now passes `DeployedCode { runtime: Some(..) }`).

`collect_stack_trace` requires the failing arena the same way: its callers
split it off `execution_traces` themselves, so the reasoning about whether
a failing arena was recorded lives at each call site.

One deliberate behavioural change falls out: an invariant campaign that
fails without recording any EVM call (e.g. an ABI error or too many
`vm.assume` rejects) now reports no stack trace, where it previously
pointed the stack-trace heuristics at the last *successful* setup arena
(`setUp()` has necessarily succeeded to reach invariant execution) and
produced a heuristic failure at best. The result's reason string already
explains such failures.

The stronger invariant — only the failing arena's steps are ever read —
is what allows later commits to strip steps from retained arenas.
@Wodann
Wodann force-pushed the refactor/explicit-failing-arena branch from ff06e01 to af03cd9 Compare August 31, 2026 08:33
@Wodann
Wodann temporarily deployed to github-action-benchmark August 31, 2026 08:33 — with GitHub Actions Inactive
@Wodann
Wodann temporarily deployed to github-action-benchmark August 31, 2026 08:37 — with GitHub Actions Inactive
@Wodann
Wodann had a problem deploying to github-action-benchmark August 31, 2026 08:37 — with GitHub Actions Failure
@Wodann
Wodann added this pull request to the merge queue Aug 31, 2026
Merged via the queue into main with commit a53e45d Aug 31, 2026
37 of 38 checks passed
@Wodann
Wodann deleted the refactor/explicit-failing-arena branch August 31, 2026 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changeset needed This PR doesn't require a changeset

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants