feat: contract-level inline configurations in Solidity tests

[popescuoctavian]

This PR adds support for contract-level inline config directives.

A NatSpec directive above a contract definition now applies to every test the contract runs, with function-level directives taking per-key precedence:

/// forge-config: default.fuzz.runs = 50
contract MyTest is Test {
    /// forge-config: default.fuzz.runs = 100
    function testFuzz_Override(uint256 a) public {} // runs = 100
    function testFuzz_Inherits(uint256 a) public {} // runs = 50
}

Key points

• Contract-level config is merged underneath each test function's own overrides by walking the contract's ABI, so it also covers test functions inherited from base contracts — mirroring Foundry.
• Merging is per key, including inside the fuzz/invariant sections (contract sets fuzz.runs, function sets fuzz.maxTestRejects → both apply).
• Both fuzz.* and invariant.* keys are valid at contract level (each section only affects tests of its kind); all other validation (syntax, keys, values, profiles, duplicates) applies as at function level. The same key at contract and function level is not a duplicate — the function wins.

[changeset-bot]

🦋 Changeset detected

Latest commit: b148776

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@nomicfoundation/edr	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[Copilot]

Pull request overview

This PR adds contract-level inline configuration directives for Solidity tests, so a NatSpec directive above a contract definition becomes the default for all tests run by that contract (including inherited ones), while function-level directives override per key.

Changes:

• Extend inline-config collection/parsing to capture contract-level directives and merge them beneath function-level overrides.
• Apply contract-level configuration to all ABI-discovered test functions (including inherited) via a new fill_from merge helper in config overrides.
• Add integration + unit tests and update NAPI/TypeScript types to reflect that directive errors may not have an associated function (contract-level directives).

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
js/integration-tests/solidity-tests/test/fuzz.ts	Adds integration coverage asserting contract-level vs function-level precedence for fuzz runs.
js/integration-tests/solidity-tests/test-contracts/ContractLevelInlineConfig.t.sol	New Solidity test contract exercising contract-level hardhat-config directive behavior.
crates/edr_solidity_tests/tests/testdata/default/inline/ContractLevelConfig.t.sol	Adds Rust testdata covering contract-level config, function overrides, and inherited tests (plus invariants).
crates/edr_solidity_tests/tests/it/inline_config.rs	Adds integration test validating contract-level config applies to direct + inherited tests and invariants.
crates/edr_solidity_tests/src/multi_runner.rs	Implements ABI-walk merge of contract-level config underneath per-function overrides.
crates/edr_solidity_tests/src/inline_config/provider.rs	Changes provider API to return a contract-level + per-function structure (ContractInlineConfig).
crates/edr_solidity_tests/src/inline_config/parse.rs	Switches from locating functions to locating contracts + their functions, enabling contract-level NatSpec scanning.
crates/edr_solidity_tests/src/inline_config/overrides.rs	Collects contract-level directives and function directives into ContractInlineConfig, with improved error localization.
crates/edr_solidity_tests/src/inline_config/error.rs	Updates error types/rendering to support directive problems without a function (contract-level).
crates/edr_solidity_tests/src/inline_config/directives.rs	Adds DirectiveTarget to validate keys differently for contract vs function directives; exposes is_test_function.
crates/edr_solidity_tests/src/inline_config.rs	Updates module docs and re-exports new inline-config types/utilities.
crates/edr_solidity_tests/src/config.rs	Adds fill_from merging logic for per-key fallback behavior across nested fuzz/invariant sections.
crates/edr_napi/src/solidity_tests/inline_config.rs	Makes directive error “function” optional for contract-level directives in the NAPI surface.
crates/edr_napi/index.d.ts	Updates TypeScript declarations to match optional function for contract-level directive errors.
.changeset/salty-colts-tie.md	Declares a minor release for the new contract-level inline-config feature.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov]

Codecov Report

❌ Patch coverage is 92.50720% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.94%. Comparing base (568f28a) to head (b148776).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/edr_solidity_tests/src/config.rs	86.79%	14 Missing ⚠️
.../edr_solidity_tests/src/inline_config/overrides.rs	92.00%	6 Missing ⚠️
...edr_solidity_tests/src/inline_config/directives.rs	93.47%	0 Missing and 3 partials ⚠️
...s/edr_solidity_tests/src/inline_config/provider.rs	93.18%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1627      +/-   ##
==========================================
- Coverage   80.02%   79.94%   -0.09%     
==========================================
  Files         464      460       -4     
  Lines       80392    80344      -48     
  Branches    80392    80344      -48     
==========================================
- Hits        64332    64227     -105     
- Misses      13855    13904      +49     
- Partials     2205     2213       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[popescuoctavian]

/bench hardhat-ref=popescuoctavian/edr-contract-level-inline-config

[github-actions]

🚀 Starting regression benchmark for 8b4e1062e1f5 against Hardhat popescuoctavian/edr-contract-level-inline-config (benchmarks matching test solidity,test mocha,test vitest).

[github-actions]

❌ Regression benchmark failed for 8b4e1062e1f5 against Hardhat popescuoctavian/edr-contract-level-inline-config. This is either a detected performance regression or an infrastructure failure — see the run for details.

View workflow run

[popescuoctavian]

/bench hardhat-ref=popescuoctavian/edr-contract-level-inline-config

[github-actions]

🚀 Starting regression benchmark for 7595688a138c against Hardhat popescuoctavian/edr-contract-level-inline-config (benchmarks matching test solidity,test mocha,test vitest).

[github-actions]

❌ Regression benchmark failed for 7595688a138c against Hardhat popescuoctavian/edr-contract-level-inline-config. This is either a detected performance regression or an infrastructure failure — see the run for details.

View workflow run

[popescuoctavian]

Note: pending NomicFoundation/hardhat#8575 to pass HH3 regression benchmarks.

[Wodann]

Overall, this looks good. I left some comments that you can address without asking for re-review (unless you want to).

[Wodann]

Thought: Can there be a third variant (e.g. file-level)? If so, can we use an union type to discriminate?

[popescuoctavian]

The directives are NatSpec doc comments which bind to declarations (contracts, functions). Something like a file-level directive would require a new parsing mechanism (e.g. pragma-style comments), which I believe is unlikely to be added. Also, even if we switched to a union now, adding another variant would still be a breaking change. I'd leave it as is for now and switch to a union in the future if that's required.

[Wodann]

I orchestrated some agents for a second pair of eyes on this. They came up with the following suggestions that you can feed to an agent for the topics you agree with:

### Contract-level configuration never reaches `table*` tests

`crates/edr_solidity_tests/src/multi_runner.rs:359` gates the ABI walk on `inline_config::is_test_function` (`test*` / `invariant*` / `statefulFuzz*`). The runner classifies `table*` as a test (`crates/foundry/evm/core/src/abi/test_function_ext.rs:136`, `TestFunctionKind::TableTest`; dispatched at `runner.rs:887`) and table tests do consult the override map for `isolate` / `evmVersion` / `allowInternalExpectRevert`. So `/// forge-config: default.isolate = true` on a contract applies to every test except its table tests. (Pre-existing and related: `overrides.rs:162` uses the same predicate, so a function-level directive above a `table*` function is silently ignored today.)

Fix: use the runner's own classifier so the two definitions of "test" cannot drift.

```rust
use foundry_evm::abi::TestFunctionExt as _;
for function in contract.abi.functions() {
    if !function.is_any_test() { continue; }
    ...
}
```

Consider also redefining `directives::is_test_function` in terms of `TestFunctionKind::classify(name, false).is_any_test()` so `table*` functions can carry function-level directives; that removes the need for the new `pub use directives::is_test_function` export (see §4.3). Add a table test to `ContractLevelConfig.t.sol` and assert it receives a contract-level `isolate`/`evmVersion`.

### Hardhat `main` renders contract-level errors as `MyTest.undefined`

`packages/hardhat/src/internal/builtin-plugins/solidity-test/formatters.ts:35` and `:68` on Hardhat `main` interpolate `${error.contract}.${error.function}` unguarded. Template-literal interpolation of `string | undefined` compiles under Hardhat's `tsconfig` (`strict` without `exactOptionalPropertyTypes`), so bumping the EDR pin would silently print `- test/Foo.t.sol:4: FooTest.undefined: invalid value "-1" …` on the one new error path this feature introduces.

**Status (verified):** the fix already exists on Hardhat branch `popescuoctavian/edr-contract-level-inline-config` (commit `7b13efd53`: `formatDirectiveOwner` helper plus tests) — but no PR is open for it, and the PR body links https://github.com/NomicFoundation/hardhat/pull/8575, which is `chore(bench): update aave-v4 pin` and unrelated. Actions:

- Open the Hardhat PR from that branch and reference it from this PR's body; land it with (or before) the EDR release carrying this change.
- Fix the link in the PR body.
- That branch's test file uses `const NO_FUNCTION = undefined as unknown as string;` to compile against the pinned 0.19.0 types; drop the cast once the pin is bumped.

### Add a JS integration test for the contract-level error shape

`js/integration-tests/solidity-tests/test/unit.ts:39` (`InvalidInlineConfig`) is the only JS test asserting the N-API error shape and every entry is function-level. Nothing pins that `function` is actually absent on the object napi-rs hands over — the exact property that breaks Hardhat. Add a `ContractLevelInlineConfigInvalidTest` contract (separate file, so the existing `errors.length === 2` and line-number assertions are undisturbed) and assert `entry.function === undefined`, `entry.contract`, `entry.line`, and `problem.kind`. Note `byFunction` at `unit.ts:56` keys on `entry.function`, so an `undefined` key would currently slip through.

### Follow-up docs

No `.md` in EDR mentions `forge-config`, and Hardhat's docs live in the website repo. Open a docs issue/PR for the inline-config page covering contract-level directives, precedence, and the inheritance limitations in §3.2.

### Readability

- `overrides.rs:123` — the `located_problem` closure holds `&mut errors` for the rest of the function; a free `fn push_problem(errors: &mut Vec<_>, contract: &str, function: Option<&str>, located: LocatedDirectiveError)` avoids a future borrow surprise.
- `overrides.rs:37` — `ContractInlineConfig::contract` reads as "the contract" at call sites (`parsed.contract`, `config.contract.is_none()`); rename to `contract_level` (two reviewers independently).
- `directives.rs:411` — `is_fuzz_test` / `is_invariant_function(function)` are recomputed per key; hoist above the loop.
- `config.rs:207` — the free `fill` fn sits between `TestFunctionConfigOverride` and its `impl`; move below the three `impl` blocks.
- `config.rs:185` — `/// Test function level config override.` is now stale; the type also models contract-level configuration (see §5.1 item 6).

### Error message ergonomics (nit)

`directives.rs:411-424`: `invariant.depth` is rejected on `testFuzz_X` but accepted on the contract; the message does not mention the escape hatch. Suggest: `` key "invariant.depth" is not valid for fuzz tests; place it above the contract to apply it to the contract's invariant tests ``.

### Missing tests (Codecov gaps and asserted-but-untested behaviour)

- `provider.rs`: same key at contract and function level; a `hardhat-config:` contract-level directive (all Rust tests use `forge-config:`).
- Contract-level `allow-internal-expect-revert = true` reaching an inherited test — the only path that populates the `HashSet` in `multi_runner.rs`.
- Contract-level directive on a contract with zero test functions (collected, harmless).
- Base contract carrying its own contract-level directive → not inherited (§3.2).
- `parse.rs`: an `abstract contract` case for `locates_contracts_and_functions_with_offsets` — the `node_start` doc claims `abstract` is the start token (verified in Slang's `text_start.generated.rs:197-206`) but nothing tests it.
- `error.rs`: `DirectiveLocation` `Display` with `function: None`.
- JS: a contract-level `invariant.*` section via `hardhat-config:` plus one inherited `testFuzz_` in `ContractLevelInlineConfig.t.sol`, asserting `InvariantTestKind.runs`/`calls` (mirror `InvariantFunctionOverrides`, `fuzz.ts:575`). Contract-level `hardhat-config:` + `invariant.*` is currently untested at every layer.
- JS: §2.2.


### Line edits

Format: `path:line` → proposed replacement. Reason only where not self-evident. House rule: comments state constraints only, ≤2 clauses per sentence; change rationale lives in commit messages.

**`crates/edr_solidity_tests/src/multi_runner.rs`**

1. `:310-311` → `/// Parses the inline configuration of the given test contract from its` / `/// source, returning the overrides keyed by test selector and the set of` / `/// tests that opted into \`allowInternalExpectRevert\`.` — overrides are per selector now.
2. `:314-317` → `/// A contract-level configuration applies to every test function in the` / `/// contract's ABI, including inherited ones. Function-level directives take` / `/// per-key precedence.`
3. `:335-337` → `// Key the overrides by selector: every overload is a distinct test with a` / `// distinct selector.` — fixes the double space ("their own  entry") and the 86-column line.
4. `:340-341` → `// A name matching no ABI function (e.g. not externally callable) can't be` / `// run as a test.`
5. `:354-356` → `// Apply the contract-level configuration underneath each test's own` / `// overrides. Walking the ABI rather than the source covers inherited tests.`

**`crates/edr_solidity_tests/src/config.rs`**

6. `:185` → `/// Config override for a single test. Also used as a contract-level default` / `/// (see [\`Self::fill_from\`]).`
7. `:215-218` → `/// Fills every unset value from \`fallback\`, key by key — including within` / `/// the fuzz and invariant sections. Set values are kept, so a function-level` / `/// override wins over a contract-level \`fallback\`.`

**`crates/edr_solidity_tests/src/inline_config.rs`**

8. `:11-13` → `//! A directive above a contract definition applies to every test the contract` / `//! runs, including inherited ones. Function-level directives take per-key` / `//! precedence:`
9. After the code fence at `:18`, insert: `//!` / `//! A contract-level directive applies only to the contract that declares it;` / `//! it does not propagate to derived contracts. Directives are validated on` / `//! every contract in a test source. A function-level directive identifies its` / `//! function by name and so applies to every overload of that name.`
10. `:28` → `//!   - overrides  compose the above into a source's per-contract configuration`

**`crates/edr_solidity_tests/src/inline_config/directives.rs`**

11. `:3-5` → `//! Directives live in NatSpec comments above a test function or above the test` / `//! contract itself, and use either the \`forge-config:\` or \`hardhat-config:\`` / `//! prefix, e.g.:` — semantics belong in `inline_config.rs`, not restated here.
12. `:69-72` → `/// What a directive block is attached to. Determines which keys are valid: a` / `/// function accepts only the keys of its test kind, a contract accepts both` / `/// fuzz and invariant keys.`
13. `:81-82` → `/// The kind of test a key applies to. On a function, a key of a different kind` / `/// is rejected;` — contract targets no longer reject by kind.
14. `:407-411` → `// On a function the key must match the test kind; top-level keys are valid` / `// on both. A contract accepts both sections; each affects only the tests of` / `// its kind.`
15. `:424-426` → `// Reject duplicate keys within one target. The same key at contract and` / `// function level is not a duplicate; the function's value wins.`

**`crates/edr_solidity_tests/src/inline_config/natspec.rs`** (untouched by the diff, now stale)

16. `:4-8` → `//! them from the raw source (see [\`super::parse\`]). Given a definition's start` / `//! offset, we scan *backwards*: the scan stops at the first byte that is` / `//! neither whitespace nor part of a comment, so it reads only the comments` / `//! immediately above that contract or function definition.`
17. `:12` → `//! *All* NatSpec blocks (\`///\` runs and \`/** */\` comments) in the definition's`
18. `:24-25` → `//! one NatSpec block, possibly with plain comments between it and the` / `//! definition — all three agree.`
19. `:27` → `/// A NatSpec comment block found in a definition's leading region.`
20. `:46` → `/// previous member's \`}\`/\`;\` (or the enclosing contract's \`{\`, or the previous` / `/// source-unit member for a contract), and the work is`

**`crates/edr_solidity_tests/src/inline_config/provider.rs`**

21. `:53` → `/// inline configuration — once. Only sources that carry a directive are`

**`crates/edr_solidity_tests/src/inline_config/overrides.rs`**

22. `:30-33` → `/// The inline configuration parsed for a single contract: the directives above` / `/// the contract definition, and those above each of its test functions.`
23. `:36` → `/// The contract-level configuration, if the contract declares any. Applies` / `/// to every test the contract runs; function-level overrides win per key.` — the precedence rule lands once, on the field that carries it.
24. `:149` → delete `// Contract-level directives.` (the `DirectiveTarget::Contract` argument says it).
25. `:171-172` → `// Only the first problem per function is reported; parsing continues so` / `// every function's problem surfaces.`

**`crates/edr_solidity_tests/src/inline_config/error.rs`**

26. `:90-91` → `/// A key was used on a test of the wrong kind (e.g. \`fuzz.*\` on an invariant` / `/// test). Only function-level directives can produce this.`
27. `:180-182` → `/// test run is aborted before any suite executes. At most one problem is` / `/// reported per directive target — each test function, plus each contract's` / `/// own directives — across every source.`

**`crates/edr_napi/src/solidity_tests/inline_config.rs`** (edit here and regenerate `index.d.ts`; nothing in CI diffs the generated file, so verify by hand that the output matches)

28. `:71` → `/// The same key was specified more than once for the same function or` / `/// contract.` — the core copy was updated, the napi copy was missed (mirrors into `index.d.ts:961`).
29. `:44` — note that `InlineConfigInvalidKeyForTestType` cannot arise for a contract-level directive.
30. `:177-179` → `/// fix it. A discriminated union over \`kind\`: a \`source\`-level entry carries no` / `/// directive location, a \`directive\`-level entry carries the contract and line,` / `/// plus the function unless the directive is contract-level.` (mirrors into `index.d.ts:971-973`).
31. `:112-113` and `:167-168` → `/// The test function the directive belongs to. Undefined when the directive` / `/// is contract-level.` — TS consumers narrow on `undefined`, not "absent" (mirrors into `index.d.ts:921-923`, `:942-944`).

**Tests / testdata**

32. `crates/edr_solidity_tests/tests/it/inline_config.rs:169` → `"expected 2 runs of depth 3 (6 calls), got {:?}",`
33. `crates/edr_solidity_tests/tests/testdata/default/inline/ContractLevelConfig.t.sol:18` → delete `// Inherited: runs with the derived contract's fuzz.runs = 15.` (duplicates the comment above the base contract). Replace with `// A directive here would apply only when this contract itself runs; it is not inherited.` if §3.2's test is added.
34. `js/integration-tests/solidity-tests/test/fuzz.ts:544-546` → `// The contract-level directive in \`ContractLevelInlineConfig.t.sol\` applies` / `// to every test in the contract; function-level directives win per key.`
35. `fuzz.ts:561`, `:565` → `// From the contract-level directive` / `// Function-level directive wins`.
36. `fuzz.ts:557` — `testResult.kind as FuzzTestKind` is cast before the name check; neighbours cast inside each branch.

### Structural notes

- "Applies to every test it runs" appears six times across `inline_config.rs`, `directives.rs` (×3), `overrides.rs`, `multi_runner.rs`, plus test comments. Keep it in two places: the module doc (`inline_config.rs`, edit 8) and the field doc on `ContractInlineConfig::contract` (edit 23). Everywhere else state only the local constraint.
- Terminology: *directive* = the NatSpec line as written; *contract-level configuration* / *function-level override* = the parsed result; *target* = what a directive is attached to (only inside `directives.rs`). Replace bare "contract level" as a noun and "contract-level inline config" accordingly.
- `TestFunctionConfigOverride` now also models the contract level; edit 6 patches the doc. If a rename is on the table, `InlineConfigOverride` removes the caveat.
- Fixture naming diverges: `ContractLevelConfig.t.sol` (Rust) vs `ContractLevelInlineConfig.t.sol` (JS). Pick one.
- PR title: singular "configuration", matching the code's terminology.

[popescuoctavian]

All the other doc-related suggestions were addressed by 4906d21. The other suggestions were either already fixed, unfounded, or require additional work - table test suggestion is valid, but not trivial, and is not yet supported and documented in Hardhat; supporting it with inline config in EDR would silently add support in Hardhat, which I'm not sure is desired.

[Wodann]

Thanks for addressing feedback! LGTM!

[popescuoctavian]

/bench hardhat-ref=popescuoctavian/edr-contract-level-inline-config

[github-actions]

🚀 Starting regression benchmark for b148776d1fa6 against Hardhat popescuoctavian/edr-contract-level-inline-config (benchmarks matching test solidity,test mocha,test vitest).