Repository navigation
Code and functional coverage with VCS - #1918
Draft
davidharrishmc wants to merge 7 commits into
Draft
davidharrishmc wants to merge 7 commits into
davidharrishmc wants to merge 7 commits into
Conversation
wsim accepted --ccov and --fcov only with Questa. run_vcs now compiles for code coverage with line, cond, branch and fsm metrics on testbench.dut.core (coverage.hier), the counterparts of Questa's +cover=sbecf: condition coverage in its default mode counts the 1-bit bitwise operators Wally's logic is written with (allops would add a condition per bit of every vector operation), and -cm_cond obs counts a subexpression only where it reaches the output, as Questa's focused expression coverage does. --fcov compiles in the riscv-arch-test covergroups the same way wally-compile.do does, with riscv-arch-test's RVVI trace in place of ImperasDV's. Each simulation writes its own database, sim/vcs/ucdb/<cfg>_<name>.vdb or fcov_ucdb/<cfg>_<name>.vdb, named like Questa's .ucdb, so that runs can go in parallel and be merged or discarded one at a time. A code coverage database starts as a copy of the coverage model the compile wrote, so it can be merged on its own. Functional coverage runs with -vcs_assert off, as riscv-arch-test's own VCS flow does, because VCS W-2024.09 otherwise stops with an internal assertion while sampling some covergroups (I_beq_cg). wsim keys the VCS compile on the covergroups, as it does for Questa, and recompiles when run_vcs or coverage.hier changes. Lockstep with ImperasDV works with --ccov. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: David Harris <David_Harris@hmc.edu>
…ions VcsCodeCoverage and VcsFunctCoverage are the urg counterparts of QuestaCodeCoverage and QuestaFunctCoverage: they merge the per-run .vdb databases and write text reports, a per-instance summary for code coverage (rv64gc_coverage_hierarchical.rpt, with covered/total counts) and the total covergroup score for functional coverage. URG takes exclusions only as an exclusion file whose entries name each object by its id and checksum, which only urg itself can produce. Rather than keep a second list of exclusions, vcs/exclusions2el translates the ones Questa uses: it replays coverage-exclusions-rv64gc.do with tclsh to get the coverage exclude commands, scans the RTL for // coverage off pragmas (which VCS ignores), and picks the matching objects out of urg's full exclusion dump of the merged database, using urg's text report for the source line of each branch condition. Statements, branch arms, whole conditions, FSM states and transitions, and expression and condition rows all have a counterpart. A row maps to the vector of the subexpression that has the row's input as an operand; for an & or | operand at its non-controlling value that vector is shared with the other operands, so it is excluded only when all of their rows are. The exclusions are applied to the merged database, as they now are for Questa, and the translation log (vcs/cov/exclusions-rv64gc.log) says for each exclusion what it became or why VCS has no counterpart. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: David Harris <David_Harris@hmc.edu>
--coveragesim {questa,vcs} picks the simulator for --ccov and --fcov; Questa stays the default. With
vcs the same tests run (code coverage one ELF per simulation in lockstep with ImperasDV, functional
coverage in chunks of 16 ELFs), each writing a .vdb under sim/vcs, and VcsCodeCoverage or
VcsFunctCoverage merges and reports them. Reuse of up-to-date code coverage, removal of databases no
selected test writes, and the check for runs that left no database work the same way; a .vdb counts as
complete only once the simulation has added its results, since run_vcs creates it before the run, and
the tool versions and run scripts that invalidate reuse are VCS's. A dry run no longer deletes the
coverage databases of an earlier run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: David Harris <David_Harris@hmc.edu>
…erage VCS counted the statements, branches and conditions behind a constant condition, such as the XLEN=32 arm of if (P.XLEN == 64) in csrc or ~P.ZICCLSM_SUPPORTED in trap, which Questa's optimizer removes before it counts coverage, so they showed up as holes no test can fill. -cm_noconst leaves them out. On rv64i/I, priv/Sm and tests/coverage, the uncovered priv lines drop from 37 to 1, as in Questa. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: David Harris <David_Harris@hmc.edu>
davidharrishmc
marked this pull request as draft
October 2, 2026 22:11
Take main's final wsim/regression-wally/run_vcs (commands as argument lists without a shell, coverage per simulation with a record of its ELFs) and put the VCS coverage back on top: - wsim: --ccov/--fcov accepted with --sim vcs; the VCS build hash includes fcovKey, and run_vcs and coverage.hier count as compile inputs. - run_vcs: coverage compile/run options, fcov sources and per-run .vdb as argument lists; with --lockstep --fcov, leave ImperasDV's rvviTrace.sv out of main's glob of the RVVI sources; per-build compile log name. - regression-wally: --coveragesim; databases under sim/<coveragesim>/ (.vdb for VCS), coverageMtime/removeCoverage for .vdb directories and their .elfs records, the VCS scripts and urg version among the coverage inputs, a dry run discards no coverage, and VcsCodeCoverage/VcsFunctCoverage. - sim/Makefile: keep the VCS targets; main's removal of the riscv-dv targets stands. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: David Harris <David_Harris@hmc.edu>
rosethompson
marked this pull request as ready for review
October 13, 2026 20:02
rosethompson
marked this pull request as draft
October 13, 2026 20:03
rosethompson
approved these changes
Oct 13, 2026
Keep main's coverage32Dir next to the --coveragesim comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: David Harris <David_Harris@hmc.edu>
Contributor
Author
|
Review notes: one fix before taking this out of draft. Questa, Verilator and non-coverage VCS runs are unaffected.
🤖 Generated with Claude Code |
-code b -allfalse now excludes only the all-false branch (the implicit else or missing case default) instead of every statement, branch arm and condition on the line; -code limits -linerange to the named kinds, and any other option is reported NOT PORTED instead of being ignored. An exclusion that matches nothing in VCS and whose source line holds nothing of the kind it excludes is reported STALE and makes the script, and so VcsCodeCoverage, fail, as Questa does for an exclusion that matches nothing. tclsh is found on the caller's PATH, and the package install script installs tcl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Review items addressed in 527d837:
The stale 🤖 Generated with Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds code and functional coverage on Synopsys VCS.
wsim --sim vcs --ccov/--fcov(with lockstep for--ccov) writes per-run.vdbdatabases undersim/vcs,make -C sim VcsCodeCoverage/VcsFunctCoveragemerge them with urg, andregression-wally --coveragesim vcsruns the same coverage regressions as Questa. Code coverage uses line, cond (observability), branch and fsm on the core; functional coverage compiles the same riscv-arch-test covergroups. Rather than keep a second exclusion list,sim/vcs/exclusions2eltranslatescoverage-exclusions-rv64gc.doand the RTL// coverage offpragmas into a URG exclusion file. It logs the exclusions VCS has no exact counterpart for (43 of 299, mostly where Questa excludes one row of a vector VCS scores as a whole), and, like Questa, fails on an exclusion that names a line with nothing to exclude.Closes #1179.
🤖 Generated with Claude Code