Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
5995693
Update generate_runt_configs.py to add Runt test suite for bi
ngernest Jul 11, 2026
42c80d5
Update Justfile to run bi tests
ngernest Jul 11, 2026
15cb68a
Add auto-generated Runt BI config
ngernest Jul 11, 2026
8ea4731
Add expected test output for BI test cases
ngernest Jul 11, 2026
c1cff14
Formatting
ngernest Jul 11, 2026
d4c8557
Redirect stderr to stdout so that bi error messages appear in expecte…
ngernest Jul 11, 2026
d383c7b
Update auto-generated BI Runt config
ngernest Jul 11, 2026
1a6a54f
Update some .expect files
ngernest Jul 11, 2026
87e6d3c
Add --color CLI arg to BI to suppress colors
ngernest Jul 11, 2026
2ed33f9
Pass --color never to BI when running Runt tests
ngernest Jul 11, 2026
f2e9e3b
Update auto-generated Runt BI config
ngernest Jul 11, 2026
cf63102
Update .expect files
ngernest Jul 11, 2026
f4fc347
Register custom panic hook to suppress extraneous information from pa…
ngernest Jul 11, 2026
9efd6a4
Update .expect file output
ngernest Jul 11, 2026
a1881e7
Formatting
ngernest Jul 11, 2026
78ae7bb
Add BI tests to CI
ngernest Jul 11, 2026
f40648e
Remove monitor code + test cases
ngernest Jul 18, 2026
b5c4fe7
Update scripts for generating runt catalog
ngernest Jul 18, 2026
224ef81
update readme to refer to BI instead of monitor
ngernest Jul 18, 2026
975ee63
Change .monitor.prot test cases to bi.prot
ngernest Jul 18, 2026
b0924d1
Update case_stem function to refer to .bi suffix instead of .monitor
ngernest Jul 18, 2026
ec880b3
Regenerate Runt config
ngernest Jul 18, 2026
469e496
Merge branch 'main' into remove-monitor
ngernest Jul 18, 2026
c9f6f89
update some outdated comments
ngernest Jul 18, 2026
5401963
[BI] Replace panics with error messages, update Runt output (#289)
ngernest Jul 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
[alias]
interp = "run --quiet --offline --package protocols-interp --"
monitor = "run --quiet --offline --package protocols-monitor --"
bi = "run --quiet --package bi --"
parse = "run --quiet --package protocols-cli -- -p"
29 changes: 3 additions & 26 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ jobs:
run: runt runt/interp
- name: Run Runt tests for graph-interpreter
run: runt runt/graph_interp
- name: Run Runt tests for monitor
run: runt runt/monitor
- name: Run Runt tests for bi
run: runt runt/bi
- name: Run Runt tests for waveform
run: runt runt/waveform
- name: Run Runt tests for fail
Expand Down Expand Up @@ -120,30 +120,7 @@ jobs:
- name: Use ruff (via uv) to format Python files
# We pass in `--preview-features format` to suppress
# uv's warning messages about the formatter being experimental
run: uv format --check --preview-features format

# benchmark:
# name: Performance Benchmarks
# runs-on: ubuntu-24.04
# timeout-minutes: 30

# steps:
# - uses: actions/checkout@v4
# with:
# fetch-depth: 0 # Fetch all history to access baseline
# - uses: ./.github/actions/setup-test-environment
# with:
# github_token: ${{ secrets.GITHUB_TOKEN }}
# - name: Install hyperfine
# run: |
# wget https://github.com/sharkdp/hyperfine/releases/download/v1.19.0/hyperfine_1.19.0_amd64.deb
# sudo dpkg -i hyperfine_1.19.0_amd64.deb
# - name: Build monitor (release)
# run: cargo build --release --package protocols-monitor
# - name: Run benchmarks
# run: uv run scripts/benchmark_monitor.py
# - name: Check for performance regressions
# run: uv run scripts/check_performance_regression.py
run: uv format --check --preview-features format


verilog:
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[workspace]
resolver = "3"
members = ["bi", "interp", "monitor", "protocols", "cli", "graph-interp"]
members = ["bi", "interp", "protocols", "cli", "graph-interp"]


[workspace.package]
Expand Down
58 changes: 26 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ A protocol is described using an `prot` definition containing a sequence of stat

This repository contains:
- An *interpreter* for the DSL
- A *monitor* tool, which given a specification of a hardware module's communication behavior (written in our DSL)
- A *backwards interpreter* (BI) tool, which given a specification of a hardware module's communication behavior (written in our DSL)
and a `.fst` waveform file, infers a transaction-level trace that is consistent with the waveform data
- Common infrastructure shared between the interpreter & the monitor, such as a parser, type-checker and pretty-printer for our DSL
- Common infrastructure shared between the interpreter & the BI, such as a parser, type-checker and pretty-printer for our DSL
These tools are all implemented in Rust, with some auxiliary benchmarking scripts written in Python.

## Installation + Build Instructions
Expand All @@ -29,17 +29,7 @@ Note: the instructions below assume a macOS environment.
- (Optional) Run `brew install hyperfine` to install [Hyperfine](https://github.com/sharkdp/hyperfine), a command-line benchmarking tool
- Run `brew install just` to install [Just](https://github.com/casey/just), a command runner
- Run `cargo install --git https://github.com/Nikil-Shyamsunder/runt.git` to install our fork of [Runt](https://github.com/rachitnigam/runt), a command-line tool we use for snapshot tests. Runt compares the output of our tools to expected outputs stored in dedicated files.

**Dependencies for benchmarking the monitor**:
- Run `uv sync` to install the Python dependencies specified in `pyproject.toml`
- Then, from the top-level directory, run the following scripts (note that they take ~3 minutes to run):
```bash
$ uv run scripts/benchmark_monitor.py
$ uv run scripts/plot_benchmark_results.py
```
- This produces a CSV and a scatter plot measuring the performance of the monitor in
`benchmark_results/benchmark_results.csv` and `benchmark_results/benchmark_plot.png` respectively

- Some of the benchmarks correspond to real-world bugs taken from the [artifact for *Debugging in the Brave New World of Reconfigurable Hardware* (Ma et al. ASPLOS '22)](https://github.com/efeslab/asplos22-hardware-debugging-artifact) -- these can be found in the `tests/fpga-debugging` sub-directory
(more details in the `README` of the sub-directories corresponding to each bug)

Expand All @@ -49,51 +39,55 @@ $ uv run scripts/plot_benchmark_results.py
**Building the source code**:
- Run `cargo build` to build the Rust code
- Run `just test` to execute all unit tests (`cargo test`) + snapshot tests (via Runt)
- To only run the subset of tests for the monitor, run `just monitor`
- To generate HTML documentation, run `just doc` (this opens Cargo-generated docs in your browser)

## Monitor CLI
The CLI for the monitor can be used as follows:
## Backwards Interpreter CLI
The CLI for the BI can be used as follows:
```bash
$ cargo monitor --help
$ cargo bi --help
analyze signal traces to extract transactions

Usage: protocols-monitor [OPTIONS] --protocol <PROTOCOLS_FILE> --wave <WAVE_FILE>
Usage: bi [OPTIONS] --wave <WAVE_FILE>

Options:
-p, --protocol <PROTOCOLS_FILE>
Path to a Protocol (.prot) file
One or several protocol files.
-w, --wave <WAVE_FILE>
Path to a waveform trace (.fst, .vcd, .ghw) file
-i, --instances <INSTANCES>...
A mapping of DUT struct in the protocol file to an instance in the signal trace. Multiple arguments can be passed if they're seperated by whitespace. Format is: `${instance_name}:${dut_struct_name}`
A mapping of DUT struct in the protocol file to an instance in the signal trace. Can be used multiple times. Format is: `${instance_name}:${dut_struct_name}
-v, --verbose...
Increase logging verbosity
-q, --quiet...
Decrease logging verbosity
--color <COLOR_CHOICE>
To suppress colors in error messages, pass in `--color never` Otherwise, by default, error messages are displayed w/ ANSI colors [default: auto] [possible values: auto, always, never]
-d, --display-hex
If enabled, displays integer literals using hexadecimal notation
--sample-posedge <SIGNAL_TO_SAMPLE_ON_RISING_EDGE>
Optional argument which specifies the name of the signal to sample on a rising edge (posedge). If enabled, this flag acts as the "clock" signal for the monitor. Note: the full path to the signal should be passed as this argument, e.g. `uut_rx.clk`, where `uut_rx` is an instance in the signal trace
--show-waveform-time
If enabled, displays the start & end waveform time for each inferred transaction
--show-thread-ids
If enabled, displays the thread ID corresponding to each inferred transaction
--time-unit <TIME_UNIT>
Specifies the time unit for displaying waveform times. Can only be used with --show-waveform-time. Valid options: fs, ps, ns, us, ms, s, auto Default is 'auto' which selects the unit based on the maximum time in the waveform
--print-num-steps
Optional flag: if enabled, prints the no. of (logical) steps (i.e. clock cycles) taken by the montior
--include-idle
Optional flag: if enabled, always prints out idle transcations regardless of whether the protocol has been annotated with `#[idle]`
--show-steps
If enabled, displays the start & end step for each inferred transaction
--include-in-progress
This flag will make the bi append any in-progress transactions to the trace
--show-waveform-time

--time-unit <TIME_UNIT>

--max-traces <MAX_TRACES>
Limit the number of traces written to stdout
--color <COLOR_CHOICE>
To suppress colors in error messages, pass in `--color never`. Otherwise, by default, error messages are displayed with colors [default: auto] [possible values: auto, always, never]
-d, --display-hex
If enabled, displays integer literals using hexadecimal notation
-r, --rename <RENAME>...
Rename pins of the DUT. &{field name in the struct} = ${verilog name}
-h, --help
Print help
```

Example usage:
```bash
$ cd tests
$ cargo monitor -p adders/add_d1.prot --wave adders/add_d1.fst --instances add_d1:Adder
$ cargo bi -p adders/add_d1.prot --wave adders/add_d1.fst --instances add_d1:Adder
```

## Interpreter CLI
Expand Down
33 changes: 0 additions & 33 deletions benchmark_results/baseline_results.csv

This file was deleted.

33 changes: 0 additions & 33 deletions benchmark_results/benchmark_results.csv

This file was deleted.

14 changes: 9 additions & 5 deletions bi/src/bi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -632,11 +632,15 @@ impl Thread {
}
}
Stmt::Fork => {
assert!(self.step > 0, "[{}] Cannot fork at step zero!", self.name);
self.has_forked = true;
self.next_stmt = ti.next_stmt[&stmt];
self.effectful_stmt_in_step = true;
Fork
if self.step > 0 {
self.has_forked = true;
self.next_stmt = ti.next_stmt[&stmt];
self.effectful_stmt_in_step = true;
Fork
} else {
eprintln!("[{}] Cannot fork at step zero!", self.name);
std::process::exit(1);
}
}
Stmt::While(cond, body) => {
let cond_value = self
Expand Down
Loading
Loading