Repro It Research

reproit-research runs verified experiments and evaluations. It keeps research commands separate from the production reproit CLI and the campaign reproit-fuzzer CLI.

Commands

Command	Result
add DEFINITION	Verify and save one authored research experiment.
gate --config CONFIG	Compare baseline and candidate outputs.
verify BUNDLE	Verify a saved evidence bundle without Cloud access.

Use --details to show bounded failure facts and stable error codes.

Add an authored experiment

The add command validates a strict research recipe. It runs the recipe in a new temporary workspace and verifies each declared input digest before the measurement starts.

reproit-research add experiment.json
reproit check REPRO_ID

The command reports the Repro ID, claim digest, observed range, median, expected range, and unit. The production CLI owns the later regression check.

The recipe uses explicit programs and argument arrays. It does not use a shell. Setup commands can get an exact source, data, or model revision. The run command can use a repository-relative program or an installed tool.

Each measured run emits one JSON line for each criterion. Repro It records the minimum, median, and maximum. A result passes when each median is inside its inclusive expected range.

Repro It removes the temporary workspace after the operation. It does not store cloned repositories, downloaded data, model weights, package environments, or compiled executables as authored evidence.

Compare a candidate

The gate command runs a baseline command and a candidate command. Each command receives the same JSON Lines suite on standard input.

reproit-research gate --config release.toml

The configuration uses relative paths, explicit executables, argument arrays, and fixed resource limits. The command writes one content-addressed evidence bundle with the suite, both model runs, the verdict, and digest bindings.

The result is PASS, REGRESSION, or UNKNOWN. Exit code 0 means PASS. Exit code 1 means REGRESSION. Exit code 2 means UNKNOWN or an invalid operation.

Verify evidence

Verify a saved bundle without Cloud access:

reproit-research verify evidence.json

The command checks the raw evidence, model runs, suite, verdict, release decision, and all digest bindings.

Install

Choose one installation method.

Run the installer

git clone https://github.com/ReproIt/reproit-research.git
cd reproit-research
./install.sh

Windows users can run .\install.ps1 from PowerShell.

Build from source

git clone https://github.com/ReproIt/reproit-research.git
cd reproit-research
cargo build --locked --release --bin reproit-research

The executable is at target/release/reproit-research on Linux and macOS. Windows uses target\release\reproit-research.exe.

The build pins the shared application implementation from reproit-cli. It also pins the Core and Experiments contracts to exact Git revisions. The repository does not copy those implementations.

GitHub Action

The root action.yml runs the installed research gate. The default configuration path is .reproit/release.toml.

Verify the repository

Run the complete repository check:

./tools/test.sh