Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,16 @@ otherwise falls back to a 4.5-second budget meant for interactive use, and a
large repository answers with partial usage sets that score as failures rather
than as the timeouts they are. The runner also holds a backstop deadline of the
budget plus a minute, so Bifrost's own budget expires first and reports
incomplete evidence instead of the call being abandoned. When `--output` is set, the runner also
incomplete evidence instead of the call being abandoned.

The runner also launches Bifrost with `BIFROST_MAX_LINE_LENGTH=0` and
`BIFROST_ADMIT_NUL_BYTES=1`. Bifrost's interactive defaults refuse to parse a
file with a line over 16,000 bytes or with a NUL byte, and one refused file can
make a language's whole usage index unavailable. Benchmark repositories contain
both in valid source (a long SVG path string in JSX, a raw NUL inside a string
literal), so the benchmark admits them and measures the repository as written.

When `--output` is set, the runner also
atomically updates a sibling `*.partial.json` checkpoint after each completed
benchmark document so interrupted batch runs retain valid evidence. Checkpoints
record `completed: false` plus the full `requestedCaseFiles` scope and are
Expand Down
7 changes: 7 additions & 0 deletions docs/src/content/docs/reproduce.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,13 @@ Checkpoints identify themselves with `completed: false`, retain the full
`requestedCaseFiles` scope, and cannot be used as completed snapshot or release
evidence.

The runner also launches Bifrost with `BIFROST_MAX_LINE_LENGTH=0` and
`BIFROST_ADMIT_NUL_BYTES=1`. Bifrost's interactive defaults refuse to parse a
file with a line over 16,000 bytes or with a NUL byte, and one refused file can
make a language's whole usage index unavailable. Benchmark repositories contain
both in valid source (a long SVG path string in JSX, a raw NUL inside a string
literal), so the benchmark admits them and measures the repository as written.

Keep `--work-dir` stable across focused profiling invocations to reuse an exact
Bifrost build and its verified executable digest. Reuse is invalidated by a
different resolved revision, toolchain, build plan/environment, or executable
Expand Down
16 changes: 15 additions & 1 deletion src/runners/bifrost.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,19 @@ pub const DEFAULT_SCAN_USAGES_MAX_DURATION_SECS: u64 = 300;
/// Bifrost's per-request analyzer budget, chosen by the frontend at server
/// launch. It supersedes the removed `max_duration_secs` request argument.
const BIFROST_REQUEST_BUDGET_ENV: &str = "BIFROST_MCP_REQUEST_BUDGET_SECS";
/// Source-admission settings for the Bifrost server the runner launches.
///
/// Bifrost's interactive defaults refuse to parse a file with a physical line
/// over 16,000 bytes or with any NUL byte, and a refused file can leave a whole
/// language's usage index unavailable. Real benchmark repositories contain both
/// in otherwise valid source: a long SVG path string in JSX, or a raw NUL inside
/// a string literal. The benchmark measures usage lookup on the repository as
/// written, so it turns the line limit off and admits NUL bytes as whitespace.
/// A Bifrost revision that predates a setting ignores it.
const BIFROST_SOURCE_ADMISSION_ENV: [(&str, &str); 2] = [
("BIFROST_MAX_LINE_LENGTH", "0"),
("BIFROST_ADMIT_NUL_BYTES", "1"),
];
/// How the requested usage-scan budget reaches the Bifrost server.
///
/// The mechanism changed across Bifrost releases: through v0.10.7 the scan
Expand Down Expand Up @@ -791,7 +804,8 @@ fn run_document_cases(
.arg("--root")
.arg(source_root)
.arg("--server")
.arg("searchtools");
.arg("searchtools")
.envs(BIFROST_SOURCE_ADMISSION_ENV);
if !workspace_models.is_empty() {
command.env("BIFROST_WORKSPACE_SEMANTIC_MODELS", "on");
}
Expand Down
Loading