Feature: compare a failed Browser Use run with the last successful run

[aaronlab]

A Browser Use discussion comment suggested that the most useful trace viewer is not only replaying a failed run, but comparing it with the last successful run and showing where the path diverged.

This came from Browser Use community feedback on the BrowserTrace discussion:
browser-use/browser-use#4816

First useful version

• keep one stable run_id for each browser task
• store enough version/config metadata to compare runs fairly: Browser Use version, BrowserTrace version, model/provider, prompt/template version, and relevant adapter config
• compute a step-level comparison between a failed run and a selected successful run
• highlight the first divergent step by URL/title/action/target element summary/model output/error
• link retry or repair attempts back to the failed step they attempted to fix
• show a diffable final result summary when extracted/clicked/submitted/changed output is available

Suggested PR slices

1. Metadata slice: record comparable run metadata without changing the UI. Include fixture coverage for missing or partial metadata.
2. CLI prototype slice: add a JSON-first comparison command or helper that compares two selected runs and returns the first divergent step.
3. Browser Use adapter slice: capture the Browser Use fields that make comparison useful, such as task text, model/provider, action summary, URL/title, extracted content, and error boundary when exposed by the callback or run-hook surface.
4. UI slice: show a compact failed-vs-successful comparison view after the CLI/data shape is stable.
5. Retry-link slice: link retry/repair attempts back to the failed step they attempted to fix.

A good first PR here should probably target slice 1 or a small part of slice 2. Please keep the first change narrow and include a short fixture or regression test.

Non-goals for the first version

• no hosted telemetry
• no automatic sharing
• no assumption that two runs are comparable unless their metadata makes that explicit
• no full DOM/session replay requirement for the first iteration

Useful feedback

If you have a real Browser Use failure where a previous successful run would have helped, please share which fields would make the divergence obvious. Public examples should avoid private trace data, tokens, cookies, customer data, and screenshots with sensitive content.

[aaronlab]

Progress update: v0.1.19 now includes an explicit CLI comparison path:

browsertrace compare <failed_run_id> <success_run_id>
browsertrace compare <failed_run_id> <success_run_id> --json

This first slice compares existing step fields (action, url, status, and error) and reports the first divergent step between two recorded runs. It is intentionally explicit: no automatic "last successful run" selection, no SQLite schema change, and no UI comparison yet.

Leaving this issue open for the next slices: automatic baseline selection, richer Browser Use metadata comparison, and a UI affordance once the CLI shape has enough real usage feedback.

[aaronlab]

Progress update: v0.1.20 now exposes the same failed-vs-good comparison payload through the local web API while the BrowserTrace UI is running:

GET /api/compare/<failed_run_id>/<success_run_id>

This matches the CLI JSON path:

browsertrace compare <failed_run_id> <success_run_id> --json

Current scope remains explicit and conservative:

• user selects the failed run and known-good run
• BrowserTrace compares existing step fields: action, url, status, and error
• response includes the first divergent step and per-run step details
• no automatic baseline selection yet
• no UI diff affordance yet

Useful next feedback: for real Browser Use pass/fail pairs, should comparison add final result, extracted content, tool output, retry/repair attempts, selected element summaries, or version/config metadata first?

Release: https://github.com/aaronlab/browsertrace/releases/tag/v0.1.20
API docs are in the README and browsertrace compare --help. No stars/upvotes requested; concrete run-shape feedback is the useful signal.

[laolaoqi]

I can work on this issue. I have experience with similar tasks and can produce a clean, well-tested PR.

Timeline: Within 24-48 hours of assignment.
Skills: Python, TypeScript, full-stack development
Payment: USDT/USDC TRC20
Wallet: 0x6c21794E27C34E93cd0049918d04Da58FC72595c

Let me know if you'd like me to proceed.

[aaronlab]

Thanks for offering.

This repository is not offering paid work or bounties, and I am not assigning paid implementation work from this issue.

This feature issue is mainly for real Browser Use pass/fail feedback and future scoped slices. If you have concrete run-shape feedback from a real Browser Use failed-vs-good pair, please share that context here. If you want to contribute code voluntarily, please first propose a narrow scope before opening a PR.