Skip to content

fix(measure): report why a server died instead of waiting out the timeout - #130

Merged
VickyXAI merged 1 commit into
mainfrom
fix/harness-fast-fail
Sep 2, 2026
Merged

fix(measure): report why a server died instead of waiting out the timeout#130
VickyXAI merged 1 commit into
mainfrom
fix/harness-fast-fail

Conversation

@VickyXAI

@VickyXAI VickyXAI commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Found while verifying the published 0.45.1 end-to-end.

The bug

Pointing scripts/measure-tool-schema.mjs at a command that fails to start gave you a silent 60-second wait and then timed out — for what was sh: blockrun-mcp: command not found in the first 200ms.

We advertise this script for measuring other people's servers, where a mistyped command or a package that 404s is the common case. A tool whose most likely failure takes a minute and then blames the wrong thing is one people try once.

The fix

  • stderr is piped, not ignored (tail only, so a chatty server isn't buffered whole).
  • A child that exits before the handshake rejects immediately, carrying its exit code and its own error text.
  • Timeout raised 60s → 120s, and the message now says what to do about it — a cold npx install of a package this size genuinely exceeds a minute, which is how the bad message got found in the first place.

Failure goes from 60s and a wrong reason to 0.25s and the right one:

Error: node /nonexistent-server.js exited with code 1 before completing the handshake.
Error: Cannot find module '/nonexistent-server.js'

Also worth recording

npx -y @blockrun/mcp@<version> run from inside this repo resolves to the local checkout and dies with exit 127. Measure published packages from a neutral directory. Done that way, 0.45.1 measures 12,900 — matching the card exactly.

Verification

  • 431/431 tests pass (+1: asserts the failure names the exit, carries the child's stderr, and returns in under 10s rather than on the timeout).
  • Typecheck and sync-brand-numbers --check clean; happy path unchanged at 12,900.

No version bump: scripts/ is not in the npm tarball, so this changes nothing for installed users and does not need a release.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WUL3ExR4Nz7uebxKaKwjDi

…eout

Pointing the harness at a command that fails to start gave you a silent
60-second wait and then "timed out" — for what was `sh: blockrun-mcp: command
not found` in the first 200ms. Since we advertise this for measuring OTHER
people's servers, where a mistyped command is the common case, that is the
difference between a tool someone uses twice and one they use once.

stderr is now piped rather than ignored (tail only, so a chatty server is not
buffered whole) and a child that exits before the handshake completes rejects
immediately with its exit code and its own error text. Failure went from 60s
and a wrong reason to 0.25s and the right one.

The timeout is also raised 60s -> 120s and now says what to do about it: a cold
`npx` install of a package this size genuinely exceeds a minute, which is how
the bad message got found.

Found while verifying the published 0.45.1 end-to-end. Worth recording that
`npx -y @blockrun/mcp@<v>` run from inside this repo resolves to the local
checkout and fails that way — measure published packages from a neutral
directory. From one, 0.45.1 measures 12,900, matching the card.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WUL3ExR4Nz7uebxKaKwjDi
@VickyXAI
VickyXAI merged commit 3d91bcc into main Sep 2, 2026
1 check passed
@VickyXAI
VickyXAI deleted the fix/harness-fast-fail branch September 2, 2026 04:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant