fix(bench): stop hardcoding the run date and replace GNU-only grep -oP in run_bench.sh

[yujung7768903]

Summary

benchmarks/run_bench.sh has two defects that make its CSV output wrong.

The run date is hardcoded (run_bench.sh:10). DATE="2026-03-15" is written straight into the date column, so every row carries that fixed date no matter when the benchmark ran.

grep -oP is GNU-only (run_bench.sh:24,26). The tok/s extraction is grep -oP '[\d.]+(?= tok/s)', and the BSD grep shipped with macOS has no -P. Both tok/s fields come back empty, the [ -z "$DECODE_TOKS" ] branch is taken, and a benchmark that completed normally is recorded as a FAILED row. Those two lines are the only use of grep -P across the shell scripts and the Makefile at upstream HEAD (6976e1df).

item	before	after
date	DATE="2026-03-15"	DATE=$(date +%F)
hardware	HARDWARE="NVIDIA_GB10_CUDA13.0"	${HARDWARE:-NVIDIA_GB10_CUDA13.0}
MLX version	MLX_VERSION="0.31.1"	${MLX_VERSION:-0.31.1}
tok/s and ms parsing	grep -oP (GNU-only)	sed -n -E (BSD and GNU)
error guards	none	set -euo pipefail + argument count check

Defaults are unchanged, so existing invocations behave as before.

One note on the approach: applying the issue's four items literally collides with itself. Under -e/pipefail, grep exiting 1 on no match kills the script before it can reach the existing FAILED-row branch. Moving the parsing to sed -n ...p avoids that — no match means no output and exit 0, so the empty string still flows into [ -z "$DECODE_TOKS" ] and the existing FAILED handling is preserved. A failing binary invocation is caught with || true so it takes the same path.

Related issues

resolves #1666

Test plan

Validated by running real inference, not a stub or faked output.

• OS: macOS 15.1 (Darwin 24.1.0, build 24B83)
• Arch: arm64 (Apple M3, 10-core GPU, Metal 3, 16 GB unified memory)
• Shell: GNU bash 3.2.57 (macOS default)
• grep: /usr/bin/grep — BSD grep 2.6.0-FreeBSD (no GNU grep on PATH)
• mlxcel: v0.7.0 (mlxcel-macos-aarch64.zip official release binary, sha256 verified)
• Model: mlx-community/Qwen3-0.6B-4bit
• Command: ./benchmarks/run_bench.sh mlx-community/Qwen3-0.6B-4bit out.csv

Both versions were run back to back with no environment variables set. Only the columns that differ are shown; the other eight are identical.

CSV field	before (current main)	after
prefill_ms	(empty)	104.12
prefill_tok_s	(empty)	182.48
decode_ms	(empty)	547.64
decode_tok_s	(empty)	182.60
date	2026-03-15 (hardcoded)	2026-09-15 (run date)
prompt	FAILED	"Explain the concept of..."

The before run produced that row even though inference finished normally. The cause is printed on stderr:

>>> Benchmarking: Qwen3-0.6B-4bit
grep: invalid option -- P
grep: invalid option -- P
    FAILED or no output

Also checked:

• Environment overrides — passing HARDWARE=Apple_M3_Metal MLX_VERSION=mlxcel-0.7.0 records those two columns with those values; omitting them records the previous defaults
• Missing arguments — running with no arguments prints usage to stderr and exits 1
• Failing binary — when the binary exits 1, the script does not abort: it writes the FAILED row and exits 0
• Syntax — bash -n passes

Notes for reviewers

No Rust sources are touched, so cargo fmt, cargo clippy, cargo test and cargo deny are unaffected by this change and were not run — this machine has no Rust toolchain, which is also why validation used the official v0.7.0 release binary rather than a local build.

Type of change: fix.

🤖 Generated with Claude Code