update light/dark mode

Author: atharvas

DATATABLE.md

@@ -43,7 +43,7 @@ Create `src/data/results.json` with structure:
     {
       "id": "3",
       "agent": "terminus-2,oracle",
-      "displayName": "Oracle (Human)",
+      "displayName": "Expert Human (Human)",
       "levels": {
         "param-level": 0.2500,
         "func-level": 0.2200,

docs/_app/immutable/chunks/BT2_t_HU.js

@@ -42,7 +42,7 @@
 git clone https://github.com/formula-code/terminal-bench.git
 cd terminal-bench
 pip install -e .</code></pre></div> <h2 class="svelte-s2t5c8">Verify Installation</h2> <div class="code-block svelte-s2t5c8"><div class="code-header svelte-s2t5c8">bash</div> <pre class="svelte-s2t5c8"><code class="svelte-s2t5c8">harbor --version
-harbor list -d formulacode</code></pre></div></article>  <article id="running" class="doc-section svelte-s2t5c8"><h1 class="svelte-s2t5c8">Running an Evaluation</h1> <h2 class="svelte-s2t5c8">Quick Start</h2> <p class="svelte-s2t5c8">Run an evaluation using the Oracle (human expert) agent to verify your
+harbor list -d formulacode</code></pre></div></article>  <article id="running" class="doc-section svelte-s2t5c8"><h1 class="svelte-s2t5c8">Running an Evaluation</h1> <h2 class="svelte-s2t5c8">Quick Start</h2> <p class="svelte-s2t5c8">Run an evaluation using the Expert Human (human expert) agent to verify your
 				setup:</p> <div class="code-block svelte-s2t5c8"><div class="code-header svelte-s2t5c8">bash</div> <pre class="svelte-s2t5c8"><code class="svelte-s2t5c8">harbor run -d formulacode@0.1.0.post20251025 -a oracle</code></pre></div> <h2 class="svelte-s2t5c8">Evaluating a Custom Agent</h2> <p class="svelte-s2t5c8">To evaluate your own agent, specify the agent configuration:</p> <div class="code-block svelte-s2t5c8"><div class="code-header svelte-s2t5c8">bash</div> <pre class="svelte-s2t5c8"><code class="svelte-s2t5c8"># Run with a specific agent and model
 harbor run -d formulacode@0.1.0.post20251025 \
     -a your_agent \

docs/_app/immutable/nodes/3.DyOdMTp5.js

@@ -32,7 +32,7 @@
 	<a href="#content" class="skip-to-main">Skip to main content</a>
 	<div>  <nav class="navbar svelte-16kkjp5" aria-label="Main navigation"><div class="navbar-inner svelte-16kkjp5"><a class="navbar-brand svelte-16kkjp5" href="/" data-svelte-h="svelte-169ip8e"><span class="brand-icon svelte-16kkjp5">⚡</span> <span class="brand-text svelte-16kkjp5">FormulaCode</span></a> <button class="hamburger svelte-16kkjp5" aria-label="Toggle navigation menu" aria-expanded="false"><span class="bar svelte-16kkjp5"></span> <span class="bar svelte-16kkjp5"></span> <span class="bar svelte-16kkjp5"></span></button> <ul class="nav-links svelte-16kkjp5"><li><a href="/" class="svelte-16kkjp5">Overview</a> </li><li><a href="/blog/" class="svelte-16kkjp5">Blog</a> </li><li><a href="/leaderboard/" class="svelte-16kkjp5 active">Leaderboard</a> </li><li><a href="/docs/" class="svelte-16kkjp5">Getting Started</a> </li></ul></div> </nav> <main id="content"> <section class="leaderboard-page svelte-1h6jcx3"><div class="container svelte-1h6jcx3"> <div class="page-header svelte-1h6jcx3" data-svelte-h="svelte-fz6f60"><h1 class="svelte-1h6jcx3">FormulaCode Leaderboard</h1> <p class="subtitle svelte-1h6jcx3">Performance of frontier LLM agents on repository-level code
 				optimization, compared against human expert patches. Updated monthly.</p></div>  <div class="metrics-section svelte-1h6jcx3"><h2 class="svelte-1h6jcx3" data-svelte-h="svelte-152aplx">Metrics</h2> <div class="metrics-grid svelte-1h6jcx3"><div class="metric-card svelte-1h6jcx3" style="--accent: var(--wine-gold)"><div class="metric-symbol svelte-1h6jcx3">RP</div> <div class="metric-body svelte-1h6jcx3"><h3 class="svelte-1h6jcx3">RP Rank</h3> <p class="svelte-1h6jcx3">Ranked Pairs position computed over pairwise advantage comparisons across all agent-model configurations. Lower is better.</p></div> </div><div class="metric-card svelte-1h6jcx3" style="--accent: var(--score-good)"><div class="metric-symbol svelte-1h6jcx3">Adv</div> <div class="metric-body svelte-1h6jcx3"><h3 class="svelte-1h6jcx3">Advantage</h3> <p class="svelte-1h6jcx3">Speedup_agent − Speedup_expert. Measures performance relative to human experts. A positive value means the agent outperformed the human.</p></div> </div><div class="metric-card svelte-1h6jcx3" style="--accent: var(--category-cyan)"><div class="metric-symbol svelte-1h6jcx3">Ã</div> <div class="metric-body svelte-1h6jcx3"><h3 class="svelte-1h6jcx3">Normalized Advantage</h3> <p class="svelte-1h6jcx3">Signal-to-noise ratio of the agent advantage, accounting for variance across workloads. Rewards consistent improvements across many benchmarks.</p></div> </div><div class="metric-card svelte-1h6jcx3" style="--accent: var(--category-blue)"><div class="metric-symbol svelte-1h6jcx3">S</div> <div class="metric-body svelte-1h6jcx3"><h3 class="svelte-1h6jcx3">Speedup</h3> <p class="svelte-1h6jcx3">Geometric mean of per-workload speedup ratios (baseline / agent execution time). Values > 1 indicate the agent made the code faster.</p></div> </div></div></div>  <div class="table-section svelte-1h6jcx3"><h2 class="svelte-1h6jcx3" data-svelte-h="svelte-16gwwlq">Global Leaderboard</h2> <p class="table-note svelte-1h6jcx3" data-svelte-h="svelte-t5qm0y">Ranked by Ranked Pairs ordering on Advantage. Click headers to sort.</p> <div class="table-wrapper svelte-1h6jcx3"><table class="svelte-1h6jcx3"><thead class="svelte-1h6jcx3"><tr><th class="text-col svelte-1h6jcx3" data-svelte-h="svelte-1j9cto">Agent</th> <th class="text-col svelte-1h6jcx3" data-svelte-h="svelte-1cw35u8">Model</th> <th class="num-col sortable svelte-1h6jcx3 active">RP Rank <span class="sort-arrow svelte-1h6jcx3">↑</span> </th><th class="num-col sortable svelte-1h6jcx3">Adv <span class="sort-arrow svelte-1h6jcx3">↑</span> </th><th class="num-col sortable svelte-1h6jcx3">Ã <span class="sort-arrow svelte-1h6jcx3">↑</span> </th><th class="num-col sortable svelte-1h6jcx3">Speedup <span class="sort-arrow svelte-1h6jcx3">↑</span> </th></tr></thead> <tbody><tr class="svelte-1h6jcx3"><td class="agent-name svelte-1h6jcx3">OpenHands</td> <td class="model-name svelte-1h6jcx3">Claude 4.0 Sonnet</td> <td class="rank-cell svelte-1h6jcx3">1</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0112</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0483</td> <td class="num-cell svelte-1h6jcx3">1.0539</td> </tr><tr class="svelte-1h6jcx3"><td class="agent-name svelte-1h6jcx3">OpenHands</td> <td class="model-name svelte-1h6jcx3">Qwen 3 Coder</td> <td class="rank-cell svelte-1h6jcx3">2</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0301</td> <td class="num-cell svelte-1h6jcx3  negative">-0.1529</td> <td class="num-cell svelte-1h6jcx3">1.0346</td> </tr><tr class="svelte-1h6jcx3"><td class="agent-name svelte-1h6jcx3">OpenHands</td> <td class="model-name svelte-1h6jcx3">GPT-5</td> <td class="rank-cell svelte-1h6jcx3">3</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0209</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0702</td> <td class="num-cell svelte-1h6jcx3">1.0825</td> </tr><tr class="svelte-1h6jcx3"><td class="agent-name svelte-1h6jcx3">Terminus 2</td> <td class="model-name svelte-1h6jcx3">Claude 4.0 Sonnet</td> <td class="rank-cell svelte-1h6jcx3">4</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0410</td> <td class="num-cell svelte-1h6jcx3  negative">-0.1065</td> <td class="num-cell svelte-1h6jcx3">1.0987</td> </tr><tr class="svelte-1h6jcx3"><td class="agent-name svelte-1h6jcx3">Terminus 2</td> <td class="model-name svelte-1h6jcx3">Qwen 3 Coder</td> <td class="rank-cell svelte-1h6jcx3">5</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0454</td> <td class="num-cell svelte-1h6jcx3  negative">-0.1257</td> <td class="num-cell svelte-1h6jcx3">1.0677</td> </tr><tr class="svelte-1h6jcx3"><td class="agent-name svelte-1h6jcx3">Terminus 2</td> <td class="model-name svelte-1h6jcx3">Gemini 2.5 Pro</td> <td class="rank-cell svelte-1h6jcx3">6</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0433</td> <td class="num-cell svelte-1h6jcx3  negative">-0.1138</td> <td class="num-cell svelte-1h6jcx3">1.0963</td> </tr><tr class="svelte-1h6jcx3"><td class="agent-name svelte-1h6jcx3">Terminus 2</td> <td class="model-name svelte-1h6jcx3">GPT-5</td> <td class="rank-cell svelte-1h6jcx3">7</td> <td class="num-cell svelte-1h6jcx3  negative">-0.0504</td> <td class="num-cell svelte-1h6jcx3  negative">-0.1387</td> <td class="num-cell svelte-1h6jcx3">1.0585</td> </tr><tr class="svelte-1h6jcx3 human-row"><td class="agent-name svelte-1h6jcx3">Human Expert</td> <td class="model-name svelte-1h6jcx3">—</td> <td class="rank-cell svelte-1h6jcx3">—</td> <td class="num-cell svelte-1h6jcx3">0.0000</td> <td class="num-cell svelte-1h6jcx3">0.0000</td> <td class="num-cell svelte-1h6jcx3">1.1040</td> </tr></tbody></table></div></div>  <div class="stratified-section svelte-1h6jcx3"><h2 class="svelte-1h6jcx3" data-svelte-h="svelte-1xa4n95">Stratified Advantage Breakdown</h2> <p class="table-note svelte-1h6jcx3" data-svelte-h="svelte-l0ylna">Advantage computed at different code hierarchy levels — module, class,
-				and function — to reveal multi-scale optimization profiles.</p> <section id="leaderboard" class="svelte-itw7ld"><div class="leaderboard-container svelte-itw7ld"><h2 class="svelte-itw7ld" data-svelte-h="svelte-16hmka3">Leaderboard</h2> <p class="description svelte-itw7ld" data-svelte-h="svelte-crdgsv">This leaderboard displays the agent advantage scores by aggregation level. Higher scores indicate better performance relative to the oracle.</p> <center data-svelte-h="svelte-rcl2ku"><p class="instructions">Use the thresholding filters above and see how they change the leaderboard.</p></center> <div class="table-wrapper svelte-itw7ld"><table class="svelte-itw7ld"><thead class="svelte-itw7ld"><tr><th class="agent-col svelte-itw7ld" data-svelte-h="svelte-11w4vv4">Agent</th> <th class="level-col svelte-itw7ld">L1: Parameter</th><th class="level-col svelte-itw7ld">L2: Function</th><th class="level-col svelte-itw7ld">L3: Class</th><th class="level-col svelte-itw7ld">L4: Module</th> <th class="overall-col svelte-itw7ld" data-svelte-h="svelte-rn5hbs">Overall</th></tr></thead> <tbody class="svelte-itw7ld"><tr class="svelte-itw7ld"><td class="agent-name svelte-itw7ld">Terminus 2 - Claude Sonnet 4.0</td> <td class="score-cell medium svelte-itw7ld">0.0955 </td><td class="score-cell medium svelte-itw7ld">0.0291 </td><td class="score-cell high svelte-itw7ld">0.1424 </td><td class="score-cell low svelte-itw7ld">-0.0002 </td> <td class="score-cell overall-cell medium svelte-itw7ld">0.0749</td> </tr><tr class="svelte-itw7ld"><td class="agent-name svelte-itw7ld">Terminus 2 - GPT-5</td> <td class="score-cell medium svelte-itw7ld">0.0090 </td><td class="score-cell medium svelte-itw7ld">0.0092 </td><td class="score-cell medium svelte-itw7ld">0.0060 </td><td class="score-cell medium svelte-itw7ld">0.0034 </td> <td class="score-cell overall-cell medium svelte-itw7ld">0.0079</td> </tr><tr class="svelte-itw7ld"><td class="agent-name svelte-itw7ld">Terminus 2 - Oracle</td> <td class="score-cell medium svelte-itw7ld">0.0000 </td><td class="score-cell medium svelte-itw7ld">0.0000 </td><td class="score-cell medium svelte-itw7ld">0.0000 </td><td class="score-cell medium svelte-itw7ld">0.0000 </td> <td class="score-cell overall-cell medium svelte-itw7ld">0.0000</td> </tr></tbody></table></div></div> </section></div>  <div class="submit-section svelte-1h6jcx3" data-svelte-h="svelte-ozhudu"><h2 class="svelte-1h6jcx3">Submit Your Agent</h2> <p class="svelte-1h6jcx3">To evaluate your agent on FormulaCode, follow the <a href="https://github.com/formula-code/terminal-bench/" target="_blank" rel="noopener" class="svelte-1h6jcx3">installation instructions</a> and run:</p> <div class="code-block svelte-1h6jcx3"><code class="svelte-1h6jcx3">harbor run -d formulacode@0.1.0.post20251025 -a oracle</code></div> <p class="svelte-1h6jcx3">See the <a href="/docs/" class="svelte-1h6jcx3">Getting Started guide</a> for complete documentation.</p></div></div> </section></main> <footer class="svelte-126cfes"><div class="c svelte-126cfes"> <div class="paper-footer-sections svelte-126cfes"> <section class="paper-section citation-section svelte-126cfes"><h3 class="paper-section-title svelte-126cfes">Citation</h3> <div class="bibtex-wrapper svelte-126cfes"><pre class="bibtex-code svelte-126cfes"><code class="svelte-126cfes">@misc{sehgal2025formulacode,
+				and function — to reveal multi-scale optimization profiles.</p> <section id="leaderboard" class="svelte-itw7ld"><div class="leaderboard-container svelte-itw7ld"><h2 class="svelte-itw7ld" data-svelte-h="svelte-16hmka3">Leaderboard</h2> <p class="description svelte-itw7ld" data-svelte-h="svelte-crdgsv">This leaderboard displays the agent advantage scores by aggregation level. Higher scores indicate better performance relative to the oracle.</p> <center data-svelte-h="svelte-rcl2ku"><p class="instructions">Use the thresholding filters above and see how they change the leaderboard.</p></center> <div class="table-wrapper svelte-itw7ld"><table class="svelte-itw7ld"><thead class="svelte-itw7ld"><tr><th class="agent-col svelte-itw7ld" data-svelte-h="svelte-11w4vv4">Agent</th> <th class="level-col svelte-itw7ld">L1: Parameter</th><th class="level-col svelte-itw7ld">L2: Function</th><th class="level-col svelte-itw7ld">L3: Class</th><th class="level-col svelte-itw7ld">L4: Module</th> <th class="overall-col svelte-itw7ld" data-svelte-h="svelte-rn5hbs">Overall</th></tr></thead> <tbody class="svelte-itw7ld"><tr class="svelte-itw7ld"><td class="agent-name svelte-itw7ld">Terminus 2 - Claude Sonnet 4.0</td> <td class="score-cell medium svelte-itw7ld">0.0955 </td><td class="score-cell medium svelte-itw7ld">0.0291 </td><td class="score-cell high svelte-itw7ld">0.1424 </td><td class="score-cell low svelte-itw7ld">-0.0002 </td> <td class="score-cell overall-cell medium svelte-itw7ld">0.0749</td> </tr><tr class="svelte-itw7ld"><td class="agent-name svelte-itw7ld">Terminus 2 - GPT-5</td> <td class="score-cell medium svelte-itw7ld">0.0090 </td><td class="score-cell medium svelte-itw7ld">0.0092 </td><td class="score-cell medium svelte-itw7ld">0.0060 </td><td class="score-cell medium svelte-itw7ld">0.0034 </td> <td class="score-cell overall-cell medium svelte-itw7ld">0.0079</td> </tr><tr class="svelte-itw7ld"><td class="agent-name svelte-itw7ld">Terminus 2 - Expert Human</td> <td class="score-cell medium svelte-itw7ld">0.0000 </td><td class="score-cell medium svelte-itw7ld">0.0000 </td><td class="score-cell medium svelte-itw7ld">0.0000 </td><td class="score-cell medium svelte-itw7ld">0.0000 </td> <td class="score-cell overall-cell medium svelte-itw7ld">0.0000</td> </tr></tbody></table></div></div> </section></div>  <div class="submit-section svelte-1h6jcx3" data-svelte-h="svelte-ozhudu"><h2 class="svelte-1h6jcx3">Submit Your Agent</h2> <p class="svelte-1h6jcx3">To evaluate your agent on FormulaCode, follow the <a href="https://github.com/formula-code/terminal-bench/" target="_blank" rel="noopener" class="svelte-1h6jcx3">installation instructions</a> and run:</p> <div class="code-block svelte-1h6jcx3"><code class="svelte-1h6jcx3">harbor run -d formulacode@0.1.0.post20251025 -a oracle</code></div> <p class="svelte-1h6jcx3">See the <a href="/docs/" class="svelte-1h6jcx3">Getting Started guide</a> for complete documentation.</p></div></div> </section></main> <footer class="svelte-126cfes"><div class="c svelte-126cfes"> <div class="paper-footer-sections svelte-126cfes"> <section class="paper-section citation-section svelte-126cfes"><h3 class="paper-section-title svelte-126cfes">Citation</h3> <div class="bibtex-wrapper svelte-126cfes"><pre class="bibtex-code svelte-126cfes"><code class="svelte-126cfes">@misc{sehgal2025formulacode,
         title={FormulaCode: Evaluating Agentic Optimization on Large Codebases}, 
         author={Atharva Sehgal and James Hou and Akanksha Sarkar and Ishaan Mantripragada and Swarat Chaudhuri and Jennifer J. Sun and Yisong Yue},
         year={2025},

docs/_app/immutable/nodes/7.Ck3xZhtG.js

@@ -24,6 +24,7 @@
 		"@sveltejs/vite-plugin-svelte": "3.0.1",
 		"archieml": "^0.5.0",
 		"autoprefixer": "^10.4.17",
+		"baseline-browser-mapping": "^2.9.19",
 		"d3": "^7.8.5",
 		"layercake": "^8.0.2",
 		"lodash.debounce": "^4.0.8",