Note. Upstream scoring rules from the Partcl/HRT Macro Placement Challenge, kept for context on how this placer was judged. Not documentation for this repository — see
README.md.
This document defines the evaluation system for the Grand Prize ($20,000).
The Grand Prize evaluation consists of two stages:
- Feasibility — hard timing constraints (pass/fail)
- Scoring — deterministic ranking via geometric mean of improvement ratios
Only submissions that pass Stage 1 on all designs are eligible for the Grand Prize.
The top 7 submissions by Tier 1 proxy cost (IBM benchmarks) are evaluated through the full OpenROAD flow on NG45 designs.
Designs evaluated:
ariane133(public)ariane136(public)mempool_tile(public)nvdla(public)- 1–2 hidden NG45 designs (to prevent overfitting)
Baselines are generated by running Simulated Annealing (SA) and RePlAce (RP) through the same OpenROAD evaluation flow used for submissions.
For each design, we measure:
- WNS — Worst Negative Slack (ns). Higher (closer to 0) is better.
- TNS — Total Negative Slack (ns). Higher (closer to 0) is better.
- Area — Core area (μm²). Lower is better.
Baseline numbers will be published for all public designs once generated.
A submission must not regress timing below both baselines on any design.
For each design:
WNS_sub ≥ min(WNS_SA, WNS_RP)
TNS_sub ≥ min(TNS_SA, TNS_RP)
- You cannot be worse than both baselines on WNS or TNS for any design
- You may match one baseline and beat the other
- If either condition fails on any design → submission is disqualified from the Grand Prize
If OpenROAD fails on any design (exits non-zero before producing WNS/TNS/Area), the submission is technically disqualified from the Grand Prize. In practice, we will reach out and work with the team to resolve the failure — whether the cause is on our side (evaluator bug, netlist issue) or on the placer's side (a placement choice ORFS can't route through).
The evaluator auto-handles the following on behalf of placers:
- Snap macro positions to the manufacturing grid
- Iteratively push macros apart to maintain ≥12 μm clearance (PDN channel
routing needs ~10 μm between adjacent macros). This happens only at
Tier 2, after scaling the placement to ORFS core coordinates; Tier 1
proxy cost uses your submitted coordinates unchanged. The pre- and
post-push coordinates are logged to a sidecar
macros.tcl.spacing_diff.txtin the ORFS results directory so you can see exactly what was moved and by how much. To keep full control over Tier 2 coordinates, leave ≥12 μm between macros in your submitted placement. - Instance name escaping for Genus netlists
The placer is responsible for: zero overlaps, staying within canvas bounds, macro orientation choices, and any other decisions that affect downstream ORFS routability.
Tier 1 (Proxy Prize) ranking is unaffected by Tier 2 / ORFS outcomes.
All feasible submissions are ranked using a single scalar score computed from improvement ratios. This approach:
- Naturally handles metrics on different scales
- Prioritizes timing improvement (WNS most, then TNS) while keeping Area in the score
- Uses a weighted geometric mean with fixed weights (WNS:3, TNS:2, Area:1) reflecting the relative importance of each metric in the final design
For each design, compute the average of SA and RePlAce:
WNS_avg = (WNS_SA + WNS_RP) / 2
TNS_avg = (TNS_SA + TNS_RP) / 2
Area_avg = (Area_SA + Area_RP) / 2
For each design:
R_WNS = WNS_avg / WNS_sub (both negative; less-negative numerator is the baseline, so ratio > 1 means submission is better)
R_TNS = TNS_avg / TNS_sub (both negative; ratio > 1 means submission is better)
R_Area = Area_avg / Area_sub (lower area is better, so avg over sub; ratio > 1 means submission is smaller)
Using a weighted geometric mean with weights (WNS:3, TNS:2, Area:1):
Design_Score = (R_WNS^3 × R_TNS^2 × R_Area^1) ^ (1/6)
- Score > 1.0 → better than average baseline
- Score = 1.0 → matched baseline exactly
- Score < 1.0 → worse than average baseline
WNS is the single most important objective — a chip that fails its worst-case path fails, regardless of average behavior or area. TNS captures the breadth of timing violations and is next most important. Area matters but is secondary to timing closure at this stage of the flow. The 3:2:1 ratio reflects this ordering without zeroing out any metric.
In log terms, a 1% improvement in WNS contributes 3× as much to the score as a 1% improvement in Area, and 1.5× as much as a 1% improvement in TNS.
Total_Score = geometric mean of Design_Scores across all evaluated designs
= (Design_Score_1 × Design_Score_2 × ... × Design_Score_N) ^ (1/N)
Highest Total Score wins the Grand Prize.
Suppose for ariane133:
- SA baseline: WNS = -1.4 ns, TNS = -11,000 ns, Area = 4,300,000 μm²
- RePlAce baseline: WNS = -1.2 ns, TNS = -9,000 ns, Area = 4,100,000 μm²
- Average baseline: WNS = -1.3 ns, TNS = -10,000 ns, Area = 4,200,000 μm²
- Submission: WNS = -0.8 ns, TNS = -6,000 ns, Area = 4,000,000 μm²
R_WNS = -1.3 / -0.8 = 1.625 (62.5% better WNS)
R_TNS = -10000 / -6000 = 1.667 (66.7% better TNS)
R_Area = 4200000 / 4000000 = 1.050 (5% smaller area)
Design_Score = (1.625^3 × 1.667^2 × 1.05^1) ^ (1/6)
= (4.291 × 2.779 × 1.05) ^ (1/6)
= (12.521) ^ (1/6)
= 1.524
This submission scores 1.524 on ariane133 — roughly 52% better than the average baseline, with WNS and TNS gains emphasized by the (3,2,1) weighting. Under equal (1,1,1) weighting the same submission would have scored 1.417.
If two submissions have identical Total Scores:
- Higher geometric mean of R_TNS across designs
- Higher geometric mean of R_WNS across designs
- Higher geometric mean of R_Area across designs
- Submission achieves WNS = 0 and TNS = 0: The clock period is tightened (frequency raised) and the design is re-evaluated, iteratively, until WNS < 0 reappears. Ranking then uses the WNS/TNS measured at that tighter clock. This ensures that submissions which close timing at the nominal clock are still differentiable on how much frequency headroom they provide, rather than being capped at a ceiling.
- The tightening schedule is fixed in advance (e.g., reduce period by 5% each step, up to a bounded number of steps) and applied identically to all submissions for that design.
- Baselines are re-evaluated at the tightened clock only if at least one baseline also reached zero-slack at the nominal clock; otherwise the original baseline numbers are used.
- WNS or TNS = 0.0 for both baselines (but submission has violations): That metric's ratio is excluded from the geometric mean for that design; the weights of the remaining metrics are renormalized so they still sum to 6.
Submissions may be disqualified if they:
- Intentionally degrade timing to game scoring
- Modify or bypass the evaluation flow
- Hardcode design-specific hacks targeting known benchmarks
- Evaluation uses a versioned OpenROAD flow; the exact version used for final scoring will be documented with the results
- The evaluation pipeline will be released alongside final results
- All submissions in a given evaluation round are run under the same conditions on the same hardware
To win the Grand Prize:
- Be in the top 7 by proxy cost (Tier 1)
- Pass feasibility — don't regress timing below both baselines on any design
- Maximize your weighted geometric mean score — prioritize WNS, then TNS, then Area (weights 3:2:1) across all designs