Skip to content

templates.md

Latest commit

 

History

History
257 lines (186 loc) · 7.21 KB

templates.md

File metadata and controls

257 lines (186 loc) · 7.21 KB
Raw

Templates

BenchBox templates bundle predefined chart sets for common content workflows. Instead of manually selecting chart types, choose a template that matches your content goal.

Template Overview

Template Charts Generated Best For
flagship up to 11 charts Multi-platform comparisons, comprehensive analysis
head_to_head up to 7 charts Two-platform shootouts
trends up to 3 charts Performance evolution over time
cost_optimization 2 charts ROI and cost analysis
comparison 3 charts Side-by-side run comparison with change annotations
latency_deep_dive 5 charts Deep latency distribution and tail analysis
regression_triage up to 5 charts Two-run regression investigation
executive_summary up to 5 charts High-level aggregate reporting
default up to 6 charts Baseline single-run summaries

Flagship Template

Use case: Comprehensive multi-platform comparisons for detailed analysis.

Generates:

  1. Performance Bar Chart (platform comparison)
  2. Power@Size Bar Chart (TPC throughput score, if available)
  3. Query Variance Heatmap (per-query breakdown)
  4. Query Latency Histogram
  5. Cost-Performance Scatter (if cost data available)
  6. Distribution Box Plot (latency consistency)
  7. Percentile Ladder (P50/P90/P95/P99)
  8. Normalized Speedup (log₂-scaled baseline comparison)
  9. Sparkline Table (compact metric overview)
  10. CDF Chart (cumulative latency distribution)
  11. Rank Table (per-query platform rankings)

See Chart Types for examples of each chart generated by this template.

Command:

benchbox visualize results/*.json --template flagship

Ideal for:

  • "State of the Data Warehouse 2025" style analysis
  • 4+ platform comparison
  • Detailed benchmark reports

Head-to-Head Template

Use case: Direct two-platform comparisons for shootout-style analysis.

Generates:

  1. Performance Bar Chart (side-by-side comparison)
  2. Power@Size Bar Chart (TPC throughput score, if available)
  3. Distribution Box Plot (consistency comparison)
  4. Query Latency Histogram
  5. Query Variance Heatmap (per-query winners)
  6. Normalized Speedup (log₂-scaled baseline comparison)
  7. Rank Table (per-query platform rankings)

See Chart Types for examples of each chart generated by this template.

Command:

benchbox visualize duckdb-results.json snowflake-results.json --template head_to_head

Ideal for:

  • "Platform A vs Platform B" comparisons
  • Migration decision support
  • Quick competitive analysis

Trends Template

Use case: Performance evolution over time for tracking improvements or detecting regressions.

Generates:

  1. Time-Series Line Chart
  2. Power@Size Bar Chart (TPC throughput progression, if available)
  3. Performance Bar Chart (latest snapshot)

See Chart Types for examples of each chart generated by this template.

Command:

benchbox visualize runs/2024/*.json runs/2025/*.json --template trends

Ideal for:

  • Quarterly performance reports
  • CI/CD regression monitoring
  • Platform version upgrade impact analysis

Cost Optimization Template

Use case: Price/performance analysis for cloud platform cost optimization.

Generates:

  1. Cost-Performance Scatter Plot
  2. Performance Bar Chart (raw performance comparison)

See Chart Types for examples of each chart generated by this template.

Command:

benchbox visualize cloud-results/*.json --template cost_optimization

Ideal for:

  • Cloud cost comparison analysis
  • ROI assessments
  • Platform selection recommendations

Comparison Template

Use case: Side-by-side comparison of two runs with percentage change annotations.

Generates:

  1. Comparison Bar Chart (paired bars per query with % change)
  2. Diverging Bar Chart (percentage change centered on zero)
  3. Summary Box (key statistics)

Command:

benchbox visualize baseline.json current.json --template comparison

Ideal for:

  • Before/after optimization comparisons
  • Version-over-version analysis
  • Regression investigation

Default Template

Use case: Quick visualization of a single benchmark run without specifying chart types.

Generates (varies by data):

  • Performance Bar Chart (always)
  • Distribution Box Plot (if query timings available)
  • Query Variance Heatmap (if 2+ platforms)

Command:

benchbox visualize  # Uses latest result automatically

Ideal for:

  • Quick result inspection
  • Development/debugging
  • Single-platform analysis

Latency Deep Dive Template

Use case: Deep analysis of latency distributions, percentile tails, and per-query hotspots.

Generates:

  1. Distribution Box Plot (quartiles and outliers)
  2. Query Latency Histogram (per-query bars)
  3. Percentile Ladder (P50/P90/P95/P99 across platforms)
  4. CDF Chart (cumulative latency distribution)
  5. Query Variance Heatmap

See Chart Types for examples of each chart generated by this template.

Command:

benchbox visualize results/*.json --template latency_deep_dive

Ideal for:

  • SLA compliance analysis (P95/P99 tail latency)
  • Identifying high-variance queries
  • Consistency investigations

Regression Triage Template

Use case: Quickly pinpoint and prioritize regressions after a version upgrade or code change.

Generates:

  1. Power@Size Bar Chart (headline throughput change, if available)
  2. Comparison Bar Chart (per-query paired bars with % change)
  3. Diverging Bar Chart (regression/improvement distribution)
  4. Summary Box (aggregate impact statistics)
  5. Rank Table (per-query platform rankings)

See Chart Types for examples of each chart generated by this template.

Command:

benchbox visualize baseline.json new_version.json --template regression_triage

Ideal for:

  • Post-upgrade regression investigation
  • PR performance review
  • Identifying the worst-impacted queries first

Executive Summary Template

Use case: Compact, high-level performance overview suitable for reports and stakeholder communication.

Generates:

  1. Performance Bar Chart (total time comparison)
  2. Power@Size Bar Chart (TPC throughput score, if available)
  3. Normalized Speedup (relative performance vs baseline)
  4. Sparkline Table (compact metric overview across platforms)
  5. Summary Box (key aggregate statistics)

See Chart Types for examples of each chart generated by this template.

Command:

benchbox visualize results/*.json --template executive_summary

Ideal for:

  • Blog post and report supporting charts
  • Stakeholder performance summaries
  • TPC benchmark result announcements

Template Customization

Templates set defaults but can be overridden:

# Use flagship template with dark theme
benchbox visualize results/*.json --template flagship --theme dark

# Use head_to_head template with no colors (for piping)
benchbox visualize a.json b.json --template head_to_head --no-color > comparison.txt

# Use trends template with ASCII-only characters
benchbox visualize results/*.json --template trends --no-unicode