Skip to content

feat(perf): compare frozen queries with Athena - #1154

Merged
bill-ph merged 4 commits into
mainfrom
codex/perf-test-athena
Sep 7, 2026
Merged

feat(perf): compare frozen queries with Athena#1154
bill-ph merged 4 commits into
mainfrom
codex/perf-test-athena

Conversation

@bill-ph

@bill-ph bill-ph commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add Athena on-demand as a third protocol in the frozen performance scenario so Duckgres raw Parquet, DuckLake via Duckgres/Trino, and Athena external-table latency are measured over the same immutable files.

The base infrastructure in PostHog/posthog-cloud-infra#10231 is applied. Merge and apply PostHog/posthog-cloud-infra#10273 before this PR: it publishes the Athena configuration to SSM and grants the existing CI role read access. No additional Athena GitHub secrets or variables are required.

Changes

  • add an AWS SDK Athena driver with result reuse disabled, full wall-time measurement, best-effort cancellation, pagination, output-root validation, and provider timing/scan metrics
  • preserve available terminal service metrics on failed/cancelled queries and result-retrieval or validation errors
  • extend paired catalogs with an Athena external-table relation and route each relation only to protocols that can query it
  • keep query_results.csv v1 unchanged and add query_service_metrics.csv for queue, planning, engine, scanned-byte, DPU, and engine-version data
  • give the frozen scenario a dedicated ServiceAccount and create/delete its EKS Pod Identity association per run
  • load Terraform-managed configuration from /duckgres/perf/athena after AWS authentication and before deployment; validate all four fields before exporting any environment settings
  • run one warmup and four measured Athena iterations alongside the existing Duckgres and Trino measurements
  • document setup, defaults, validation, and recovery

Testing

  • new loader and workflow tests were run red before implementation, then green
  • just test-scenario and just test-perf pass
  • bash -n scripts/scenario_athena_config.sh passes
  • golangci-lint run --new-from-rev=HEAD before committing: zero new issues
  • just lint was run; it reports six pre-existing SA4023 findings in main.go, cmd/duckgres-controlplane/main.go, and controlplane/control.go, outside this change

Rollout

Merge and apply PostHog/posthog-cloud-infra#10273 first, then merge this PR and manually run posthog_frozen_perf. The workflow reads the SSM configuration automatically; do not configure the former four Athena repository settings. Validate the first live Athena run against the other engines before relying on its comparisons.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

Test Impact Plan

Deterministic summary of how this PR changes tests, CI runners, and coverage-risk signals.

Summary

Area Added Changed Deleted
Test files 3 19 0
E2E/journey files 0 0 0
Workflow files 0 1 0

Signals

  • Test cases: +13 / -0
  • Assertions: +86 / -7
  • Skips or known failures added: 0
  • Workflow continue-on-error added: 0
  • Workflow path filters added: 0
  • Test commands removed from justfile: 0
  • E2E/journey retry lines added: 0

Coverage risk: neutral or increased

No coverage-reduction warnings detected.

@bill-ph
bill-ph merged commit 8a4c33d into main Sep 7, 2026
38 checks passed
@bill-ph
bill-ph deleted the codex/perf-test-athena branch September 7, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant