Skip to content

feat(benchmarks): add async-profiler support to TPC benchmark scripts#3613

Merged
andygrove merged 3 commits intoapache:mainfrom
andygrove:async-profiler-support
Mar 3, 2026
Merged

feat(benchmarks): add async-profiler support to TPC benchmark scripts#3613
andygrove merged 3 commits intoapache:mainfrom
andygrove:async-profiler-support

Conversation

@andygrove
Copy link
Copy Markdown
Member

@andygrove andygrove commented Feb 28, 2026

Summary

  • Add --async-profiler flag to run.py for profiling both Java and native (Rust/C++) code, with configurable event type (cpu, wall, alloc, lock) and output format (flamegraph, jfr, collapsed, text)
  • Install async-profiler v3.0 in the Docker image with automatic architecture detection (x64/aarch64)
  • Add comprehensive documentation in README.md covering prerequisites, usage examples, and Docker setup
  • Fix minor bug where profiling output directories were created during --dry-run

Add --async-profiler flag to run.py for profiling both Java and native
(Rust/C++) code during benchmarks. Unlike JFR, async-profiler captures
unified flame graphs across the JVM/native boundary, which is especially
useful for Comet workloads.

- Add --async-profiler, --async-profiler-dir, --async-profiler-event,
  and --async-profiler-format CLI flags to run.py
- Install async-profiler v3.0 in the Docker image with arch detection
- Set ASYNC_PROFILER_HOME in both docker-compose files
- Add documentation section in README.md with usage examples
- Fix minor bug: skip profiling dir creation during --dry-run
@andygrove andygrove marked this pull request as ready for review February 28, 2026 18:57
Copy link
Copy Markdown
Contributor

@0lai0 0lai0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andygrove!

@andygrove
Copy link
Copy Markdown
Member Author

Example flamegraph generated with python3 run.py --engine comet --benchmark tpch --query 18 --async-profiler --async-profiler-format flamegraph --async-profiler-dir=/tmp on macOS.

executor.html

Copy link
Copy Markdown
Contributor

@mbutrovich mbutrovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @andygrove!

@andygrove andygrove merged commit 0cc1e8c into apache:main Mar 3, 2026
3 checks passed
@andygrove andygrove deleted the async-profiler-support branch March 3, 2026 15:37
@andygrove
Copy link
Copy Markdown
Member Author

Thanks @0lai0 and @mbutrovich

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants