Collect community CPU/GPU benchmark results

[initial-d]

Community benchmark collection

We are collecting reproducible CPU/GPU benchmark results for the tensor factor engine. Slow or negative results are useful too; the goal is to understand machine and runtime differences, not to cherry-pick the fastest number.

Canonical CPU run: protocol v1

From the repository root, on Windows, macOS, or Linux:

python scripts/benchmark_tensor_factors.py --json-out artifacts/benchmark-v1.json

If GNU Make is installed, make benchmark is an optional shortcut for the same protocol.

Protocol v1 fixes:

• 750 dates × 1,000 stocks;
• 20-day rolling window;
• seed 42;
• 3 warmup runs and 10 measured repetitions;
• one PyTorch intra-op thread and one inter-op thread.

The console output and JSON report include the protocol, Python, PyTorch, platform, CPU, logical CPU count, thread counts, CUDA availability, panel shape, window, warmup/repeat count, seed, and per-case mean/std timings.

Useful variants

After the canonical CPU run, variants are welcome:

python scripts/benchmark_tensor_factors.py --device cuda --json-out artifacts/benchmark-cuda.json
python scripts/benchmark_tensor_factors.py --device cpu --n-dates 1500 --n-stocks 3000 --json-out artifacts/benchmark-large.json

Please label GPU, larger-panel, and multi-thread runs separately from protocol v1.

What to submit

1. Commit SHA.
2. Exact command.
3. The generated JSON report as an attachment.
4. Complete console environment and result tables.
5. CPU/GPU model, CUDA details where relevant, and any thermal, memory, or workload notes.

See the benchmarking guide, benchmark board, and benchmark result form.

[initial-d]

A first maintainer CPU baseline is now on the benchmark board:

https://github.com/initial-d/ml-quant-trading/blob/main/docs/benchmark_board.md

Next useful reports: CUDA GPU, Apple Silicon MPS if supported in future benchmark variants, Linux CPU, and larger panels.

[initial-d]

Post-v0.2.0 benchmark ask:

The most useful next reports are still external-machine runs, especially:

1. CUDA GPU benchmark

python scripts/benchmark_tensor_factors.py --device auto

2. Linux CPU benchmark on a common cloud or workstation environment

python scripts/benchmark_tensor_factors.py --device cpu

3. Larger panel stress test, if memory allows

python scripts/benchmark_tensor_factors.py --n-stocks 1000 --n-days 1000 --device auto

Please include:

• commit SHA;
• OS and CPU model;
• Python and PyTorch versions;
• GPU/CUDA availability if applicable;
• exact command;
• full Markdown table output;
• any memory/runtime warnings.

Negative or slow results are still useful if the environment is documented. The goal is to build a realistic benchmark board across machines, not to cherry-pick the fastest result.

[initial-d]

Maintainer update — August reproduction challenge is open.

I added a structured reproduction-report issue template so CPU/GPU benchmark and validation reports are easier to compare across machines and data sources:

https://github.com/initial-d/ml-quant-trading/issues/new?template=reproduction_report.yml

Most useful reports right now:

• CPU/GPU benchmark runs on a new machine
• public-data validation with a different A-share universe or date range
• failure reports where data coverage, provider availability, or transaction-cost assumptions break the experiment
• comparison notes against Qlib / vectorbt / custom factor research stacks

Please include the command, environment, data source, cost assumptions, and known limitations. A clean No-Go or data-source failure report is useful too.

[initial-d]

Maintainer update — the benchmark board now includes a second reproducible CPU snapshot:

• Windows 11, Intel Core i7-1255U, Python 3.14.4, PyTorch 2.11.0+cpu
• Synthetic panel: 750 dates × 1,000 stocks
• Exact command and raw timing table: https://github.com/initial-d/ml-quant-trading/blob/main/docs/benchmark_board.md

This is intentionally a machine snapshot, not a controlled ranking against the Apple M5 result. The highest-value next report is still a CUDA GPU run or a larger-panel stress test with the full environment and command recorded. Slow or negative results are welcome through the benchmark result form.

[initial-d]

Benchmark protocol v1 is live

The repository now has a fixed CPU benchmark protocol so reports from different machines are easier to compare.

Run the canonical command from the repository root:

make benchmark

Protocol v1 fixes a 750 × 1,000 synthetic panel, 20-day window, seed 42, 3 warmups, 10 measured repetitions, and one PyTorch intra-op plus one inter-op thread. GPU, larger-panel, and multi-thread runs are welcome, but please label them as separate variants.

The first comparable maintainer baseline is an Intel Core i7-1255U on Windows 11: compute_legacy_set(6 factors) measured 551.0 ms and ts_corr(close,returns,20) measured 152.3 ms.

Please paste the complete output table, environment, commit SHA, and any thermal or workload notes. The issue template and benchmark board have been updated for this protocol. Successful and unusual results are both useful.