Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/topic06-benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Topic 06 Compiler Benchmark

on:
push:
pull_request:

jobs:
topic06-benchmark:
runs-on: ubuntu-latest
timeout-minutes: 15

steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install ScratchV and benchmark dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -e ".[topic06]"

- name: Verify Topic 06 integration contracts
run: python -m pytest tests/topic06/test_integration.py -q

- name: Run supported correctness gates
run: |
python scripts/run_topic06_benchmarks.py --category activation --fail-on-test-failure
python scripts/run_topic06_benchmarks.py --category elementwise --fail-on-test-failure
python scripts/run_topic06_benchmarks.py --category loop --fail-on-test-failure

- name: Generate full diagnostic benchmark report
run: python scripts/run_topic06_benchmarks.py --benchmark 3

- name: Upload Topic 06 reports
if: always()
uses: actions/upload-artifact@v4
with:
name: topic06-benchmark-reports
path: |
benchmark_reports/topic06/report.md
benchmark_reports/topic06/report.json
benchmark_reports/topic06/failures/
benchmarks/topic06/baseline.json
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ScratchV developer makefile
.POSIX:

.PHONY: quick-start install test bench bench-cnn clean lint
.PHONY: quick-start install test bench bench-topic06 bench-cnn clean lint

# ── Beginner quick-start ─────────────────────────────────────────────────────

Expand Down Expand Up @@ -48,6 +48,11 @@ bench:
--output-json benchmark_reports/dsl_bench.json \
--output-html benchmark_reports/dsl_bench.html

# ── Topic 06 DSL correctness + TinyFive benchmark ─────────────────────────

bench-topic06:
python3 scripts/run_topic06_benchmarks.py --benchmark 3

# ── CNN RISC-V 编译 + 估算 ────────────────────────────────────────────────

bench-cnn:
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
```bash
make quick-start # 打印新手引导
make test # 运行全部测试
make bench-topic06 # 运行课题 06 TinyFive 性能测试套件
make bench-cnn # 编译 CNN 模型 + 性能估算
make bench-ci # 完整 CI 对比 (ScratchV vs LLVM)
make bench-reports # 生成 Dashboard + 优化历史
Expand All @@ -62,9 +63,9 @@ ScratchV/
│ ├── 00~04-*.md ← 新手入门 5 篇
│ ├── topics/ ← 30 个模块详解
│ └── topics/html/ ← 🌐 交互式课程站点
├── benchmarks/ ← 23 个 DSL 基准用例
├── tests/ ← 348 个单元测试
├── scripts/ ← 工具脚本
├── benchmarks/ ← 通用基准与课题 06 性能基线
├── tests/ ← 单元测试及课题 06 的 23 个 DSL 用例
├── scripts/ ← 工具脚本及课题 06 测试驱动
└── models/ ← 测试用 ONNX 模型
```

Expand Down Expand Up @@ -104,6 +105,7 @@ ONNX 模型 (.onnx)
| [📖 文档导航](docs/INDEX.md) | 全部 Markdown 文档索引 |
| [🏗️ 架构总览](docs/ARCHITECTURE.md) | ONNX→RISC-V 双路径详解 |
| [📊 性能仪表盘](https://scratchv-compiler.github.io/ScratchV/dashboard.html) | LLVM vs ScratchV 对比 |
| [课题 06 测试套件](docs/topics/06-性能测试套件使用说明.md) | TinyFive 正确性验证、Benchmark 与回归报告 |
| [📢 项目海报](https://scratchv-compiler.github.io/ScratchV/ScratchV.html) | 招募信息 + 3个月学习路线 + 课题精选 |

---
Expand Down
102 changes: 102 additions & 0 deletions benchmarks/topic06/baseline.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
{
"add_relu_relu": {
"category": "activation",
"avg_instr_count": 7.0,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个baseline是怎么测出来的

"runs": 3
},
"relu_add": {
"category": "activation",
"avg_instr_count": 5.0,
"runs": 3
},
"relu_only": {
"category": "activation",
"avg_instr_count": 5.0,
"runs": 3
},
"relu_twice": {
"category": "activation",
"avg_instr_count": 6.0,
"runs": 3
},
"add_chain": {
"category": "elementwise",
"avg_instr_count": 4.0,
"runs": 3
},
"add_chain_3": {
"category": "elementwise",
"avg_instr_count": 5.0,
"runs": 3
},
"add_fan_in_4": {
"category": "elementwise",
"avg_instr_count": 5.0,
"runs": 3
},
"add_reuse": {
"category": "elementwise",
"avg_instr_count": 4.0,
"runs": 3
},
"vector_add": {
"category": "elementwise",
"avg_instr_count": 3.0,
"runs": 3
},
"loop_add_4": {
"category": "loop",
"avg_instr_count": 22.0,
"runs": 3
},
"loop_add_chain_4": {
"category": "loop",
"avg_instr_count": 26.0,
"runs": 3
},
"loop_relu_add_4": {
"category": "loop",
"avg_instr_count": 30.0,
"runs": 3
},
"dot_4": {
"category": "reduction",
"avg_instr_count": 3.0,
"runs": 3
},
"dot_8": {
"category": "reduction",
"avg_instr_count": 3.0,
"runs": 3
},
"dot_relu_4": {
"category": "reduction",
"avg_instr_count": 5.0,
"runs": 3
},
"dot_relu_8": {
"category": "reduction",
"avg_instr_count": 5.0,
"runs": 3
},
"matmul_2x2": {
"category": "tensor",
"avg_instr_count": 3.0,
"runs": 3
},
"matmul_4x4": {
"category": "tensor",
"avg_instr_count": 3.0,
"runs": 3
},
"matmul_add_2x2": {
"category": "tensor",
"avg_instr_count": 4.0,
"runs": 3
},
"matmul_relu_2x2": {
"category": "tensor",
"avg_instr_count": 5.0,
"runs": 3
}
}
Loading
Loading