Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
## <!-- PR Title: brief descriptive title -->

**<!-- One-sentence bold summary of the purpose and scope of this PR -->**

### Changes

- **New samples:** <!-- List newly added samples, or delete this line if none -->
- **Updated samples:** <!-- List modified samples, or delete this line if none -->
- **Updated kits:** <!-- List updated kits/libraries (e.g., ATGTK, OpenSource/imgui), or delete this line if none -->
- **General:** <!-- Other notable changes: config updates, build changes, docs, removals -->

### Additional Notes

<!-- Optional: any extra context, breaking changes, migration notes, or testing instructions -->
53 changes: 53 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Copilot Instructions for PR Summaries

When generating a pull request summary for this repository, follow the format used
in our release notes. The summary should help reviewers quickly understand what
changed, which samples were affected, and any infrastructure/kit updates.

## Required Format

Use this structure for every PR summary:

```
## <Brief descriptive title of the change>

**<One-sentence bold summary>** explaining the purpose and scope of the PR.

### Changes
- **New samples:** <list any newly added samples, or omit this line if none>
- **Updated samples:** <list samples that were modified, or omit if none>
- **Updated kits:** <list any kit/library updates (e.g., ATGTK, OpenSource/imgui), or omit if none>
- **General:** <any other notable changes — config updates, build changes, docs, removals>
```

## Formatting Rules

- Use an H2 (`##`) heading for the PR title/theme
- The first line after the heading must be a **bold summary sentence**
- Use a bulleted list under a `### Changes` subheading
- Each bullet should have a **bold category prefix** followed by a colon
- Sample and kit names should be wrapped in **bold**
- Omit category lines that have no items (do not include empty categories)
- Keep the summary concise — aim for clarity over length

## Category Detection

Determine categories by inspecting the diff:
- Files under `Samples/` that are newly added → **New samples**
- Files under `Samples/` that are modified → **Updated samples**
- Files under `Kits/` that are modified → **Updated kits** (ATGTK or OpenSource/imgui)
- Changes to build files (`.vcxproj`, `.sln`), configs, docs, `Media/`, or vcpkg manifests → **General**

## Examples

Good summary:
```
## ARM64 Build Target Support

**Added ARM64 (Desktop) build configurations across all samples** to enable building and running on ARM64-based Windows PCs.

### Changes
- **Updated samples:** All samples (ARM64 platform targets added to project files)
- **Updated kits:** ATGTK
- **General:** Updated `.sln` solution files with ARM64 platform entries, updated vcpkg configuration
```
127 changes: 127 additions & 0 deletions .github/scripts/eval-pr-summary.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
// PR Summary Format Evaluator
// Scores a PR body against the expected release-note format criteria.

const CRITERIA = [
{
id: 'h2_heading',
name: 'H2 Heading',
description: 'PR body starts with an H2 (##) heading',
weight: 20,
test: (body) => /^##\s+\S/m.test(body),
},
{
id: 'bold_summary',
name: 'Bold Summary',
description: 'Contains a bold (**...**) summary sentence after the heading',
weight: 25,
test: (body) => /^##\s+.+\n+\*\*.+\*\*/m.test(body),
},
{
id: 'changes_section',
name: 'Changes Section',
description: 'Contains a "### Changes" subheading',
weight: 15,
test: (body) => /^###\s+Changes/mi.test(body),
},
{
id: 'bullet_list',
name: 'Bullet List',
description: 'Contains at least one bullet point (- or *)',
weight: 15,
test: (body) => /^[\-\*]\s+/m.test(body),
},
{
id: 'bold_category',
name: 'Bold Category Prefix',
description: 'At least one bullet has a bold category prefix (e.g., **Updated samples:**)',
weight: 15,
test: (body) => /^[\-\*]\s+\*\*.+?\*\*:?/m.test(body),
},
{
id: 'recognized_category',
name: 'Recognized Category',
description: 'Uses at least one recognized category (New samples, Updated samples, Updated kits, General)',
weight: 10,
test: (body) => {
const categories = ['new samples', 'updated samples', 'updated kits', 'general'];
const lower = body.toLowerCase();
return categories.some((cat) => lower.includes(cat));
},
},
];

function evaluate(prBody) {
if (!prBody || prBody.trim().length === 0) {
return {
score: 0,
maxScore: 100,
pass: false,
results: CRITERIA.map((c) => ({
...c,
passed: false,
earned: 0,
})),
feedback: 'PR body is empty. Please add a summary following the release-note format.',
};
}

const results = CRITERIA.map((criterion) => {
const passed = criterion.test(prBody);
return {
id: criterion.id,
name: criterion.name,
description: criterion.description,
weight: criterion.weight,
passed,
earned: passed ? criterion.weight : 0,
};
});

const score = results.reduce((sum, r) => sum + r.earned, 0);
const maxScore = results.reduce((sum, r) => sum + r.weight, 0);
const pass = score >= 60;

const missing = results.filter((r) => !r.passed);
let feedback = '';
if (pass && missing.length === 0) {
feedback = 'PR summary follows the expected format.';
} else if (pass) {
feedback =
'PR summary mostly follows the format. Consider adding: ' +
missing.map((r) => r.name).join(', ') +
'.';
} else {
feedback =
'PR summary does not meet the format requirements. Missing: ' +
missing.map((r) => r.name).join(', ') +
'. See .github/copilot-instructions.md for the expected format.';
}

return { score, maxScore, pass, results, feedback };
}

// Main: read PR body from environment or stdin
async function main() {
const prBody = process.env.PR_BODY || '';
const result = evaluate(prBody);

console.log(JSON.stringify(result, null, 2));

// Output for GitHub Actions
const githubOutput = process.env.GITHUB_OUTPUT;
if (githubOutput) {
const fs = require('fs');
fs.appendFileSync(githubOutput, `score=${result.score}\n`);
fs.appendFileSync(githubOutput, `max_score=${result.maxScore}\n`);
fs.appendFileSync(githubOutput, `pass=${result.pass}\n`);
fs.appendFileSync(githubOutput, `feedback=${result.feedback}\n`);
}

process.exit(result.pass ? 0 : 1);
}

module.exports = { evaluate, CRITERIA };

if (require.main === module) {
main();
}
101 changes: 101 additions & 0 deletions .github/workflows/pr-summary-eval-reusable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
name: Evaluate PR Summary

on:
workflow_call:
inputs:
pr-number:
required: true
type: number

permissions:
pull-requests: write

jobs:
eval:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'

- name: Fetch current PR body
uses: actions/github-script@v7
with:
script: |
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: ${{ inputs.pr-number }},
});
const fs = require('fs');
fs.writeFileSync('/tmp/pr-body.txt', pr.body || '');

- name: Evaluate PR summary format
run: |
set +e
export PR_BODY="$(cat /tmp/pr-body.txt)"
node .github/scripts/eval-pr-summary.js > eval-result.json 2>&1
echo "Eval exit code: $?"

- name: Post evaluation comment
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
let result;
try {
result = JSON.parse(fs.readFileSync('eval-result.json', 'utf8'));
} catch (e) {
result = { score: 0, maxScore: 100, pass: false, feedback: 'Could not parse eval results.', results: [] };
}

const icon = result.pass ? '✅' : '⚠️';
const status = result.pass ? 'PASS' : 'NEEDS IMPROVEMENT';

let body = `## ${icon} PR Summary Format Eval: ${status}\n\n`;
body += `**Score:** ${result.score}/${result.maxScore}\n\n`;
body += `${result.feedback}\n\n`;

if (result.results && result.results.length > 0) {
body += `### Criteria Breakdown\n\n`;
body += `| Criterion | Status | Points |\n`;
body += `|-----------|--------|--------|\n`;
for (const r of result.results) {
const checkmark = r.passed ? '✅' : '❌';
body += `| ${r.name} | ${checkmark} | ${r.earned}/${r.weight} |\n`;
}
}

body += `\n---\n*Format reference: [.github/copilot-instructions.md](.github/copilot-instructions.md)*`;

const comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: ${{ inputs.pr-number }},
});

const marker = '## ✅ PR Summary Format Eval';
const markerAlt = '## ⚠️ PR Summary Format Eval';
const existing = comments.data.find(c =>
c.body.startsWith(marker) || c.body.startsWith(markerAlt)
);

if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: ${{ inputs.pr-number }},
body,
});
}
15 changes: 15 additions & 0 deletions .github/workflows/pr-summary-eval.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: PR Summary Eval

on:
pull_request:
types: [opened, edited, synchronize]

jobs:
evaluate-summary:
# Skip the initial 'opened' event for bot PRs — the generate workflow will
# handle eval after updating the body.
if: >-
!(github.event.action == 'opened' && endsWith(github.event.pull_request.user.login, '[bot]'))
uses: ./.github/workflows/pr-summary-eval-reusable.yml
with:
pr-number: ${{ github.event.pull_request.number }}
Loading