ci: add Copilot automation + security workflows (14 workflows) - #32
ci: add Copilot automation + security workflows (14 workflows)#32m0nk1111 wants to merge 1 commit into
Conversation
Adapted from m0nklabs/cryptotrader workflow suite: Copilot SWE Agent automation: - auto-approve-workflows: bypass first-time contributor approval - copilot-auto-assign: auto-assign Copilot to priority issues - copilot-branch-ci: lightweight lint CI for copilot/** branches - copilot-autofix: auto-fix style issues on CI failure - copilot-ci-feedback: notify Copilot of CI failures with fix requests - copilot-ready-pr: mark draft PRs ready after CI passes - copilot-rebase: keep Copilot branches up-to-date with master - copilot-request-review: request Copilot Reviewer after agent finishes - copilot-auto-review-llm: LLM triage of Copilot Reviewer feedback - copilot-review-feedback: forward review suggestions summary General automation: - automation-cleanup-pr-body: clean HTML comments and empty sections - automation-stale: mark/close stale issues and PRs Security: - security-codeql: Python CodeQL analysis - security-gitleaks: secret scanning with Gitleaks All workflows use [self-hosted, Linux] runners. Required secrets: GH_PAT, APP_ID, APP_PRIVATE_KEY
There was a problem hiding this comment.
Pull request overview
Adds a suite of GitHub Actions workflows to automate Copilot SWE agent lifecycle (assignment → lint CI → autofix → review routing) and introduce baseline security scans (CodeQL + Gitleaks), plus general repo hygiene automation.
Changes:
- Add Copilot automation workflows for CI feedback loops, review requesting/forwarding, rebase help, and auto-assignment.
- Add repo maintenance workflows (stale issue/PR handling, PR body cleanup).
- Add security scanning workflows (CodeQL, Gitleaks) on
masterpush/PR and schedules.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/auto-approve-workflows.yml | Scheduled dispatcher to re-trigger blocked Copilot-related workflows. |
| .github/workflows/automation-cleanup-pr-body.yml | Cleans PR bodies on open (removes HTML comments/empty sections). |
| .github/workflows/automation-stale.yml | Marks/closes stale issues/PRs on a schedule. |
| .github/workflows/copilot-auto-assign.yml | Auto-selects and assigns Copilot to eligible priority issues. |
| .github/workflows/copilot-auto-review-llm.yml | Uses local LLM + heuristics to triage Copilot Reviewer feedback and post fix requests. |
| .github/workflows/copilot-autofix.yml | Attempts to auto-fix ruff issues after branch CI failures and push changes. |
| .github/workflows/copilot-branch-ci.yml | Lightweight lint + syntax CI for copilot/** branches and PR feedback posting. |
| .github/workflows/copilot-ci-feedback.yml | Posts fix requests to Copilot when main CI fails (with loop protection). |
| .github/workflows/copilot-ready-pr.yml | Marks eligible Copilot draft PRs ready after CI success + minimum age. |
| .github/workflows/copilot-rebase.yml | Helper to update Copilot PR branches against master. |
| .github/workflows/copilot-request-review.yml | Requests Copilot Reviewer once Copilot signals work is finished. |
| .github/workflows/copilot-review-feedback.yml | Summarizes new Copilot review suggestions back onto the PR. |
| .github/workflows/security-codeql.yml | Runs CodeQL analysis for Python. |
| .github/workflows/security-gitleaks.yml | Runs secret scanning via Gitleaks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - name: Install Gitleaks | ||
| run: | | ||
| curl -sSfL https://github.com/gitleaks/gitleaks/releases/download/v8.21.2/gitleaks_8.21.2_linux_x64.tar.gz | tar -xz | ||
| sudo mv gitleaks /usr/local/bin/ | ||
|
|
||
| - name: Run Gitleaks (full scan) | ||
| run: gitleaks detect --source . --log-opts="--all" --verbose --exit-code 1 |
There was a problem hiding this comment.
Installing Gitleaks by piping curl output directly into tar and moving it with sudo is supply-chain risky and bypasses integrity verification. Prefer using the official gitleaks action or download the release and verify its checksum/signature before executing, and avoid requiring sudo on the runner.
| - name: Install Gitleaks | |
| run: | | |
| curl -sSfL https://github.com/gitleaks/gitleaks/releases/download/v8.21.2/gitleaks_8.21.2_linux_x64.tar.gz | tar -xz | |
| sudo mv gitleaks /usr/local/bin/ | |
| - name: Run Gitleaks (full scan) | |
| run: gitleaks detect --source . --log-opts="--all" --verbose --exit-code 1 | |
| - name: Run Gitleaks (full scan) | |
| uses: gitleaks/gitleaks-action@v2 | |
| with: | |
| args: detect --source . --log-opts="--all" --verbose --exit-code 1 |
|
|
||
| jobs: | ||
| cleanup: | ||
| runs-on: [self-hosted, Linux] |
There was a problem hiding this comment.
This workflow triggers on pull_request:opened but runs on a self-hosted runner. Even though it doesn’t checkout code, it allows any PR opener (including forks) to consume self-hosted runner capacity. Consider moving this job to a GitHub-hosted runner unless there is a specific need for self-hosted.
| runs-on: [self-hosted, Linux] | |
| runs-on: ubuntu-latest |
|
|
||
| jobs: | ||
| stale: | ||
| runs-on: [self-hosted, Linux] |
There was a problem hiding this comment.
This scheduled maintenance workflow doesn’t appear to require self-hosted resources, but it will occupy the self-hosted runner daily. Consider running it on a GitHub-hosted runner to reduce contention and avoid coupling repo hygiene automation to the availability of your self-hosted machine.
| runs-on: [self-hosted, Linux] | |
| runs-on: ubuntu-latest |
| runs-on: [self-hosted, Linux] | ||
| permissions: | ||
| actions: read | ||
| contents: read | ||
| security-events: write |
There was a problem hiding this comment.
This workflow runs on pull_request but uses a self-hosted runner. CodeQL analysis may execute parts of the project during extraction/autobuild, which is unsafe for untrusted PRs on a self-hosted machine. Consider running CodeQL on a GitHub-hosted runner instead (or restrict execution to trusted branches/events).
| if (!hasCopilotLabel && !(hasPriorityLabel && issue.assignees.length === 0)) { | ||
| return false; | ||
| } |
There was a problem hiding this comment.
Candidates can include issues that already have non-Copilot assignees when the issue has the "copilot" label (there’s no check that issue.assignees is empty in that case). This matters because the assignment step later replaces assignees. Consider requiring issue.assignees.length === 0 for all candidates you intend to auto-assign.
| if (!hasCopilotLabel && !(hasPriorityLabel && issue.assignees.length === 0)) { | |
| return false; | |
| } | |
| const hasNoAssignees = issue.assignees.length === 0; | |
| if (!hasNoAssignees) { | |
| return false; | |
| } | |
| if (!hasCopilotLabel && !hasPriorityLabel) { | |
| return false; | |
| } |
| run: | | ||
| if git diff --quiet; then | ||
| echo "No style fixes needed - failure was from something else" | ||
| exit 1 |
There was a problem hiding this comment.
If no style changes are produced, this exits with status 1, which makes the autofix workflow fail even though it successfully determined there was nothing to fix. Consider exiting 0 (and logging that the failure was non-style) to avoid persistent red workflow runs/noise.
| exit 1 | |
| exit 0 |
| MUTATION_JSON=$(printf '{"query":"mutation { replaceActorsForAssignable(input: {assignableId: \\"%s\\", actorIds: [\\"%s\\"]}) { assignable { ... on Issue { assignees(first: 10) { nodes { login } } } } } }"}' "$ISSUE_ID" "$COPILOT_BOT_ID") | ||
| RESULT=$(echo "$MUTATION_JSON" | gh api graphql --input - --jq '.data.replaceActorsForAssignable.assignable.assignees.nodes[].login') |
There was a problem hiding this comment.
replaceActorsForAssignable will overwrite the entire assignee list with only the Copilot bot. If the issue had existing assignees, they will be removed. Prefer an API/mutation that adds Copilot as an additional assignee, or ensure the issue has zero assignees before calling this mutation.
| MUTATION_JSON=$(printf '{"query":"mutation { replaceActorsForAssignable(input: {assignableId: \\"%s\\", actorIds: [\\"%s\\"]}) { assignable { ... on Issue { assignees(first: 10) { nodes { login } } } } } }"}' "$ISSUE_ID" "$COPILOT_BOT_ID") | |
| RESULT=$(echo "$MUTATION_JSON" | gh api graphql --input - --jq '.data.replaceActorsForAssignable.assignable.assignees.nodes[].login') | |
| MUTATION_JSON=$(printf '{"query":"mutation { addAssigneesToAssignable(input: {assignableId: \\"%s\\", assigneeIds: [\\"%s\\"]}) { assignable { ... on Issue { assignees(first: 10) { nodes { login } } } } } }"}' "$ISSUE_ID" "$COPILOT_BOT_ID") | |
| RESULT=$(echo "$MUTATION_JSON" | gh api graphql --input - --jq '.data.addAssigneesToAssignable.assignable.assignees.nodes[].login') |
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| token: ${{ secrets.GH_PAT }} | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Configure git | ||
| run: | | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "github-actions[bot]@users.noreply.github.com" | ||
|
|
There was a problem hiding this comment.
These checkout/fetch steps don’t appear to be used since the workflow rebases via the GitHub API (pulls.updateBranch) and doesn’t run local git commands. Consider removing checkout (and git config) to reduce runtime and avoid using a write token unnecessarily.
| - name: Checkout | |
| uses: actions/checkout@v4 | |
| with: | |
| token: ${{ secrets.GH_PAT }} | |
| fetch-depth: 0 | |
| - name: Configure git | |
| run: | | |
| git config user.name "github-actions[bot]" | |
| git config user.email "github-actions[bot]@users.noreply.github.com" |
|
|
||
| MARKER="<!-- copilot-branch-ci:sha=$SHA -->" | ||
|
|
||
| if gh api "repos/$REPO/pulls/$PR_NUMBER/reviews?per_page=20" --jq '.[].body // ""' | grep -Fq "$MARKER"; then |
There was a problem hiding this comment.
The duplicate-prevention check only fetches the last 20 PR reviews (per_page=20). On branches with more review comments over time, the marker could fall out of that window and the workflow may post duplicate reviews. Consider searching issue comments instead, or paginate/search all review bodies for the marker.
| if gh api "repos/$REPO/pulls/$PR_NUMBER/reviews?per_page=20" --jq '.[].body // ""' | grep -Fq "$MARKER"; then | |
| FOUND_MARKER=0 | |
| PAGE=1 | |
| while :; do | |
| BODIES="$(gh api "repos/$REPO/pulls/$PR_NUMBER/reviews?per_page=100&page=$PAGE" --jq '.[].body // ""')" | |
| # Stop if no more reviews are returned | |
| if [ -z "$BODIES" ]; then | |
| break | |
| fi | |
| if echo "$BODIES" | grep -Fq "$MARKER"; then | |
| FOUND_MARKER=1 | |
| break | |
| fi | |
| PAGE=$((PAGE + 1)) | |
| done | |
| if [ "$FOUND_MARKER" -eq 1 ]; then |
| COPILOT_BOT_ID=$(gh api graphql -f query=' | ||
| query($owner: String!, $name: String!) { | ||
| repository(owner: $owner, name: $name) { | ||
| suggestedActors(first: 100, capabilities: CAN_BE_ASSIGNED) { | ||
| nodes { |
There was a problem hiding this comment.
This step depends on the gh CLI being present on the self-hosted runner. To avoid runner-specific failures, explicitly install gh (and any JSON tooling you rely on) in the workflow or guarantee it in runner provisioning.
Copilot Automation + Security Workflows
Adapted from
m0nklabs/cryptotraderworkflow suite for the Mycodo fork.What's included (14 workflows)
Copilot SWE Agent Automation (10)
auto-approve-workflowscopilot-auto-assigncopilot-branch-cicopilot/**branches (ruff + syntax check)copilot-autofixcopilot-ci-feedbackcopilot-ready-prcopilot-rebasecopilot-request-reviewcopilot-auto-review-llmcopilot-review-feedbackGeneral Automation (2)
automation-cleanup-pr-bodyautomation-staleSecurity (2)
security-codeqlsecurity-gitleaksWhat's NOT included (and why)
ci.yml—main.ymlalready handles full Mycodo CIautomation-release.yml— fork doesn't do releasesautomation-label-issues.yml— would need full label remap for Mycodo domaincustom-agent.yml— requiresscripts/custom_agent.pywhich doesn't exist yetAdaptations from cryptotrader
[self-hosted, Linux](noorglabel since this is a personal repo)"CI"→"Mycodo"(matchesmain.ymlname)run-headercomposite action references (doesn't exist in this repo)BITFINEX_*,OPENROUTER_*)@m0nk111(not@m0nk1111)Required Secrets
GH_PATRequired Setup
m0nk111/Mycodo(Settings → Actions → Runners → New self-hosted runner)self-hosted,Linux192.168.1.26)GH_PATsecret (Settings → Secrets → Actions → New repository secret)Copilot Agent Loop (how it works)