feat: Consolidate DevOps and CI/CD infrastructure with Docker CI, Python automation, and unified documentation#36
feat: Consolidate DevOps and CI/CD infrastructure with Docker CI, Python automation, and unified documentation#36danelkay93 with Copilot wants to merge 9 commits into
Conversation
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. Comment |
|
- Add Docker-based CI workflow (docker-compose.yml) - Add lock file validation script - Add Python branch manager for automation - Add branch management workflow - Add Pulumi IaC workflow - Create automation and infrastructure directories with documentation Co-authored-by: danelkay93 <24777308+danelkay93@users.noreply.github.com>
- Deprecate azure-staging-cleanup.yml in favor of branch-management.yml - Simplify ci.yml to lightweight quick checks - Create comprehensive DEVCONTAINER_AND_AUTOMATION.md as single source of truth - Remove redundant documentation files (CI_CD_GUIDE.md, CI_CD_QUICK_REFERENCE.md, IMPLEMENTATION_CHECKLIST.md) - Update docs/README.md to point to consolidated documentation Co-authored-by: danelkay93 <24777308+danelkay93@users.noreply.github.com>
- Fix multiline string issue in branch-management.yml - Format all workflow files with Prettier - Format all source files - Verify build and lint pass successfully Co-authored-by: danelkay93 <24777308+danelkay93@users.noreply.github.com>
Create detailed proposal for integrating Claude Code with PR #36's DevOps consolidation work, including options, conflict resolution strategy, and questions for multi-agent consensus. Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Create detailed proposal for integrating Claude Code with PR #36's DevOps consolidation work, including options, conflict resolution strategy, and questions for multi-agent consensus. Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Filter by date | ||
| cutoff = datetime.now() - timedelta(days=days) | ||
| recent_prs = [] | ||
| for pr in prs: | ||
| if pr.get('closedAt'): | ||
| closed_date = datetime.fromisoformat(pr['closedAt'].replace('Z', '+00:00')) | ||
| if closed_date > cutoff: | ||
| recent_prs.append(pr) |
There was a problem hiding this comment.
The comparison between closed_date and cutoff will raise TypeError: can't compare offset-naive and offset-aware datetimes the first time get_closed_prs executes. closed_date is timezone-aware because GitHub timestamps include Z and are parsed with fromisoformat(...+00:00), but cutoff = datetime.now() is naive. As a result the branch-cleanup workflow will crash before deleting any branches or inspecting staging environments. Consider generating the cutoff with datetime.now(timezone.utc) or stripping the timezone from closed_date so both operands are consistent.
Useful? React with 👍 / 👎.
| echo "Running branch manager..." | ||
| echo "Action: $ACTION" | ||
| echo "Dry run: ${{ inputs.dry_run || 'true' }}" | ||
|
|
||
| python3 automation/branch_manager.py \ | ||
| --owner "${{ github.repository_owner }}" \ | ||
| --repo "${{ github.event.repository.name }}" \ | ||
| --action "$ACTION" \ |
There was a problem hiding this comment.
The workflow passes --repo "${{ github.event.repository.name }}" to the Python script. github.event.repository is not populated for schedule or manual dispatch events, so this expression resolves to an empty string and all gh commands inside the script fail with “unknown repository”. Because the job is configured to run weekly via on.schedule, the automation will currently fail every time. Use github.repository (e.g. split on /) or default to the job’s repository when the event payload doesn’t include a repository object.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull Request Overview
This PR consolidates DevOps and CI/CD infrastructure by creating a comprehensive, production-ready system with Docker-based CI, Python automation, and unified documentation.
Key Changes
- Docker CI as primary pipeline: New
docker-compose.ymlworkflow integrates lock file validation, retry mechanisms, comprehensive checks, and security scanning - Python-based automation:
branch_manager.pyscript provides branch lifecycle management and Azure staging environment cleanup - Documentation consolidation: Merged 27.6KB of scattered docs into single 19KB
DEVCONTAINER_AND_AUTOMATION.mdguide
Reviewed Changes
Copilot reviewed 37 out of 40 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/docker-compose.yml |
New primary CI workflow with comprehensive validation and security scanning |
.github/workflows/branch-management.yml |
Orchestrates Python automation for branch and environment management |
.github/workflows/pulumi.yml |
Infrastructure as Code workflow ready for IaC implementation |
.github/workflows/ci.yml |
Simplified to lightweight quick checks (60% size reduction) |
.github/workflows/azure-staging-cleanup.yml |
Deprecated with migration notices to new branch management |
scripts/validate-lockfile.sh |
Lock file validation with JSON checks and sync verification |
scripts/setup-agent-environment.sh |
Multi-agent environment configuration with tool verification |
scripts/get_pr_reviews.py |
PR review access via GitHub CLI with exit code conventions |
scripts/agent-check.mjs |
Unified QA helper for ESLint, build, and test execution |
automation/branch_manager.py |
Comprehensive branch and staging environment management |
automation/README.md |
Automation scripts documentation and usage guide |
infrastructure/README.md |
IaC setup and deployment documentation |
DEVCONTAINER_AND_AUTOMATION.md |
Single source of truth for all DevOps documentation |
docs/README.md |
Updated to reference consolidated documentation |
docs/AGENT_TOOLKIT.md |
Quick command reference for multi-agent collaboration |
.vscode/settings.json |
VS Code settings for consistent formatting and linting |
.vscode/extensions.json |
Extended recommended extensions for multi-agent development |
.mcp/config.json |
GitHub MCP server configuration for enhanced agent capabilities |
.gitignore |
Updated with agent artifacts and Python cache patterns |
.gitattributes |
Git attributes for consistent line endings across agents |
.env.example |
Environment variable template for multi-agent and GitHub configuration |
.devcontainer/* |
Complete devcontainer setup with multi-agent support |
.github/AGENT_COLLABORATION.md |
Simplified multi-agent collaboration guidelines |
.github/ACCESSING_PR_REVIEWS.md |
Guide for programmatic PR review access |
.github/copilot-instructions.md |
Enhanced with MCP integration and QA helper documentation |
.claude/README.md |
Claude Code configuration referencing main documentation |
package.json |
Added qa and qa:with-typecheck scripts |
README.md |
Added multi-agent development section and consolidated guide references |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Remove unused imports (os, Optional) from branch_manager.py - Replace scripts/validate-lockfile.sh with validate-lockfile.py - Update workflows (ci.yml, docker-compose.yml) to use Python script - Update documentation to reference new Python script - Use only Python standard library (no external dependencies needed) Addresses review comments: - Comment 2532065461: Remove unused 'os' import - Comment 2532065466: Remove unused 'Optional' import - Comment 2532065426: Replace bash script with Python Co-authored-by: danelkay93 <24777308+danelkay93@users.noreply.github.com>
| # Binary files | ||
| *.png binary | ||
| *.jpg binary | ||
| *.jpeg binary | ||
| *.gif binary | ||
| *.ico binary | ||
| *.svg binary | ||
| *.woff binary |
There was a problem hiding this comment.
Marking *.svg as binary prevents useful diffs/merges for the many SVG source assets in src/assets/ (they will appear as binary changes). Consider treating SVG as text (keep text eol=lf) and reserve binary for actual raster assets.
| output = run_gh_command([ | ||
| "api", | ||
| f"/repos/{{owner}}/{{repo}}/pulls/{pr_number}/reviews", | ||
| "--jq", "." | ||
| ]) |
There was a problem hiding this comment.
The GitHub API path uses literal {owner} / {repo} placeholders, so gh api will call an invalid endpoint. Replace these with the actual repo (e.g., via --repo owner/repo, $GITHUB_REPOSITORY, or by passing owner/repo into this function).
| if git diff --cached --name-only | xargs grep -n "TODO\|FIXME" 2>/dev/null; then | ||
| echo "::warning::Found TODO/FIXME comments in changes" | ||
| fi | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci | ||
| # Check for large files | ||
| if git diff --cached --name-only | xargs du -h 2>/dev/null | awk '$1 ~ /M$/ && $1 > 1'; then | ||
| echo "::warning::Found large files in changes" |
There was a problem hiding this comment.
git diff --cached is typically empty in CI, and xargs du -h without a --no-run-if-empty/-r guard will run du with no paths (walking the entire repo). This can produce noisy output and slow CI. Consider diffing the PR range instead of --cached, and add xargs -r (or equivalent) so nothing runs when there are no changed files.
| - `.github/workflows/docker-compose.yml` (4.4KB) - Docker CI | ||
| - `.github/workflows/branch-management.yml` (4.3KB) - Python automation orchestrator | ||
| - `.github/workflows/pulumi.yml` (3.9KB) - IaC workflow | ||
| - `scripts/validate-lockfile.sh` (1.5KB) - Lock file validation |
There was a problem hiding this comment.
This file list still mentions scripts/validate-lockfile.sh, but the actual implementation appears to be scripts/validate-lockfile.py. The inventory should match the current scripts in the repo.
| - `scripts/validate-lockfile.sh` (1.5KB) - Lock file validation | |
| - `scripts/validate-lockfile.py` (1.5KB) - Lock file validation |
| # Create or update issue if not in dry run | ||
| if [ "${{ inputs.dry_run || 'true' }}" = "false" ]; then | ||
| gh issue list --label "infrastructure,staging-limit" --state open --limit 1 > existing_issues.txt | ||
|
|
||
| if [ ! -s existing_issues.txt ]; then | ||
| ISSUE_BODY="Active staging environments: $ACTIVE_COUNT | ||
|
|
||
| Azure Static Web Apps free tier has a limit of 10 staging environments. | ||
|
|
||
| **Action Required:** | ||
| - Review and close stale PRs | ||
| - Merge or close PRs that are ready | ||
|
|
||
| **Current Status:** Automatically detected by branch management workflow" | ||
|
|
||
| gh issue create \ | ||
| --title "⚠️ Staging Environment Limit Approaching ($ACTIVE_COUNT/10)" \ | ||
| --body "$ISSUE_BODY" \ | ||
| --label "infrastructure,staging-limit,automated" | ||
|
|
||
| echo "Created alert issue" | ||
| else | ||
| echo "Alert issue already exists" | ||
| fi | ||
| fi |
There was a problem hiding this comment.
The workflow inputs default dry_run to true, and the scheduled run has no inputs, so this condition prevents alert issues from ever being created on the weekly schedule. If the goal is proactive alerting, consider allowing issue creation even in dry-run mode (or default scheduled runs to create alerts while still avoiding destructive cleanup).
| """Get list of remote branches""" | ||
| logger.info("Fetching branches...") | ||
| cmd = ["gh", "api", f"/repos/{self.repo_full}/branches", "--jq", ".[].name"] | ||
| output = self.run_command(cmd) | ||
| return output.split('\n') if output else [] |
There was a problem hiding this comment.
gh api /repos/.../branches is paginated (default page size 30). Without --paginate, this will miss branches in repos with >30 branches and can lead to incomplete cleanup/status output. Consider adding --paginate (and potentially querying only needed fields).
| output = run_gh_command([ | ||
| "api", | ||
| f"/repos/{{owner}}/{{repo}}/pulls/{pr_number}/comments", | ||
| "--jq", "." | ||
| ]) |
There was a problem hiding this comment.
Same issue as above: this uses literal {owner} / {repo} placeholders, so the endpoint is invalid. Use a real repo identifier (e.g., --repo owner/repo) or substitute owner/repo dynamically.
| python-version: '3.11' | ||
|
|
||
| - name: Install Pulumi CLI | ||
| uses: pulumi/actions@v5 |
There was a problem hiding this comment.
pulumi/actions@v5 typically expects required inputs (like command/stack-name). As written, this step is likely to fail at runtime rather than "install" Pulumi. Consider switching to an install/setup action (or provide the required inputs and let the action run preview/up instead of invoking pulumi manually).
| uses: pulumi/actions@v5 | |
| uses: pulumi/setup-pulumi@v3 |
| check_tool "node" | ||
| check_tool "npm" | ||
| check_tool "git" | ||
| check_tool "python3" | ||
|
|
There was a problem hiding this comment.
With set -e enabled, any missing required tool here will cause the script to exit immediately because check_tool returns non-zero. If the intent is to report all missing tools (or treat only some as fatal), wrap these calls to avoid early exit and/or aggregate failures before exiting with a clear error message.
| ```json | ||
| { | ||
| "mcpServers": { | ||
| "github": { | ||
| "command": "npx", | ||
| "args": ["-y", "@modelcontextprotocol/server-github"], | ||
| "env": { | ||
| "GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_TOKEN}" | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The MCP configuration example here launches the GitHub MCP server via npx -y @modelcontextprotocol/server-github while passing GITHUB_PERSONAL_ACCESS_TOKEN (from GITHUB_TOKEN) directly into the process environment. This means unpinned third-party code fetched from npm at runtime is executed with full access to a GitHub personal access token, creating a high‑impact supply chain risk if the package or registry is ever compromised. To reduce this risk, pin the MCP server to an immutable version or digest and avoid giving dynamically downloaded tooling a high‑privilege PAT (use least‑privilege tokens and a vetted, version‑pinned installation method in .mcp/config.json).
Pull Request
Description
This PR consolidates and modernizes the project's DevOps and CI/CD infrastructure by creating a comprehensive, production-ready system that unifies build processes, automates branch management, and establishes a single source of truth for documentation.
Related Issues
Related to consolidating features from conceptual PRs #28 and #30 for improved DevOps infrastructure.
Type of Change
Changes Made
Problem Statement
The project needed a cohesive DevOps foundation with:
Solution
🐳 Docker-Based CI as Primary Pipeline
Created
.github/workflows/docker-compose.ymlas the single source of truth for CI/CD:The legacy
ci.ymlworkflow has been simplified to lightweight quick validation checks (60% size reduction from 120 to 48 lines), running basic pre-checks before the comprehensive Docker CI.🤖 Python-Based Automation
Created
automation/branch_manager.py(265 lines, optimized with unused imports removed) for comprehensive management:Orchestrated by
.github/workflows/branch-management.yml:azure-staging-cleanup.ymlworkflow☁️ Infrastructure as Code Support
Created
.github/workflows/pulumi.ymlready for IaC implementation:📝 Consolidated Documentation
Created
DEVCONTAINER_AND_AUTOMATION.md(19KB) as the single source of truth:Removed redundant files:
docs/CI_CD_GUIDE.md(13.5KB)docs/CI_CD_QUICK_REFERENCE.md(4.8KB)docs/IMPLEMENTATION_CHECKLIST.md(9.3KB)🔐 Lock File Validation (Python Implementation)
Created
scripts/validate-lockfile.pyfor ensuring package integrity:Modified Files
.github/workflows/ci.yml- Updated to use Python validation script.github/workflows/docker-compose.yml- Updated to use Python validation script.github/workflows/azure-staging-cleanup.yml- Deprecated with migration noticedocs/README.md- Updated to reference consolidated documentationautomation/branch_manager.py- Removed unused imports (os, Optional)DEVCONTAINER_AND_AUTOMATION.md- Updated script referencesREVIEW_REQUEST.md- Updated script referencesNew Files
.github/workflows/docker-compose.yml- Primary CI workflow.github/workflows/branch-management.yml- Python automation orchestrator.github/workflows/pulumi.yml- IaC workflowscripts/validate-lockfile.py- Python lock file validation (90 lines)automation/branch_manager.py- Branch and environment managerautomation/README.md- Automation documentationinfrastructure/README.md- IaC documentationDEVCONTAINER_AND_AUTOMATION.md- Consolidated guideDeleted Files
scripts/validate-lockfile.sh- Replaced with Python implementationdocs/CI_CD_GUIDE.md- Merged into consolidated guidedocs/CI_CD_QUICK_REFERENCE.md- Merged into consolidated guidedocs/IMPLEMENTATION_CHECKLIST.md- Merged into consolidated guideTesting
Test Coverage
Testing Steps
python3 scripts/validate-lockfile.pynpm run lintnpm run buildTest Results
Build Status: ✅ Pass
Screenshots/Recordings
N/A - Infrastructure and automation changes only
Breaking Changes
Breaking Changes: No
All existing workflows continue to function. The bash validation script is replaced with a Python version, but this is called by workflows and doesn't affect user-facing functionality.
Documentation
DEVCONTAINER_AND_AUTOMATION.md, updatedREVIEW_REQUEST.md,automation/README.md,infrastructure/README.mdChecklist
Code Quality
Testing & Validation
npm run build)npm run lint)npm run type-check) or errors are documented - Known type errors exist (documented)npm run test:unit) or N/A - No unit tests currently existDocumentation
Review Readiness
Impact Assessment
Performance Impact: Neutral - Python script has similar performance to bash
Bundle Size Impact: No change - Infrastructure changes only
Backward Compatibility: Maintained - All workflows updated to reference new script
Deployment Notes
Requires:
Reviewer Notes
Review Focus:
Questions for Reviewers:
scripts/or move elsewhere?Additional Context
Benefits
Review Comments Addressed
This PR addresses the following review feedback:
scripts/validate-lockfile.pyusing Python standard libraryautomation/branch_manager.pyFor AI Agents:
Multi-Agent Collaboration: Single agent implementation. Review requested from @danelkay93 and @coderabbitai per
.github/AGENT_COLLABORATION.md.Original prompt
This pull request was created as a result of the following prompt from Copilot chat.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.