Skip to content

Add Copilot agent and skills for AzValidation customer guidance#14

Open
vinushree-git wants to merge 1 commit into
Azure:mainfrom
vinushree-git:feature/copilot-agent-skills
Open

Add Copilot agent and skills for AzValidation customer guidance#14
vinushree-git wants to merge 1 commit into
Azure:mainfrom
vinushree-git:feature/copilot-agent-skills

Conversation

@vinushree-git

Copy link
Copy Markdown
Contributor

Summary

Adds GitHub Copilot agent and skill files so customers can use *@AzValidation* in VS Code Copilot Chat to get guided ARM CLI commands for the full Microsoft.Validate Compute Validation lifecycle.

What's added

File Purpose
.github/agents/AzValidation.agent.md\ @AzValidation\ agent — orchestrates onboarding, CRUD, monitoring, and debug in a conversational multi-step flow
.github/skills/azvalidation-onboard/SKILL.md\ Step-by-step RP registration → first run (triggered by: 'onboard', 'register Microsoft.Validate')
.github/skills/azvalidation-monitor/SKILL.md\ Poll run status, fetch test results, list runs (triggered by: 'check status', 'get test results')
.github/skills/azvalidation-debug/SKILL.md\ Triage failures: VHD SAS issues, quota, LISA test fixes, orphaned resources (triggered by: 'debug run', 'why did it fail')

Customer experience

Customers clone this repo, open it in VS Code with GitHub Copilot, and can immediately type:

  • @AzValidation onboard my subscription abc-123\
  • @AzValidation check status of run my-run under plan my-plan\
  • @AzValidation why did my LinuxQualityValidations fail?\

No binary installation required. Works with any Azure MCP or az CLI.

Motivation

  • Zero-friction alternative to distributing a custom MCP binary
  • Agent carries ARM path knowledge; customers only need \�z login\
  • Debug skill encodes common failure patterns from private preview support cases

Testing

Validated against the ARM paths and API version (\2026-02-01-preview) documented in the onboarding guide.

Adds GitHub Copilot agent and skills to help customers onboard, run,
monitor, and debug Microsoft.Validate Compute Validation via ARM APIs.

Files added:
- .github/agents/AzValidation.agent.md     - @AzValidation agent: full workflow orchestrator
- .github/skills/azvalidation-onboard/     - Onboarding skill (register RP → first run)
- .github/skills/azvalidation-monitor/     - Monitoring skill (poll status, fetch results)
- .github/skills/azvalidation-debug/       - Debug skill (failure triage, LISA fixes, quota)

Customers with VS Code + GitHub Copilot can clone this repo and immediately
use @AzValidation in chat to get guided ARM CLI commands for all validation
lifecycle steps — no binary installation required.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
```

### Supported Region
`southcentralus` — only supported region for public preview.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preview-stage mislabeled. Repo + onboarding guide are Private Preview, but this says "only supported region for public preview" (PR body too). Make the preview stage consistent everywhere.


---

## Mandatory Workflow Order

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agent workflow omits required steps. The "Mandatory Workflow Order" skips Grant RP Contributor and Enable Defender, which azvalidation-onboard lists as Steps 2–3. Agent-driven customers will hit failures. Align the orchestrator with the skill.

```bash
az rest --method GET \
--url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans?api-version=2026-02-01-preview" \
--query "value[].{name:name, state:properties.provisioningState, overall:properties.overallState}" \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Property typo. Uses properties.overallState; every other query uses properties.overallStatus. Copy-paste yields a null column.

--container-name {container} \
--name {vhdBlob} \
--permissions rl \
--expiry $(date -u -d "+7 days" +%Y-%m-%dT%H:%MZ) \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bash/GNU-only. date -u -d "+7 days" fails on macOS/BSD; poll loop is bash. Add PowerShell equivalents or a portability note for Windows customers.

--container-name {containerName} \
--name {vhdBlobName} \
--permissions rl \
--expiry $(date -u -d "+7 days" +%Y-%m-%dT%H:%MZ) \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bash/GNU-only. date -u -d "+7 days" fails on macOS/BSD; poll loop is bash. Add PowerShell equivalents or a portability note for Windows customers.

--output tsv
```

Requirements: **≥1 day expiry**, permissions **`rl` (read + list)**, publicly accessible storage endpoint.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SAS expiry inconsistent. Example uses +7 days; prose says "≥1 day". Align example with stated minimum.

Same in onboard L30, agent L180

### Supported Validations
- `MalwareValidation` — malware scan
- `BootValidation` — VM boot test (testNames: `VM-Boot-Test`, `Boot-Diagnostics-Available`)
- `VulnerabilityValidation` — Microsoft Defender scan (requires Defender for Cloud Plan 2)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defender naming drift. Agent says "Defender for Cloud Plan 2"; skills use --tier Standard. Use one term.

│ ├── MalwareValidation failed → Clean image of malware, re-run
│ ├── BootValidation failed → Check VHD SAS URI validity (≥1 day, read+list)
│ │ Check Azure Agent installed in image
│ ├── VulnerabilityValidation → Enable Defender for Cloud Plan 2 on subscription

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, Defender naming drift. Agent says "Defender for Cloud Plan 2"; skills use --tier Standard. Use one term.

Use when: "debug run", "why did it fail", "run failed", "test failed",
"quota exceeded", "vhd error", "orphaned resources", "deployment failed",
"provisioning failed", "how to fix".
triggers:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicated trigger metadata. Triggers appear in description: "Use when…" and again in a triggers: block. Confirm the host even consumes triggers:; if not, it's dead metadata.

Same in monitor L7, onboard L8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants