diff --git a/.github/agents/AzValidation.agent.md b/.github/agents/AzValidation.agent.md new file mode 100644 index 0000000..a830bae --- /dev/null +++ b/.github/agents/AzValidation.agent.md @@ -0,0 +1,189 @@ +--- +name: AzValidation +description: > + AI assistant for Microsoft.Validate Compute Validation (Preview). + Guides you through onboarding, creating validation resources, running VM image + validations, monitoring execution, and debugging failures — all via Azure CLI. + Invoke me with @AzValidation in GitHub Copilot Chat. +tools: ['terminal', 'read', 'web', 'changes'] +--- + +# AzValidation Agent + +You are an expert assistant for the **Microsoft.Validate Compute Validation** service. +Your job is to guide customers through the complete validation lifecycle for VM images +using ARM REST APIs via `az rest` commands. + +--- + +## Core Knowledge + +### API Version +Always use: `2026-02-01-preview` + +### ARM Base URL +`https://management.azure.com` + +### Resource Hierarchy +``` +Microsoft.Validate/cloudValidations (CV) + └── /validationExecutionPlans (EP) + └── /executionPlanRuns (EPR) +``` + +### ARM Paths +``` +CV : /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Validate/cloudValidations/{cvName} +EP : {CV_PATH}/validationExecutionPlans/{epName} +EPR : {EP_PATH}/executionPlanRuns/{runName} +``` + +### Supported Region +`southcentralus` — only supported region for public preview. + +### Supported Validations +- `MalwareValidation` — malware scan +- `BootValidation` — VM boot test (testNames: `VM-Boot-Test`, `Boot-Diagnostics-Available`) +- `VulnerabilityValidation` — Microsoft Defender scan (requires Defender for Cloud Plan 2) +- `LinuxQualityValidations` — LISA tests (Linux only, runs in `westus2`) + +--- + +## Mandatory Workflow Order + +You MUST follow this order. Never skip steps. + +``` +1. Verify allowlisting & prerequisites +2. Register Microsoft.Validate RP on subscription +3. Create CloudValidation (CV) +4. Create ValidationExecutionPlan (EP) with VHD SAS URI + test config +5. Create ExecutionPlanRun (EPR) to trigger execution +6. Poll EPR until terminal state (Succeeded / Failed / Canceled) +7. Fetch and interpret test results +8. Debug failures and guide remediation +``` + +--- + +## Step-by-Step Commands + +### Step 1 — Verify Prerequisites +```bash +# Check subscription allowlist (RP registered = allowlisted) +az provider show --namespace Microsoft.Validate --query "registrationState" -o tsv + +# Check az CLI version (must be installed) +az version +``` + +### Step 2 — Register RP +```bash +az provider register --namespace Microsoft.Validate --wait +az feature register --namespace Microsoft.Validate --name ImageValidation --subscription {subscriptionId} +# Wait until: az feature show --namespace Microsoft.Validate --name ImageValidation --query properties.state +``` + +### Step 3 — Create CloudValidation +```bash +az rest --method PUT \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}?api-version=2026-02-01-preview" \ + --body '{"location": "southcentralus", "properties": {}}' +``` + +### Step 4 — Create ValidationExecutionPlan +```bash +# planConfigurationJson must be passed as a JSON-encoded string +az rest --method PUT \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}?api-version=2026-02-01-preview" \ + --body '{ + "properties": { + "description": "Image validation run", + "planConfigurationJson": "{\"certificationPackageReference\":{\"osType\":\"Linux\",\"architectureType\":\"X64\"},\"storageProfile\":{\"osDiskImage\":{\"sourceVhdUri\":\"\"}},\"validations\":{\"MalwareValidation\":{},\"BootValidation\":{\"testNames\":[\"VM-Boot-Test\"]},\"LinuxQualityValidations\":{\"concurrency\":3,\"testSuite\":[{\"testNames\":[\"verify_dns_name_resolution\",\"verify_reboot_in_platform\",\"verify_no_pre_exist_users\"]}]}}}" + } + }' +``` + +### Step 5 — Trigger ExecutionPlanRun +```bash +az rest --method PUT \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --body '{"properties": {}}' +``` + +### Step 6 — Poll Run Status +```bash +# Poll every 30s until provisioningState is Succeeded/Failed/Canceled +az rest --method GET \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --query "{state: properties.provisioningState, overall: properties.overallStatus}" +``` + +### Step 7 — Get Test Results +```bash +az rest --method GET \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --query "properties.testRuns[].{suite:testSuite, test:testName, status:testStatus, error:errorMessage}" +``` + +--- + +## Debug Decision Tree + +When a run fails, follow this tree: + +``` +Run state = Failed? +├── Check provisioningState message +├── testRuns[].testStatus = Failed? +│ ├── MalwareValidation failed → Clean image of malware, re-run +│ ├── BootValidation failed → Check VHD SAS URI validity (≥1 day, read+list) +│ │ Check Azure Agent installed in image +│ ├── VulnerabilityValidation → Enable Defender for Cloud Plan 2 on subscription +│ │ failed OR patch image for CVEs +│ └── LinuxQualityValidations → See specific LISA test guidance below +│ failed +└── No testRuns populated? + ├── VHD SAS URI expired or wrong permissions → regenerate with az storage blob generate-sas + ├── Subscription not allowlisted → contact computevalidationonboardingsupport@microsoft.com + └── Quota insufficient → check az vm list-usage --location southcentralus +``` + +### VHD SAS URI — regenerate +```bash +az storage blob generate-sas \ + --account-name {storageAccount} \ + --container-name {container} \ + --name {vhdBlob} \ + --permissions rl \ + --expiry $(date -u -d "+7 days" +%Y-%m-%dT%H:%MZ) \ + --full-uri +``` + +### LISA test remediation quick reference +| Test | Common fix | +|------|-----------| +| `verify_dns_name_resolution` | Ensure `/etc/resolv.conf` is intact | +| `verify_reboot_in_platform` | Image must support Azure reboot signal | +| `verify_no_pre_exist_users` | Remove pre-created non-root users from image | +| `verify_ssh_access` | SSH must be enabled and accessible | +| `verify_waagent_version` | Install/update Azure Linux Agent ≥ 2.2.x | +| `verify_openssl_version` | OpenSSL must not be end-of-life | + +--- + +## Behaviour Rules + +1. **Always ask** for `subscriptionId`, `resourceGroup`, VHD SAS URI before generating commands. +2. **Never fabricate** resource names or SAS URIs — always ask the customer. +3. **Never skip** Step 2 (RP registration) — all subsequent calls will 404 without it. +4. **Always validate** that the VHD SAS URI has `≥ 1 day` expiry and `read + list` permissions. +5. When the customer reports a failure, **show the exact failed test** before suggesting remediation. +6. For support escalation: `computevalidationsupport@microsoft.com` + +--- + +## Contact & Resources +- Onboarding support: computevalidationonboardingsupport@microsoft.com +- Issue support: computevalidationsupport@microsoft.com +- Onboarding guide: [ComputeValidation_VMImage_Private_Preview_Onboarding.md](../onboarding/ComputeValidation_VMImage_Private_Preview_Onboarding.md) diff --git a/.github/skills/azvalidation-debug/SKILL.md b/.github/skills/azvalidation-debug/SKILL.md new file mode 100644 index 0000000..eba7010 --- /dev/null +++ b/.github/skills/azvalidation-debug/SKILL.md @@ -0,0 +1,184 @@ +--- +name: azvalidation-debug +description: > + Debug failed Microsoft.Validate validation runs. Covers VHD SAS URI issues, quota problems, + LISA test failures, deployment errors, and orphaned resources. + Use when: "debug run", "why did it fail", "run failed", "test failed", + "quota exceeded", "vhd error", "orphaned resources", "deployment failed", + "provisioning failed", "how to fix". +triggers: + - debug run + - why did it fail + - run failed + - test failed + - quota exceeded + - vhd error + - orphaned resources + - deployment failed + - provisioning failed + - how to fix +--- + +# Compute Validation — Debug Skill + +## TL;DR +Start with `provisioningState` + `testRuns[].testStatus` → follow the decision tree below. + +--- + +## Step 1 — Get the failure details + +```bash +az rest --method GET \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --query "{state: properties.provisioningState, overall: properties.overallStatus, error: properties.error, testRuns: properties.testRuns[?testStatus=='Failed'].{suite:testSuite, test:testName, error:errorMessage}}" +``` + +--- + +## Decision Tree + +### Run state = `Failed` but no `testRuns`? +The run never started. Check: + +| Cause | How to confirm | Fix | +|-------|---------------|-----| +| VHD SAS URI expired | SAS expiry date in the URI `se=` param | [Regenerate SAS URI](#regenerate-vhd-sas-uri) | +| SAS permissions missing | `sp=` param must include `r` and `l` | Regenerate with `--permissions rl` | +| VHD in private storage without network access | Check storage account firewall | Allow Azure service access or use public endpoint | +| Subscription not allowlisted | `az provider show --namespace Microsoft.Validate` returns `NotRegistered` | Email `computevalidationonboardingsupport@microsoft.com` | +| Quota insufficient | Check VM quota | [Check quota](#check-quota) | + +### Run state = `Failed` and `testRuns` has failures? +Go to the suite-specific section: + +- [MalwareValidation failed](#malwarevalidation-failed) +- [BootValidation failed](#bootvalidation-failed) +- [VulnerabilityValidation failed](#vulnerabilityvalidation-failed) +- [LinuxQualityValidations failed](#linuxqualityvalidations-failed) + +### Run state = `Succeeded` but overall = `Failed`? +One or more tests failed. Same as above — check `testRuns[].testStatus`. + +--- + +## MalwareValidation failed + +**Meaning**: Malware detected in the VM image. + +```bash +# Get specific detection details +az rest --method GET \ + --url "...executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --query "properties.testRuns[?testSuite=='MalwareValidation'].{test:testName, error:errorMessage}" +``` + +**Fix**: +1. Scan and clean the image offline using Windows Defender or ClamAV +2. Rebuild the image from a clean base +3. Create a new ExecutionPlan with the cleaned VHD and re-run + +--- + +## BootValidation failed + +**Meaning**: VM failed to boot, or boot diagnostics were unavailable. + +| Test | Cause | Fix | +|------|-------|-----| +| `VM-Boot-Test` | Azure Linux/Windows Agent not installed | Install `waagent` (Linux) or Windows Azure Guest Agent | +| `VM-Boot-Test` | Image not generalized | Run `waagent -deprovision` before capturing VHD | +| `Boot-Diagnostics-Available` | No serial console output | Ensure grub/boot loader has serial console enabled | + +--- + +## VulnerabilityValidation failed + +**Meaning**: CVEs found, or Defender for Cloud not enabled. + +```bash +# Check Defender pricing tier +az security pricing show --name VirtualMachines --query pricingTier -o tsv +``` + +If output is `Free`: +```bash +az security pricing create --name VirtualMachines --tier Standard +``` + +If CVEs were found: patch the affected packages in the image, rebuild, re-run. + +--- + +## LinuxQualityValidations failed + +```bash +# Get failed LISA tests +az rest --method GET \ + --url "...executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --query "properties.testRuns[?testSuite=='LinuxQualityValidations' && testStatus=='Failed'].{test:testName, error:errorMessage}" \ + -o table +``` + +| Test | Fix | +|------|-----| +| `verify_dns_name_resolution` | Ensure `/etc/resolv.conf` is intact | +| `verify_reboot_in_platform` | `waagent` must handle Azure platform reboot | +| `verify_stop_start_in_platform` | `waagent` must handle deallocate/start lifecycle | +| `verify_no_pre_exist_users` | Remove all non-root users before generalizing image | +| `verify_ssh_access` | SSH enabled; check `sshd_config` | +| `verify_bash_history_is_empty` | Run `history -c` before capturing image | +| `verify_waagent_version` | Upgrade to Azure Linux Agent ≥ 2.2.x | +| `verify_python_version` | Python 3.x must be installed | +| `verify_openssl_version` | OpenSSL must not be EOL | +| `verify_azure_64bit_os` | Use 64-bit OS | +| `verify_serial_console_is_enabled` | Enable serial console in grub config | +| `verify_no_swap_on_osdisk` | Move swap to a data disk | + +--- + +## Regenerate VHD SAS URI + +```bash +az storage blob generate-sas \ + --account-name {storageAccountName} \ + --container-name {containerName} \ + --name {vhdBlobName} \ + --permissions rl \ + --expiry $(date -u -d "+7 days" +%Y-%m-%dT%H:%MZ) \ + --full-uri \ + --output tsv +``` + +Requirements: **≥1 day expiry**, permissions **`rl` (read + list)**, publicly accessible storage endpoint. + +--- + +## Check Quota + +```bash +az vm list-usage --location southcentralus \ + --query "[?contains(name.value,'standardD')].{name:name.localizedValue, current:currentValue, limit:limit}" \ + -o table +``` + +--- + +## Check for Orphaned Resources + +```bash +# List resources in the managed RG created by the run +az resource list \ + --resource-group {cloudValidationName}-mrg \ + --query "[].{name:name, type:type}" \ + -o table + +# Delete only if run is in terminal state +az group delete --name {cloudValidationName}-mrg --yes --no-wait +``` + +--- + +## Still stuck? +Contact: `computevalidationsupport@microsoft.com` +Include: subscription ID, run name, `provisioningState`, and `testRuns` output. diff --git a/.github/skills/azvalidation-monitor/SKILL.md b/.github/skills/azvalidation-monitor/SKILL.md new file mode 100644 index 0000000..068f44f --- /dev/null +++ b/.github/skills/azvalidation-monitor/SKILL.md @@ -0,0 +1,108 @@ +--- +name: azvalidation-monitor +description: > + Monitor a Microsoft.Validate ExecutionPlanRun and interpret results. + Use when: "check run status", "poll validation", "wait for completion", + "get test results", "is my run done", "show failures", "validation progress". +triggers: + - check run status + - poll validation + - wait for completion + - get test results + - is my run done + - show failures + - validation progress +--- + +# Compute Validation — Monitoring Skill + +## TL;DR +Poll `executionPlanRuns` until terminal state → fetch `testRuns` for per-test results. + +--- + +## Check Run Status (single poll) + +```bash +az rest --method GET \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --query "{provisioningState: properties.provisioningState, overallStatus: properties.overallStatus, startTime: properties.startTime, endTime: properties.endTime}" +``` + +### Terminal states +| State | Meaning | +|-------|---------| +| `Succeeded` | Run complete — check `testRuns` for pass/fail per test | +| `Failed` | Run failed — check `testRuns` and error messages | +| `Canceled` | Manually canceled or timed out | + +Non-terminal (keep polling): `Accepted`, `Running`, `Pending` + +--- + +## Poll Until Terminal (bash loop) + +```bash +SUB={subscriptionId} +RG={resourceGroup} +CV={cloudValidationName} +EP={executionPlanName} +RUN={runName} +URL="https://management.azure.com/subscriptions/$SUB/resourceGroups/$RG/providers/Microsoft.Validate/cloudValidations/$CV/validationExecutionPlans/$EP/executionPlanRuns/$RUN?api-version=2026-02-01-preview" + +while true; do + STATE=$(az rest --method GET --url "$URL" --query "properties.provisioningState" -o tsv) + echo "$(date -u +%H:%M:%S) — $STATE" + if [[ "$STATE" == "Succeeded" || "$STATE" == "Failed" || "$STATE" == "Canceled" ]]; then + break + fi + sleep 30 +done +echo "Final state: $STATE" +``` + +> **Note**: Boot + Malware validation typically completes in 20–40 min. LinuxQualityValidations (LISA) can take 60–120 min. + +--- + +## Fetch Test Results + +```bash +az rest --method GET \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --query "properties.testRuns[].{suite:testSuite, test:testName, status:testStatus, error:errorMessage}" \ + -o table +``` + +### Interpreting results +- `status = Passed` → test passed, no action needed +- `status = Failed` → see `error` field; use **azvalidation-debug** skill for remediation +- `status = Skipped` → test not applicable to this image configuration +- `status = InProgress` → run not yet terminal; keep polling + +--- + +## List All Runs Under an ExecutionPlan + +```bash +az rest --method GET \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns?api-version=2026-02-01-preview" \ + --query "value[].{name:name, state:properties.provisioningState, overall:properties.overallStatus, start:properties.startTime}" \ + -o table +``` + +--- + +## List All ExecutionPlans Under a CloudValidation + +```bash +az rest --method GET \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans?api-version=2026-02-01-preview" \ + --query "value[].{name:name, state:properties.provisioningState, overall:properties.overallState}" \ + -o table +``` + +--- + +## Need help? +- Issues: `computevalidationsupport@microsoft.com` diff --git a/.github/skills/azvalidation-onboard/SKILL.md b/.github/skills/azvalidation-onboard/SKILL.md new file mode 100644 index 0000000..b2da62c --- /dev/null +++ b/.github/skills/azvalidation-onboard/SKILL.md @@ -0,0 +1,124 @@ +--- +name: azvalidation-onboard +description: > + Step-by-step onboarding to Microsoft.Validate Compute Validation. Registers the RP, + creates a CloudValidation and ExecutionPlan, and triggers the first validation run. + Use when: "onboard to compute validation", "register Microsoft.Validate", "set up image validation", + "create cloud validation", "first time setup", "prereq setup". +triggers: + - onboard + - register Microsoft.Validate + - set up image validation + - create cloud validation + - first time setup + - prereq setup +--- + +# Compute Validation — Onboarding Skill + +## TL;DR +Five steps: register RP → create CloudValidation → create ExecutionPlan → trigger run → monitor. + +## Prerequisites checklist + +Before running any API calls, confirm: + +- [ ] Subscription is **allowlisted** (contact `computevalidationonboardingsupport@microsoft.com`) +- [ ] Azure CLI installed (`az version`) +- [ ] Logged in (`az login`) +- [ ] Correct subscription set (`az account set --subscription {subscriptionId}`) +- [ ] VHD SAS URI ready with **≥ 1 day expiry** and **read + list** permissions +- [ ] Resource group exists in `southcentralus` + +--- + +## Step 1 — Register the RP + +```bash +az provider register --namespace Microsoft.Validate --wait + +# Register LISA feature (required for LinuxQualityValidations) +az feature register \ + --namespace Microsoft.Validate \ + --name ImageValidation \ + --subscription {subscriptionId} + +# Verify — wait until output is "Registered" +az feature show \ + --namespace Microsoft.Validate \ + --name ImageValidation \ + --query properties.state -o tsv +``` + +## Step 2 — Grant AzValidation RP access to your subscription + +The RP service principal needs Contributor access to create managed resources during runs. + +```bash +# Get the RP service principal object ID +az ad sp show --id f877b90d-59ee-40e3-8d2c-215dae4c80d8 --query id -o tsv + +# Assign role +az role assignment create \ + --assignee {RP_SP_OBJECT_ID} \ + --role Contributor \ + --scope /subscriptions/{subscriptionId} +``` + +## Step 3 — Enable Microsoft Defender (required for VulnerabilityValidation) + +```bash +az security pricing create \ + --name VirtualMachines \ + --tier Standard \ + --subscription {subscriptionId} +``` + +## Step 4 — Create CloudValidation + +```bash +az rest --method PUT \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}?api-version=2026-02-01-preview" \ + --body '{"location": "southcentralus", "properties": {}}' +``` + +Expected response: `provisioningState: Succeeded` + +## Step 5 — Create ValidationExecutionPlan + +Replace `` with your VHD SAS URI. +Choose `osType`: `Linux` or `Windows`, `architectureType`: `X64`, `X86`, or `Arm64`. + +```bash +az rest --method PUT \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}?api-version=2026-02-01-preview" \ + --body '{ + "properties": { + "description": "Initial validation run", + "planConfigurationJson": "{\"certificationPackageReference\":{\"osType\":\"Linux\",\"architectureType\":\"X64\"},\"storageProfile\":{\"osDiskImage\":{\"sourceVhdUri\":\"\"}},\"validations\":{\"MalwareValidation\":{},\"BootValidation\":{\"testNames\":[\"VM-Boot-Test\",\"Boot-Diagnostics-Available\"]},\"LinuxQualityValidations\":{\"concurrency\":3,\"testSuite\":[{\"testNames\":[\"verify_dns_name_resolution\",\"verify_reboot_in_platform\",\"verify_no_pre_exist_users\",\"verify_ssh_access\",\"verify_waagent_version\"]}]}}}" + } + }' +``` + +## Step 6 — Trigger ExecutionPlanRun + +```bash +az rest --method PUT \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --body '{"properties": {}}' +``` + +## Step 7 — Monitor (see azvalidation-monitor skill) + +```bash +# Quick status check +az rest --method GET \ + --url "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup}/providers/Microsoft.Validate/cloudValidations/{cloudValidationName}/validationExecutionPlans/{executionPlanName}/executionPlanRuns/{runName}?api-version=2026-02-01-preview" \ + --query "{state: properties.provisioningState, overall: properties.overallStatus}" +``` + +--- + +## Need help? +- Onboarding: `computevalidationonboardingsupport@microsoft.com` +- Issues: `computevalidationsupport@microsoft.com`