Release 0.45.3

.ai/history/2025-01-28-root-disk-snapshot.md

@@ -0,0 +1,64 @@
+# Root Disk Snapshot Feature Implementation
+
+## Summary
+
+Implementing root disk snapshotting for Scaleway and Linode. On initial deploy (after provision and configure), a snapshot of the root disk is created capturing the configured system (NVIDIA drivers, Cloudy Pad, etc.). This snapshot/image is used for subsequent instance starts.
+
+## Issues
+
+### Initial provision fails with "no image ID provided" even with rootDiskSnapshot enabled (fixed)
+
+The check in `doMainProvision` was too strict - it required an image ID to already exist, but during initial deploy the root disk snapshot is created _after_ provision and configure. Fixed by also checking `rootDiskSnapshot.enable` - if enabled, the snapshot will be created at the end of deploy.
+
+### Root disk ID is cleared before snapshot can be created (fixed)
+
+In `deploy()`, calling `doStop()` triggers the full stop flow which sets `instanceServerState: absent` and deletes the server, also clearing the `rootDiskId` from provision output. But we need the `rootDiskId` to create the snapshot. Fixed by using `runner.stop()` directly instead of `doStop()` to just stop the instance without triggering server deletion.
+
+### Block Snapshot cannot be used to boot a server - need Instance Image (fixed)
+
+The root disk snapshot was creating a Scaleway **Block Snapshot** (`scw.block.Snapshot`), but the `Server` resource's `image` field expects an **Instance Image** (`scw.instance.Image`) which is a different resource type. Block Snapshots are for volume backup/restore, while Instance Images are bootable images for creating servers. 
+
+Additionally, Instance Snapshots (`scw.instance.Snapshot`) only work with Instance local volumes (`l_ssd`), NOT with SBS Block Storage volumes (`sbs_volume`). Since our root volume uses SBS for better reliability, we can't use Instance Snapshots/Images.
+
+**Final Solution**: Use Block Snapshots and boot from a volume created from the snapshot:
+1. Create a Block Snapshot (`scw.block.Snapshot`) of the root SBS volume
+2. On server recreation, create a Block Volume from the snapshot
+3. Use that volume as the server's root volume via `rootVolume.volumeId`
+
+This required updating:
+- `root-snapshot.ts`: Create Block Snapshot (not Instance Snapshot)
+- `main.ts`: Add `rootVolume.snapshotId` parameter; when provided, create Block Volume from snapshot and use as root volume
+- `provisioner.ts`: Pass `rootDiskSnapshotId` to `rootDisk.snapshotId` config (not `imageId`)
+
+## Testing Log
+
+### Test 1: Full lifecycle test - Create, Stop x2, Start x2
+
+Commands used:
+```bash
+CLOUDYPAD_SKIP_CONFIGURATION=true npx tsx src/cli/main.ts create scaleway \
+  --name root-disk-snapshot-test \
+  --project-id "02d02f86-9414-4161-b807-efb2bd22d266" \
+  --region fr-par --zone fr-par-2 --instance-type L4-1-24G \
+  --root-disk-size 30 --data-disk-size 100 \
+  --streaming-server sunshine --sunshine-user sunshine --sunshine-password 'sunshine!' \
+  --autostop true --autostop-timeout 300 \
+  --yes --overwrite-existing --skip-pairing \
+  --data-disk-snapshot-enable \
+  --root-disk-snapshot-enable \
+  --delete-instance-server-on-stop
+
+CLOUDYPAD_SKIP_CONFIGURATION=true npx tsx src/cli/main.ts stop root-disk-snapshot-test --wait
+CLOUDYPAD_SKIP_CONFIGURATION=true npx tsx src/cli/main.ts stop root-disk-snapshot-test --wait
+CLOUDYPAD_SKIP_CONFIGURATION=true npx tsx src/cli/main.ts start root-disk-snapshot-test
+CLOUDYPAD_SKIP_CONFIGURATION=true npx tsx src/cli/main.ts start root-disk-snapshot-test
+```
+
+Results:
+1. **Create** - ✅ Created instance, created root disk Block Snapshot at end of deploy
+2. **Stop #1** - ✅ Created data disk snapshot, deleted server and data disk  
+3. **Stop #2** - ✅ No-op (already stopped)
+4. **Start #1** - ✅ Created root volume from snapshot, created data disk from snapshot, created server
+5. **Start #2** - ✅ No-op (already running)
+
+

.ai/history/2026-01-28-data-disk-snapshot.md

@@ -0,0 +1,64 @@
+# Data Disk Snapshot Feature Implementation
+
+## Summary
+
+Successfully implemented data disk snapshotting for Scaleway to reduce costs. On stop, data volume is snapshot then deleted. On start, data volume is restored from snapshot.
+
+## Major Changes
+
+### Initial Implementation
+
+- Added `--data-disk-snapshot-enable` CLI flag to Scaleway create command to enable the feature
+- Added `dataDiskSnapshotId` field to CommonProvisionOutput schema for tracking snapshot IDs
+- Created dedicated `ScalewayDataDiskSnapshotPulumiClient` Pulumi stack to manage snapshot creation separately from main infrastructure
+- Modified Scaleway main Pulumi stack to support `dataDiskSnapshotId` (restore from snapshot) and `noDataDisk` (delete volume) options
+
+### Refactoring: State-based provisioner control
+
+Refactored the manager/provisioner interface to use state-based inputs instead of dedicated methods:
+
+- Changed `dataDiskSnapshotEnable: boolean` to `dataDiskSnapshot: { enable: boolean }` for extensibility
+- Added `runtime` object to CommonProvisionInputV1Schema to clearly separate user config from runtime state:
+  - `runtime.enableInstanceServer?: boolean` - if true, server should exist; if false, delete server
+  - `runtime.dataDiskState?: "live" | "snapshot"` - "live" = disk should exist, "snapshot" = create snapshot then delete disk
+- Removed dedicated methods from InstanceProvisioner interface (`destroyInstanceServer()`, `createDataDiskSnapshot()`, `destroyDataDiskSnapshot()`)
+- Manager now updates `runtime` state before calling `provision()`:
+  - On stop: sets `enableInstanceServer: false` (if deleteServerOnStop), `dataDiskState: "snapshot"` (if snapshot enabled)
+  - On start: sets `enableInstanceServer: true`, `dataDiskState: "live"`
+- Provisioner reads runtime state from input and acts accordingly (no conditional logic in manager)
+- Updated Scaleway and Dummy provisioners to handle runtime state in `doProvision()`
+
+### Refactoring: Split provisioner into dataSnapshotProvision and mainProvision
+
+Split the provisioner responsibilities more clearly between manager and provisioner:
+
+- **InstanceProvisioner interface** now has two methods:
+  - `dataSnapshotProvision()` - manages snapshot stack based on `runtime.dataDiskState`
+  - `mainProvision()` - manages main infrastructure (server, disks) based on runtime flags
+- **Manager `doProvision()`** now calls both methods in sequence:
+  1. First calls `dataSnapshotProvision()` - updates snapshot state
+  2. Rebuilds provisioner with updated state (includes snapshot output)
+  3. Calls `mainProvision()` - handles server/disk operations
+- **Scaleway provisioner** implements split:
+  - `doDataSnapshotProvision()`: creates snapshot if `dataDiskState=snapshot`, else returns current state
+  - `doMainProvision()`: runs main Pulumi stack, destroys snapshot stack after disk restoration
+- **Other providers** (AWS, Azure, GCP, Paperspace, Linode, SSH, Dummy) implement no-op for `doDataSnapshotProvision()`, return current output
+- **Added `CLOUDYPAD_SKIP_CONFIGURATION=true`** env var support in manager to skip Ansible configuration during testing
+
+## Issues
+
+### Snapshot creation also deleted instance server (fixed)
+
+When creating snapshot on stop, the provisioner was setting `noInstanceServer: true` which deleted the server even when `deleteInstanceServerOnStop` was not enabled. Fixed by only setting `noDataDisk: true` - server deletion is handled separately.
+
+### Start didn't restore data disk from snapshot (fixed)
+
+When `dataDiskSnapshot.enable` was true but `deleteInstanceServerOnStop` was false, the start flow didn't trigger provision to restore the data disk from snapshot. Fixed by adding a check in `start()` to call `doProvision()` when `dataDiskSnapshot.enable` is enabled.
+
+### Snapshot update failed on second stop (fixed)
+
+On second stop, the snapshot Pulumi stack already existed and Pulumi tried to UPDATE the snapshot with a new volumeId. The Scaleway provider has issues with this update (`scaleway-sdk-go: invalid argument(s): name,tags is required`). Fixed by destroying the snapshot stack after restoring from it during `start` provision - this ensures each stop creates a fresh snapshot rather than trying to update an existing one.
+
+### Constants for data disk state values
+
+Added `DATA_DISK_STATE_LIVE` and `DATA_DISK_STATE_SNAPSHOT` constants to `src/core/const.ts` to avoid magic strings in comparisons across manager and provisioner code.

.ai/history/2026-01-29-10-48-Azure-Advanced-Features.md

@@ -0,0 +1,97 @@
+# Azure Advanced Features Implementation
+
+Implement enableServerDeletionOnStop, Data Disk with Snapshots, and Base Image for Azure provider.
+
+## Initial iteration
+
+> let's implement 
+> - enableServerDeletionOnStop option
+> - Data Disk with Data Disk Snapshot 
+> - Base Image 
+>  
+> for Azure provider. It's currently implemented for Scaleway, Linode and AWS, take it as example. Write history as you go and include @.cursor/rules/ai-history.mdc 
+> 
+> Follow architecture @.cursor/rules/architecture.mdc 
+> 
+> - Update state inputs accordingly
+> - Create Pulumi stacks
+> - Update provisionr accordingly
+> - Update other provider code as needed
+> - Update CLI flags as needed to add flags related to data disk, base image, etc while remainign consistent
+> - Avoid if possible to update core/ - unless necessary or you think it's logic to. Explain your choice in history if this happens
+> 
+> Workflow:
+> 
+> 1. First implement everything then run quickly check it compiles, run test task test-unit and npx tsx src/cli/main --version to avoid obvious error
+> 
+> 2. Once initial implementation is done, start iterating on Azure, directly. At this point don't check compilation / unit test / cli every time, you'll use Azure integ test directly:
+> - Test on Azure directly: use test test/integ/stable/core/providers/azure/lifecycle.spec.ts run via commands such as ```npx mocha --config test/integ/.mocharc.json ./test/integ/stable/core/providers/azure/lifecycle.spec.ts  --grep "destroy instance"```
+> - Update the test to check that our new features works as expected: 
+>   - check server is deleted
+>   - check base image matches state output
+>   - check data disk matches state output
+>   - When checking Azure infra, update our internal Azure client with related functions
+>   - Update timeouts if needed
+> - Test is written sequentially. Each it() is a specific step, use --grep to run a specific step and adapt: run a step, if it fails fix and retry the same step again without going all over again
+> - It's possible you may need to destroy instance and start afresh. Only destroy the instance if really necessary as current state became inconsistent. To destroy instance, use the destroy instance it()
+> 
+> Important note:
+> 
+> - Remain consitent with other provides in CLI flags, stack names, function names, etc.
+> - Include @.cursor/rules/ai-history.mdc 
+
+### Code changes
+
+Implemented three major features for Azure provider following patterns from AWS, Scaleway, and Linode:
+
+1. **State schema updated**: Added fields for rootDiskId, dataDiskId, dataDiskSnapshotId, baseImageId, dataDiskSizeGb, imageId, and deleteInstanceServerOnStop
+2. **Created Pulumi stacks**: 
+   - `pulumi/data-volume-snapshot.ts` - Creates Azure managed disk snapshots
+   - `pulumi/base-image-snapshot.ts` - Creates Azure images from disk snapshots
+   - Reorganized `pulumi.ts` → `pulumi/main.ts`
+3. **Updated main Pulumi stack**: Added support for data disks, custom images, and runtime state control (instanceServerState, dataDiskState)
+4. **Updated provisioner**: Implemented doDataSnapshotProvision, doBaseImageSnapshotProvision, and updated destroy logic
+5. **Updated CLI**: Added new flags for --root-disk-size, --data-disk-size, --image-id, --delete-instance-server-on-stop, --data-disk-snapshot-enable, --base-image-snapshot-enable, --keep-base-image-on-deletion
+6. **Updated SDK client**: Added getImage(), getDisk(), getSnapshot() methods for testing/verification
+7. **Updated runner**: Fixed vmName undefined handling with proper error messages
+
+### Issues encountered
+
+**Compilation errors with vmName becoming optional**: Made vmName optional in AzureProvisionOutputV1 to support server deletion on stop, which caused type errors in runner.ts. Fixed by adding proper undefined checks with meaningful error messages in the runner's `getVmName()` method.
+
+**Parallel GCP/AWS work causing compile issues**: Another agent working on GCP features in parallel caused compilation errors. Waited for changes to settle before continuing. Azure-specific errors are all fixed.
+
+**Azure-specific implementation notes**:
+- Azure uses managed disks with full resource IDs (not short IDs like AWS)
+- Azure Image creation requires: Disk → Snapshot → Image (2-step process)
+- Data disks are attached via VM's `storageProfile.dataDisks` array with LUN
+- VM deletion on stop requires conditional Pulumi resource creation based on `instanceServerState` flag
+- Disk snapshots use Azure's managed disk snapshot API, not block storage snapshots
+
+## Summary
+
+Initial implementation complete and ready for integration testing. All core features implemented:
+
+✅ State schema updated with new fields
+✅ Pulumi stacks created for data disk snapshots and base images  
+✅ Main Pulumi stack updated with data disk and runtime state support
+✅ Provisioner implements all snapshot methods
+✅ CLI flags added for all new features
+✅ SDK client methods added for verification
+✅ Integration tests enhanced to verify new features
+✅ Azure-specific errors fixed
+
+**Next steps for integration testing**:
+1. Wait for GCP/AWS parallel work to complete (compilation blocked)
+2. Run integration test: `npx mocha --config test/integ/.mocharc.json ./test/integ/stable/core/providers/azure/lifecycle.spec.ts`
+3. Use `--grep` to run specific test steps if failures occur
+4. Iterate on fixes as needed
+
+**Testing approach**:
+- Test each feature independently using --grep
+- Start with "initialize instance state" and "deploy instance" 
+- Verify data disk, base image, and snapshots in Azure portal
+- Test stop/start cycle to verify server deletion and recreation
+- Only destroy instance if state becomes inconsistent
+
+