Make sandbox runner pluggable with Kubernetes driver support

[eshulman2]

Summary

• Extract SandboxDriver ABC from monolithic ContainerRunner, creating a pluggable driver interface (ExecutionSpec → ExecutionResult)
• Refactor Podman-specific logic into PodmanDriver (command building, timeout handling, image management)
• Add KubernetesDriver that runs tasks as Kubernetes Jobs with PVC-based workspace sharing
• Select the driver with FORGE_SANDBOX_DRIVER (default: podman; also supports kubernetes)
• Add a Helm chart for the Forge API, worker, Redis, RBAC, OpenShift Route, and shared workspace PVC
• Support optional Google ADC Secret mounts in both the worker and sandbox Jobs
• Keep caller behavior unchanged across existing workflow nodes

Closes #30

Kubernetes deployment

The Helm chart is under charts/forge. It configures:

• Separate API and worker Deployments
• Redis Stack
• Worker and sandbox service accounts with namespace-scoped RBAC
• A shared workspace PVC, retained on Helm uninstall
• Recreate worker updates for ReadWriteOnce storage
• Optional OpenShift Route
• Optional Google ADC Secret propagation to the worker and sandbox Jobs
• Writable shared skills storage

The Forge application image installs the k8s optional dependency.

Configuration

# Default
FORGE_SANDBOX_DRIVER=podman

# Kubernetes
FORGE_SANDBOX_DRIVER=kubernetes
WORKSPACE_BASE_DIR=/mnt/workspaces
K8S_NAMESPACE=forge
K8S_WORKSPACE_PVC=forge-workspaces
K8S_WORKSPACE_BASE_PATH=/mnt/workspaces
K8S_IMAGE_PULL_SECRETS=
K8S_SERVICE_ACCOUNT=forge-runner

# Optional Vertex AI credential Secret mounted into sandbox Jobs
K8S_GOOGLE_CREDENTIALS_SECRET=google-adc
K8S_GOOGLE_CREDENTIALS_KEY=forge-gcp-credentials.json
K8S_GOOGLE_CREDENTIALS_MOUNT_PATH=/etc/forge-gcp-credentials.json

For Vertex AI deployments:

oc create secret generic google-adc \
  --from-file=forge-gcp-credentials.json=/path/to/forge-gcp-credentials.json \
  -n forge

helm upgrade --install forge charts/forge -n forge \
  --set googleCredentials.enabled=true

Test results

• Kubernetes driver unit tests: 23 passed
• Broader sandbox suite: 59 passed; 3 pre-existing unrelated review import failures
• Ruff formatting and lint checks pass for changed Python files
• Helm lint passes with Google credentials enabled and disabled
• Helm resources validated against a real OpenShift 4.20 cluster
• Forge API, worker, Redis, Route, RBAC, and RWO PVC deployed successfully
• Worker selected KubernetesDriver and used in-cluster Kubernetes authentication
• Google ADC and writable skills deployment issues reproduced and fixed
• Complete an implementation sandbox Job against a real cluster
• Manually verify unchanged Podman execution

The temporary OpenShift test reached model-powered task triage. The final sandbox Job was not reached because the temporary ADC identity lacked aiplatform.endpoints.predict; the cluster was subsequently deleted. The chart now supports mounting the production Forge service-account credential into both the worker and sandbox Jobs for the next test.