Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ services:
container_name: shipyard-worker
volumes:
- shipyard_sites:/var/lib/shipyard/sites
- /var/run/docker.sock:/var/run/docker.sock
environment:
DATABASE_URL: postgres://${POSTGRES_USER:-shipyard}:${POSTGRES_PASSWORD:-shipyard}@postgres:5432/${POSTGRES_DB:-shipyard}
REDIS_URL: redis://redis:6379
Expand Down
34 changes: 17 additions & 17 deletions docs/adr/0004-spa-fallback.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# ADR-0004: SPA Fallback via nginx and Caddy
# ADR-0004: SPA Fallback

## Status

Expand All @@ -8,32 +8,32 @@ Accepted (updated)

Shipyard deploys static sites (React, Vue, Svelte, etc.) that use client-side routing. When a user navigates to `/dashboard` directly, the browser requests that path. If there is no literal file, the server must return `index.html` so the SPA can boot and handle routing.

The API previously served `public/index.html` as a fallback, but this mixed concerns — the API should not serve static files. The separation is:

- **Web container** (nginx) serves the SPA built output
- **Caddy** reverse-proxies user domains to Garage storage
- **API** serves only REST endpoints

## Decision

### Development
### Caddy pass_thru

Vite dev server handles SPA fallback natively. The API has no static file serving.
Static sites with SPA mode use Caddy's `file_server` with `pass_thru: true`:

### Production
```json
{
"handle": [
{"handler": "file_server", "root": "/var/lib/shipyard/sites/{appId}", "pass_thru": true},
{"handler": "rewrite", "uri": "/index.html"},
{"handler": "file_server", "root": "/var/lib/shipyard/sites/{appId}"}
]
}
```

nginx in the web container serves the SPA via `try_files $uri $uri/ /index.html`. The `nginx.conf` proxies `/api/` and `/health` to the API container.
The first `file_server` attempts to serve the file. On 404, `pass_thru: true` lets the request fall through to the next handler, which rewrites to `/index.html` and serves it via a second `file_server`.

Caddy routes custom domains to Garage storage with its own `handle_errors` for SPA fallback when serving from object storage.
This replaced a broken `subroute` + `errors` pattern that Caddy v2 doesn't support at the route level.

## Consequences

- Clean separation: API never deals with static files.
- Web container is self-contained SPA serving with its own nginx.
- Caddy handles domain routing independently.
- Clean SPA routing without custom nginx or subrequest logic.
- One caveat: genuine 404s (missing static assets) return `index.html` with 200 status. Standard SPA behavior.

## Alternatives Considered

- **API serves SPA (rejected):** Mixed concerns. API container becomes responsible for frontend.
- **Single Caddy for everything (considered for v2):** Caddy could serve the SPA and proxy API. Simpler topology but requires Caddy config complexity.
- **subroute + errors (rejected):** Caddy v2 doesn't support `errors` at the route level. The subroute errors pattern doesn't bubble 404s correctly.
- **Custom nginx container (not adopted for Shipyard):** Dokploy uses Traefik for SPA routing. Shipyard uses Caddy exclusively.
83 changes: 40 additions & 43 deletions docs/adr/0006-static-caddy-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

## Status

Accepted
Accepted (updated)

## Context

Caddy needs to route `myapp.bigboss.dev` to the correct Garage prefix. Two approaches:
Caddy needs to route `myapp.bigboss.dev` to the correct backend. Two approaches:

1. **Dynamic (request-time):** Caddy queries the database per request to resolve `active_deployment_id` → S3 path.
2. **Static (deploy-time):** Send JSON config to Caddy API when deployment succeeds or rollback happens.
Expand All @@ -19,53 +19,51 @@ If two deploys finish at the same time, both sending config to Caddy API, could

**Solution:** Use Caddy's per-route API (not full config):
```
POST /config/apps/http/servers/{server_name}/routes/{route_id}
PATCH /id/{route_id} → update existing route
POST /config/apps/http/servers/srv0/routes → create new route
```
- Each app gets unique route_id based on app.id
- Each app gets unique `@id` based on `app-{appId}`
- Deploy updates ONLY that app's route, not entire config
- No lock needed — Caddy's API is atomic per-route

**Alternative (if simpler for MVP):**
- Single-threaded Caddy config writer (queue config updates)
- Workers call queueCaddyUpdate(appId) after deploy
- Background job processes queue serially
- No concurrent writes to Caddy API

We'll use per-route updates for simplicity.

## Decision

Static config generation at deploy time:

1. Deployment succeeds (or rollback triggered)
2. Read `active_deployment_id` from DB
3. Generate Caddy JSON config via string-replace template:

```json
{
"apps": {
"http": {
"servers": {
"sites": {
"listen": [":443"],
"routes": [{
"match": [{"host": ["myapp.bigboss.dev"]}],
"handle": [{
"handler": "reverse_proxy",
"upstreams": [{"dial": "garage:3900"}]
}]
}]
}
}
}
}
}
```

4. Send JSON payload to Caddy Admin API (`POST /config/`)
5. `Caddy auto-reloads upon receiving the config API update`

No database lookup per request. Caddy serves purely from static config.
3. Generate Caddy JSON config:

**Static sites (isStatic=true):**
```json
{
"@id": "app-{appId}",
"match": [{"host": ["myapp.bigboss.dev"]}],
"handle": [
{"handler": "file_server", "root": "/var/lib/shipyard/sites/{appId}", "pass_thru": true},
{"handler": "rewrite", "uri": "/index.html"},
{"handler": "file_server", "root": "/var/lib/shipyard/sites/{appId}"}
],
"terminal": true
}
```

The `pass_thru` on the first `file_server` lets unmatched requests fall through to the rewrite handler, which serves `index.html` for SPA routing. This replaces the broken subroute+errors pattern that doesn't work in Caddy v2.

**Server apps (dockerfile, nixpacks isStatic=false):**
```json
{
"@id": "app-{appId}",
"match": [{"host": ["myapp.bigboss.dev"]}],
"handle": [
{"handler": "reverse_proxy", "upstreams": [{"dial": "shipyard-app-{appId}:{port}"}]}
],
"terminal": true
}
```

4. Send JSON payload to Caddy Admin API
5. Caddy auto-reloads upon receiving the config API update

## Consequences

Expand All @@ -77,7 +75,6 @@ No database lookup per request. Caddy serves purely from static config.

## Alternatives Considered

- **Dynamic via Lua/OpenResty (rejected):** Flexible but adds complexity and per-request overhead. Overkill for a static site platform.
- **Dynamic via `auth_request` to API (rejected):** Adds API dependency in request path. If API is down, sites break.
- **Symlink-style S3 paths (rejected):** S3 doesn't support symlinks. Would require copying files, which is slow on rollback.

- **Dynamic via Lua/OpenResty (rejected):** Flexible but adds complexity and per-request overhead.
- **Dynamic via `auth_request` to API (rejected):** Adds API dependency in request path.
- **subroute + errors for SPA (rejected):** Caddy v2 doesn't support `errors` at the route level. `pass_thru` on `file_server` is the correct pattern.
105 changes: 28 additions & 77 deletions docs/adr/0012-build-pack-abstraction.md
Original file line number Diff line number Diff line change
@@ -1,115 +1,66 @@
# ADR-0012: Build Pack Abstraction — Five Build Pack Types
# ADR-0012: Build Pack Abstraction — Four Build Pack Types

## Status

Accepted (updated)

## Context

The original Shipyard spec scoped the project to static sites only, with a simple `package.json` dependency scan for framework detection (Vite, CRA, Next.js, Vue). During triage of Issue #3 (app creation), two things became clear:
The original Shipyard spec scoped the project to static sites only. During triage of Issue #3 (app creation), two things became clear:

1. The maintainer's goal is a general-purpose deployment platform (Coolify/Dokploy-like), not a static-site-only tool.
2. Platforms like Coolify use a **build pack** model where users choose how their project is built: Nixpacks (auto-detect), static (nginx), Dockerfile (custom), Docker Compose (multi-service), or pre-built Docker image.
1. The maintainer's goal is a general-purpose deployment platform (Coolify/Dokploy-like).
2. Platforms like Coolify use a **build pack** model where users choose how their project is built.

The build pack is defined as a PostgreSQL enum on the `apps` table:
Originally five build pack types were defined: `nixpacks`, `static`, `dockerfile`, `dockercompose`, `dockerimage`. The `static` and `nixpacks` packs were later merged — a "static site" is now a mode of nixpacks (isStatic=true), not a separate build path.

## Decision

### Build Pack Enum

```sql
CREATE TYPE build_pack AS ENUM (
'nixpacks',
'static',
'dockerfile',
'dockercompose',
'dockerimage'
);
```

## Decision

### Build Pack Enum

The deployment engine routes to one of five implementations based on the app's `build_pack` field:

| Value | Description | Use Case |
|-------|-------------|----------|
| `nixpacks` | Nixpacks auto-detects framework from repo, generates Dockerfile, builds and runs it | Most applications; zero-config deployments |
| `static` | nginx:alpine serves pre-built static assets | SPAs, documentation sites, plain HTML |
| `nixpacks` | Nixpacks auto-detects framework, builds image. Serves via nginx (isStatic=true) or runs as container (isStatic=false) | Most applications; zero-config deployments |
| `dockerfile` | User provides Dockerfile in repo, Shipyard builds and runs it | Custom builds with specific OS deps |
| `dockercompose` | User provides docker-compose.yml, Shipyard deploys the stack | Multi-service apps with bundled services |
| `dockerimage` | Pull a pre-built image from a registry and run it | Deploy without source/build pipeline |

### Build Pack Interface

```typescript
interface BuildPackResult {
imageName: string; // Built or pulled Docker image name
containerPort: number; // Port to expose
outputDir?: string; // For static: dir to pull assets from
healthCheck?: string; // Optional health check path
composeFile?: string; // For dockercompose: the compose file content
}
```

### Build Pack Implementations

#### 1. Nixpacks

- **Flow:** Git clone → `nixpacks build .` (generates Dockerfile) → `docker build` → `docker run`
- **Detection:** Nixpacks auto-detects framework from repo contents (Next.js, Django, Rails, Go, etc.)
- **Config:** User can override build/start commands and output directory
- **Dependency:** Requires `nixpacks` binary installed on the worker host

#### 2. Static

- **Base image:** `nginx:alpine`
- **Flow:** Clone → (optional build command) → copy assets from `output_dir` to `/usr/share/nginx/html/` → serve on port 80
- **Config:** User specifies `output_dir` (default: `/dist`), SPA fallback toggle, custom nginx config
- **No framework detection** — assumes pre-built assets

#### 3. Dockerfile
### Nixpacks Strategy

- **Flow:** Git clone → `docker build -f Dockerfile` → `docker run`
- **Config:** User commits a `Dockerfile` in their repo. Shipyard builds it as-is.
- **Port:** User configures the container port Shipyard should route to.
Nixpacks handles both static sites and server apps:

#### 4. Docker Compose
| Mode | `isStatic` | Flow | Caddy Route |
|------|-----------|------|-------------|
| Static site | `true` (default) | nixpacks build → extract output dir → serve via nginx file_server with pass_thru SPA fallback | File route |
| Server app | `false` | nixpacks build → run long-lived container → reverse proxy | Proxy route |

- **Flow:** Git clone → `docker compose -f docker-compose.yml up -d`
- **Config:** User provides `docker-compose.yml` in their repo.
- **Use case:** Apps that bundle a database, cache, or need multiple services.
- **Routing:** Caddy routes to the primary service's port.
**Flow:** Git clone → `nixpacks build --cache-key shipyard-{appId} --inline-cache` → (static: extract /app/{outputDir} to sites dir) or (server: runLongLived) → Caddy route

#### 5. Docker Image
**Caching:** `--cache-key` uses a stable per-app identifier so nixpacks restores `~/.npm` and `~/.cache` between deploys. `--inline-cache` embeds Docker layer metadata for faster rebuilds.

- **Flow:** `docker pull <image>` → `docker run`
- **Config:** User specifies image name + tag from a registry (Docker Hub, GHCR, etc.)
- **No repository needed** — deploy without connecting a git repo.
- **Use case:** Deploy existing images without rebuilding from source.
**Subdirectory support:** `--subdirectory` clones the full repo but nixpacks builds from `{repo}/{subdirectory}`. Used for monorepos.

### User Configuration Per App
### SPA Fallback

| Field | nixpacks | static | dockerfile | dockercompose | dockerimage |
|-------|----------|--------|------------|---------------|-------------|
| `output_dir` | Optional | Required | N/A | N/A | N/A |
| `build_command` | Override | Optional | N/A | N/A | N/A |
| `run_command` | Override | N/A | N/A | N/A | N/A |
| `port` | Optional | 80 | Required | Optional | Required |
| `dockerfile_path` | N/A | N/A | `./Dockerfile` | N/A | N/A |
| `image` | N/A | N/A | N/A | N/A | Required |
| `compose_file` | N/A | N/A | N/A | `docker-compose.yml` | N/A |
| `is_spa` | N/A | Configurable | N/A | N/A | N/A |
| `custom_nginx_config` | N/A | Optional | N/A | N/A | N/A |
Static sites with `isSpa=true` use Caddy's `file_server` with `pass_thru: true`, followed by a rewrite to `/index.html`. This avoids the broken subroute+errors pattern that doesn't work in Caddy v2.

## Consequences

- **Five build paths** share the same deployment state machine. Only execution differs.
- **No migration cost** when adding a new build pack — implement the interface, add the enum value.
- **Nixpacks is optional** — don't need the binary unless using that pack.
- **Docker Compose adds significant complexity** — multi-container lifecycle, networking, service dependencies. Post-MVP.
- **Docker Image is the simplest** — just pull and run. No build step at all.
- **Container routing required** — once you support long-lived containers, Caddy needs to reverse-proxy to them. ADR-0006 needs updating.
- **Four build paths** share the same deployment state machine. Only execution differs.
- **Nixpacks replaces the static build pack** — one fewer strategy to maintain.
- **Build caching** speeds up repeated deploys (~60% faster on npm installs).
- **Old step files** (clone, install, build, verify, copy) are marked dead code and will be removed after nixpacks stabilises.

## Alternatives Considered

- **Three packs only (static, dockerfile, nixpacks) (rejected):** Misses dockercompose for multi-service apps and dockerimage for pre-built image deploys.
- **Merge dockerimage into dockerfile (rejected):** Different semantics — one builds from source, one pulls a pre-built artifact. Different validation, different config.
- **Skip dockercompose for MVP (accepted):** Will be implemented after the single-container packs are stable. Multi-container orchestration is a separate complexity class.
- **Keep static as separate pack (rejected):** Duplicate of nixpacks with isStatic=true. Manual step pipeline was fragile (missing package managers like pnpm).
- **Railpack instead of Nixpacks (noted):** Nixpacks is in maintenance mode, Railpack is the successor. Will migrate when Railpack matures.
- **--provider docker flag (noted):** May be needed to bypass broken Nix derivations for certain language packs like bun.
6 changes: 6 additions & 0 deletions drizzle/0007_medical_terror.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
ALTER TABLE "apps" ALTER COLUMN "build_pack" SET DEFAULT 'nixpacks';--> statement-breakpoint
ALTER TABLE "apps" ADD COLUMN "is_static" boolean DEFAULT true;--> statement-breakpoint
ALTER TABLE "public"."apps" ALTER COLUMN "build_pack" SET DATA TYPE text;--> statement-breakpoint
DROP TYPE "public"."build_pack";--> statement-breakpoint
CREATE TYPE "public"."build_pack" AS ENUM('nixpacks', 'dockerfile', 'dockercompose', 'dockerimage');--> statement-breakpoint
ALTER TABLE "public"."apps" ALTER COLUMN "build_pack" SET DATA TYPE "public"."build_pack" USING "build_pack"::"public"."build_pack";
Loading
Loading