Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 168 additions & 2 deletions arcane/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,21 @@ Do not recursively change restored database ownership.

## Quick Start

These steps are for an **empty** database directory. If PostgreSQL 17 has ever
used `/mnt/SSD/Containers/arcane-postgres`, follow
[the major upgrade procedure](#postgresql-17-to-18) instead. Changing the image
tag or moving database files does not convert a PostgreSQL 17 cluster to 18.

The reviewed images are Arcane **v2.17.1** and PostgreSQL **18.6 Alpine**, frozen
by multi-platform digest in Compose. On 2026-10-31, the Arcane `latest` and
`v2.17.1` tags resolved to the same digest; freezing it does not establish what
an existing installation is running. Arcane's tagged
[PostgreSQL test stack](https://github.com/getarcaneapp/arcane/blob/v2.17.1/tests/setup/compose-postgres.yaml)
uses PostgreSQL 18 and its
[CI matrix](https://github.com/getarcaneapp/arcane/blob/v2.17.1/.github/workflows/ci.yml)
runs that stack. This is compatibility evidence for this release, not a
guarantee for older deployed versions.

### 1. Copy the environment template

```bash
Expand Down Expand Up @@ -312,8 +327,10 @@ PostgreSQL initialization behavior is documented by the

### Updating Arcane

Select a reviewed compatible exact release first; this stack still needs
running-image evidence before its floating references can be frozen safely.
Review an exact release and update its digest deliberately. Pulling the pinned
references below fetches the reviewed content; it does not select a newer
release. For an existing installation, record its running image first and
review intervening application migrations separately from a database upgrade.

```bash
# Pull only after reviewing image changes and migrations
Expand All @@ -326,6 +343,155 @@ docker compose up -d
docker compose logs -f arcane
```

### PostgreSQL 17 to 18

**Do not auto-deploy this major upgrade onto existing data.** Pause Git polling,
image updaters and scheduled Arcane operations first. Rehearse with an isolated
restore before scheduling downtime. The commands below are for the dedicated
single-database stack in this repository, in one root Bash session on the Linux
Docker host, from the **currently deployed PostgreSQL 17** `arcane/` directory.
Verify the Docker context and storage paths privately first. Do not run these
commands merely to review the PR.

This procedure keeps the Arcane version and original keys unchanged while
moving its database. It requires the deployed Arcane image to already match
the pinned v2.17.1 content. If it differs, stop and plan/test that application
upgrade independently on PostgreSQL 17 first. If the database has extra roles,
databases, tablespaces, extensions or custom authentication/configuration,
prepare a tailored migration: this single-database dump does not preserve
cluster-wide roles or server configuration. Keep an independent, verified
backup of both data paths and original settings before beginning.

PostgreSQL documents [logical dump/restore for major upgrades](https://www.postgresql.org/docs/18/upgrading.html).
The [official container image](https://hub.docker.com/_/postgres) changes the
mount to `/var/lib/postgresql` for 18, with data under `18/docker`. The old
`/var/lib/postgresql/data` mount must not be carried forward, nor should
`PGDATA` be overridden to make 18 open 17's files.

1. Record the deployed configuration and immutable local images **before
replacing Compose**. The backup contains secrets; keep it private and
encrypt it for off-host storage. Ensure sufficient free space for the
dump, a cold Arcane copy, image archive and a second database cluster.

```bash
set -eu
umask 077
command -v mountpoint >/dev/null
if mountpoint -q /mnt/SSD/Containers/arcane-postgres || \
mountpoint -q /mnt/SSD/Containers/arcane; then
echo 'Stop: data paths include a mountpoint; use a dataset-aware snapshot/clone migration.' >&2
exit 1
fi
stamp=$(date -u +%Y%m%dT%H%M%SZ)
backup="/mnt/SSD/Backups/arcane-pg18-$stamp"
old_db_dir="/mnt/SSD/Containers/arcane-postgres.pg17-$stamp"
test ! -e "$backup"
test ! -e "$old_db_dir"
test "$(cat /mnt/SSD/Containers/arcane-postgres/PG_VERSION)" = 17
mkdir -p "$backup"
cp -p .env "$backup/settings.env"
docker compose config > "$backup/compose.pg17.yaml"
old_arcane=$(docker inspect --format '{{.Image}}' arcane)
old_postgres=$(docker inspect --format '{{.Image}}' arcane-postgres)
project=$(docker inspect --format '{{index .Config.Labels "com.docker.compose.project"}}' arcane-postgres)
test -n "$project"
test "$project" = "$(docker inspect --format '{{index .Config.Labels "com.docker.compose.project"}}' arcane)"
docker pull ghcr.io/getarcaneapp/arcane:v2.17.1@sha256:3d7057b7e3b3e0be4b3ba34dd1b137b02a055f97ddc232c32817666658643842
reviewed_arcane=$(docker image inspect --format '{{.Id}}' ghcr.io/getarcaneapp/arcane:v2.17.1@sha256:3d7057b7e3b3e0be4b3ba34dd1b137b02a055f97ddc232c32817666658643842)
test "$old_arcane" = "$reviewed_arcane"
printf 'services:\n arcane:\n image: %s\n postgres:\n image: %s\n' \
"$old_arcane" "$old_postgres" > "$backup/images.pg17.yaml"
printf '%s\n' "$project" > "$backup/project-name"
printf '%s\n' "$old_db_dir" > "$backup/old-db-path"
docker image save -o "$backup/images.tar" "$old_arcane" "$old_postgres"
```

The rename-based procedure below requires ordinary directories. If either
path is a mounted ZFS dataset, stop and prepare a dataset-aware snapshot/clone
and rollback procedure instead; do not rename a mounted dataset directory.

2. Stop Arcane and all other writers, then dump the still-running PostgreSQL
17 database. Stop on any error. Listing an archive is a format check; only
a successful isolated restore and application checks prove recoverability.

```bash
docker compose stop arcane
docker compose exec -T postgres sh -ec \
'pg_dump --format=custom -U "$POSTGRES_USER" -d "$POSTGRES_DB"' \
> "$backup/arcane.dump"
test -s "$backup/arcane.dump"
docker compose exec -T postgres pg_restore --list \
< "$backup/arcane.dump" > "$backup/dump-contents.txt"
docker compose down # Never add --volumes / -v
cp -a /mnt/SSD/Containers/arcane "$backup/arcane-data"
mv /mnt/SSD/Containers/arcane-postgres "$old_db_dir"
mkdir /mnt/SSD/Containers/arcane-postgres
```

Keep the entire old database directory intact with numeric ownership. The
new directory is empty; PostgreSQL initializes its own ownership. Do not
copy PostgreSQL 17 files into it or delete the existing named volume.

3. Replace the deployed `compose.yaml` with this reviewed PostgreSQL 18 version,
keeping the same `.env` and Compose project name. Do not start Arcane yet.
If initialization or restore fails, keep it stopped and use rollback.

```bash
docker compose -p "$project" config --quiet
docker compose -p "$project" pull postgres
docker compose -p "$project" up -d postgres
# Wait until this succeeds; inspect PostgreSQL logs if it does not.
docker compose -p "$project" exec -T postgres sh -ec \
'pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"'
docker compose -p "$project" exec -T postgres sh -ec \
'pg_restore --exit-on-error --single-transaction --no-owner -U "$POSTGRES_USER" -d "$POSTGRES_DB"' \
< "$backup/arcane.dump"
docker compose -p "$project" exec -T postgres sh -ec \
'psql -v ON_ERROR_STOP=1 -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "ANALYZE;" -c "SHOW server_version;" -c "SHOW data_directory;"'
```

Confirm the server is 18 and its data directory is
`/var/lib/postgresql/18/docker`. The restore puts objects under the original
configured database user; no password or encryption-key rotation is needed.

4. Start Arcane with `docker compose -p "$project" up -d arcane`. Keep other
access and automation paused while checking startup logs, login, projects,
settings, and access to saved encrypted credentials. Recreate PostgreSQL
and recheck those results to verify persistence. Only resume normal writes
and automation after acceptance. Retain the old directory and backup through
the agreed rollback window.

**Rollback before normal writes resume:** stop the 18 stack, preserve its tree,
restore the cold Arcane copy and use the original database, image IDs and
settings. Never point PostgreSQL 17 at the PostgreSQL 18 directory. In the
same session, with the saved `$backup`, `$project`, `$old_db_dir` and `$stamp`:

```bash
docker compose -p "$project" down # No -v
test ! -e "/mnt/SSD/Containers/arcane-postgres.pg18-failed-$stamp"
test ! -e "/mnt/SSD/Containers/arcane.pg18-failed-$stamp"
mv /mnt/SSD/Containers/arcane-postgres "/mnt/SSD/Containers/arcane-postgres.pg18-failed-$stamp"
mv "$old_db_dir" /mnt/SSD/Containers/arcane-postgres
mv /mnt/SSD/Containers/arcane "/mnt/SSD/Containers/arcane.pg18-failed-$stamp"
cp -a "$backup/arcane-data" /mnt/SSD/Containers/arcane
docker image load -i "$backup/images.tar"
docker compose -p "$project" --env-file "$backup/settings.env" \
-f "$backup/compose.pg17.yaml" -f "$backup/images.pg17.yaml" \
up -d --pull never postgres
# Wait for PostgreSQL readiness before starting Arcane.
docker compose -p "$project" --env-file "$backup/settings.env" \
-f "$backup/compose.pg17.yaml" -f "$backup/images.pg17.yaml" \
exec -T postgres sh -ec 'pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"'
docker compose -p "$project" --env-file "$backup/settings.env" \
-f "$backup/compose.pg17.yaml" -f "$backup/images.pg17.yaml" \
up -d --pull never arcane
```

Recheck login, projects and saved credentials. Restore the deployed Compose and
settings to the original version before allowing any polling/updater to resume.
If normal writes have already resumed on 18, this rollback discards those writes;
stop and plan reconciliation instead of treating the old directory as current.

### Backing Up Data

Your data is stored in:
Expand Down
10 changes: 6 additions & 4 deletions arcane/compose.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@

services:
arcane:
image: ghcr.io/getarcaneapp/arcane:latest
image: ghcr.io/getarcaneapp/arcane:v2.17.1@sha256:3d7057b7e3b3e0be4b3ba34dd1b137b02a055f97ddc232c32817666658643842
container_name: arcane
restart: unless-stopped
networks:
Expand All @@ -28,7 +28,7 @@ services:
- postgres

postgres:
image: postgres:17-alpine
image: postgres:18.6-alpine@sha256:10a5c69a83035ab5239f5620d708520e8b28f5aa1f2b20143e6024532edadfb5
container_name: arcane-postgres
restart: unless-stopped
networks:
Expand All @@ -38,7 +38,9 @@ services:
- POSTGRES_USER=${POSTGRES_USER:-arcane}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
volumes:
- postgres-data:/var/lib/postgresql/data
# PostgreSQL 18 stores PGDATA in /var/lib/postgresql/18/docker.
# Existing PostgreSQL 17 data MUST be dumped/restored first; see README.
- postgres-data:/var/lib/postgresql
#ports: # No port exposure needed - Arcane can access via arcane-network via postgres:5432
#- 5432:5432

Expand All @@ -58,4 +60,4 @@ volumes:

networks:
arcane-network:
driver: bridge
driver: bridge
Loading