Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,18 @@ on:
jobs:
build:
runs-on: ubuntu-latest
services:
backup-postgres:
image: postgres:16-alpine
env:
POSTGRES_PASSWORD: backup-ci-only
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 5s
--health-timeout 5s
--health-retries 10
steps:
- uses: actions/checkout@v4

Expand All @@ -35,6 +47,18 @@ jobs:
- name: Test
run: go test ./...

- name: Verify backup command against PostgreSQL 16
run: |
CGO_ENABLED=0 go build -o "$RUNNER_TEMP/beacon-backup" ./cmd/beacon-backup
CGO_ENABLED=0 go test -c -o "$RUNNER_TEMP/backup.test" ./internal/backup
docker run --rm --network host \
-e PGHOST=127.0.0.1 -e PGUSER=postgres -e PGDATABASE=postgres \
-e PGPASSWORD=backup-ci-only -e PGSSLMODE=disable \
-e BEACON_BACKUP_TEST_POSTGRES=1 -e BEACON_BACKUP_TEST_BINARY=/beacon-backup \
-v "$RUNNER_TEMP/beacon-backup:/beacon-backup:ro" \
-v "$RUNNER_TEMP/backup.test:/backup.test:ro" \
postgres:16-alpine /backup.test -test.run '^TestExportPostgres$' -test.v

- name: Install govulncheck
run: go install golang.org/x/vuln/cmd/govulncheck@latest

Expand All @@ -54,4 +78,3 @@ jobs:
exit 1
fi
rm docs/swagger.json.bak

4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ in PostgreSQL, and streams live events to WebSocket clients.

For deployment instructions including the frontend app, see the deployment docs.

For a bounded private database and saved-config bundle, see
[backup export](docs/backup-export.md). This is a standalone export tool; the
backup web interface and import workflow are separate follow-ups.

---

## Stack
Expand Down
39 changes: 39 additions & 0 deletions cmd/beacon-backup/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
// Copyright 2026 Beacon Contributors
// SPDX-License-Identifier: AGPL-3.0-or-later

// beacon-backup creates a private database and saved-config export.
package main

import (
"context"
"flag"
"fmt"
"os"
"os/signal"
"syscall"

"github.com/MeshCore-Beacon/beacon-server/internal/backup"
)

var version = "dev"

func main() {
var opts backup.Options
flag.StringVar(&opts.ConfigPath, "config", "config.yaml", "saved YAML file to include verbatim (may contain secrets)")
flag.StringVar(&opts.OutputPath, "output", "", "new private .tar.gz destination (required; never overwritten)")
flag.Int64Var(&opts.MaxBytes, "max-bytes", backup.DefaultMaxBytes, "maximum uncompressed database dump size")
flag.DurationVar(&opts.Timeout, "timeout", backup.DefaultTimeout, "maximum export duration")
flag.Parse()
if flag.NArg() != 0 {
fmt.Fprintln(os.Stderr, "unexpected positional arguments; connection settings use PG* environment variables")
os.Exit(2)
}
opts.Version = version
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
if err := backup.Export(ctx, opts); err != nil {
fmt.Fprintln(os.Stderr, "backup failed:", err)
os.Exit(1)
}
fmt.Println("Backup complete. Store this bundle privately; it may contain keys and message data.")
}
89 changes: 89 additions & 0 deletions docs/backup-export.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Database and saved-config export

`beacon-backup` is the export foundation for issue #72. It produces a private,
versioned `.tar.gz` using PostgreSQL's `pg_dump`. It does not yet provide a web
interface, account login, scheduled/remote storage or automatic import.

Build it with `go build ./cmd/beacon-backup`. Install `pg_dump` in the same runtime
as this command; an installation on the Docker host does not install it inside an
app container. Use a client of the same major version as the source PostgreSQL
server; an older client cannot dump a newer server.

Set libpq's standard `PGHOST`, `PGPORT`, `PGDATABASE`, `PGUSER` and TLS settings.
`PGDATABASE` is required and must be a plain database name. Prefer a private
`PGPASSFILE` (0600 on Unix) or an existing libpq service configuration for secrets.
The command does not load `.env`, read `POSTGRES_DSN`, start Beacon, run migrations,
subscribe to MQTT or connect to Redis. Connection settings are not command-line
arguments and client stderr is not printed because it can contain private data.

For example, after configuring those connection settings:

```sh
beacon-backup -config /private/config.yaml -output /private/beacon-20260913.tar.gz
```

Use an output directory controlled by the operator. Staging directories are
0700 and files are 0600 on Unix; Windows operators must use a directory with
appropriately restricted ACLs. Existing destinations, including symlinks, are
never replaced. A hard link publishes the finished archive atomically, so the
destination filesystem must support hard links. Unsupported filesystems fail
without publishing an output. Normal failure, timeout and handled interruption
remove temporary files; a power loss or SIGKILL can leave a private
`.beacon-backup-*` staging directory for the operator to inspect.

## Format 1

The tar contains exactly three regular files with fixed names:

- `manifest.json`: format version, creation time, exporter version, dump format,
uncompressed payload sizes/SHA-256 hashes and explicit exclusions.
- `database.sql`: one consistent `pg_dump` snapshot of schema and data, including
Beacon's migration journal. Ownership, ACLs and tablespace placement are omitted
so objects can be restored under the destination operator.
- `config.yaml`: the supplied saved file, byte-for-byte, including comments and
any keys. It is captured before the database snapshot; avoid configuration edits
during export if the files must describe the same deployment state.

This is sensitive, unencrypted data. Store and transfer it privately. The bundle
does not include deployment environment variables, `.env`, external files such as
`borderFile` inputs or TLS keys, runtime-only changes, PostgreSQL roles/cluster
settings, Redis or service/deployment files. Retain those separately. A config
export is not the sanitized admin-config response and is not a complete server
recovery package by itself.

Defaults are ten minutes and a 1 GiB uncompressed SQL limit. Saved YAML is capped
at 1 MiB. `-timeout` and `-max-bytes` set finite positive limits (SQL maximum 1 TiB).
Allow disk space for both the uncompressed SQL and compressed bundle, roughly
twice the chosen SQL limit plus overhead. A five-second lock-wait limit prevents
waiting indefinitely behind schema changes. Export failure publishes no backup;
check the client version, privileges, connection settings and available capacity
privately. The command deliberately does not expose raw client diagnostics.

## Restore verification

Use trusted bundles only: PostgreSQL dumps can contain executable SQL. Inspect
the fixed members, verify the gzip stream and the manifest's payload sizes and
hashes, and extract into private staging. For a **new, empty disposable database**,
with its own explicit `PGDATABASE` and target-role connection settings:

```sh
psql -X --set ON_ERROR_STOP=on --single-transaction --file database.sql
```

Restore with a compatible PostgreSQL version and the required extensions already
available. Reconcile the migration journal and representative records/relationships
before relying on the bundle. Review saved configuration and restore external
secrets/files separately before starting a new Beacon server. Never restore over
live data simply to test an export; overwrite/import requires a separate workflow.

PostgreSQL references: [pg_dump](https://www.postgresql.org/docs/16/app-pgdump.html),
[connection environment](https://www.postgresql.org/docs/16/libpq-envars.html),
[password files](https://www.postgresql.org/docs/16/libpq-pgpass.html).

CI runs the compiled command and its PostgreSQL round-trip test with a dedicated
PostgreSQL 16 service. To repeat it privately, build the command, set the `PG*`
connection variables for an isolated test server and set
`BEACON_BACKUP_TEST_POSTGRES=1` plus `BEACON_BACKUP_TEST_BINARY` to the command's
absolute path. Run `go test ./internal/backup -run '^TestExportPostgres$' -v`.
The test role needs permission to create/drop its two randomly named databases;
the test migrates and restores only those databases and removes them afterward.
Loading
Loading