Skip to content

deploy.md

Latest commit

 

History

History
252 lines (210 loc) · 16.6 KB

deploy.md

File metadata and controls

252 lines (210 loc) · 16.6 KB
Raw

Phase 7 — Hardening & deployment

Stack (docker-compose)

  • postgres — primary store (projects/members/topics/comments/viewpoints/attachments/audit).
  • minio — object storage (source IFC, .frag tiles, props.json, attachments).
  • api — FastAPI; image bundles services/data so exports/clash/validate/drawings work.
  • worker — the durable job queue, in its own process (python -m aec_api.worker). Not optional under this compose file: it sets AEC_JOB_WORKER=off on the API, so with no worker running every enqueue succeeds, every row is written, every caller is told the work is under way, and none of it happens. Nothing raises. Scale it independently — --scale worker=3 — when conversions back up. Serves nothing, so it binds no port and its healthcheck is disabled.
  • web — Vite build served by nginx (COOP/COEP headers for web-ifc threading).
  • converter — Node IFC→Fragments, run as a job (docker compose run).
# core stack (api + worker + web + postgres + minio)
docker compose --profile full up --build
#   web → http://localhost:8080    api → http://localhost:8000    minio console → :9001

# convert a model (drop it in ./data first)
docker compose --profile tools run --rm converter /data/model.ifc /data/model.frag

# enforce project roles
AEC_RBAC=1 docker compose --profile full up --build

# smoke test a running stack
API=http://localhost:8000 bash scripts/smoke-stack.sh samples/school_str.ifc

Configuration (env)

Var Service Purpose
DATABASE_URL api postgresql+psycopg://… (sqlite if unset, dev)
S3_ENDPOINT / S3_ACCESS_KEY / S3_SECRET_KEY / S3_BUCKET api MinIO/S3 object storage; unset → local STORAGE_DIR
AEC_RBAC api 1 enforces project-scoped roles
AEC_API_KEY api bearer treated as admin (service-to-service)
AEC_AUTH_SECRET api set in prod — signs auth tokens; unset = forgeable dev secret (logs a warning)
AEC_RATE_LIMIT_RPM api global per-IP request cap/min (0/unset = off); needs AEC_REDIS_URL to share across workers
AEC_REDIS_URL api Redis for the shared rate-limit + login-lockout counters (fail-open to in-process)
AEC_THROTTLE_REVIEW_RPM api per-caller cap on AI-review endpoints (default 30; 0 disables)
AEC_THROTTLE_CONVERT_RPM api per-caller cap on convert endpoints (default 12; 0 disables)
AEC_PROPS_CACHE_PROJECTS api in-process property-index LRU size (default 16 projects/worker)
SPECKLE_SERVER / SPECKLE_TOKEN api optional Speckle bridge (must be https://; see SSRF note below)
SPECKLE_ALLOW_PRIVATE api 1 allows a private/LAN Speckle host (otherwise SSRF guard blocks non-public IPs)
APS_CLIENT_ID / APS_CLIENT_SECRET api optional Autodesk APS bridge (RVT→IFC, paid, per-translation cost)

SSRF note: SPECKLE_SERVER is admin-settable from the Settings UI, so the bridge requires https:// and refuses hosts that resolve to private/loopback/link-local/cloud-metadata addresses before making any request. Point it at a public Speckle host, or set SPECKLE_ALLOW_PRIVATE=1 for a trusted self-hosted server on your LAN.

Auth & roles (RBAC)

Project-scoped roles, least→most: viewer < reviewer < editor < admin (rbac.py).

  • viewer: read models/properties/issues/drawings/exports
  • reviewer: + create/comment topics & viewpoints, attachments (RFIs, markup)
  • editor: + author IFC (/edit, /publish), clash-with-topics, BCF import
  • admin: + project settings, manage members The project creator becomes admin. Caller identified by X-User (swap for your IdP/JWT in prod). Off by default (AEC_RBAC unset) so local dev stays open. Verified: test_rbac.py.

Accounts & identity (independent of AEC_RBAC): the built-in auth issues signed bearer tokens + an httpOnly cookie (auth.py). The first /auth/register bootstraps a global admin; after that, admins manage accounts via /auth/users (create / list / set role / activate / deactivate / reset password) — surfaced in the web app under the account menu → Manage users. Users change their own password at /auth/password (account menu → Change password). Deactivating an account blocks new logins and invalidates its existing tokens immediately; the last active admin can't be removed. Verified: test_auth.py.

Object storage & streaming

storage.py has Local and S3 (boto3) backends behind one interface incl. byte-range reads. .frag tiles and attachments are served with HTTP range requests (serving.py): 206 Partial Content, Accept-Ranges, Content-Range, immutable cache headers — so the viewer/ CDN stream large models. Verified: test_serving.py (200 full / 206 ranged / 416).

Note: /publish reconvert spawns the Node converter; in the container that step is best-effort (reindex always runs). For prod, run conversion via the converter service and write the .frag to MinIO under <project_id>/model.frag.

Audit & backups

Every write records an AuditLog row (actor, action, method, path, topic, detail) — RFIs/ punchlist are contractual records. The DB + attachments + uploaded source IFCs are the system of record; .frag tiles are reproducible from source IFC.

Backup/restore runbook (scripts/backup.sh / scripts/restore.sh, run from the repo root with the stack up; Windows: Git Bash or WSL):

# back up DB (pg_dump) + MinIO objects + uploaded source IFCs → one timestamped tarball
./scripts/backup.sh                      # → ./backups/aec-backup-<ts>.tgz

# schedule it (crontab): nightly at 02:00, then prune backups older than 14 days
0 2 * * *  cd /srv/massing && ./scripts/backup.sh >> /var/log/aec-backup.log 2>&1
0 3 * * *  find /srv/massing/backups -name 'aec-backup-*.tgz' -mtime +14 -delete

# restore (DESTRUCTIVE — overwrites DB, objects, IFCs; stops the app while restoring)
./scripts/restore.sh ./backups/aec-backup-<ts>.tgz

backup.sh logically dumps Postgres (pg_dump --clean) and tars the MinIO + IFC volumes via a throwaway alpine container (--volumes-from, so no volume-name or S3-credential coupling). Verify a backup by restoring into a throwaway compose project and hitting /health + a known project before trusting it. Keep backups off-box (e.g. aws s3 cp, rclone) for DR.

Deploy to a cloud VM with HTTPS (turnkey demo / production)

The base stack runs anywhere Docker does. For a public, TLS-secured demo, layer the production overlay — it adds a Caddy reverse proxy that fetches + renews a Let's Encrypt cert automatically, enforces auth (AEC_RBAC=1), and sets restart policies.

Before exposing anything, work through PRODUCTION_CHECKLIST.md and run the preflight: docker compose … run --rm api python /app/scripts/validate_prod_config.py (exit 0 = go). The API also refuses to boot on Postgres without RBAC + a real auth secret.

# 1. a small VM (1–2 vCPU, 2–4 GB) with Docker + a DNS A record for your domain → the VM IP
# 2. firewall: allow only 80 + 443
sudo ufw allow OpenSSH && sudo ufw allow 80,443/tcp && sudo ufw enable

# 3. clone + configure secrets (REQUIRED in prod)
git clone https://github.com/ibuilder/massing.git && cd Massing
cp .env.example .env       # set POSTGRES_PASSWORD, S3_ACCESS_KEY/SECRET, AEC_API_KEY, AEC_AUTH_SECRET

# 4. bring it up behind Caddy (auto-HTTPS for $DOMAIN)
DOMAIN=app.example.com docker compose -f docker-compose.yml -f docker-compose.prod.yml \
    --profile full up -d --build

# 5. create the first admin (bootstraps as admin), then sign in at https://app.example.com
curl -s -X POST https://app.example.com/api/auth/register \
    -H "Content-Type: application/json" -d '{"username":"admin","password":"<strong-password>"}'

# optional: seed a demo project across all relation chains
DOMAIN=app.example.com docker compose -f docker-compose.yml -f docker-compose.prod.yml \
    --profile full --profile seed run --rm seed

Caddy is the only public entrypoint (web/api/postgres/minio stay on the Docker network — keep 8000/8080/5432/9000 off the public firewall). Cookie auth + the SSE feed + downloads all work because everything is same-origin behind the proxy. Managed hosts (Fly.io, Render, Railway) work too — run the same images, swap MinIO for their S3-compatible bucket + managed Postgres, and front the web service with their TLS.

GitHub note: Pages is static-only and can't host this stack (it needs the API + Postgres

  • MinIO). Pages can serve the marketing page (docs/index.html) and, with extra setup, a viewer-only build; the full app needs a Docker host as above.

Free single-project desktop app (.exe)

The whole platform runs in one process for a single operator — FastAPI serving both the API and the web SPA on 127.0.0.1:8765, backed by SQLite + local files (no Docker / Postgres / MinIO), in local mode (no login — the operator owns the one site). This is the free, single-seat desktop build; projects are saved/opened as portable .mmproj bundles (Open/Save menu).

  • Run from source: python -m aec_api.desktop (from services/api, with the venv) — builds the web first with npm run build:desktop so the SPA calls the same-origin API (.env.desktop sets VITE_API_URL=).
  • Package the .exe: services/api/build-desktop.ps1 runs the desktop web build, then PyInstaller (desktop.spec) into services/api/dist_desktop/AEC-BIM/AEC-BIM.exe. The bundle includes the SPA (web/), the 68 module definitions (modules/), and ifcopenshell. Data lives under %LOCALAPPDATA%\AEC-BIM (override with AEC_DATA_DIR); uninstall = delete that folder + the app.
  • Env knobs: AEC_PORT (8765), AEC_OPEN_BROWSER (1), AEC_DATA_DIR, AEC_HOST. The frozen build resolves the bundled SPA + module catalog via _MEIPASS (AEC_WEB_DIST / AEC_MODULES_DIR).

A native-window wrapper (Tauri, below) can host this same backend; the PyInstaller .exe already ships a complete, double-clickable app on its own (it opens the system browser).

Auto-update

  • Update check (built-in, no setup): on launch the app checks the project's GitHub Releases for a newer version than the one baked in at build (VITE_APP_VERSION ← package.json), and shows a dismissible banner with a download link; there's also a manual "Check for updates" in Settings (apps/web/src/ui/update.ts). Works for the .exe, the Tauri build, and the browser. It links to the new installer rather than hot-swapping (an OS can't replace a running .exe in place).
  • Silent self-install (Tauri, optional): for true in-app install, enable the Tauri updater — it requires a signing keypair:
    1. npx @tauri-apps/cli signer generate -w aec.key → keep the public key, guard the private one.
    2. Put the public key in tauri.conf.json under plugins.updater.pubkey and set bundle.createUpdaterArtifacts: true + plugins.updater.endpoints to the release latest.json.
    3. Add repo secrets TAURI_SIGNING_PRIVATE_KEY + TAURI_SIGNING_PRIVATE_KEY_PASSWORD and pass them to the tauri-action step (tauri-action then emits the signed updater bundles + latest.json). Not enabled by default because createUpdaterArtifacts makes the build require the signing key — so flip it on only once the secret is configured (otherwise the release build fails).

Mobile apps (Capacitor — iOS / Android)

The same web build (dist) wraps into native iOS/Android apps via Capacitor (capacitor.config.ts, appId com.ibuilder.aecbim). A phone has no local Python backend, so a mobile build must target a hosted API.

  1. Point at your API: edit apps/web/.env.mobile → VITE_API_URL=https://your-cloud-api (blank = viewer-only offline build; the portal/proforma need the API).
  2. Build the web + sync: npm run build:mobile then npx cap sync (or the one-shot npm run mobile:android / npm run mobile:ios).
  3. Add a platform once: npx cap add android / npx cap add ios (folders are gitignored; regenerate anytime).
  4. Build the binary (your toolchain): Android needs the Android SDK + JDK (cap open android → Gradle build/APK/AAB); iOS needs macOS + Xcode (cap open ios).
  5. Validate on device: the model viewer uses threaded WASM (web-ifc, SharedArrayBuffer) — confirm it runs in the device WebView before relying on 3D; the portal/proforma/2D work everywhere.
  6. Store submission: Apple Developer + Google Play accounts (signing + listings are store-side).

Scaffolding is wired and verified (cap add android syncs the web build); the binary build, on-device WASM validation, and store accounts are the external pieces.

Desktop installers (Tauri) & code signing

.github/workflows/desktop.yml builds Windows / macOS (arm64) / Linux installers. Push a tag (v0.1.0) for a draft Release; run it manually (Actions → Desktop release → Run workflow) for artifact-only smoke builds. Without the secrets below, builds are unsigned (Gatekeeper / SmartScreen warn on first launch) — everything still works; add the secrets to sign + notarize.

Native window + bundled backend (sidecar). The Tauri shell (src/lib.rs) spawns the Python backend as a sidecar (binaries/aec-bim-server, declared in tauri.conf.json externalBin), waits for 127.0.0.1:8765, then points the WebView at it — so the installed app is the full platform (API + SPA + SQLite, local mode), same-origin, fully offline, in a native window. CI builds that sidecar per-platform with services/api/build_sidecar.py (PyInstaller can't cross-compile, so each runner builds its own; the binary is named with the Rust target triple). The shell's beforeBuildCommand is npm run build:desktop so the bundled SPA targets the same-origin API. To build locally: npm run build:desktop → python services/api/build_sidecar.py → npm --prefix apps/web run tauri build (needs the Rust toolchain).

Platform Repo secrets Notes
macOS APPLE_CERTIFICATE (base64 of a Developer ID .p12), APPLE_CERTIFICATE_PASSWORD, APPLE_SIGNING_IDENTITY (e.g. Developer ID Application: Name (TEAMID)), APPLE_ID, APPLE_PASSWORD (app-specific password), APPLE_TEAM_ID tauri-action imports the cert and notarizes automatically when all are set.
Windows WINDOWS_CERTIFICATE (base64 of an Authenticode .pfx), WINDOWS_CERTIFICATE_PASSWORD The workflow imports the PFX and writes its thumbprint into tauri.conf.json before building. For an EV/HSM or Azure Trusted Signing cert, replace that step with the vendor's signtool flow.
Linux — .deb/.AppImage are not signed; distribute over HTTPS / via checksums.

Generate APPLE_CERTIFICATE / WINDOWS_CERTIFICATE with base64 -w0 cert.p12 (Linux) or [Convert]::ToBase64String([IO.File]::ReadAllBytes("cert.pfx")) (PowerShell), and add them under Settings → Secrets and variables → Actions. The workflow then signs automatically on the next tag — no code change needed (the import + thumbprint + notarize steps are already wired and guarded so unsigned builds stay green until the secrets exist).

Where to get the certs (the only external piece):

  • Windows — an Authenticode code-signing cert from a CA (DigiCert, Sectigo, SSL.com; OV ≈ $200–400/yr). OV still shows a SmartScreen prompt until it earns reputation; an EV cert (HSM/ token) or Azure Trusted Signing (~$10/mo, cloud, no physical token) clears SmartScreen immediately — for those, swap the PFX-import step for the vendor's signtool/Trusted-Signing action.
  • macOS — Apple Developer Program ($99/yr) → a Developer ID Application cert (export .p12)
    • an app-specific password for notarization. Fills all six APPLE_* secrets above.
  • Linux — .deb/.AppImage aren't OS-signed; publish SHA-256 checksums (GitHub shows asset digests) and/or GPG-detach-sign if your channel needs it.

Observability

GET /metrics exposes Prometheus text (request counts + latency summary by method/route template, in-flight gauge, uptime) — point a Prometheus scrape at it. Each request also emits a structured JSON access line on the aec.access logger (method/route/status/dur_ms) for log aggregation. Metrics are per-process; with multiple uvicorn workers use a multiprocess collector or scrape each worker. (Endpoint is unauthenticated — keep it on an internal network or front it with auth at the proxy.)

Offline / jobsite

web-ifc WASM + the Fragments worker are bundled into the web image; tiles serve from your own MinIO. No external CDN — the viewer runs fully offline.

Licensing — see ../THIRD-PARTY-NOTICES.md

Bonsai/Blender GPL (separate process), IfcOpenShell LGPL, That Open MIT-style.