Skip to content

README.md

Latest commit

 

History

History
529 lines (459 loc) · 28.4 KB

README.md

File metadata and controls

529 lines (459 loc) · 28.4 KB
Raw

Running Roomote on Railway

Two official templates are published, one per image channel:

Channel Tracks Deploy
main (stable) :main Deploy on Railway
develop (latest builds) :develop Deploy on Railway

This guide covers deploying Roomote on Railway — either through an official Roomote template or by composing the services manually. Both templates are mirrored from the same maintained service specification, template.yaml, and differ only in which image alias the five app services track; everything in this guide applies to both. Railway does not read that file directly, but it is the source of truth that maintainers mirror into Railway's Template Composer.

For single-host Docker deployments, use the one-command installer or the Compose paths in SELF_HOSTING.md instead.

How the Railway shape differs from single-host

  • No Caddy edge. Railway terminates HTTPS and gives every service its own public domain. The web app and the API run on separate origins, so TRPC_URL points at the api service's public domain with no /_roomote-api path prefix. GitHub webhooks and hosted-sandbox workers call that API origin directly, and Slack webhooks arrive at the web origin and are proxied to the API internally.
  • No Docker socket. The docker sandbox provider cannot run on Railway. Task execution must use a hosted sandbox provider: Modal (template default), E2B, or Daytona. Those only need outbound HTTPS plus API credentials. The template sets EXCLUDED_COMPUTE_PROVIDERS=docker so the unusable provider never appears in setup or sandbox selection.
  • No openssl provisioning step. The template sets R_AUTO_GENERATE_KEYS=true, so Roomote generates the JOB_AUTH_* / PREVIEW_AUTH_* P-256 keypairs at first boot and persists them encrypted (with ENCRYPTION_KEY) in Postgres. Every other secret is a random string that Railway's ${{secret(n)}} template function generates.
  • No template-level shared variables. Railway's Template Composer has no environment-level shared-variable store, so every value that must match across services is defined once on the api service and referenced from the other services as ${{api.NAME}}. Never paste ${{secret(n)}} into more than one service: each occurrence generates a different value, and ENCRYPTION_KEY must be identical everywhere.
  • Provider credentials live in the app, not the template. The template ships with zero required deploy-time inputs. Modal tokens and the model provider API key are entered in the /setup wizard (or Settings) after first boot and stored encrypted in Postgres. Optional deploy-screen inputs are R_APP_URL, for deployers who want a custom domain from the start (see Attaching a custom domain), and R_LICENSE_KEY, for operators provisioning a paid-seat license. An env license key takes precedence over a key stored through Settings > Users.
  • Live previews are off by default, but the proxy ships ready. Preview subdomains need a wildcard domain, and Railway-generated service domains are single-label. The preview-proxy service is part of the template and boots healthy with the PREVIEW_* variables empty; enabling previews later is a wildcard custom domain plus three variable values (see below). Everything else works without them.

What you need

Image access

Both published images (ghcr.io/roocodeinc/roomote-app and ghcr.io/roocodeinc/roomote-worker) are public. Railway pulls the app image anonymously, and hosted sandbox providers (Modal's remote builder, E2B/Daytona worker builds) pull the worker image anonymously; no registry credentials or MODAL_REGISTRY_USERNAME/MODAL_REGISTRY_PASSWORD are needed.

Image channel and versions

Each template tracks one mutable channel alias, which the publish workflow moves on every matching build:

  • The main-channel template tracks :main, moved on every build of the main branch (each build also publishes an immutable main-<sha> tag). This is the stable choice.
  • The develop-channel template tracks :develop, moved on every build of the develop branch (immutable tag: develop-<sha>). Tagged v* releases additionally move :latest.

Nothing else in either template encodes a version:

  • The images bake RELEASE_VERSION at build time, and the app derives DOCKER_WORKER_IMAGE and MODAL_BASE_IMAGE_REF from it when those are unset. Set them only to override the derived worker image.
  • The controller reads the worker release version from the VERSION file inside worker-current.tar.gz, so DOCKER_WORKER_RELEASE_PATH is a constant.

To pin instead (recommended for production deployments): put the same immutable tag (v*, main-<sha>, or develop-<sha>) in the five app-service image fields. No other edits are needed — the derived values follow the image.

Service topology

<channel> below is main or develop, per template.

Railway service Source Start command Public domain Healthcheck
Postgres Railway managed PostgreSQL — no managed
Redis Railway managed Redis — no managed
minio pinned roomote-minio + /data minio server /data --console-address :9001 yes (HTTP proxy port 9000) —
gbrain roomote-gbrain:<channel> + /data image entrypoint no /health
web roomote-app:<channel> /roomote/.docker/app/entrypoint.sh web yes (HTTP proxy port 8080) /health
api roomote-app:<channel> /roomote/.docker/app/entrypoint.sh api yes (HTTP proxy port 8080) /health/liveness
controller roomote-app:<channel> /roomote/.docker/app/entrypoint.sh controller no —
bullmq roomote-app:<channel> /roomote/.docker/app/entrypoint.sh bullmq no —
preview-proxy roomote-app:<channel> /roomote/.docker/app/entrypoint.sh preview-proxy yes (HTTP proxy port 8080) /health

Railway's custom start command bypasses the image entrypoint and executes the command directly, so the full /roomote/.docker/app/entrypoint.sh <service> form is required — a bare web fails with The executable `web` could not be found. The same applies to minio (minio server ..., not server ...) and to the api pre-deploy command below.

  • Rename the minio service to exactly minio: the image-derived default name contains a slash, which breaks ${{minio.RAILWAY_PRIVATE_DOMAIN}} references.
  • Set the api service's pre-deploy command to /roomote/.docker/app/entrypoint.sh db-migrate so schema migrations run before each new deploy starts serving. On a fresh project the other app services boot in parallel with that first migration pass and wait for it with bounded backoff (up to ~90 seconds, logging one [auth-keypairs] waiting-for-migrations line per retry) instead of crash-looping; Railway's restart policy still recovers them in the rare case migrations take longer.
  • The HTTP proxy port becomes the Railway-injected PORT, which all app services honor. Any value works for web/api (the template uses 8080); the minio proxy must target 9000 (the S3 API — the console on 9001 is not exposed).

Environment variables

The api service is the anchor: it defines every shared value once, and the other app services reference them with ${{api.NAME}}.

api:

R_APP_ENV=production
ROOMOTE_DOCKER_LOAD_ENV_FILE=false
R_AUTO_GENERATE_KEYS=true
ENCRYPTION_KEY=${{secret(32)}}
R_DISCORD_GATEWAY_SECRET=${{secret(32)}}
R_LICENSE_KEY=
ARTIFACT_SIGNING_KEY=${{secret(32)}}
DASHBOARD_PASSWORD=${{secret(24)}}
S3_SECRET_ACCESS_KEY=${{secret(32)}}
S3_ACCESS_KEY_ID=roomote
S3_REGION=us-east-1
S3_BUCKET_ARTIFACTS=roomote-artifacts
S3_AUTO_CREATE_BUCKET=true
SETUP_TOKEN=${{secret(32)}}
R_PING_BASE_URL=https://ping.roomote.dev
PREVIEW_PROXY_BASE_URL=
NEXT_PUBLIC_PREVIEW_PROXY_BASE_URL=
PREVIEW_DOMAINS=
DEFAULT_COMPUTE_PROVIDER=modal
EXCLUDED_COMPUTE_PROVIDERS=docker
DATABASE_URL=${{Postgres.DATABASE_URL}}
REDIS_URL=${{Redis.REDIS_URL}}
R_APP_URL=https://${{web.RAILWAY_PUBLIC_DOMAIN}}
TRPC_URL=https://${{api.RAILWAY_PUBLIC_DOMAIN}}
S3_ENDPOINT=http://${{minio.RAILWAY_PRIVATE_DOMAIN}}:9000
S3_PRESIGN_ENDPOINT=https://${{minio.RAILWAY_PUBLIC_DOMAIN}}

web, bullmq, and preview-proxy (identical block):

R_APP_ENV=${{api.R_APP_ENV}}
ROOMOTE_DOCKER_LOAD_ENV_FILE=${{api.ROOMOTE_DOCKER_LOAD_ENV_FILE}}
R_AUTO_GENERATE_KEYS=${{api.R_AUTO_GENERATE_KEYS}}
ENCRYPTION_KEY=${{api.ENCRYPTION_KEY}}
R_DISCORD_GATEWAY_SECRET=${{api.R_DISCORD_GATEWAY_SECRET}}
R_LICENSE_KEY=${{api.R_LICENSE_KEY}}
ARTIFACT_SIGNING_KEY=${{api.ARTIFACT_SIGNING_KEY}}
DASHBOARD_PASSWORD=${{api.DASHBOARD_PASSWORD}}
S3_SECRET_ACCESS_KEY=${{api.S3_SECRET_ACCESS_KEY}}
S3_ACCESS_KEY_ID=${{api.S3_ACCESS_KEY_ID}}
S3_REGION=${{api.S3_REGION}}
S3_BUCKET_ARTIFACTS=${{api.S3_BUCKET_ARTIFACTS}}
SETUP_TOKEN=${{api.SETUP_TOKEN}}
R_PING_BASE_URL=${{api.R_PING_BASE_URL}}
PREVIEW_PROXY_BASE_URL=${{api.PREVIEW_PROXY_BASE_URL}}
NEXT_PUBLIC_PREVIEW_PROXY_BASE_URL=${{api.NEXT_PUBLIC_PREVIEW_PROXY_BASE_URL}}
PREVIEW_DOMAINS=${{api.PREVIEW_DOMAINS}}
DEFAULT_COMPUTE_PROVIDER=${{api.DEFAULT_COMPUTE_PROVIDER}}
EXCLUDED_COMPUTE_PROVIDERS=${{api.EXCLUDED_COMPUTE_PROVIDERS}}
DATABASE_URL=${{Postgres.DATABASE_URL}}
REDIS_URL=${{Redis.REDIS_URL}}
R_APP_URL=${{api.R_APP_URL}}
TRPC_URL=${{api.TRPC_URL}}
S3_ENDPOINT=${{api.S3_ENDPOINT}}
S3_PRESIGN_ENDPOINT=${{api.S3_PRESIGN_ENDPOINT}}

controller: the same block plus

DOCKER_WORKER_RELEASE_PATH=/roomote/releases/worker-current.tar.gz

The worker release archive is baked into the app image, and the controller uploads it into hosted sandboxes at spawn time — no shared volume is needed. The version-less worker-current.tar.gz name works because the controller reads the release version from the VERSION file inside the archive. Do not leave DOCKER_WORKER_RELEASE_PATH unset: without it the controller falls back to fetching GitHub worker releases, which only exist for tagged v* releases — not for develop or main branch builds.

minio:

MINIO_ROOT_USER=${{api.S3_ACCESS_KEY_ID}}
MINIO_ROOT_PASSWORD=${{api.S3_SECRET_ACCESS_KEY}}

Notes:

  • R_APP_URL on api is the single canonical-origin knob: it is the URL users browse, and web/controller/bullmq/preview-proxy reference ${{api.R_APP_URL}} rather than repeating the value. It is also one of the template's optional deploy-time prompts — the deploy screen shows it pre-filled with the generated-domain reference so a custom domain can be entered before first boot. Do not set R_PUBLIC_URL — it is optional and the app falls back to R_APP_URL everywhere it would apply. See Attaching a custom domain.
  • Leave DOCKER_WORKER_IMAGE and MODAL_BASE_IMAGE_REF unset. The app derives both from the RELEASE_VERSION baked into the running image, so they always match the deployed build. Setting them explicitly overrides the derivation and silently pins the worker to whatever version the value encodes — the exact multi-place version bump this template design removes.
  • MODAL_TOKEN_ID, MODAL_TOKEN_SECRET, and model provider keys such as OPENROUTER_API_KEY are not template variables. Enter them in the /setup wizard (or Settings → Sandboxes / Models) after first boot; they are stored encrypted in Postgres. Setting them as env vars still works and takes precedence, but is unnecessary.
  • SETUP_TOKEN gates the pre-auth /setup wizard and is required: the app refuses tokenless first-admin bootstrap everywhere except local development, so a deployment without it cannot complete setup at all. The template generates it with ${{secret(32)}}. Visitors to /setup are prompted for the token, which the operator copies from the api service's Variables tab in the deployed project. The URL form /setup?token=<value> also works.
  • R_PING_BASE_URL is the endpoint for Roomote's anonymous analytics and version checks (the image default is https://ping.roomote.dev; the template points at the Roomote ping service). Admins can opt out of anonymous analytics in the setup wizard or Settings → Misc; version checks ignore that setting.
  • The three PREVIEW_* variables ship empty (mark them optional in the Template Composer so the deploy screen does not prompt for them). Roomote and the preview-proxy service boot fine with them empty; previews report as not configured in the task page's preview pane until the values are filled in on api (see Enabling live previews).
  • Leave JOB_AUTH_* and PREVIEW_AUTH_* unset — R_AUTO_GENERATE_KEYS=true manages them. If you later provide explicit env values, they take precedence over the persisted keypairs.

The artifact bucket

MinIO does not auto-create buckets, so the template sets S3_AUTO_CREATE_BUCKET=true on api: at boot, the api creates the artifacts bucket when it is missing. Nothing to do after the first deploy — watch for [artifacts-bucket] Created S3 bucket ... in api's logs.

To create it manually instead (or if the automatic creation warned in the logs), run once from any machine with mc:

mc alias set roomote https://<minio-public-domain> roomote <S3_SECRET_ACCESS_KEY>
mc mb --ignore-existing roomote/roomote-artifacts

The generated S3_SECRET_ACCESS_KEY value is on the api service's Variables tab in the deployed project.

Alternatively, skip bundled MinIO entirely and point the S3 values at an external S3-compatible store (AWS S3 or Cloudflare R2, with S3_REGION=auto for R2). Roomote uses path-style addressing, and S3_PRESIGN_ENDPOINT must be reachable from hosted-sandbox workers. On an external store, either pre-create the bucket and remove S3_AUTO_CREATE_BUCKET, or keep the flag if the configured credentials are allowed to create buckets (the api only logs a warning when creation fails).

First boot

  1. Wait for web and api to report healthy. On a brand-new project the first boot also generates and persists the auth keypairs (watch for [auth-keypairs] Generated ... in an app service's logs — whichever app service boots first wins the race and generates them).
  2. Open https://<web-domain>/setup and paste the SETUP_TOKEN value from the api service's Variables tab into the wizard's token step (or append ?token=<SETUP_TOKEN> to the URL). If you entered a custom domain in the R_APP_URL prompt at deploy time, attach that domain to the web service and finish DNS first, then open /setup on the custom domain — the generated domain will reject auth with 403 Invalid origin.
  3. Create the founding admin account (email/password works immediately; Slack or Microsoft sign-in can be added later).
  4. Connect GitHub with Create GitHub App — the manifest flow derives the callback and webhook URLs from R_APP_URL and TRPC_URL, so no manual URL entry is needed.
  5. Enter the sandbox provider credentials (Modal token pair for the default) and the model provider key when the wizard asks. When swapping to E2B or Daytona instead, the wizard and Settings → Sandboxes can build the E2B template or Daytona snapshot in your provider account after credentials are saved.
  6. Pick repositories, create an environment, and run a small task end to end (the SELF_HOSTING.md verification checklist applies from step 2 onward).

Attaching a custom domain

By default the template boots on Railway-generated domains, and R_APP_URL — the origin users browse — derives from the web service's generated domain. A custom domain can be set either at deploy time (through the R_APP_URL prompt) or after deploy (a one-variable edit). Setting it at deploy time is preferable when you already own the domain: everything the setup wizard registers — in particular the GitHub App's OAuth callback and webhook URLs — derives from the canonical origin, so getting it right before /setup avoids reconnecting integrations later.

At deploy time. The deploy screen shows R_APP_URL on the api service pre-filled with the generated-domain reference (https://${{web.RAILWAY_PUBLIC_DOMAIN}}). Replace it with your domain, for example https://app.example.com. Then, after the project deploys:

  1. Add app.example.com as a custom domain on the web service and complete the DNS setup.
  2. Open /setup on the custom domain — not the generated one. Once R_APP_URL points at the custom domain, the generated web domain rejects signup and login with 403 {"error":"Invalid origin"}, which is expected: only the canonical origin is trusted.

After deploy. When you attach a custom domain to the web service on a running deployment, Railway does not update RAILWAY_PUBLIC_DOMAIN, so the app keeps treating the generated domain as canonical. The symptom is a working dashboard that rejects signup, login, and OAuth flows with 403 {"error":"Invalid origin"}: the browser sends the custom domain as its Origin, and the auth layer only trusts R_APP_URL. The fix is a one-variable edit, because the app services all reference ${{api.R_APP_URL}}:

  1. Add the custom domain (for example app.example.com) to the web service in Railway and complete the DNS setup.
  2. On the api service, set R_APP_URL=https://app.example.com and accept Railway's prompt to redeploy the app services.

Everything derived from the canonical origin follows: auth origins, OAuth callback URLs, and the absolute links Roomote posts to Slack, GitHub, and other integrations. Connectors registered before the switch (for example a GitHub App created by the wizard) keep the callback URLs they were created with, so reconnect or update those in their provider settings if you change the domain after onboarding.

Leave TRPC_URL on the api service's own domain — hosted-sandbox workers and webhooks call it directly, and it never needs to match the domain users browse. (A custom domain on the api or MinIO services works the same way if you want one: they are separate values on api, TRPC_URL and S3_PRESIGN_ENDPOINT.)

Enabling live previews (optional)

The preview-proxy service is already part of the template and boots healthy with previews unconfigured. Enabling previews needs a wildcard domain, which requires a domain you control:

  1. Point previews.<your-domain> and *.previews.<your-domain> at Railway as custom domains on the preview-proxy service, following Railway's wildcard-domain docs.
  2. Fill in the three empty PREVIEW_* values on api: PREVIEW_PROXY_BASE_URL=https://previews.<your-domain>, NEXT_PUBLIC_PREVIEW_PROXY_BASE_URL=https://previews.<your-domain>, and PREVIEW_DOMAINS=previews.<your-domain>, then redeploy the app services. The other services already reference ${{api.*}} for all three; the NEXT_PUBLIC_ variant is what the web client uses to build preview links, so it must reach the web service.
  3. Open any task's preview pane as an admin to validate the wildcard hostname. Previews are always enabled and publish for every environment that defines preview ports.

Enabling Memory (optional)

The gbrain service is part of the template and boots idle. It gives this deployment shared memory: completed tasks and activity from connected integrations (pull requests, public Slack channels, meeting notes, GitHub issues) become pages that agents consult with citations, and agents record what they decided and why when they finish substantial work.

One variable turns it on: set R_BRAIN_OPENROUTER_API_KEY or R_BRAIN_OPENAI_API_KEY on api (or as a Settings environment value). The explicit R_BRAIN_* name is the on switch for Memory: a deployment that only configured task models under Settings → Models keeps no Memory, no ingestion, and agents are never told one exists. The value can be the same key you use for tasks, or a separate one to bill Memory independently. Memory has no provider key of its own; it asks the api service for embeddings and synthesis using this key, and changing it later takes effect on Memory's next request, with no redeploy.

Within a minute of the Memory key being configured, the deployment registers its own scoped clients against Memory (a read-only one for agents, a write-only one for ingestion), backfills the task history it already has, and starts delivering Memory to new tasks. Watch the bullmq service logs for [brain] provisioned scoped clients.

Nothing else is required, and nothing is exposed: Memory has no public domain, so it is reachable only over Railway's private network, and task sandboxes never touch it directly — they go through the api service with their run token, which grants read access only.

Leaving every provider unconfigured is a supported state: Memory sits idle, agents are told nothing about it, and no ingestion runs. Deployments that will never want memory can delete the gbrain service entirely.

Two operational notes:

  • The template backs up the volume. The /data volume holds the corpus, and new template deployments schedule Railway's daily and weekly backups for it. Losing it is recoverable — task history and integration sources re-ingest, and Roomote re-registers its clients automatically when Memory no longer recognizes them — but the deployment starts cold until that finishes.
  • Model choice is a variable, not a rebuild. R_BRAIN_MODEL selects the synthesis model and R_BRAIN_EMBEDDING_MODEL the embedding model, both set on api. Leave them empty for the defaults. The synthesis model can change at any time; the embedding model sizes Memory's vector storage when it is first created, so set it (with R_BRAIN_EMBEDDING_DIMENSIONS) before first boot or not at all. A later change is ignored and reported in Memory's logs rather than silently applied.

Upgrades, backups, and costs

  • Template edits do not propagate. A deployed project is a snapshot; Railway's template-update notifications only exist for GitHub-repo-based templates, not Docker-image-based ones like this. Template changes affect new deploys only.
  • Upgrade a deployment on a channel alias (:main or :develop) by redeploying the five app services — they pull the current alias, and everything version-coupled derives from the new image. On an immutable pin, bump the tag in the five image fields first. The api service's db-migrate pre-deploy applies any schema changes, and the auto-generated keypairs persist in Postgres, so sessions, job tokens, and preview tokens survive redeploys. These redeploys can be automated against Railway's GraphQL API — see Auto-deploying every develop build.
  • One-time Discord gateway secret (existing projects). New template deploys generate R_DISCORD_GATEWAY_SECRET on the api service and share it with the other app services. Older projects without that variable must set a distinct random value once on api, then reference the same value on bullmq (and any other service that already copies api.* secrets) before enabling or continuing Discord. Template edits do not rewrite variables on existing projects.
  • Back up the Railway Postgres database, the MinIO volume or external bucket, and the Memory volume. The template schedules volume backups for all three Railway-managed copies; keep equivalent coverage if you replace any of them with external infrastructure.
  • Costs split three ways: Railway hosts the control plane (web, api, preview-proxy, controller, bullmq, Postgres, Redis, MinIO), while task execution bills through your hosted sandbox provider and model usage bills through your model provider.
  • Node memory is capped by default. Railway bills memory on usage, and an uncapped Node process lets collectable garbage accumulate for hours before a full GC, inflating that usage several times over. The app image therefore caps each Node service's V8 heap (--max-old-space-size): 768 MB for web and api, 512 MB for controller and bullmq, lowered to ~75% of the container's memory when a cgroup limit is set. To override, set NODE_OPTIONS=--max-old-space-size=<MB> on the service — an explicit value always wins. Images published before this change need that variable set manually to get the same effect.

Auto-deploying every develop build (optional)

Railway never redeploys on its own when a mutable alias like :develop or :main moves, so a deployment tracking a channel alias only picks up new builds when the app services are redeployed.

Those redeploys can be automated from any CI system against Railway's public GraphQL API using a project token (scoped to a single environment). Resolve the environment from the token itself (query { projectToken { environmentId } }), find the services whose image starts with ghcr.io/<owner>/roomote-app, then for each issue serviceInstanceUpdate pointing at an immutable tag (e.g. develop-<short-sha> or v1.2.3) followed by serviceInstanceDeploy. Both mutations are idempotent, so a retried or partially failed run is safe to re-run, and pinning immutable tags makes rollback a tag switch — point the image field back at an older tag and deploy.

Maintaining the marketplace template

Two published templates mirror the one spec in template.yaml: the main-channel template (railway.com/deploy/Rj2cFo) and the develop-channel template (railway.com/deploy/bP3Lsu). They differ only in the app image alias (:main vs :develop); everything else must stay identical. When the spec changes, edit template.yaml first, then mirror the change into both published templates through Railway's Template Composer (the Templates list has a Duplicate action for seeding a new template, but edits to existing templates are applied to each by hand). Re-run the first-boot verification on a scratch Railway project before publishing an update (one scratch run on either channel covers a change that does not touch the image fields). When the change touches reference variables — in particular the ${{api.*}} references to values that are themselves references, like R_APP_URL — also open each app service's Variables tab on the scratch project and confirm the resolved values are real URLs, not literal ${{...}} strings.

For Memory storage changes, also open the gbrain service's Backups tab and confirm its /data volume has both Daily and Weekly schedules before publishing the template update.

The R_APP_URL deploy-time prompt needs its own check on the scratch deploy: on the deploy screen, open the api service's Configure step and expand its pre-configured environment variables — R_APP_URL must appear as an editable field with the description from template.yaml and the reference default pre-filled. Confirm that leaving the default still resolves to the generated web domain after deploy, and that overriding it with a test value reaches the app services as that literal value.