Skip to content
urbforgePublic

About

The urbex CLI (Go). Architecture and ADRs: github.com/urbforge/urbex

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

urbex-cli

The urbex CLI — implementation repo. For architecture, ADRs, and the manifest/platform spec, see urbforge/urbex: getting started (with one guide for the CLI, one for any AI agent and one for Claude Code), the cookbook, and the reference (every command and flag in CLI reference).

Status

The model (ADR-0020): projects are developed trunk-based on Gitea; every push to main is built by Komodo and deployed to staging, which follows main (ADR-0021); a release is a tag (vX.Y.Z) that gives a commit's images the version, and production runs releases. What each environment runs - version, configuration, SOPS-encrypted secrets - is a folder of the GitOps repo, which Komodo deploys whenever it changes. Every LXC sends its logs and resource metrics to the platform's Loki and Prometheus, labelled platform or app (ADR-0023). Every project environment also gets an internal DNS record in Technitium (ADR-0026, ADR-0027), independently of Cloudflare. Without a Cloudflare account, cloudflare.quickTunnel: true exposes every service marked public: true on its own *.trycloudflare.com hostname, for trying Urbex (ADR-0025). With Cloudflare configured (ADR-0022), the platform and the services marked public: true are published through a Cloudflare Tunnel - Gitea and Komodo behind Cloudflare Access - and web frontends are deployed to Cloudflare Workers.

Commands:

  • urbex init — scaffolds a starter urbex.yaml app manifest, or validates an existing one against the manifest schema.
  • urbex bootstrap — scaffolds urbex.platform.yaml if missing, validates it, checks Proxmox for base-service hostnames that already exist and aren't in local Terraform state (aborting rather than risk adopting/duplicating them), allocates static IPs for the 5 base-service LXCs, materializes the embedded Terraform/Ansible assets into the GitOps repo, runs terraform init/apply and ansible-playbook (skippable with --dry-run), then sets up the platform: generates admin passwords and secrets into ~/.urbex/credentials.yaml; creates a Gitea token, the org and its gitops repo; a Komodo API key, the onboarding key project LXCs join Komodo with, Komodo's Gitea git/registry accounts and URBEX_* variables; the Komodo Action that builds main and publishes release tags, and the Procedure (webhook + every 15 minutes) that deploys the GitOps repo; .sops.yaml; and pushes the GitOps repo to Gitea. Re-runs keep existing secrets and tokens.
  • urbex plan <env> — reads urbex.yaml, allocates a VMID/IP for each declared service's LXC (one per service per environment, from a ledger shared by every project in the GitOps repo), materializes the per-service Terraform module into terraform/projects/<project>/<env>/, and runs terraform plan.
  • urbex apply <env> — like plan, then terraform apply, then Docker, the Komodo Periphery agent and sops on each LXC (Ansible), the project's repo on Gitea with a Komodo Build per service and its webhook, and the environment's folder in the GitOps repo (environments/<env>/<project>/<service>/) with the Komodo Stack that deploys it. Also registers every service's internal DNS name in Technitium. A new staging follows main and runs its head; a new prod runs nothing until promoted. A service with image and version instead of runtime is an application from a published image: it is pulled, not built, and runs the digest its tag resolved to. Named volumes are mount points of the LXC; apply refuses a change that would replace or delete an LXC that has them. --delete-data allows deleting one (an application removed from urbex.yaml); a replacement stays refused. With provides and uses, applications are linked: a provider (a PostgreSQL) gets a generated user, password and database, a consumer its connection and ${app.key} references resolved, and a Stack starts once what it uses answers its probe. The reconciler, the Komodo Action urbex-reconcile run every minute, deploys again what is down and holds back what uses a provider that restarted.
  • urbex release — tags the next release (vX.Y.Z, or --version) on the commit staging runs (or main, or --ref) and waits for Komodo to publish its images - the ones main built, retagged. Pushing such a tag by hand does the same.
  • urbex deploy <env> — sets what the environment runs in the GitOps repo: the latest release, --version (a release or a commit; also how to roll back, and it stops the environment following main), or --follow-main; pushes, and waits for Komodo to run it. No Terraform, no SSH.
  • urbex promote — sets prod to the release staging runs: prod runs the very images staging ran.
  • urbex secret set|unset|list <env> <service> [KEY] — manages a service's secrets in the GitOps repo, encrypted with SOPS for the platform's age key and decrypted only on the LXC at deploy time. Needs the sops binary.
  • urbex status [env] — with no argument, which base-service LXCs exist on Proxmox, every platform endpoint (LAN and public) and where the logins are; with an environment, per service, its allocation, whether its LXC exists, the version the GitOps repo asks for, what Komodo reports running, its LAN and public addresses and its internal DNS name, read from Technitium (flagged if missing or pointing elsewhere). --progress follows bootstrap and apply live, phase by phase. With linked applications, urbex status <env> also shows the reconciler.
  • urbex users <env> — in staging, the test users per role of the environment's Keycloak realm.
  • urbex destroy <env> — takes the environment's Stacks down, removes its routes and DNS records from Cloudflare and its web Worker, removes each service's internal DNS record from Technitium, runs terraform destroy, then removes its folder, Terraform files and ledger entries from the GitOps repo. The project's Builds go with its last environment. Resumable after a failure. Volumes are deleted only with --delete-data.
  • urbex teardown [--yes] — destroys the platform bootstrap created: the base-service LXCs (with everything on Gitea), the platform's Cloudflare tunnel, records and Access objects, and the generated credentials. Refuses while project environments exist; the GitOps repo is kept as a local commit.

Known limitations

The main ones - the full list, with workarounds, is docs/known-limitations.md in the docs repo:

  • Not implemented: the services' own metrics and alerting beyond the reconciler's (logs and LXC resource metrics of every service do reach Loki and Prometheus - ADR-0023), backups beyond Proxmox's, roles and schemas of a linked PostgreSQL (urbex-cli#56), mobile frontend deploys, several frontends per project, per-project Proxmox isolation, promotion by pull request, image pruning.
  • One Proxmox node; static IPs are neither checked nor reserved; LXCs need proxmox.network.dnsServers if the host resolves through Tailscale.
  • Keycloak answers ~4 minutes after bootstrap reports the platform ready (development mode).
  • nested public hostnames need Advanced Certificate Manager, enabled and ordered by hand.
  • Gitea and Komodo behind Access are for browsers: git and this CLI use the LAN.
  • urbex teardown destroys Gitea and every repo on it.

Planned next: see docs/roadmap.md.

What has been validated

e2e/run.sh runs the real binary through all of the above - bootstrap, apply, continuous deployment to staging, release, promote, secrets, a hand-edited configuration, pinning, rollback, destroy, re-apply onto a new machine - against Debian 13 systemd machines reachable over SSH, with only terraform and the Proxmox API stubbed. terraform validate passes with the real bpg/proxmox provider, and the whole flow - Cloudflare included, then destroy and teardown down to nothing - has been run on a real Proxmox VE 9.2 node with a pool-scoped, non-root token and a real Cloudflare account. The Cloudflare parts are covered by unit tests against a fake API (internal/cloudflare/cloudflaretest), not by the e2e.

Run it in a container

tools/operator/urbex-op runs urbex - and terraform, ansible, sops, git, ssh - in a container, against a workspace folder holding the credentials, the keys, the GitOps repo and your projects. Only podman or docker is needed, on Linux or on macOS (docker with colima, or podman machine). See tools/operator/README.md.

Each release attaches urbex-op and publishes its image, ghcr.io/urbforge/urbex-op:<version>, which urbex-op pulls the first time: download the one file and run it. From a checkout, build the image first:

tools/operator/urbex-op build && tools/operator/urbex-op -w ~/urbex-work init
export URBEX_PROXMOX_TOKEN=...
tools/operator/urbex-op -w ~/urbex-work 'urbex bootstrap'

Build & test

The tools and their versions are pinned in mise.toml - Go, Terraform, sops, age, Python, Ansible - and installed with mise, without root:

mise install           # once after cloning, and when mise.toml changes
mise run build         # ./urbex
mise run test          # unit tests
mise run lint          # go vet + gofmt check (mise run fmt formats)
mise run e2e           # end-to-end, needs podman - see e2e/README.md
mise run release-snapshot  # the release archives in dist/, publishing nothing
mise tasks             # every task

Every pull request and push to main runs mise run lint and mise run test on GitHub Actions (.github/workflows/ci.yml), with Go and Deno from mise.toml: run both before you push. The e2e needs a Proxmox node and runs on ours, not in CI.

What changed in each release is in CHANGELOG.md: a pull request that changes what users see adds a line under Unreleased, and a release moves them under its version.

A release is a v* tag, pushed by a maintainer: .github/workflows/release.yml runs the tests, then GoReleaser (.goreleaser.yaml) builds urbex for Linux and macOS (amd64, arm64), with checksums, provenance, the version's notes from CHANGELOG.md (the workflow fails without them) and the list of commits, and publishes the GitHub release - a pre-release while urbex is 0.x or the tag has a suffix (v0.1.0-rc.1).

With mise activated in your shell, go, terraform, ansible-playbook... are the pinned ones whenever you are in this directory; otherwise prefix commands with mise exec --. Personal settings go in mise.local.toml, which git ignores, e.g.:

[env]
URBEX_GITOPS_REPO = "/home/me/urbex-gitops"

Credentials don't: they stay in ~/.urbex/credentials.yaml. The operator image (tools/operator) and the e2e runner install their tools from the same mise.toml, so changing a version there changes it everywhere.

Try it

go run ./cmd/urbex init          # scaffolds urbex.yaml in the current directory
# ... edit urbex.yaml to declare a frontend and/or services ...
go run ./cmd/urbex init          # validates it

go run ./cmd/urbex bootstrap --gitops-repo /path/to/gitops-repo
# scaffolds urbex.platform.yaml there; edit it, then run again
export URBEX_PROXMOX_TOKEN=... URBEX_AGE_KEY=...
go run ./cmd/urbex bootstrap --gitops-repo /path/to/gitops-repo --dry-run
go run ./cmd/urbex bootstrap --gitops-repo /path/to/gitops-repo   # provisions + sets up Gitea/Komodo

# from an app repo containing urbex.yaml:
export URBEX_GITOPS_REPO=/path/to/gitops-repo
go run ./cmd/urbex plan staging
go run ./cmd/urbex apply staging     # LXCs, builds, folder and stacks; staging runs main's head
git push gitea main                  # ...and every push to main is deployed to staging
go run ./cmd/urbex release           # tag what staging runs as the next release
go run ./cmd/urbex status            # base-service status
go run ./cmd/urbex status staging    # per service: wanted version against what Komodo runs
printf %s "$TOKEN" | go run ./cmd/urbex secret set staging api API_TOKEN

go run ./cmd/urbex apply prod                 # apply prod first
go run ./cmd/urbex promote                    # prod runs the release staging runs
go run ./cmd/urbex deploy prod --version 0.1.0 # roll back
go run ./cmd/urbex deploy staging --version 0.1.0  # pin staging; --follow-main resumes
go run ./cmd/urbex destroy staging            # tears down staging (LXCs, Stacks, Cloudflare routes)
go run ./cmd/urbex teardown --yes             # then the whole platform, once no environment is left

Note on the vendored schema

internal/manifest/schema/urbex.schema.json is a copy of the schema maintained in urbforge/urbex. It must be kept in sync manually for now; see ADR-0016 in the docs repo for the reasoning and the plan to automate this.

License

urbex-cli is licensed under the Apache License 2.0, and so is the Urbex docs and Claude Code plugin.

About

The urbex CLI (Go). Architecture and ADRs: github.com/urbforge/urbex

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages