[Self-hosting]: no DNS-01 path, and no way to opt out of on-demand preview certificates

[MrGeorgen]

Preflight

• I searched existing issues and docs before opening this.
• I removed secrets, tokens, private keys, customer data, and private repository details from logs and screenshots.
• I included the exact command, service, or upgrade step that failed.

Summary

A Roomote deployment that is not reachable from the public internet cannot obtain
certificates. install.sh offers exactly two TLS modes: letsencrypt, which relies on
HTTP-01 and therefore needs inbound port 80 from the ACME server, and internal, which
issues a self-signed CA that every client has to trust manually.

There is no third option for the case in between — a deployment on a public host whose
Roomote instance is deliberately bound to a private address and reached only over a VPN.
The domain is public and DNS-01 would work; the host simply must not answer on 80/443 for
that name.

Two separate things follow from this, and the second one is not specific to our setup:

1. No DNS-01 path exists, so this deployment cannot get a certificate at all.
2. Preview certificates are issued on demand, one per hostname. That is the only
   option today, and it carries costs that every self-hosted deployment pays — public
   Certificate Transparency entries for every preview host, ACME rate limits, and a
   round trip on first hit. A deployment that can solve DNS-01 could use a single
   wildcard instead, but install.sh currently prevents that.

Problem area

Reverse proxy, TLS, or public URL

Installation method

Docker Compose

Roomote version

v1.4.1

Previous version

not applicable

Environment

• Host OS / architecture: Debian, amd64
• Container runtime: Docker with Compose
• Public URL / reverse proxy: bundled Caddy, bound to a private tunnel address
• DNS: Cloudflare
• Everything else: as shipped by install.sh

Command or upgrade path

# TLS mode "letsencrypt": Caddy never gets a certificate, because the ACME server
# cannot reach the instance for the HTTP-01 challenge.
deploy/install.sh --domain <internal-domain> --tls-mode letsencrypt

# TLS mode "internal": works, but every client must trust a private CA.
deploy/install.sh --domain <internal-domain> --tls-mode internal

Expected behavior

A deployment that owns a public DNS zone should be able to solve an ACME DNS-01 challenge
and get a real certificate, without being publicly reachable over HTTP.

Actual behavior

Not possible with the shipped images. Two things stand in the way, and neither is a bug —
just a missing seam:

1. The Caddy image cannot do DNS-01. Caddy compiles DNS provider modules into the
   binary rather than loading them at runtime, so caddy:2.10-alpine has no provider
   available no matter what the Caddyfile says.
2. The Caddyfile has no place for global TLS configuration. acme_dns belongs in the
   global options block, and there is no environment placeholder there — only
   {$ROOMOTE_CADDY_LOCAL_CERTS:}, which selects the internal CA.

What on-demand issuance costs, independent of our case

This part applies to any self-hosted deployment, not just one behind a VPN. The preview
block resolves to tls { on_demand }, so Caddy issues one certificate per preview
hostname on first access:

1. Certificate Transparency. Every preview hostname a deployment ever serves is
   published in public CT logs. For a self-hosted instance that is a running list of
   internal task names, readable by anyone watching the logs.
2. ACME rate limits. Issuance is counted per registered domain, so a deployment that
   creates many previews can run into the limit — and the failure surfaces to a user
   opening a preview, not to an operator running a command.
3. Latency on first hit. The first request to each preview host blocks on a
   certificate issuance round trip instead of being served immediately.

On-demand is clearly the right default when DNS-01 is not available, and none of this is
fatal. But a deployment that can solve DNS-01 would be better served by a single
wildcard certificate for *.<preview domain> — and today install.sh rewrites
ROOMOTE_CADDY_WILDCARD_TLS_SNIPPET on every run, so that choice cannot be made even by
editing .env.

Workarounds tried

We are running this successfully with four small additive changes, all inert unless the new
variables are set — a deployment that sets nothing keeps the stock image and today's
behaviour exactly. The whole thing is about 35 lines across four files. We are ready to
open a PR; per CONTRIBUTING we are raising the issue first and will wait for an invitation.

1. image: ${CADDY_IMAGE:-<current pinned image>} in docker-compose.prod.yml, so a
   deployment can supply its own Caddy build. Default unchanged.
2. A {$ROOMOTE_CADDY_GLOBAL_TLS:} placeholder in the global options block of the
   Caddyfile, plus passing that variable (and CLOUDFLARE_API_TOKEN) through to the
   container.
3. A small xcaddy Dockerfile that builds Caddy with a DNS provider module compiled in.
4. install.sh no longer overwrites ROOMOTE_CADDY_WILDCARD_TLS_SNIPPET when the .env
   already sets it, so a deployment can clear it deliberately and get the wildcard
   described above. Note env_has_value was not usable here — it treats an intentionally
   empty value as unset.

Proposal

We would like to contribute both parts upstream, and we think the second matters more than
the first.

The DNS-01 seam is narrow: two hooks (an overridable Caddy image and a global TLS
placeholder) unblock every deployment that cannot use HTTP-01, and they cost nothing when
unused. Whether Roomote also ships a DNS-enabled Caddy build is a separate call — the hooks
alone are enough for us.

The on-demand behaviour is the part we would raise even if our own deployment were publicly
reachable. Publishing every internal preview hostname to public CT logs, with no way to
turn it off, is a property most operators would not expect from a self-hosted install, and
it is decided today by a single line in install.sh that rewrites itself on every upgrade.
Making that line respect an existing value is a two-line change and takes nothing away from
the default.

We have all four changes running and are ready to open a PR — per CONTRIBUTING we are
waiting for an invitation rather than sending one unsolicited. Happy to cut it to whatever
shape you prefer, split it into two PRs, or hand over the diff for someone on the team to
take from here.

[roomote-community]

I've analyzed this against the current develop deployment path, and here's my proposed implementation plan:

1. Add narrow Caddy extension points in deploy/caddy/Caddyfile: keep the existing ACME/on-demand behavior as the default, but allow an operator-supplied global TLS directive so a DNS-enabled Caddy build can configure acme_dns and obtain app/wildcard certificates before traffic arrives.
2. Update deploy/compose/docker-compose.prod.yml and the repository production Compose overlay to support an optional CADDY_IMAGE override while retaining the currently pinned stock image as the default. Pass the global TLS setting and the Cloudflare credential through only to Caddy, rather than exposing the full Roomote environment to that container.
3. Change deploy/install.sh so an explicitly customized or intentionally empty ROOMOTE_CADDY_WILDCARD_TLS_SNIPPET survives reinstall/upgrade. The installer should still update values it previously generated when --tls-mode changes; comparing the saved value with the known default for the saved mode avoids breaking an internal → acme transition while preserving real operator overrides.
4. Extend deploy/ci/validate-deployment-artifacts.mjs to verify the stock-image default and custom-image override, Caddy environment passthrough, installer-managed versus operator-owned wildcard settings, and successful Caddy adaptation for default ACME, internal TLS, and custom DNS-01/wildcard configurations.
5. Document the supported operator flow in SELF_HOSTING.md, .env.production.example, and the public self-hosting docs: build or select a Caddy image containing the chosen DNS provider module, scope the provider token to DNS changes, set the global DNS challenge directive, clear the on-demand preview snippet, and verify that one wildcard certificate covers the configured preview hostname layout.

I recommend keeping the first change provider-neutral and not shipping a provider-specific xcaddy image in the initial patch. The image/configuration hooks are enough to support DNS-01 without adding another Roomote-maintained image or making Cloudflare the product-level default; the stock image and on-demand issuance remain unchanged for operators who set nothing.

This addresses both parts of the issue: private-ingress deployments gain a public-certificate path, and operators with DNS-01 can opt out of per-preview issuance, CT entries, first-request issuance latency, and avoidable rate-limit pressure.

Please tag @roomote if you'd like me to implement this, or reply with feedback on the plan.