diff --git a/.github/canary-pagerduty-handoff.md b/.github/canary-pagerduty-handoff.md new file mode 100644 index 00000000..e03d775d --- /dev/null +++ b/.github/canary-pagerduty-handoff.md @@ -0,0 +1,111 @@ +# Handoff: PagerDuty service + routing key for the GitHub Actions workflow canary + +**For:** the agent doing terraform work in `~/git/my-infra`. +**From:** the canary-implementer in `Szer/bots` (this repo). +**Status:** awaiting your terraform work — do not start the bots-side canary until the items in **"What I need back"** below are answered. + +--- + +## 1. Why this exists (context) + +All scheduled agent workflows in `Szer/bots` silently failed for ~1.5 weeks (08-21 → 09-02): +`openai/codex-action@v1` moved its tag to a commit (86365089) that rejects our +`codex-args` network override under `safety-strategy: drop-sudo`, breaking every +`codex-action` invocation (Monitor 4h, Project daily, Product Tue/Fri, SRE). The runs +went **red but nobody was notified** — there is no human-notification path for a +failed/stalled workflow in this repo. + +We are building a **canary**: an independent sentinel workflow in `Szer/bots` that runs +on its own schedule, checks `gh run list` for every scheduled workflow (failed OR +stalled = no run in longer than its cadence), and on a problem calls **PagerDuty Events +API v2** (`POST https://events.pagerduty.com/v2/enqueue`, plain `curl`, `dedup_key` + +auto-`resolve` on recovery) so one incident pages once and self-closes. + +The sentinel needs a **PagerDuty routing key**. That key is provisioned by PagerDuty for +an **Events API v2 integration** on a PagerDuty service. The PagerDuty configuration +lives in your terraform repo (`~/git/my-infra`). This document is the exact work order. + +## 2. What I need you to do (terraform side) + +1. **Explore `~/git/my-infra/terraform/`** — read `pagerduty.tf`, `alerts-monitoring.tf`, + `provider.tf`, `outputs.tf`, and any module dir they reference. Report the existing + PagerDuty shape: what services exist, what integrations each has, what escalation + policies exist, and how outputs/secrets are conventionally declared. + +2. **Decide: new dedicated service vs reuse.** My recommendation is a **dedicated + "GitHub Actions"** service with its own Events API v2 integration, so GitHub + workflow alerts are a separate, quiet channel from your infra alerts. If you find a + strong reason to reuse an existing service instead (e.g., an existing Events API v2 + routing key that already pages the right people), say so and justify — but a + dedicated key is preferred. + +3. **Implement terraform** (follow the repo's existing conventions — modules, naming, + tags, provider config, output style): + - (if new) a `pagerduty_service` for "GitHub Actions"; + - a `pagerduty_service_integration` with `type = "events_api_v2_inbound_integration"`. + The `integration_key` is **auto-generated by PagerDuty** (read-only attribute); + - expose the routing key as a **sensitive** terraform output (e.g. + `output "github_actions_routing_key" { value = ; sensitive = true }`); + - attach/reuse an **escalation policy** so the incident actually pages the right + people — confirm which one, or flag if a new one is needed. + +4. **Do NOT run terraform locally.** The repo's CI does `plan` on PR and `apply` on + merge. Open a my-infra PR that plans cleanly. + +5. **Security:** never print the routing key anywhere — not in logs, PR title/body, + comments, or commits. The key will live in terraform state (sensitive attribute); + that is acceptable. You do **not** need to deliver the key value to me — the repo + owner will copy it into a GitHub secret (see below). You only need to tell me *how to + retrieve it*. + +6. **Do not touch anything outside terraform** in my-infra. Do not modify the bots repo. + Do not re-delegate. + +## 3. What I need back (the blocking questions for my side) + +Answer these explicitly in your final report — the bots-side canary cannot proceed +until they are answered: + +1. **Approach chosen** — new dedicated service, or reused existing one (name + PD + service ID either way). +2. **Integration type** — confirm `events_api_v2_inbound_integration` and the exact + resource names. +3. **How to retrieve the routing key** — exact command (e.g. `terraform output -raw + github_actions_routing_key` after apply) or the PagerDuty console path + (Service → Integrations → Events API v2 → Integration Key), so the owner can store it + as GitHub secret **`PAGERDUTY_ROUTING_KEY`** in `Szer/bots`. +4. **Escalation policy** — which policy the service uses, and who it pages (or "default", + if that's the convention). +5. **my-infra PR link/number** and its plan status. +6. **Anything that surprises you** in the existing PD setup that affects this (plan + limits, an existing routing key that should be used instead, existing Grafana→PD + alerting to align with, etc.). + +## 4. Sequence after you finish (for the owner / bots side — not your job) + +1. Owner applies your PR (via merge), retrieves the key, adds it as the + `PAGERDUTY_ROUTING_KEY` secret in `Szer/bots`. +2. The bots-side agent (separate handoff) adds the sentinel workflow + optional + in-workflow failure hooks, then tests with a dry-run and a forced-failure run. + +Do not start step 2 until your answers land. + +--- + +## 5. Answers from the my-infra side (2026-09-02) + +Work done by the trusted agent that holds `~/git/my-infra`. The bots-side canary may proceed once the owner confirms the secret is set. + +1. **Approach chosen:** NEW dedicated PagerDuty service **"GitHub Actions"** (`pagerduty_service.github_actions`). Reuse was rejected: the only candidate, "Azure Monitor", is hard-wired to constant low urgency and semantically wrong for GitHub. PD service ID is assigned on apply; retrieve it from the PD console (Services → GitHub Actions) if needed, it is not required for the Events API. +2. **Integration:** `pagerduty_service_integration.github_actions_events_v2`, `type = "events_api_v2_inbound_integration"`, display name "GitHub Actions Events API V2". Routing key exposed as sensitive output `pagerduty_github_actions_routing_key`. +3. **How to retrieve the key (owner only, after apply):** + `gh secret set PAGERDUTY_ROUTING_KEY --repo Szer/bots --body "$(terraform -chdir=terraform output -raw pagerduty_github_actions_routing_key)"` run from `~/git/my-infra` (reads state only, applies nothing). Console fallback: PagerDuty → Services → GitHub Actions → Integrations → Events API V2 → Integration Key. +4. **Escalation policy:** the existing single policy "Default" (`pagerduty_escalation_policy.default`): one rule, 30 min, targets the owner's user directly. No teams or schedules exist on this PD plan (Teams returns HTTP 402). +5. **my-infra PR:** https://github.com/Szer/my-infra/pull/146 — plan check green, `Plan: 2 to add, 0 to change, 0 to destroy`, no existing resource touched. Merge = CI apply (owner merges; PR is mergeable and CLEAN). +6. **Things that affect the canary design (BINDING for the bots side):** + - The service uses **severity_based urgency**. The canary chooses per event: `"severity": "warning"` → low urgency → silent mobile push; `"severity": "critical"` → high urgency → loud page. Owner's notification model: push for "worth knowing", page only for "must act now". Recommended mapping: a single failed/stalled workflow = `warning`; ALL scheduled agent workflows failed or stalled at once (the 08-21 pattern) = `critical`. + - PD account is **EU region**. `https://events.pagerduty.com/v2/enqueue` routes cross-region fine, but `https://events.eu.pagerduty.com/v2/enqueue` is what the in-cluster Alertmanager uses; either works, prefer the EU host for consistency. + - Precedent to copy: `Szer/my-infra/.github/workflows/terraform-apply.yml` lines ~150-176 already does an `if: failure()` curl to Events API v2 with a repo secret and no-ops quietly when the secret is unset. Same shape, same silence-on-missing-secret behavior expected. + - Use one `dedup_key` per monitored workflow (e.g. `gh-actions/`) and send `event_action: resolve` on recovery so incidents self-close. The service has `auto_resolve_timeout = null`, so nothing resolves unless the canary says so. + - Payload `source` should identify the repo + workflow; `component`/`group` are free-form and show up in the PD mobile UI, use them. + - The secret name `PAGERDUTY_ROUTING_KEY` in Szer/bots is free (no PagerDuty secret exists there today). diff --git a/.github/canary-sentinel-handoff.md b/.github/canary-sentinel-handoff.md new file mode 100644 index 00000000..547c043a --- /dev/null +++ b/.github/canary-sentinel-handoff.md @@ -0,0 +1,74 @@ +# Handoff: build the scheduled-workflow canary in Szer/bots + +**For:** the agent implementing the bots-side sentinel (this repo only). +**From:** the agent that did the PagerDuty terraform in my-infra. +**Status:** ALL prerequisites are DONE. Start now. Background and my answers to the original work order are in `.github/canary-pagerduty-handoff.md` (sections 1 and 5). + +## 1. What already exists (do not redo, do not touch) + +- PagerDuty service **"GitHub Actions"** with an Events API v2 integration is live (my-infra PR #146, applied 2026-09-02, run 33629978524 green). +- Repo secret **`PAGERDUTY_ROUTING_KEY`** is set in Szer/bots (2026-09-02 12:30 UTC). Never print, echo, mask-test, or log it. If a run shows it is missing, no-op quietly and print one line saying so; never fail the canary because of a missing secret. +- The service has `auto_resolve_timeout = null`: an incident stays open until YOU send `event_action: resolve`. There is no time-based cleanup. Every trigger path must have a matching resolve path. +- Urgency is **severity_based**. `severity: warning` = low urgency = silent mobile push. `severity: critical` = high urgency = loud page. This mapping is BINDING (owner's notification model: push for "worth knowing", page only for "must act now"). +- The my-infra repo is out of scope. Do not open it, do not reference its files. + +## 2. What to build + +One new workflow, `.github/workflows/workflow-canary.yml` (name it `Workflow Canary`), on its own schedule, that: + +1. Lists recent runs of every scheduled workflow in this repo via `gh run list` with `GITHUB_TOKEN` (`permissions: actions: read` is enough; add `contents: read` if you check out the repo). +2. Classifies each workflow as `healthy`, `failed` (latest completed scheduled run concluded `failure`, `timed_out`, or `startup_failure`), or `stalled` (no completed run within its expected cadence + grace). +3. Sends PagerDuty Events API v2 events: `trigger` for failed/stalled, `resolve` when healthy again. Endpoint `https://events.eu.pagerduty.com/v2/enqueue` (EU account; the global host also routes, but prefer EU). +4. Uses one stable `dedup_key` per workflow, e.g. `bots/`, so repeated polls update one incident and recovery closes exactly that incident. Resolving a dedup key that has no open incident is harmless; do it unconditionally on healthy. + +Plain `curl` + `gh` + `jq`. No `openai/codex-action`, no LLM, no VPN, no `aks-vpn` concurrency group. The canary must be dumber than the things it watches. + +## 3. Workflows to watch (ground truth from `.github/workflows/`, 2026-09-02) + +| Workflow file | `name:` | cron (UTC) | Cadence | Suggested stall threshold | +|---|---|---|---|---| +| `monitor.yml` | Monitor | `20 */4 * * *` | 4h | 5h30m | +| `project.yml` | Daily Project Assessment | `37 4 * * *` | 24h | 26h | +| `product.yml` | Product Analysis | `15 10 * * 2,5` | Tue/Fri (up to 4d gap) | 4d + 2h = 98h | +| `vahter-mirror-reconcile.yml` | Vahter: Mirror Reconcile | `10 7 * * *` | 24h | 26h | + +Keep this as a static table in the workflow (a small JSON or bash array with file, threshold), not cron parsing. Adding a workflow = adding a row. Grace must be at least 60 min: GitHub delays scheduled runs by minutes to over an hour under load, and schedules are silently disabled after 60 days of repo inactivity (stall detection covers that case, which is why it is mandatory). + +Notes per workflow: +- **`vahter-mirror-reconcile.yml` fails BY DESIGN** as its signal (it fails the run when a mirror sync PR is stuck or check-less). A failure there is a real "someone must look" condition and today it notifies nobody. Treat it exactly like the others: warning. +- Monitor, Project, Product share the `aks-vpn` concurrency group, so a run can conclude `cancelled`. Treat `cancelled` as neither failed nor healthy: it does not trigger, and it does not count as a fresh healthy run for stall purposes. Use the latest run whose conclusion is `success`/`failure`/`timed_out`/`startup_failure`. +- Filter to `--event schedule` when judging cadence. A manual `workflow_dispatch` success may be counted as healthy for the failed check but should not reset the stall clock unless you decide otherwise and say so in the PR. +- An unloadable workflow (bad YAML) produces a jobless `startup_failure` with no error text in the API, or no run at all. Both must surface: `startup_failure` via the failed path, no-run via the stalled path. + +## 4. Severity rules (BINDING) + +- One workflow failed or stalled: `severity: warning` (silent push). +- Two or more of the three agent workflows (Monitor, Project, Product) failed or stalled in the same poll: `severity: critical` (loud page). This is the 2026-08-21 pattern, when a single upstream action change killed all of them at once. +- Send severity per dedup key. When the count drops back to one, the next trigger for that key should carry `warning` again (PagerDuty updates urgency on re-trigger with a new severity). +- Payload: `source` = `Szer/bots/`, `summary` = one line with workflow name, state, and the run URL or "no run in N hours", `component` = workflow file, `group` = `github-actions`, `custom_details` = latest run id, conclusion, created_at, threshold. These fields show in the PD mobile UI; use them. + +## 5. Canary schedule and self-reporting + +- Cron: hourly at an odd minute, e.g. `50 * * * *`. Also `workflow_dispatch` with inputs `dry_run` (boolean, default `true`: print every event that would be sent, send nothing) and `test_event` (choice: `none` / `warning-roundtrip`: send one `warning` trigger with dedup key `bots/canary-selftest` and immediately resolve it). Never send `critical` from a test path. +- The canary's own failure is the remaining blind spot. Mitigate cheaply: add `if: failure()` last step that sends a `warning` trigger with dedup key `bots/workflow-canary` (same secret), and resolve that key at the start of every successful run. If PagerDuty itself is unreachable nothing can help; accept it. +- `timeout-minutes: 10`. Runner `ubuntu-latest`. `gh`, `jq`, `curl` are preinstalled. +- On any non-2xx from PagerDuty, print the response body (it never contains the key) and fail the step. Do not retry blindly. + +## 6. Repo rules that WILL fail your PR if ignored + +- **Comment gate** (`.comment-ratio.conf`, enforced by `.githooks/pre-commit` and `comment-lint.yml`): YAML ratio 0.30 with floor 5, and **no comment block longer than 2 lines anywhere, ever**. Existing workflows have long headers; those predate the gate. Yours cannot. Comments = constraints only, no narrative, no process history. Run `make install-hooks` once so the pre-commit hook checks you locally. +- **actionlint** runs on every workflow via `lint-workflows.yml`. Run `actionlint .github/workflows/workflow-canary.yml` locally before pushing if it is installed; otherwise rely on CI and read the failure. +- Do not add the canary to `.github/bots.yml` (that registry is for bots, not workflows) and do not edit any existing workflow. Optional in-workflow `if: failure()` hooks in the agent workflows are a separate follow-up PR, not this one. +- Do not run any real-Telegram or paid-LLM test workflows (`*-real-test.yml`). Not related, listed because agents keep tripping on it. +- PR title style: `CI: workflow canary → PagerDuty for failed/stalled scheduled workflows`. Keep the PR body factual: what is watched, thresholds, severity rules, how to test, how to add a workflow. + +## 7. Test plan (do this, report the evidence) + +1. Push the branch, open the PR, confirm `lint-workflows` and `comment-lint` are green. +2. `workflow_dispatch` the canary from the PR branch with `dry_run=true`: paste the classification table it printed for all four workflows into the PR. All four should be healthy today (Monitor and Project were re-verified green on 2026-09-02; check Product and Mirror Reconcile yourself and report what you see). +3. `workflow_dispatch` with `test_event=warning-roundtrip`, `dry_run=false`: report the two HTTP responses (status only, no bodies with keys). The owner confirms a silent push arrived on the phone and that the incident auto-closed. Do not proceed to merge until the owner confirms. +4. Do NOT force a real workflow to fail to test the critical path. Describe in the PR how to test it by hand if the owner wants to (e.g. temporarily set one threshold to 1 minute on a branch dispatch with `dry_run=true`). + +## 8. Report back + +State: PR number + CI conclusions, the dry-run classification table, the roundtrip HTTP statuses, every deviation from sections 2 to 6 with the reason, and anything in the run history that surprised you (e.g. a workflow that is already stalled today). diff --git a/src/Fizruk/Config.fs b/src/Fizruk/Config.fs index b2f05db6..38899faa 100644 --- a/src/Fizruk/Config.fs +++ b/src/Fizruk/Config.fs @@ -18,6 +18,13 @@ type PlayersConfig = /// Env var holding the RCON password. Set only when Type = Rcon. PasswordEnv: string option } +/// Extra game-specific facts appended to a running game's /status reply. Each case +/// has its own fetcher in StatusDetails; add a case per new kind of detail. +[] +type StatusDetail = + /// Current research and its progress, read over the game's RCON endpoint. + | FactorioResearch + /// The shared Envoy Gateway a game's ListenerSet attaches to. type GatewayConfig = { Name: string; Namespace: string } @@ -46,6 +53,7 @@ type GameConfig = /// Empty when the game has no public port. `Gateway` is `Some` iff non-empty. Listeners: ListenerConfig list Gateway: GatewayConfig option + Details: StatusDetail list ActivityRegex: Regex IdleGraceMinutes: int IdleWindowMinutes: int @@ -176,6 +184,20 @@ module Config = | Some g -> listeners, Some g | None -> raise (ConfigError $"{ctx}: listeners require a top-level 'gateway' to be configured") + /// The optional `details` array: `[{ "type": "factorio-research" }]`. Details that + /// talk RCON reuse the endpoint declared in `players`, so they require type=rcon. + let private parseDetails (ctx: string) (players: PlayersConfig) (el: JsonElement option) : StatusDetail list = + let parseEntry (entry: JsonElement) = + match (requireString entry ctx "type").ToLowerInvariant() with + | "factorio-research" -> + if players.Type <> ProbeType.Rcon then + raise (ConfigError $"{ctx}: details type 'factorio-research' requires players.type=rcon") + StatusDetail.FactorioResearch + | other -> raise (ConfigError $"{ctx}: unknown details type '{other}'") + match el with + | None -> [] + | Some arr -> arr.EnumerateArray() |> Seq.map parseEntry |> List.ofSeq + let private parseGame (gateway: GatewayConfig option) (id: string) (el: JsonElement) : GameConfig = let ctx = $"games.{id}" let regexStr = requireString el ctx "activityRegex" @@ -184,6 +206,7 @@ module Config = with :? ArgumentException as ex -> raise (ConfigError $"{ctx}: invalid activityRegex '{regexStr}': {ex.Message}") let listeners, gameGateway = parseListeners ctx id gateway (prop el "listener") (prop el "listeners") + let players = parsePlayers ctx (prop el "players") { Id = id DisplayName = requireString el ctx "displayName" Namespace = requireString el ctx "namespace" @@ -192,9 +215,10 @@ module Config = Container = requireString el ctx "container" NodeLabelSelector = requireString el ctx "nodeLabelSelector" Address = requireString el ctx "address" - Players = parsePlayers ctx (prop el "players") + Players = players Listeners = listeners Gateway = gameGateway + Details = parseDetails ctx players (prop el "details") ActivityRegex = regex IdleGraceMinutes = requireInt el ctx "idleGraceMinutes" IdleWindowMinutes = requireInt el ctx "idleWindowMinutes" diff --git a/src/Fizruk/Core.fs b/src/Fizruk/Core.fs index c27c8952..2fe349e2 100644 --- a/src/Fizruk/Core.fs +++ b/src/Fizruk/Core.fs @@ -37,6 +37,7 @@ module private StatusText = (pod: PodInfo option) (node: NodeInfo option) (players: PlayersProbeResult) + (detailLines: string list) (listenerLines: string list) (now: DateTimeOffset) : string = @@ -53,6 +54,7 @@ module private StatusText = [ $"{prefix}: running." formatNodeLine node now formatPlayersLine players + yield! detailLines $"Address: {game.Address}" ] |> String.concat "\n" @@ -166,7 +168,11 @@ type GameCore(config: FizrukConfig, k8s: IK8sGateway, notifier: INotifier, time: let game = gameConfig gameId let! desired, pod, node, players = this.GetState gameId let! listenerLines = this.GetListenerLines game - return StatusText.formatStatus game desired pod node players listenerLines (time.GetUtcNow()) + let! detailLines = + match pod with + | Some p when p.Ready -> StatusDetails.fetchAll game + | _ -> Task.FromResult [] + return StatusText.formatStatus game desired pod node players detailLines listenerLines (time.GetUtcNow()) } /// Deletes a game's ListenerSet, if it has one, swallowing (and logging) any diff --git a/src/Fizruk/Fizruk.fsproj b/src/Fizruk/Fizruk.fsproj index 1caaafef..f8950053 100644 --- a/src/Fizruk/Fizruk.fsproj +++ b/src/Fizruk/Fizruk.fsproj @@ -10,6 +10,7 @@ + diff --git a/src/Fizruk/PlayerProbe.fs b/src/Fizruk/PlayerProbe.fs index 22193030..e9dd875a 100644 --- a/src/Fizruk/PlayerProbe.fs +++ b/src/Fizruk/PlayerProbe.fs @@ -13,10 +13,9 @@ module PlayerProbe = match game.Players.Type with | ProbeType.None -> return PlayersProbeResult.NotConfigured | ProbeType.Rcon -> - let envName = game.Players.PasswordEnv |> Option.defaultValue "" - match Environment.GetEnvironmentVariable envName with - | null -> return PlayersProbeResult.Error $"env var {envName} not set" - | password -> + match Rcon.passwordFor game.Players with + | Error e -> return PlayersProbeResult.Error e + | Ok password -> let! result = Rcon.probeAsync game.Players.Host game.Players.Port password timeout return match result with diff --git a/src/Fizruk/Rcon.fs b/src/Fizruk/Rcon.fs index 27fd1cac..62405642 100644 --- a/src/Fizruk/Rcon.fs +++ b/src/Fizruk/Rcon.fs @@ -7,8 +7,8 @@ open System.Text open System.Threading open System.Threading.Tasks -/// Source RCON client for Factorio's "/players online" console command. Packet -/// framing and response parsing are pure so they're unit-testable without a socket. +/// Source RCON client: authenticates, runs one console command, returns its output. +/// Packet framing and response parsing are pure so they're unit-testable without a socket. module Rcon = [] @@ -72,9 +72,9 @@ module Rcon = | None -> return failwith "RCON auth response not received within packet budget" } - /// Runs the auth+exec protocol over any Stream — a real NetworkStream in - /// production, a fake in-memory one in tests — so this needs no socket to test. - let runProtocol (stream: Stream) (password: string) (ct: CancellationToken) : Task> = + /// Runs the auth+exec protocol for one command over any Stream — a real + /// NetworkStream in production, a fake in-memory one in tests. + let runCommand (stream: Stream) (password: string) (command: string) (ct: CancellationToken) : Task> = task { let readExact (buf: byte[]) = task { @@ -101,22 +101,43 @@ module Rcon = if authId = -1 then return Error "RCON authentication failed" else - let execPacket = encodePacket 2 ServerdataExecCommand "/players online" + let execPacket = encodePacket 2 ServerdataExecCommand command do! stream.WriteAsync(execPacket.AsMemory(), ct) let! _, _, body = readPacket () - return Ok(parsePlayersResponse body) + return Ok body } - /// Connects, then delegates to runProtocol for the auth+exec exchange. - let probeAsync (host: string) (port: int) (password: string) (timeout: TimeSpan) : Task> = + /// "/players online" over any Stream, parsed into player names. + let runProtocol (stream: Stream) (password: string) (ct: CancellationToken) : Task> = + task { + let! result = runCommand stream password "/players online" ct + return result |> Result.map parsePlayersResponse + } + + /// Connects and runs one command, mapping timeouts and socket errors to Error. + let execAsync (host: string) (port: int) (password: string) (command: string) (timeout: TimeSpan) : Task> = task { use cts = new CancellationTokenSource(timeout) try use client = new TcpClient() do! client.ConnectAsync(host, port, cts.Token) use stream = client.GetStream() - return! runProtocol stream password cts.Token + return! runCommand stream password command cts.Token with - | :? OperationCanceledException -> return Error "RCON probe timed out" + | :? OperationCanceledException -> return Error "RCON request timed out" | ex -> return Error ex.Message } + + /// Connects and returns the names from "/players online". + let probeAsync (host: string) (port: int) (password: string) (timeout: TimeSpan) : Task> = + task { + let! result = execAsync host port password "/players online" timeout + return result |> Result.map parsePlayersResponse + } + + /// Reads the RCON password from the env var a game's players block names. + let passwordFor (players: PlayersConfig) : Result = + let envName = players.PasswordEnv |> Option.defaultValue "" + match Environment.GetEnvironmentVariable envName with + | null -> Error $"env var {envName} not set" + | password -> Ok password diff --git a/src/Fizruk/StatusDetails.fs b/src/Fizruk/StatusDetails.fs new file mode 100644 index 00000000..b1e52a4e --- /dev/null +++ b/src/Fizruk/StatusDetails.fs @@ -0,0 +1,83 @@ +namespace Fizruk + +open System +open System.Globalization +open System.Text.RegularExpressions +open System.Threading.Tasks + +/// Factorio's current research over RCON. The command and parsing are pure so +/// they're testable without a server; `fetch` does the network call. +module FactorioResearch = + + type Research = { Name: string; Progress: float } + + /// Silent command, so players' in-game consoles don't echo it on every /status. + /// Prints "\t\t", or "none" when idle. + let command = + "/sc local f = game.forces.player local r = f.current_research " + + "if r then rcon.print(r.name .. '\\t' .. r.level .. '\\t' .. f.research_progress) " + + "else rcon.print('none') end" + + let private levelSuffix = Regex(@"^(.*)-(\d+)$", RegexOptions.Compiled) + + /// "mining-productivity-3" at level 12 -> "Mining productivity 12"; + /// "steel-processing" -> "Steel processing". + let displayName (technology: string) (level: int) : string = + let m = levelSuffix.Match technology + let baseName = if m.Success then m.Groups[1].Value else technology + let words = baseName.Replace('-', ' ') + let capitalized = + if words = "" then words else string (Char.ToUpperInvariant words[0]) + words.Substring 1 + if m.Success || level > 1 then $"{capitalized} {level}" else capitalized + + let parse (body: string) : Result = + match body.Trim() with + | "none" -> Ok None + | "" -> Error "empty response" + | text -> + match text.Split('\t') with + | [| name; level; progress |] -> + match Int32.TryParse level, Double.TryParse(progress, NumberStyles.Float, CultureInfo.InvariantCulture) with + | (true, lvl), (true, p) -> Ok(Some { Name = displayName name lvl; Progress = p }) + | _ -> Error $"unexpected research output '{text}'" + | _ -> Error $"unexpected research output '{text}'" + + let format (result: Result) : string = + match result with + | Ok None -> "Research: nothing queued." + | Ok(Some r) -> + let percent = (r.Progress * 100.0).ToString("0.0", CultureInfo.InvariantCulture) + $"Research: {r.Name} ({percent}%%)." + | Error e -> $"Research: unknown ({e})." + + let fetch (game: GameConfig) (timeout: TimeSpan) : Task = + task { + match Rcon.passwordFor game.Players with + | Error e -> return format (Error e) + | Ok password -> + let exec () = Rcon.execAsync game.Players.Host game.Players.Port password command timeout + let! first = exec () + // A save that still has achievements enabled silently swallows its first + // Lua command (empty output) and runs it only when repeated. + let! result = + match first with + | Ok body when String.IsNullOrWhiteSpace body -> exec () + | other -> Task.FromResult other + return format (result |> Result.bind parse) + } + +/// A running game's configured `details` lines, in config order. Fetchers render +/// their own failures as "unknown", so one broken detail never hides /status. +module StatusDetails = + + let private timeout = TimeSpan.FromSeconds 5.0 + + let fetchLine (game: GameConfig) (detail: StatusDetail) : Task = + match detail with + | StatusDetail.FactorioResearch -> FactorioResearch.fetch game timeout + + let fetchAll (game: GameConfig) : Task = + task { + let! lines = game.Details |> List.map (fetchLine game) |> Task.WhenAll + return List.ofArray lines + } diff --git a/tests/Fizruk.Tests/ConfigTests.fs b/tests/Fizruk.Tests/ConfigTests.fs index 0826b97b..791c585b 100644 --- a/tests/Fizruk.Tests/ConfigTests.fs +++ b/tests/Fizruk.Tests/ConfigTests.fs @@ -171,3 +171,26 @@ let ``an empty listeners array configures no listener and doesn't require a gate let config = Config.parse json Assert.True(config.Games.["g"].Listeners.IsEmpty) Assert.True(config.Games.["g"].Gateway.IsNone) + +let private rconPlayers = + """"players": { "type": "rcon", "host": "h", "port": 27015, "passwordEnv": "PW" },""" + +[] +let ``a game without a details block has no status details`` () = + let config = Config.parse (minimalGame "x" rconPlayers) + Assert.Empty(config.Games.["g"].Details) + +[] +let ``factorio-research detail parses for an rcon game`` () = + let config = Config.parse (minimalGame "x" (rconPlayers + """ "details": [ { "type": "factorio-research" } ],""")) + Assert.Equal([ StatusDetail.FactorioResearch ], config.Games.["g"].Details) + +[] +let ``factorio-research detail without an rcon players block raises ConfigError`` () = + let json = minimalGame "x" """"players": { "type": "none" }, "details": [ { "type": "factorio-research" } ],""" + Assert.Throws(fun () -> Config.parse json |> ignore) |> ignore + +[] +let ``unknown details type raises ConfigError`` () = + let json = minimalGame "x" (rconPlayers + """ "details": [ { "type": "weather" } ],""") + Assert.Throws(fun () -> Config.parse json |> ignore) |> ignore diff --git a/tests/Fizruk.Tests/Fizruk.Tests.fsproj b/tests/Fizruk.Tests/Fizruk.Tests.fsproj index 2e45d8a2..50a66ddf 100644 --- a/tests/Fizruk.Tests/Fizruk.Tests.fsproj +++ b/tests/Fizruk.Tests/Fizruk.Tests.fsproj @@ -14,6 +14,7 @@ + diff --git a/tests/Fizruk.Tests/RconTests.fs b/tests/Fizruk.Tests/RconTests.fs index 07fe8656..0fc2889c 100644 --- a/tests/Fizruk.Tests/RconTests.fs +++ b/tests/Fizruk.Tests/RconTests.fs @@ -88,3 +88,22 @@ let ``runProtocol reports auth failure only once the real auth response id is -1 Assert.Equal(Error "RCON authentication failed", result) } + +[] +let ``runCommand sends the given command and returns the raw response body`` () = + task { + let authOkPacket = Rcon.encodePacket 1 Rcon.ServerdataAuthResponse "" + let responsePacket = Rcon.encodePacket 2 Rcon.ServerdataResponseValue "steel-processing\t1\t0.5\n" + use readSource = new MemoryStream(Array.append authOkPacket responsePacket) + use writeSink = new MemoryStream() + use duplex = new DuplexTestStream(readSource, writeSink) + + let! result = Rcon.runCommand duplex "secret" "/sc rcon.print(1)" CancellationToken.None + + Assert.Equal(Ok "steel-processing\t1\t0.5\n", result) + let written = writeSink.ToArray() + let execPayload = written.[(Rcon.encodePacket 1 Rcon.ServerdataAuth "secret").Length + 4 ..] + let _, packetType, body = Rcon.decodePacket execPayload + Assert.Equal(Rcon.ServerdataExecCommand, packetType) + Assert.Equal("/sc rcon.print(1)", body) + } diff --git a/tests/Fizruk.Tests/StatusDetailsTests.fs b/tests/Fizruk.Tests/StatusDetailsTests.fs new file mode 100644 index 00000000..92ed5155 --- /dev/null +++ b/tests/Fizruk.Tests/StatusDetailsTests.fs @@ -0,0 +1,44 @@ +module Fizruk.Tests.StatusDetailsTests + +open Fizruk +open Xunit + +[] +[] +[] +[] +let ``displayName humanizes the technology name and shows the level`` (name: string, level: int, expected: string) = + Assert.Equal(expected, FactorioResearch.displayName name level) + +[] +let ``parse reads name, level and progress`` () = + let result = FactorioResearch.parse "promethium-science-pack\t1\t0.3745\n" + Assert.Equal(Ok(Some ({ Name = "Promethium science pack"; Progress = 0.3745 }: FactorioResearch.Research)), result) + +[] +let ``parse treats "none" as no research`` () = + Assert.Equal(Ok None, FactorioResearch.parse "none\n") + +[] +let ``parse treats empty output as an error`` () = + Assert.Equal(Error "empty response", FactorioResearch.parse "") + +[] +let ``parse rejects unexpected output`` () = + match FactorioResearch.parse "Cannot execute command" with + | Error _ -> () + | Ok r -> Assert.Fail $"expected Error, got {r}" + +[] +let ``format shows progress with one decimal`` () = + let line = FactorioResearch.format (Ok(Some ({ Name = "Railgun damage 3"; Progress = 0.3745 }: FactorioResearch.Research))) + Assert.Equal("Research: Railgun damage 3 (37.5%).", line) + +[] +let ``format reports idle labs and errors`` () = + Assert.Equal("Research: nothing queued.", FactorioResearch.format (Ok None)) + Assert.Equal("Research: unknown (RCON request timed out).", FactorioResearch.format (Error "RCON request timed out")) + +[] +let ``command is silent so players' consoles don't echo it`` () = + Assert.StartsWith("/sc ", FactorioResearch.command)