diff --git a/CLAUDE.md b/CLAUDE.md index 19a316a..a50a7a3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -213,7 +213,7 @@ One line per area: enough to know whether it is what you are touching. The long | **One renderer, live mimic** (#75, #63, #82, #129) | `TrackDiagram` is the **only** thing that draws the railway. State wins the colour channel; a route is a halo drawn from a **walk** along held track; three global layers, not per-tile groups. | `liveness.md` | | **The control plane** (#165) | `ControlView` (was `MonitorView`) is the mimic **and** the screen the layout is driven from: `ThrottleCard` overlays per loco, `Normal`/`Reverse` on each point-key row. **Track is never a button**; a card for an `auto`-route loco opens *armed*; a refused command is shown, not just logged. Tab reads *Control*, or *Monitor* for the `monitor` role, which gets no controls. | `liveness.md` M10–M16 | | **Control view e2e** | `tests/e2e/control-view.spec.ts` — the one spec that mounts this view, reading the rendered SVG back in one `evaluate`, plus which controls each role is offered. | `liveness.md` | -| **Point position feedback** (#25) | `commandedPosition` / `confirmedPosition` / six-state `confirmation`; **`effectivePosition` decides which is trusted**. Nothing is persisted. Every fault kind Safe-Stops; the escape hatch is `positionFeedback: 'none'`. | `point-feedback.md` | +| **Point position feedback** (#25, #167) | `commandedPosition` / `confirmedPosition` / seven-state `confirmation`; **`effectivePosition` decides which is trusted**. Nothing is persisted. Every *fault* kind Safe-Stops; the escape hatch is `positionFeedback: 'none'`. A controller must **re-assert every 30 s** — silence past the window degrades the point to `stale`, which latches nothing. | `point-feedback.md` | | **Per-loco braking** (#6) | `domain/braking.ts` is pure; `BrakingService` plans and `LayoutService` executes one chained timer on `IClock`. Unmeasured intermediate track **refuses** — never guess a stopping distance. | `braking.md` | | **Sub-block sensor position** (#77) | `position_toward_block_id` + `position_offset_mm`, anchored to an operator-owned **block id**. A **rising edge** sets the fix and it decays by a worst-case travel allowance; **a clear beam still clears nothing.** Consumed as the `lead` term in `remainingRouteDistanceMm`. | `sensor-position.md` | | **Automation engine** (#7) | Drives an `auto`-authority route: departs, runs, brakes, crawls, berths. The invariant is that **a train under automation never passes the end of its authority**. A 250 ms sweep, not an occupancy hook; the stop target is a **berthing beam**. | `automation.md` | @@ -260,6 +260,13 @@ the id (`grep -n "^### D9" docs/sensor-trust.md`) before concluding any of these - A point `command` arms the confirmation deadline; a **`query` deliberately does not** — arming on query would halt the layout at boot for any instrumented point whose controller was powered off (`point-feedback.md` D6). +- **A `stale` point latches no `PointFault` and does not Safe-Stop, unlike every other + non-confirmed state** — silence is a device dying, not lying, so it degrades like a stale + sensor rather than faulting like a timeout. It still suspends any route holding it, which + is the *route's* fault, not the point's (`point-feedback.md` D11, D12). +- **`resumeRoute` deliberately has no staleness precondition** — resuming re-commands every + held point, so the resume itself probes whether the controller is alive and settles it in + 8 s. Adding one would block the probe (`point-feedback.md` D12). - A **`retained` point reading is dropped with a warn, not faulted** — it confirms nothing and arms nothing. A malformed or id-mismatched payload still faults, retained or not (`point-feedback.md`). diff --git a/README.md b/README.md index 61a20af..5548245 100644 --- a/README.md +++ b/README.md @@ -54,12 +54,14 @@ Implemented: - Latched route faults (`SystemHealth.routeFaults`) with per-route operator acknowledge, covering an unexpected occupancy, a rejected point command, a reserved block whose occupancy stops being determinable mid-route, and a held point that fails to confirm -- Point position confirmation (#25) — a point controller reports its observed position on - `point/{pointId}/reading` (never retained; restart recovery is a live `point/*/query`), - and the backend keeps *commanded* and *confirmed* position as separate fields. Opt in per - point: `positionFeedback: 'required'` means commanded position is no longer trusted on its - own, and a point that fails to confirm within 8 seconds latches a fault and Safe-Stops — - see `docs/point-feedback.md` +- Point position confirmation (#25, #167) — a point controller reports its observed + position on `point/{pointId}/reading` (never retained; restart recovery is a live + `point/*/query`), and the backend keeps *commanded* and *confirmed* position as separate + fields. Opt in per point: `positionFeedback: 'required'` means commanded position is no + longer trusted on its own, and a point that fails to confirm within 8 seconds latches a + fault and Safe-Stops. The controller must also **re-assert every 30 s**; one that goes + quiet degrades the point to `stale`/`unknown` past a 90 s window, which refuses routes + over it without faulting anything — see `docs/point-feedback.md` - WebSocket state streaming to the frontend - Local username/password authentication with role-based access (`admin` / `operator` / `monitor`) — see `docs/auth.md` for the scheme and its threat model @@ -586,7 +588,13 @@ feedback hardware is fitted yet. For those the old statement stands exactly: the promises no other software authority will command the point, not that the blades have moved, and the position drawn is commanded. The Layout panel and the point key say which kind each point is rather than the system claiming one uniform level of trust it does not -have. The firmware side is not written yet — see `docs/project-plan.md`. +have. + +A controller that dies quietly is caught too, as of #167: a `'required'` point whose +controller stops re-asserting inside 90 s degrades to `stale`, its position becomes +`unknown`, and routes over it are refused. That is a **degrade, not a fault** — nothing is +latched and nothing Safe-Stops, because silence is a device dying rather than lying. The +firmware side is designed and unblocked but not built — `bazauto/layout-feedback#15`. **A failed point names the route it invalidated, and that route cannot be resumed until the point is dealt with.** A held point that times out, reports the wrong position, or reports @@ -737,10 +745,11 @@ preamble in `docs/sensor-simulation.md`. 2. ESP firmware, as **one** flash-and-test cycle rather than three. Three obligations are now waiting on the same boards, and all three are specified against a stable contract: the WiThrottle→MQTT migration (#9); answering `point/*/query` for point position - feedback (#50 — #25 is complete in this repo, but nothing answers a query yet and no - point has a feedback switch wired); and #28's **30 s re-assert** on - `sensor/*/reading`, which is roughly ten lines and without which every sensor-backed - block reads `unknown` + feedback (#50 — #25 and #167 are complete in this repo, but nothing answers a query yet + and no point has a feedback switch wired); and the **30 s re-assert** now required on + *both* `sensor/*/reading` (#28) and `point/*/reading` (#167), which is roughly ten lines + and without which every sensor-backed block reads `unknown`. The point-side node is + `bazauto/layout-feedback#15`, whose publish behaviour #167 unblocked 3. **The rest of the command-station work** (#153 is the index). #147 and #148 have landed, and #150's guard with them. Next is #149 — track power: `<1>` on connect, refusing routes over a dark layout, and an operator control to turn power back on after a cutoff, which diff --git a/docs/current-state.md b/docs/current-state.md index 60e3698..a3be316 100644 --- a/docs/current-state.md +++ b/docs/current-state.md @@ -36,7 +36,7 @@ beneath the old one.** | **One renderer, the live mimic, and the control plane** (#75, #63, #82, #165) | `TrackDiagram` is the **only** thing that draws the railway. `GridEditor` composes it with authoring chrome and passes no live state; `ControlView` composes it with a `live` prop and no-op *drawing* handlers. A second renderer is the failure mode #75 exists to prevent — on a diagram whose purpose is being trusted at a glance, divergence is not cosmetic. Supporting extractions: `diagram/tilePaths.tsx`, `diagram/diagramModel.ts` (pure derivations), `hooks/useDiagramViewport.ts` (pan/zoom + the load-bearing StrictMode-safe persistence guard — move it verbatim, an e2e spec caught it once). **State wins the colour channel**: where live state is drawn the `BLOCK_TINTS` identity wash is not, and the run label becomes the only thing naming a block. Occupancy is a fill + hatch and a commanded road is solid vs dimmed vs dashed-indeterminate — over a point tile whose **own roads are drawn faint** (`POINT_BASE_OPACITY`, live only), because the overlay only ever *added* a highlight and a point standing reverse still drew its normal road as connected track. **A route is a coloured line along the track it holds** (#129, `diagram/routePaths.ts`), drawn under the track and wider — a halo — inside each tile's group, because every tile paints its own opaque background and a layer beneath them all would be invisible. The segments come from a **walk**, not a wash (`docs/liveness.md` M9): seeded at the joins between held blocks and followed leg to leg, taking only the road a point hold selects, because a block is not a piece of track — washing the destination block lit the road out of it that the route had held shut. The walk's state is a **port** (cell + boundary entered by), which is what keeps a crossing from reading as a junction (#26). Two fallbacks, both drawing *more*: an unresolved point tile stays traversable, and a block the walk never reaches is washed whole. The renderer draws in **three layers** — every wash, then every route halo, then every piece of track and text — because one group per tile means a tile's opaque background erases its already-drawn neighbour's halo, which on a 45° run showed as a waist at each tile edge (the "carriages"). A halo is wider than a tile boundary is thin; nothing that crosses a boundary can live in a per-tile group. Sleepers were blamed for this first and are **not** the cause: they stay. **Decorative track is not dimmed where `live` is present**: "unfinished" is an authoring question, and the absent occupancy wash already says the system cannot see it. It **replaced** the dashed `perimeterEdges` outline rather than joining it: the outline could say "held" but never "held by which", and it is drawn from the same `lockedByRoute` field — a step is drawn only while its block still reports this route's lock — so the two cannot disagree. A route highlight drawn *in addition to* a lock outline would still be two marks for one fact, and is still refused. Identity is hue **and** dash (`ROUTE_TINTS` = `BLOCK_TINTS`, free because state takes the colour channel; the two never share a surface); `active` and `suspended` both draw. Decorative cells between blocks come from `CompiledEdge.via` — the control view reads `GET .../topology/compile` **once** at mount and joins `RoutePathStep.edgeId` → `block_edges` row → connection tuple → `via`; an unresolved join draws a **gap and says so in the key**, never a guessed path. Position stays block-granular: no train at a spot, nothing animated between blocks (`docs/braking.md` B7). A **point lock** is `LOCK.glyph` **beside the point's name**, in the same `` — once per point, the same mark a locked block's label carries, and **not** gated on label density, because a lock is state and "Labels: off" is an authoring control (M7); with labels off it draws alone. In the tile's corner it was a mark with nothing tying it to a point. The **point key** (`PointKeyPanel`, `diagram/pointKey.ts`, M6) is a collapsible panel the operator **drags over the canvas** — that inverts M6's original "beside, never over", because a fixed column covers track permanently and everywhere while an overlay covers only what the operator puts it over. Clamped inside the canvas on every move (a panel dragged off the edge cannot be dragged back), arrow-key movable, sized to its content. It resolves the `shortPointLabel` abbreviations a wall display cannot hover, plus commanded position and holder; control-view-only, position and collapsed state persisted per layout, and a point with no live state reads `unknown` rather than `normal`. Its drag/clamp/nudge/persist half is `hooks/useFloatingPlacement.ts` since #165, shared with the throttle cards. Occupants are a **list** so #39 populates it rather than reshaping consumers. Liveness is one word derived once in `App` (`useConnectionHealth`) and handed down; stale or disconnected **covers the canvas**, never a badge, and stacks with Safe-Stop. **The sensor layer (#76)** is a `ControlView`-owned "Sensors" checkbox, off by default: `buildLiveDiagramState`'s `showSensors` gates `live.sensors` at the source, so `TrackDiagram` needs no flag of its own — an empty map reads exactly like "no sensors placed", which is what the Track Editor already gets by never passing one. | `docs/liveness.md`, `docs/diagram-encoding.md` | | **The control plane** (#165) | The live mimic became the screen the layout is driven from — `MonitorView` is `ControlView`, and "no read-only write path" stopped being a virtue once every act meant leaving the picture of the railway for a form on another tab and coming back to a train that had moved. **Every control is an overlay; track is never a button** (M10): `TrackDiagram` still gets no-op *drawing* handlers, because this display's most valuable form is a touchscreen read from across a room, where a pan gesture and a tap are separated by a movement threshold and the cost of the threshold guessing wrong is a point moving under a train. **Throttle cards** (`ThrottleCard`) are one per loco, several at once, placed by the operator and remembered per layout. **The slider commands** (M11) — no `Set`, throttled to one command per 120 ms while dragging with the released value always sent, a trailing timer holding the *latest* value rather than dropping the final frame. Three interlocks. **A card for a loco under an `auto`-authority route opens armed** (M12): `handleThrottleCommand` cancels that route and abandons its automation run (D6, A12), which is right and which turns a brushed slider into a cancelled journey, so the controls sit inert behind a `Take control` button *naming the route it will cancel* — and the guard is re-checked in every handler, not left to a `disabled` attribute standing in for a behavioural guarantee. A `manual`-authority route is deliberately not armed against: that route **is** the operator driving their own reserved road. **Direction cannot change under a moving train** (M13) — mechanical sympathy rather than system safety, and `ThrottlePanel`'s form is left alone, its `Set` press being the same protection arrived at differently. **No function buttons at all**, because `SerialDccAdapter.setFunction` throws against PicoDCC (#150). **Points are set from the key** (M14): explicit `Normal`/`Reverse` per row, never a toggle — a toggle asks the operator to derive "the other one" from a position that may read `unknown` — and re-commanding the current position is allowed on purpose, since that is how a `mismatch` or `timed-out` point is re-asserted. A point a route holds shows the word "route" and **no buttons**: forcing it cancels the holder (`point-feedback.md` D6), which belongs on the Routes panel where that route is visible. **A refusal is on screen** (M15): the WS `ERROR` frame went to `console.warn` and nowhere else, which the code itself called still-open — `useLayoutSocket` now keeps the last refusal with a sequence number (so an identical refusal twice reads as two events) and the view shows it, dismissible and never self-clearing. Roles are unchanged and still enforced at the transport edge (M16): `canControl` is affordance, `DRIVING_MESSAGE_TYPES` is enforcement, and the `monitor` role's tab keeps the label **Monitor**. Placement is `hooks/useFloatingPlacement.ts`, extracted from the point key; the open-card list and each card's placement are **separate** entries, so closing and re-opening a card returns it to its corner. A stored address that is not a plain positive integer is dropped, never coerced. | `docs/liveness.md` M10–M16 | | **Control view e2e** (`tests/e2e/control-view.spec.ts`) | The one spec that mounts this view. Its fixture is Westgate Hollow's Engine / Goods Transfer corner — a 45° staircase block, two connective cells belonging to no block, and a point *inside* the destination block — because that shape is what every claim on this view is about. It reads the rendered SVG back in one `evaluate` (which cells carry a route line, what opacity the track under them is drawn at, what the labels say and how they are turned) rather than one locator per assertion: the diagram is one picture and the facts worth pinning are relationships between its layers. | `docs/liveness.md` | -| **Point position feedback** (#25 PR A) | A point can now say where it **is**, not just where it was told to go. `PointState.position` is deleted, not deprecated: `commandedPosition` (what we sent, `null` if never), `confirmedPosition` (what came back, `'unknown'` until it does) and a six-state `confirmation` replace it, and **`effectivePosition` (`domain/pointConfirmation.ts`, D7) is the one place that decides which is trusted** — `'required'` trusts `confirmedPosition` and nothing else, `'none'` falls back to commanded exactly as the whole system did before. It is deliberately **not** consumed by the pathfinder (`docs/pathfinding.md` P3 stands). Ingestion is one wildcard subscription to `point/+/reading`; **neither `reading` nor `query` is retained**, and the asymmetry with the retained `sensor/*/reading` is the load-bearing decision (D1): occupancy self-corrects on the next train movement, a point's position is re-asserted by nothing and can change while its controller is off. Restart recovery is a live `query`, never a broker cache, and **nothing about a confirmation is persisted** — a database row is a retained message with better durability (D2/D10). A **command** arms the 8 s deadline; a **query never does** (D6), so an unanswered query at boot leaves the point `unknown` without faulting, symmetric with how block occupancy already boots. The timeout is a pure predicate over a 250 ms sweep on the injected `IClock` (which gained `setInterval` for it) — both halves are needed: a lazy predicate emits nothing, and a per-command `setTimeout` cannot be exercised by the scenario harness. Faults are latched in `SystemHealth.pointFaults`, keyed by point, **every kind Safe-Stopping**, in the priority slot between sensor and route faults; the escape hatch is `positionFeedback: 'none'`, which is why a two-tier "some point faults halt the layout" model buys nothing. `points.position_feedback` defaults `'none'` with **no CHECK constraint** (D10) so the live layout is byte-identical after the migration. `SimulatedPointController` is a fake **ESP**, not a fake DCC callback (D9) — it subscribes to `point/+/query` over the real port, so the tests exercise the contract rather than a private channel. The UI states the guarantee **per point**, because per system it would be false. **PR B gives the failure a route** (D8): `RouteFaultKind` gains `'point-not-confirmed'` and `RouteFault` gains a `pointId` beside its `blockId` — an operator told a route is suspended is told *which point*, and a point is not a block. A held point going `timeout`/`mismatch`/`indeterminate` stops that route's loco **unconditionally** (not only under `auto` authority, matching `handleRouteOccupancyUnknown`) and latches **both** faults: "this point motor is dead" and "route R's road is no longer known to be set" are different facts, resolved by different actions. The three kinds are the whole list — `malformed-payload` and `id-mismatch` leave the confirmation untouched, so the road may still genuinely be set. **`resumeRoute` refuses against the latched fault, never against `confirmation === 'pending'`** — a `pending` check deadlocks, because resuming re-commands every held point and makes each one `pending` again. | `docs/point-feedback.md` | +| **Point position feedback** (#25 PR A, #167) | A point can now say where it **is**, not just where it was told to go. `PointState.position` is deleted, not deprecated: `commandedPosition` (what we sent, `null` if never), `confirmedPosition` (what came back, `'unknown'` until it does) and a seven-state `confirmation` replace it, and **`effectivePosition` (`domain/pointConfirmation.ts`, D7) is the one place that decides which is trusted** — `'required'` trusts `confirmedPosition` and nothing else, `'none'` falls back to commanded exactly as the whole system did before. It is deliberately **not** consumed by the pathfinder (`docs/pathfinding.md` P3 stands). Ingestion is one wildcard subscription to `point/+/reading`; **neither `reading` nor `query` is retained**, and the asymmetry with the retained `sensor/*/reading` is the load-bearing decision (D1): occupancy self-corrects on the next train movement, a point's position is re-asserted by nothing and can change while its controller is off. Restart recovery is a live `query`, never a broker cache, and **nothing about a confirmation is persisted** — a database row is a retained message with better durability (D2/D10). A **command** arms the 8 s deadline; a **query never does** (D6), so an unanswered query at boot leaves the point `unknown` without faulting, symmetric with how block occupancy already boots. The timeout is a pure predicate over a 250 ms sweep on the injected `IClock` (which gained `setInterval` for it) — both halves are needed: a lazy predicate emits nothing, and a per-command `setTimeout` cannot be exercised by the scenario harness. Faults are latched in `SystemHealth.pointFaults`, keyed by point, **every kind Safe-Stopping**, in the priority slot between sensor and route faults; the escape hatch is `positionFeedback: 'none'`, which is why a two-tier "some point faults halt the layout" model buys nothing. `points.position_feedback` defaults `'none'` with **no CHECK constraint** (D10) so the live layout is byte-identical after the migration. `SimulatedPointController` is a fake **ESP**, not a fake DCC callback (D9) — it subscribes to `point/+/query` over the real port, so the tests exercise the contract rather than a private channel. The UI states the guarantee **per point**, because per system it would be false. **PR B gives the failure a route** (D8): `RouteFaultKind` gains `'point-not-confirmed'` and `RouteFault` gains a `pointId` beside its `blockId` — an operator told a route is suspended is told *which point*, and a point is not a block. A held point going `timeout`/`mismatch`/`indeterminate` stops that route's loco **unconditionally** (not only under `auto` authority, matching `handleRouteOccupancyUnknown`) and latches **both** faults: "this point motor is dead" and "route R's road is no longer known to be set" are different facts, resolved by different actions. The three kinds are the whole list — `malformed-payload` and `id-mismatch` leave the confirmation untouched, so the road may still genuinely be set. **`resumeRoute` refuses against the latched fault, never against `confirmation === 'pending'`** — a `pending` check deadlocks, because resuming re-commands every held point and makes each one `pending` again. **#167 closed the last hole in the channel** (D11/D12): nothing anywhere checked that a point *controller* was still alive, so a controller that died quietly left its last `confirmedPosition` standing forever while the layout kept setting roads over it. The gap was three correct decisions composing — a reading is not retained (D1), it had no re-assert, and a query arms nothing (D6) — so no clock expected to hear from a controller again. The contract now obliges a `'required'` point's controller to re-publish every **30 s**, and `evaluateStaleness` degrades it to a seventh state, `'stale'` / `'unknown'`, past a 90 s window (3x the interval, the same tolerance as the sensor side). It is sharper here than for sensors because **command and feedback are two unrelated devices on two transports** — a fire-and-forget DCC accessory command keeps succeeding long after the MQTT feedback node stopped, so silence after a command carries no signal at all. **Staleness is a degrade, not a fault**: no `PointFault`, no Safe-Stop, which is #28 D10's lying-vs-dying split applied unchanged. It narrows three ways, each load-bearing: `'required'` only (a `'none'` point has nothing reporting on it, so every live Westgate Hollow point is untouched), **from `'confirmed'` only** (the one state where a reading is trusted — `'unreported'` is D6's boot case, `'pending'` belongs to the 8 s deadline, and the fault states are already sharp), and `'stale'` is its own state rather than a reuse of `'unreported'` because "never heard from" and "heard, then went quiet" send an operator to different places. The route consequence is D8's, minus the point fault, reusing `'point-not-confirmed'` — exactly the shape `runSensorTrustSweep` already has, where a stale sensor latches no `SensorFault` and reaches routes only through `occupancy-unknown`. **`resumeRoute` gains no staleness clause on purpose** (D12): resuming re-commands every held point, so a recovered controller confirms and a dead one times out into a real fault — the resume *is* the probe, and it settles in 8 s rather than another freshness window. `SimulatedPointController` gained the matching 30 s re-assert, without which the simulator would have manufactured staleness the contract forbids. | `docs/point-feedback.md` | | **Per-loco braking** (#6) | A model plus an **explicitly-invoked** execution surface — nothing decides *when* to brake, which is the whole #6/#7 boundary. `brakingFactor` means dimensionless *effectiveness* scaling `REFERENCE_STOPPING_DISTANCE_MM` quadratically in normalised speed (B1); callers depend on the `StoppingDistanceModel` **function type**, never the scalar formula, so a measured curve replaces it without touching one (B2). `domain/braking.ts` is pure: a `BrakingSchedule` is a linear step-down, 8 steps per 250 ms tick, **loco-independent** — `brakingFactor` predicts *distance*, it does not shape the ramp (B3). Available distance is the sum of `blocks.length_mm` over the **intermediate** steps only (B4); an unmeasured block **refuses**, and `DEFAULT_BLOCK_LENGTH_MM` must never be reused here — guessing a cost steers a search, guessing a stopping distance is a collision. `BrakingService` plans; `LayoutService` executes against the injected `IClock` as **one chained timer**, step 0 awaited inline so a first-command rejection can answer `started: false` *and* latch a fault (B12). B5's overrun expectation is a snapshot armed after step 0 and **outliving the ramp** — reaching speed 0 is a command, not a confirmation — cleared by a manual throttle, its route leaving `active`, a new run, or firing. It is checked in `recomputeBlock` **before** `onOccupancyChange`, which is what makes an overrun latch *two* faults (braking + the route's `unexpected-occupancy` cancel) instead of letting the route read `completed` — the false success the check exists to prevent. `SystemHealth.brakingFaults` is keyed per loco, latched, **no arming threshold** (a Safe-Stopped loco cannot prove itself), between point and route faults. Every ramp is aborted **before** any `emergencyStop()` broadcast. B7's #25 forward reference is built: a plan refuses `point-not-confirmed` on a held point whose `effectivePosition` is not the required one — inert for `'none'` points, so the live layout is unchanged. `POST .../locos/:address/brake` is B8's calibration stimulus (409 + structured reason on refusal); `GET .../braking-faults` and `POST .../locos/:address/acknowledge-fault` mirror the sensor/route pattern. No MQTT or schema change (B9). | `docs/braking.md` | | **Sub-block sensor position** (#77 PR A) | A sensor may say **where in its block it is**, and — far more carefully — what may be concluded from one having been broken. `sensors` gains `position_toward_block_id`/`position_offset_mm`: `offsetMm` of track before the boundary its own block shares with a **neighbouring block**. **The anchor is the whole decision** (D2). #77 as filed said `(blockId, endLabel, offsetMm)`, and #103 took that vocabulary away — an end label is regenerated by every compile, so a stored one comes to name a *different* end after an innocuous redraw. The issue's own three replacements are all rejected: a **port** anchors a measurement of the railway to a picture of it (moving a tile does not move a beam), an **edge id** is rewritten wholesale by every apply for the same reason a label is, and **store-and-diagnose** fails quiet. A block id is operator-owned and the compiler never rewrites it. `null` = unmeasured = exactly today's B4 worst case (D3); only an `ir_position` sensor may carry one (D4); a self-anchor, a missing block, or an offset longer than the block is a **400, never a Safe-Stop**. **The fix is the rising edge and nothing else** (D6): a repeated `occupied` moves `lastReadingAt` and deliberately not `lastRisingEdgeAt`, or a device re-publishing on a timer would look like it kept re-seeing a train. It **decays by a worst-case travel allowance** (`MAX_CREDIBLE_SPEED_MM_PER_S`, D7), not a freshness cliff — subtracting a deliberately generous speed can only *understate* the distance, and an old fix falls to zero on its own. **A clear beam still clears nothing** (D8): `deriveBlockOccupancy` clause 3 is untouched, and the tempting "but we know where the beam is now" simplification is refused in the decision record, not merely in a comment. Ambiguity resolves to *unmeasured*, never a guess (D5) — two blocks joined in two places, joined nowhere, or a half-set row left by `ON DELETE SET NULL`, which reads as unmeasured rather than as corruption (the one place `parseSensorRow`'s corruption-must-throw posture is deliberately not applied). Nothing about a fix is persisted (D11). **PR B is the consumer**: `remainingRouteDistanceMm` takes one optional `lead` term (D9) — the distance from a fix in the **confirmed** block to the boundary the train is about to cross — which is what **closes B4's adjacent-target refusal** and unblocks #7. It can only ever *add*: every way of not having a usable fix contributes `0` and falls through to the B4 sum, omitting `lead` reproduces the pre-#77 answer exactly, and `unmeasured-track` still refuses on intermediate track whatever the fix says. `BrakingService`'s `IClock` is **optional and inert when absent** (mirroring #103's `IGraphCompletenessView`), so a missed wiring refuses a run that could have been granted rather than the reverse. One knock-on: `handleSensorReading` now stamps a reading with `clock.now()`, not `new Date()` — a fix aged against `IClock` and a reading stamped from the wall clock disagree by decades under `ManualClock`. **PR C is the authoring column** (`SensorPositionCell` in Configure → Sensors, D11a): the pair is edited as a pair and written in one `PUT`, so no patch can move one half; clearing either control clears the whole measurement, because unmeasured is a real safe state and not an error to argue out of; and a row that cannot carry one says `n/a`/`no block` rather than rendering a blank, which would read as a gap. The anchor select offers only blocks the drawing joins this one to **exactly once** (`sensorPosition.ts#anchorCandidates`) — deliberately narrower than the write path, which still accepts an anchor the drawing has not connected yet. **The monitor is untouched**: a position is config, not live state, and there is still no train-at-a-spot to draw. | `docs/sensor-position.md` | | **Automation engine** (#7 — PR A the model, PR B the engine, PR C the surface) | Automation **drives** an `auto`-authority route: departs it, runs it, brakes it, crawls it, and berths it. The framing is the load-bearing part — the one invariant is that **a train under automation never passes the end of its authority**, and on this railway that *is* collision avoidance, because `planReservation` already refuses two routes over one block (A1). The stop target is a **berthing beam**, not a boundary: the operator's IR beams go where a train should stand (the platform, the goods shed), and a run brakes toward one, crawls, and stops when it breaks — the only closed loop in a system that is otherwise open-loop dead reckoning (A2). **The crawl is why B4's conservatism is affordable** (A5): the ramp may brake as early as the block-granular model forces, because the beam and not the model is what stops the train — do *not* "fix" B4 to make automation stop more precisely. `berthingBeamIn` reuses #77's measured pair from both anchor directions, nearest beam wins, and **never decays** (a screwed-down beam is not an observation of a moving train). `planBrakingSchedule` gained `toSpeedStep`, which changes what the ramp *commands* and deliberately not what it *requires* — the distance check stays the full-stop one, and the slack that leaves **is** the crawl allowance (A4). `buildBerthExpectation` replaces B5's `path.slice` for a berthing run with the graph's blocks *beyond* the destination, which closes a real hole: a train running past its destination into unreserved track previously raised nothing at all (A9). Three nullable columns, all refusing rather than defaulting (A7): `locos.auto_speed_step`, `locos.crawl_speed_step`, and `route_reservations.direction` — direction is an operator input because the DCC direction bit is loco orientation, which nothing here can derive from path geometry. `BrakingFaultKind` gains `unable-to-stop` and `berth-not-confirmed` rather than opening a fifth latched collection (A10). The loco write path gained Zod at the same time, because two of its fields are now speed steps automation issues with no operator in the loop. **PR B is `AutomationService` plus the sweep**: planning and run bookkeeping only, mirroring `PointConfirmationService` (no MQTT, no DCC, no timer, never calls back), with `LayoutService` owning `IClock.setInterval` and every command. A **sweep** rather than an occupancy hook, structurally: since #77 the available distance shrinks *continuously* as a fix decays, so the moment braking becomes necessary can fall between two events. **The trigger is a look-ahead, not just a distance** (`isLastChanceToPlan`): the ability to plan runs out before the distance does — a route completes the instant its destination reads `occupied` and `remainingRouteDistanceMm` refuses past `targetIndex`, so a train that crosses one boundary too many is one nothing will ever brake. It asks "will I still be able to do this next time I look", which is what makes the no-beam case (brake a block earlier) and the berthed case (brake on entering the last block) one rule. **Automation will not start a journey it cannot finish** (`canPlanAStop` at departure, A7) — that turns unmeasured track and an adjacent beam-less destination from Safe-Stops into config-gap reports. A run is **adopted at most once**, pruned to the active set, so an emergency stop cannot be undone by the next sweep re-departing the train, while a suspended-then-resumed route is picked up again. A missing `LocoState` deliberately does **not** block a departure (asymmetric with B6, which needs the speed as a calculation input; a departure sets it) — requiring one made automation unusable from cold, since the only way to establish one cancels the route. Teardown hooks: manual throttle, emergency stop, Safe-Stop, `stop()`, and `standDownAutomation` on the switch to `manual` — the last one *stops the train*, because a crawling train's route has usually completed already so `suspendAuto` would not touch it. **PR C is the surface** (A14): the two speed columns in Configure → Locos (clearing to `—`, because unconfigured is a real state), an authority select on the route request defaulting to `manual` with a direction select shown only for `auto`, and `AutomationRunView` on `GET .../automation`, in the WS snapshot, and on its own `AUTOMATION_STATE`. That event is **published only when it changed** — a 250 ms sweep is usually a no-op and would otherwise put 240 messages a minute on every socket — which is exactly why the view is *also* in the snapshot. The phase is a **word, not a colour** (two states are indistinguishable from outside: not-yet-departed and berthed are both a stopped train), and `blocker` carries the rendered sentence rather than the tag so the browser never re-implements `describeAutomationBlocker`. Not on the MQTT contract, and **the monitor is untouched** — a phase is a property of a journey, and there is still no train-at-a-spot to draw. One line had been the whole of "automation is unreachable from the browser": `requestRoute` hard-coded `authority: 'manual'`. | `docs/automation.md`, `docs/plans/7-automation.md` | diff --git a/docs/mqtt-contract.md b/docs/mqtt-contract.md index db85777..c29307c 100644 --- a/docs/mqtt-contract.md +++ b/docs/mqtt-contract.md @@ -32,7 +32,7 @@ layout/{layoutId}/system/heartbeat | `sensor/{sensorId}/reading` | Sensor HW (or backend sim) → Backend | 1 | YES | Current sensor occupancy. Published on change **and** re-asserted at least every 30 s | | `point/{pointId}/command` | Backend → DCC | 1 | **NO** | Point position command | | `point/{pointId}/state` | Backend → Subscribers | 1 | YES | Broadcast current point state | -| `point/{pointId}/reading` | Point Controller → Backend | 1 | **NO** | Physical position as observed by the point controller | +| `point/{pointId}/reading` | Point Controller → Backend | 1 | **NO** | Physical position as observed by the point controller. Published on change, in answer to a `query`, **and** re-asserted at least every 30 s | | `point/{pointId}/query` | Backend → Point Controller | 1 | **NO** | Request an immediate `reading` for this point | | `block/{blockId}/state` | Backend → Subscribers | 1 | YES | Block occupancy broadcast | | `system/status` | Backend → Subscribers | 1 | YES | System status (also used as LWT) | @@ -42,14 +42,17 @@ layout/{layoutId}/system/heartbeat > `loco/*/command` and `point/*/command` MUST NOT be retained. A retained throttle command would trigger a ghost movement immediately on any new subscriber connecting to the broker (e.g., after an ESP controller reboot). This is a safety requirement. > > **`point/*/reading` is also NOT retained**, for a different reason from the -> control topics. Occupancy is continuously re-asserted by a live sensor and -> self-corrects on the next movement, which is why `sensor/*/reading` **is** -> retained. A point's position is re-asserted by nothing, and it can change while +> control topics. Both `reading` topics now carry the same 30 s re-assert +> obligation (#167), so re-assertion is no longer what separates them: retention +> is decided by whether a stale copy can **mislead**, not by whether the publisher +> repeats itself. A sensor's occupancy self-corrects on the next movement, which +> is why `sensor/*/reading` **is** retained. A point's position can change while > its controller is offline — hand-thrown during a shutdown, power lost -> mid-travel, a linkage dropped. A retained point reading is therefore a -> confident assertion with no correction path, and believing a stale point -> position is the direct cause of a wrong-route movement. Restart recovery is -> provided instead by `point/*/query`, which recovers position **live**. +> mid-travel, a linkage dropped — and nothing on the layout corrects it +> afterwards. A retained point reading is therefore a confident assertion with no +> correction path, and believing a stale point position is the direct cause of a +> wrong-route movement. Restart recovery is provided instead by +> `point/*/query`, which recovers position **live**. > **Critical — Retention Is Not Evidence of Liveness:** > A retained message tells a new subscriber what was last published. It says nothing @@ -62,9 +65,12 @@ layout/{layoutId}/system/heartbeat > > `sensor/*/reading` is retained *and* its publisher MUST re-assert it at least every > 30 s (see its payload section below), so the retained copy is a bootstrap for a value -> that is about to be reconfirmed. `point/*/reading` is **not** retained, because -> nothing re-asserts a point: its position can change while its controller is offline, -> so a retained position is an archived belief with nothing behind it. +> that is about to be reconfirmed. `point/*/reading` carries the same 30 s obligation +> (#167) but is **not** retained: re-assertion is **necessary** for retention and not +> **sufficient** for it. A point's position can change while its controller is offline, +> so a retained position is an archived belief with nothing behind it — a re-assert +> says the controller is alive *now*, which is a different claim from the archived +> value still being true. > > The backend distinguishes a message delivered because of a **new subscription** > (RETAIN flag set, [MQTT-3.3.1-8]) from one delivered on an **established @@ -194,8 +200,25 @@ Sent by the backend to set a point position. NOT retained. --- ### `point/{pointId}/reading` -Published by the point controller whenever its observed position changes, and in -response to a `point/{pointId}/query`. NOT retained — see the retention callout. +Published by the point controller whenever its observed position changes, in +response to a `point/{pointId}/query`, **and** re-published unchanged at least every +**30 seconds** as a liveness assertion. NOT retained — see the retention callout. + +The re-assertion is a hard requirement, not an optimisation (#167). Command and +feedback are **two unrelated devices on two transports**: the point motors are +commanded over DCC accessory addresses and have no feedback path of their own, while +position is read from their changeover contacts by a separate MQTT node. Nothing about +a command succeeding tells you the reporting device is still alive — there is no +shared component whose failure would show up in both — so a fire-and-forget accessory +command goes on succeeding long after the feedback node has stopped. + +A point configured `positionFeedback: "required"` from which the backend has received +no reading inside its freshness window (`POINT_FRESHNESS_TIMEOUT_MS`, default 90 s = +3 × the re-assert interval) degrades to `confirmation: "stale"` with +`confirmedPosition: "unknown"`, and every edge through it becomes untraversable. That +is a **degradation, not a fault**: it latches no `PointFault` and does not Safe-Stop. +A point configured `positionFeedback: "none"` has nothing reporting on it at all and +never goes stale. ```json { @@ -313,6 +336,7 @@ Published by the backend on startup and on any status change. Also configured as | Backend restarts, sensor alive | Retained readings arrive with the RETAIN flag set and are provisional. Those blocks read `unknown` until the first re-assert, i.e. within 30 s. | | Backend restarts, sensor dead | The same retained readings arrive, but no re-assert ever follows, so those blocks stay `unknown` indefinitely. Retention alone got this case wrong — it reported a dead sensor's last `clear` as live track (#28). | | Point controller restarts | The backend receives NO retained point reading. Every point configured `positionFeedback: "required"` remains or becomes `"unknown"` until it answers a live `point/*/query`. | +| Point controller dies mid-session | No further reading arrives. Every `positionFeedback: "required"` point it reports degrades to `"stale"`/`"unknown"` inside the freshness window, and a route holding one is suspended. Before #167 no clock anywhere expected to hear from a point controller again, so its last reading stood indefinitely. | | New frontend client connects | Receives all retained `block/*/state`, `point/*/state`, `loco/*/state`, and `system/status` immediately without needing a REST poll. | --- @@ -332,23 +356,33 @@ The following MQTT conditions MUST trigger a Safe-Stop in the backend: ## Degradation Triggers -The following conditions degrade **specific track** to `"unknown"` — which the domain -layer already treats as occupied, so routes over it cannot be granted and a live route -holding it is suspended — rather than triggering a system-wide Safe-Stop. The failure is -scoped to the track the failing device actually observes, so a layout that is otherwise -fully observed keeps running. +The following conditions degrade **specific track** — a block, or a point — to +`"unknown"`, which the domain layer already treats as unusable, so routes over it cannot +be granted and a live route holding it is suspended. None of them triggers a system-wide +Safe-Stop. The failure is scoped to what the failing device actually observes, so a +layout that is otherwise fully observed keeps running. 1. No **live** sensor reading received inside the sensor freshness window (`SENSOR_FRESHNESS_TIMEOUT_MS`, default 90 s = 3 × the 30 s re-assert interval). 2. A sensor reading known only from a **retained** message, with no live reading since the backend connected. - -Both set every affected block to `"unknown"` and emit a `BLOCK_STATE` event, so the -condition is visible to the operator rather than silent. - -> **A malformed sensor payload is NOT in this list.** It remains Fail-Safe Trigger 3 -> above — a system-wide Safe-Stop, on the first message, with no tolerance. The two -> failures are different and are answered differently: a malformed payload is a device -> **lying**, which is immediate and sharp; an unrefreshed reading is a device **dying**, -> which is a freshness window and a scoped degrade. Do not merge them into one rule. -> See `docs/sensor-trust.md` and `docs/sensor-fault-recovery.md`. +3. No `point/*/reading` received inside the point freshness window + (`POINT_FRESHNESS_TIMEOUT_MS`, default 90 s = 3 × the 30 s re-assert interval), for + a point configured `positionFeedback: "required"` that had previously confirmed + (#167). + +1 and 2 set every affected block to `"unknown"` and emit a `BLOCK_STATE` event; 3 sets +the point to `confirmation: "stale"` with `confirmedPosition: "unknown"` and emits a +`POINT_STATE` event. Every one of them is visible to the operator rather than silent, +and none of them latches a fault. + +> **A malformed payload is NOT in this list**, on either `reading` topic. It remains +> Fail-Safe Trigger 3 above — a system-wide Safe-Stop, on the first message, with no +> tolerance. The two failures are different and are answered differently: a malformed +> payload is a device **lying**, which is immediate and sharp; an unrefreshed reading is +> a device **dying**, which is a freshness window and a scoped degrade. Do not merge +> them into one rule. +> +> The same split governs the point side, which is why a stale point degrades while every +> *fault* kind Safe-Stops (`docs/point-feedback.md` D4, D11). See `docs/sensor-trust.md`, +> `docs/sensor-fault-recovery.md` and `docs/point-feedback.md`. diff --git a/docs/point-feedback.md b/docs/point-feedback.md index 9a10465..724dc5b 100644 --- a/docs/point-feedback.md +++ b/docs/point-feedback.md @@ -31,7 +31,8 @@ here rather than restate it (that update lands with PR B, see below). | Contract amendment (`docs/mqtt-contract.md`) | **Shipped** — `b2b6641` | | PR A — the channel | **Shipped** 2026-08-15 | | PR B — route interaction (D8) | **Shipped** 2026-08-15 | -| ESP firmware (`bazauto/esp-layout-controller`) | Not started — see `docs/project-plan.md` | +| Controller liveness (D11, D12 — #167) | **Shipped** 2026-08-24 | +| Feedback node (`bazauto/layout-feedback#15`) | Designed, unblocked by D11 — `docs/point-position-feedback.md` there | Nothing on the live layout has feedback hardware fitted, so every point on Westgate Hollow is `positionFeedback: 'none'` and behaves exactly as it did @@ -356,6 +357,111 @@ have to agree, because they are the same argument at two different layers. --- +## D11 — A point controller must re-assert every 30 s; silence degrades the point, and only that point + +**#167, decided 2026-08-24.** Chosen from four candidate shapes: periodic +re-assert (this one), a periodic backend query, a controller-level heartbeat, +and accepting the gap knowingly. + +Before this, nothing anywhere checked that a point controller was still alive. +Three decisions, each correct on its own, composed into the hole: a +`point/*/reading` is not retained (D1), it had no periodic re-assert, and the +confirmation deadline arms on a **command** and deliberately not on a **query** +(D6). So no clock anywhere expected to hear from a point controller again, and a +controller that died quietly left its last `confirmedPosition` standing +indefinitely while the layout went on setting roads over it. + +The contract now requires a `positionFeedback: 'required'` point's controller to +re-publish its observed position at least every **30 s**, matching +`sensor/*/reading` exactly. A point from which no reading arrives inside +`POINT_FRESHNESS_TIMEOUT_MS` (default 90 s = 3 × the interval, the same tolerance +and the same reasoning as `DEFAULT_SENSOR_FRESHNESS_TIMEOUT_MS`) degrades to +`confirmation: 'stale'`, `confirmedPosition: 'unknown'`. + +**Why the re-assert and not the periodic query.** The query shape keeps the +controller dumb and the policy in one place, which is genuinely attractive — but +it would have had to make an *unanswered* query into evidence, and D6 exists +precisely to stop a query meaning anything. Reworking what a query means, on the +one path that recovers position at boot, to gain a liveness check that a timer in +the publish loop already provides, is a poor trade. A controller-level heartbeat +is cheaper on the wire again and proves less: that the node is running, not that +any particular point's sensing path is intact. Six points at one message per 30 s +is not a traffic problem worth designing around. + +**Why this is sharper here than on the sensor side.** Command and feedback are +two unrelated devices on two transports. The Cobalt iP motors are commanded over +DCC accessory addresses and have no feedback mechanism of their own; position is +read from their `S2` changeover contacts by a separate MQTT node. There is no +shared component whose failure shows up in both, so a DCC accessory command — +fire and forget — keeps succeeding long after the feedback node has stopped. The +usual reassurance that silence after a command is at least *suspicious* does not +apply, because the commanded device is not the reporting device. + +**Staleness is a degrade, not a fault.** It latches no `PointFault` and does not +Safe-Stop, which puts it on the opposite side of the line from every kind in D4. +That is #28 D10's split applied unchanged: a malformed payload is a device +**lying** — immediate, sharp, system-wide; an unrefreshed reading is a device +**dying** — a freshness window and a scoped degrade. It is also D6's organising +principle read forward: a fault means something the backend *commanded* did not +happen as promised, and nothing commanded a controller to keep talking. + +**Three narrowings, each load-bearing:** + +- **Only `positionFeedback: 'required'`.** A `'none'` point has nothing reporting + on it and would otherwise go stale immediately and permanently. All six points + on Westgate Hollow are `'none'` today, so this decision changes live behaviour + by exactly nothing until feedback hardware is fitted and a point is opted in — + the same rollout property D10's `DEFAULT 'none'` was chosen for. +- **Only from `confirmed`.** That is the one state in which a reading is actually + being *trusted*. `unreported` must stay `unreported` or a `'required'` point + whose controller was powered off at boot would degrade into a different-looking + state for no new reason (D6's boot case). `pending` is owned by the 8 s + confirmation deadline, which fires an order of magnitude sooner. `mismatch`, + `indeterminate` and `timed-out` are already latched faults and already + untrusted; re-labelling one of them `stale` would replace a sharp fact with a + vaguer one. +- **`stale` is a seventh `PointConfirmation`, not a reuse of `unreported`.** + "Never heard from" and "heard, then went quiet" are different facts about the + hardware and lead an operator to different places — the first is ordinary at + boot, the second means a node died. Collapsing them would throw away the only + evidence that distinguishes them. + +## D12 — A stale point suspends the routes holding it, but latches no point fault, and resume is the test + +A stale point does two things, and the split matters. + +**To itself:** `confirmedPosition` goes `'unknown'`, so `effectivePosition` (D7) +returns `'unknown'` for a `'required'` point, every edge through it is +untraversable, and no new route can be set over it. No fault is latched. + +**To routes already holding it:** each `active`/`suspended` reservation holding +that point gets its loco stopped and a `RouteFault` of the existing kind +`'point-not-confirmed'`. This is the same shape D8 gives a `timeout`/`mismatch`/ +`indeterminate`, minus the `PointFault` — and it is exactly the precedent the +sensor side already sets. `runSensorTrustSweep` degrades a stale sensor's blocks +and touches no fault, and the route-level consequence arrives through +`recomputeBlock`'s existing `occupancy-unknown` path: loco stopped, `RouteFault` +latched, no `SensorFault`. A stale point is the same event on the other channel, +and answering it differently would be an inconsistency with no argument behind +it. + +**The resume precondition needs no special case, because resume tests the +staleness.** D8's precondition refuses a resume while any held point carries a +latched, unacknowledged `PointFault` — and a stale point has none, so a resume +proceeds. That is correct rather than a hole: `LayoutService.resumeRoute` +re-commands every point the route holds, which puts each `'required'` one back to +`pending` and arms the 8 s deadline. A controller that has recovered answers and +the point confirms; a controller that is genuinely dead does not, the point times +out, a `PointFault` latches, and the route re-suspends. The operator's resume is +therefore a live probe of the thing that was in doubt, and it resolves in 8 s +rather than in a freshness window. + +Adding a staleness clause to the precondition would have blocked that probe and +left the operator holding a route that could not be resumed until a device that +may already be healthy happened to speak again. + +--- + ## The commanded-over-serial / confirmed-over-MQTT asymmetry This is worth stating as its own fact because it is easy to assume symmetry @@ -396,10 +502,14 @@ see. provisioning that mapping over MQTT or HTTP is a separate design question, to be raised in the firmware repository (`bazauto/esp-layout-controller`), not here. -3. **#28's sensor-side mirror.** This document writes the retention asymmetry - (D1) into `docs/mqtt-contract.md` as a recorded decision, which is #28's - minimum requirement. Whether `sensor/*/reading` additionally needs its own - liveness/staleness check remains #28's call, not this document's. +3. **#28's sensor-side mirror — settled, in both directions.** This document + wrote the retention asymmetry (D1) into `docs/mqtt-contract.md`, which was + #28's minimum requirement; #28 then answered its own half with the 30 s + re-assert and `isSensorFresh`. The *point*-side equivalent was never posed at + the time, because the only point controller then contemplated was one that + would also command the motors — the case where silence after a command is at + least suspicious. `bazauto/layout-feedback#15` made it a separate device on a + separate transport, and #167 answered it in D11/D12 above. ## Deferred, and stated so nobody has to ask diff --git a/packages/backend/src/adapters/simulator/SimulatedPointController.ts b/packages/backend/src/adapters/simulator/SimulatedPointController.ts index d1603a9..167171f 100644 --- a/packages/backend/src/adapters/simulator/SimulatedPointController.ts +++ b/packages/backend/src/adapters/simulator/SimulatedPointController.ts @@ -23,7 +23,7 @@ */ import { IMqttAdapter } from '../../ports/IMqttAdapter'; -import { IClock } from '../../ports/IClock'; +import { ClockTimer, IClock } from '../../ports/IClock'; import { LayoutId, PointId } from '../../domain/types'; /** @@ -44,6 +44,14 @@ export interface SimulatedPointControllerOptions { modes?: Record; /** D9: delay, on the injected clock, before answering a query or a command with a reading. */ confirmDelayMs?: number; + /** + * D11 (#167): how often, on the injected clock, this controller re-asserts + * every point it knows about. Defaults to the contract's 30 s. A real + * controller is obliged to do this; a simulator that did not would make + * every `positionFeedback: 'required'` point go `'stale'` 90 s after its + * last command, which is a simulator defect rather than a system one. + */ + reassertIntervalMs?: number; } export interface SimulatedPointControllerLogger { @@ -62,6 +70,15 @@ export class SimulatedPointController { * hand-thrown point happens to be sitting at". */ private readonly commanded = new Map(); + /** + * Every point this controller has heard of, by command or by query — what + * the D11 re-assert loop walks. A real controller knows its own points from + * its pin allocation; this one learns them from traffic, which is the + * closest a simulator with no wiring can get. + */ + private readonly known = new Set(); + private reassertTimer: ClockTimer | null = null; + private readonly reassertIntervalMs: number; constructor( private readonly mqtt: IMqttAdapter, @@ -72,18 +89,49 @@ export class SimulatedPointController { ) { this.defaultMode = options.defaultMode ?? 'confirm'; this.confirmDelayMs = options.confirmDelayMs ?? 150; + this.reassertIntervalMs = options.reassertIntervalMs ?? 30_000; for (const [pointId, mode] of Object.entries(options.modes ?? {})) { this.modes.set(pointId, mode); } } - /** Subscribes to `point/+/query`. The only MQTT subscription this controller ever makes. */ + /** Subscribes to `point/+/query` — the only MQTT subscription this controller ever makes — and arms the D11 re-assert loop. */ async start(): Promise { await this.mqtt.subscribe(`layout/${this.layoutId}/point/+/query`, (_payload, topic) => { const pointId = topic.split('/')[3]; + this.known.add(pointId); this.scheduleResponse(pointId); }); - this.log.info('[SimPointController] Subscribed to point queries', { layoutId: this.layoutId }); + this.reassertTimer = this.clock.setInterval(() => { + void this.reassertAll(); + }, this.reassertIntervalMs); + this.log.info('[SimPointController] Subscribed to point queries', { + layoutId: this.layoutId, + reassertIntervalMs: this.reassertIntervalMs, + }); + } + + /** Cancels the re-assert loop. On `IClock`, so it is the harness's `ManualClock` in tests and a real timer only in the simulator binary. */ + stop(): void { + this.reassertTimer?.cancel(); + this.reassertTimer = null; + } + + /** + * D11 (#167): re-publishes every known point's current reading, unchanged, + * on the contract's interval — the liveness assertion `evaluateStaleness` + * measures against. + * + * Goes through `respond` rather than publishing directly, so a controller in + * `'silent'` mode stays silent here too. That is the whole point of the mode: + * `'silent'` is a dead controller, and a dead controller does not re-assert. + * It is what lets a scenario take a point from `'confirmed'` to `'stale'` + * simply by flipping the mode and letting the clock run. + */ + private async reassertAll(): Promise { + for (const pointId of this.known) { + await this.respond(pointId); + } } /** @@ -94,6 +142,7 @@ export class SimulatedPointController { */ noteCommanded(pointId: PointId, position: 'normal' | 'reverse'): void { this.commanded.set(pointId, position); + this.known.add(pointId); this.scheduleResponse(pointId); } diff --git a/packages/backend/src/config.ts b/packages/backend/src/config.ts index b1c18d7..5d29e6a 100644 --- a/packages/backend/src/config.ts +++ b/packages/backend/src/config.ts @@ -95,6 +95,13 @@ export const config = { confirmTimeoutMs: parseInt(process.env.POINT_CONFIRM_TIMEOUT_MS ?? '8000', 10), /** D5: how often (ms) the confirmation sweep evaluates the timeout predicate. */ sweepIntervalMs: parseInt(process.env.POINT_CONFIRM_SWEEP_MS ?? '250', 10), + /** + * D11 (#167): how long (ms) a point configured `positionFeedback: + * "required"` may go without a reading before its confirmed position + * degrades to `"stale"`/`"unknown"`. 90 s = 3 x the contract's 30 s + * re-assert interval, matching `SENSOR_FRESHNESS_TIMEOUT_MS`. + */ + freshnessTimeoutMs: parseInt(process.env.POINT_FRESHNESS_TIMEOUT_MS ?? '90000', 10), /** * D4: consecutive confirming readings a latched `PointFault` needs before * an operator may acknowledge it. Layout-wide, not per-point — the diff --git a/packages/backend/src/domain/pointConfirmation.ts b/packages/backend/src/domain/pointConfirmation.ts index 099f41d..a41bff1 100644 --- a/packages/backend/src/domain/pointConfirmation.ts +++ b/packages/backend/src/domain/pointConfirmation.ts @@ -21,9 +21,26 @@ import { PointState, } from './types'; -/** Confirmation-timeout policy (D5). `timeoutMs` defaults to 8000 in `LayoutService`'s config — this module knows nothing about the default. */ +/** + * The freshness window's default, in ms (#167, D11) — 3 x the contract's 30 s + * re-assert interval, the same tolerance and the same reasoning as + * `DEFAULT_SENSOR_FRESHNESS_TIMEOUT_MS`. Three intervals absorbs two + * consecutive lost messages, which is what stops ordinary WiFi packet loss + * flapping a confirmed point to `'stale'`. + * + * `config.points.freshnessTimeoutMs` is what production reads; this is the + * fallback and the number the tests are written against. + */ +export const DEFAULT_POINT_FRESHNESS_TIMEOUT_MS = 90_000; + +/** + * Confirmation policy. `timeoutMs` is D5's confirmation deadline (8000 in + * `LayoutService`'s config); `freshnessTimeoutMs` is D11's staleness window. + * This module knows neither default — both are supplied by the caller. + */ export interface PointConfirmationPolicy { timeoutMs: number; + freshnessTimeoutMs: number; } /** @@ -147,6 +164,59 @@ export function evaluateTimeout(p: PointState, now: Date, policy: PointConfirmat }; } +/** + * Applies D11's staleness predicate: a point whose controller has stopped + * re-asserting. `null` unless the point is `positionFeedback: 'required'`, + * currently `'confirmed'`, and has a `lastReadingAt` older than + * `policy.freshnessTimeoutMs` — in which case `'stale'` with + * `confirmedPosition: 'unknown'`. + * + * Returns `null` rather than `p` for a point with nothing to evaluate, for + * exactly the reason `evaluateTimeout` does: it is what lets the sweep tell a + * transition from a no-op without a reference-equality check, and what keeps + * a 250 ms tick from re-publishing every point on the layout. + * + * The three narrowings are each load-bearing, and D11 records why: + * + * - **`'required'` only.** A `'none'` point has nothing reporting on it and + * would go stale immediately and permanently. Every point on Westgate + * Hollow is `'none'` today, so this predicate fires nowhere on the live + * layout until feedback hardware is fitted and a point is opted in. + * - **From `'confirmed'` only.** That is the one state where a reading is + * actually being trusted. `'unreported'` is D6's boot case and must stay + * itself; `'pending'` belongs to the 8 s deadline, which fires an order of + * magnitude sooner; `'mismatch'`/`'indeterminate'`/`'timed-out'` are + * already latched faults, and re-labelling one `'stale'` would replace a + * sharp fact with a vaguer one. + * - **Measured on `lastReadingAt`**, the backend's own receipt clock, never + * on the payload's `reportedAt` — that timestamp comes from the very + * device whose liveness is in question (the same rule `isSensorFresh` + * states). + * + * The boundary matches `isSensorFresh` deliberately: an age strictly greater + * than the window is stale, and **exactly equal is fresh**. A negative age (a + * reading stamped in the future, which under `ManualClock` means a test moved + * the clock backwards) reads as fresh, so a clock skew cannot silently + * untrust a healthy point. + */ +export function evaluateStaleness( + p: PointState, + now: Date, + policy: PointConfirmationPolicy, +): PointState | null { + if (p.positionFeedback !== 'required') return null; + if (p.confirmation !== 'confirmed') return null; + if (p.lastReadingAt === null) return null; + if (now.getTime() - p.lastReadingAt.getTime() <= policy.freshnessTimeoutMs) return null; + return { + ...p, + confirmation: 'stale', + confirmedPosition: 'unknown', + awaitingSince: null, + lastUpdated: now, + }; +} + /** * The single place that decides what a point's position is trusted to be * (D7). Consumed by the UI and by any road-confirmation check — deliberately diff --git a/packages/backend/src/domain/types.ts b/packages/backend/src/domain/types.ts index 7836d45..e27a22e 100644 --- a/packages/backend/src/domain/types.ts +++ b/packages/backend/src/domain/types.ts @@ -614,6 +614,12 @@ export type PointFeedbackMode = 'none' | 'required'; * `'driver'`-sourced reading on a `'required'` point (D3). * - `'timed-out'` — the confirmation deadline elapsed with no reading * (D5). + * - `'stale'` — the point HAD confirmed, and then its controller + * stopped re-asserting inside the freshness window (#167, D11). Distinct + * from `'unreported'` on purpose: "never heard from" is ordinary at boot, + * "heard, then went quiet" means a node died. Unlike every state above it + * except `'confirmed'` and `'unreported'`, this latches NO `PointFault` — + * silence is a device dying, not a device lying (D11). */ export type PointConfirmation = | 'unreported' @@ -621,7 +627,8 @@ export type PointConfirmation = | 'confirmed' | 'mismatch' | 'indeterminate' - | 'timed-out'; + | 'timed-out' + | 'stale'; /** * A point's live confirmation state (#25). Replaces the pre-#25 single @@ -642,6 +649,7 @@ export interface PointState { positionFeedback: PointFeedbackMode; /** Non-null iff `confirmation === 'pending'` — the invariant `applyPointReading`/`evaluateTimeout`/`onPointCommanded` maintain. */ awaitingSince: Date | null; + /** When the last reading landed, on the backend's own clock. `null` = never. This is what `evaluateStaleness` measures the freshness window against (#167 D11), the same role `SensorObservation.lastLiveReadingAt` plays for a sensor. */ lastReadingAt: Date | null; /** Whether this point is locked by an active route. */ locked: boolean; diff --git a/packages/backend/src/index.ts b/packages/backend/src/index.ts index 3e24add..2a7e86b 100644 --- a/packages/backend/src/index.ts +++ b/packages/backend/src/index.ts @@ -135,6 +135,7 @@ async function main() { const clock = new SystemClock(); const pointConfirmations = new PointConfirmationService(stateManager, { timeoutMs: config.points.confirmTimeoutMs, + freshnessTimeoutMs: config.points.freshnessTimeoutMs, }); // #25 D9: a genuine simulated twin of the ESP point controller — ships @@ -265,6 +266,7 @@ async function main() { const shutdown = async (signal: string) => { console.log(`[Shutdown] Received ${signal}`); await server.close(); + simulatedPointController?.stop(); await layoutService.stop(); process.exit(0); }; diff --git a/packages/backend/src/services/LayoutService.ts b/packages/backend/src/services/LayoutService.ts index 7e074b1..2615e74 100644 --- a/packages/backend/src/services/LayoutService.ts +++ b/packages/backend/src/services/LayoutService.ts @@ -90,6 +90,7 @@ import { toSensorObservationView, } from '../domain/occupancy'; import { DEFAULT_SENSOR_FRESHNESS_TIMEOUT_MS, isSensorFresh } from '../domain/sensorTrust'; +import { DEFAULT_POINT_FRESHNESS_TIMEOUT_MS } from '../domain/pointConfirmation'; import { isEmptySensorPayload } from '../domain/sensorPayload'; import { sensorPositionOf } from '../domain/sensorPosition'; import { @@ -434,6 +435,7 @@ export class LayoutService extends EventEmitter { stateManager, { timeoutMs: 8000, + freshnessTimeoutMs: DEFAULT_POINT_FRESHNESS_TIMEOUT_MS, }, ), /** @@ -2007,7 +2009,18 @@ export class LayoutService extends EventEmitter { this.emitPointFaults(); } - /** D5: applies `evaluateTimeout` to every registered point via the sweep, publishing and fault-latching each transition. */ + /** + * D5 + D11: applies the sweep's two predicates to every registered point, + * publishing each transition and giving it its consequence. + * + * The two consequences are deliberately different, and the difference is the + * whole of D11: `'timed-out'` latches a `PointFault` and Safe-Stops (a device + * **lying** about a command the backend issued), while `'stale'` latches no + * point fault at all (a device **dying**, which degrades only what it + * observes). Both stop the locos of any routes holding the point and latch a + * `RouteFault`, because in both cases route R's road is no longer known to be + * set. + */ private async runConfirmationSweep(): Promise { const transitioned = this.pointConfirmations.sweep(this.clock.now()); for (const point of transitioned) { @@ -2019,10 +2032,47 @@ export class LayoutService extends EventEmitter { 'timeout', `Point ${pointLabel(point.pointId, this.names.get())} failed to confirm within the timeout — no reading received`, ); + } else if (point.confirmation === 'stale') { + await this.handlePointWentStale(point); } } } + /** + * D11/D12 (#167): the consequence of a confirmed point's controller going + * quiet. The degrade half of `handlePointNotConfirmed` — same route + * consequence, no `PointFault`. + * + * This is `runSensorTrustSweep`'s shape on the other channel, and the + * symmetry is the argument: a stale sensor degrades its own blocks, latches + * no `SensorFault`, and reaches routes only through `recomputeBlock`'s + * `occupancy-unknown` path. A stale point degrades its own position, latches + * no `PointFault`, and reaches routes through `point-not-confirmed`. + * + * No staleness clause is added to `resumeRoute`'s precondition, and D12 + * records why: `resumeRoute` re-commands every held point, which puts a + * `'required'` one back to `'pending'` and arms the 8 s deadline. A recovered + * controller answers and the point confirms; a dead one does not, times out, + * latches a real `PointFault`, and the route re-suspends. The resume IS the + * probe, and it settles in 8 s rather than in a freshness window. + */ + private async handlePointWentStale(point: PointState): Promise { + this.log.warn( + '[LayoutService] Point stale — no reading inside the freshness window, position no longer trusted', + { + layoutId: this.layoutId, + pointId: point.pointId, + pointName: this.names.get().points.get(point.pointId), + lastReadingAt: point.lastReadingAt?.toISOString() ?? null, + }, + ); + + await this.faultRoutesHoldingPoint( + point.pointId, + `stopped re-asserting its position (stale)`, + ); + } + /** D5: started next to the heartbeat, stopped in `stop()`. Runs on `this.clock`, never a bare `setInterval` — the seam `ManualClock` drives in tests. */ private startConfirmationSweep(): void { this.confirmationSweepTimer = this.clock.setInterval(() => { @@ -2496,7 +2546,27 @@ export class LayoutService extends EventEmitter { pointFaultReason: string, ): Promise { await this.raisePointFault(pointId, kind, pointFaultReason); + await this.faultRoutesHoldingPoint(pointId, `failed to confirm (${kind})`); + } + /** + * The route half of D8, extracted so D11's staleness can reuse it without + * the `PointFault` (D12). Stops the loco of every `active`/`suspended` + * reservation holding this point and latches a `'point-not-confirmed'` + * `RouteFault` naming it. + * + * The route is never cancelled here: `raiseRouteFault` re-evaluates + * Safe-Stop, and Safe-Stop suspends every active reservation with its locks + * retained (`docs/route-locking.md` D8) — the same mechanism + * `handleRouteOccupancyUnknown` relies on rather than suspending the one + * route directly. + * + * `situation` completes the sentence "point X ..." in the operator-facing + * reason, so the two callers read as what they are: "failed to confirm + * (timeout)" for a fault, "stopped re-asserting its position (stale)" for a + * degrade. + */ + private async faultRoutesHoldingPoint(pointId: PointId, situation: string): Promise { if (!this.layoutId) return; const routes = this.reservations.routesHoldingPoint(this.layoutId, pointId); for (const route of routes) { @@ -2504,7 +2574,7 @@ export class LayoutService extends EventEmitter { await this.raiseRouteFault({ routeId: route.id, kind: 'point-not-confirmed', - reason: `Route ${route.id} suspended: point ${pointLabel(pointId, this.names.get())} failed to confirm (${kind})`, + reason: `Route ${route.id} suspended: point ${pointLabel(pointId, this.names.get())} ${situation}`, blockId: null, pointId, locoAddress: route.locoAddress, diff --git a/packages/backend/src/services/PointConfirmationService.ts b/packages/backend/src/services/PointConfirmationService.ts index 2361ef8..391432e 100644 --- a/packages/backend/src/services/PointConfirmationService.ts +++ b/packages/backend/src/services/PointConfirmationService.ts @@ -14,6 +14,7 @@ import { applyPointReading, confirmationArms, + evaluateStaleness, evaluateTimeout, onPointCommanded, onPointQueried, @@ -109,15 +110,28 @@ export class PointConfirmationService { } /** - * Applies `evaluateTimeout` to every registered point, storing and - * returning only the ones that transitioned — never the whole layout, - * so a caller publishing `point/state` per entry does not re-publish - * every point on the layout on every 250ms tick (D5). + * Applies `evaluateTimeout` (D5) and then `evaluateStaleness` (D11) to every + * registered point, storing and returning only the ones that transitioned — + * never the whole layout, so a caller publishing `point/state` per entry does + * not re-publish every point on the layout on every 250ms tick. + * + * The two predicates cannot both fire on one point: a timeout needs + * `'pending'` and staleness needs `'confirmed'`. Evaluating the timeout first + * and the staleness against its result is therefore ordering for clarity + * rather than for correctness — but it is the ordering that stays correct if + * either predicate's precondition is ever widened, because a point that has + * just timed out is already `'timed-out'` and no longer a staleness + * candidate. + * + * The caller distinguishes the two outcomes by the returned + * `confirmation` — `'timed-out'` is a fault, `'stale'` is a degrade — which is + * the same discrimination `runConfirmationSweep` already did for the one kind. */ sweep(now: Date): PointState[] { const transitioned: PointState[] = []; for (const point of this.stateManager.getState().points.values()) { - const next = evaluateTimeout(point, now, this.policy); + const timedOut = evaluateTimeout(point, now, this.policy); + const next = evaluateStaleness(timedOut ?? point, now, this.policy) ?? timedOut; if (next) { transitioned.push(this.stateManager.setPointState(point.pointId, next)); } diff --git a/packages/backend/tests/scenario/harness.ts b/packages/backend/tests/scenario/harness.ts index 4f5aa28..1bf8e6c 100644 --- a/packages/backend/tests/scenario/harness.ts +++ b/packages/backend/tests/scenario/harness.ts @@ -53,6 +53,16 @@ export const POINT_CONFIRM_TIMEOUT_MS = 8000; export const POINT_CONFIRM_SWEEP_MS = 250; export const POINT_CONFIRM_DELAY_MS = 150; +/** + * #167 D11: the point-side twin of `SENSOR_FRESHNESS_TIMEOUT_MS` / + * `SENSOR_REASSERT_MS` below. `POINT_REASSERT_MS` is not a backend setting + * either — it is the interval `docs/mqtt-contract.md` obliges point-controller + * *firmware* to re-publish on, and the number the freshness window is three of. + * A scenario simulating a healthy point controller re-asserts on it. + */ +export const POINT_FRESHNESS_TIMEOUT_MS = 90_000; +export const POINT_REASSERT_MS = 30_000; + /** * #28 D11: the same defaults `config.sensors` carries. `SENSOR_REASSERT_MS` is * not a backend setting at all — it is the interval `docs/mqtt-contract.md` @@ -466,6 +476,7 @@ export function createScenarioHarness(options?: Partial): const pointClock = new ManualClock(new Date('2026-01-01T00:00:00.000Z')); const pointConfirmations = new PointConfirmationService(stateManager, { timeoutMs: POINT_CONFIRM_TIMEOUT_MS, + freshnessTimeoutMs: POINT_FRESHNESS_TIMEOUT_MS, }); // D9: a genuine simulated twin of the ESP point controller — subscribes to // `point/+/query` and answers on `pointClock`, never a live MQTT command diff --git a/packages/backend/tests/scenario/point-liveness.scenario.test.ts b/packages/backend/tests/scenario/point-liveness.scenario.test.ts new file mode 100644 index 0000000..19dd245 --- /dev/null +++ b/packages/backend/tests/scenario/point-liveness.scenario.test.ts @@ -0,0 +1,319 @@ +/** + * Scenario: point controller liveness (#167, see docs/point-feedback.md D11 + * and D12, and docs/mqtt-contract.md's Degradation Triggers). + * + * The failure this file exists for is the one nothing caught before #167: a + * point controller that dies **quietly**. Not a stalled motor (that is a + * timeout, covered in `point-feedback-routes.scenario.test.ts`), not a + * mechanical bind (that is a mismatch) — a controller that confirmed + * correctly and then simply stopped speaking, leaving its last + * `confirmedPosition` standing while the layout went on setting roads over + * it. + * + * The distinction every scenario here turns on: **silence is a device dying, + * not a device lying.** A timeout latches a `PointFault` and Safe-Stops; a + * stale point latches nothing and degrades only itself. What it has in common + * with a timeout is the route consequence, because in both cases route R's + * road is no longer known to be set. + * + * Fixture, mirroring `point-feedback-routes.scenario.test.ts`: + * + * b1 --e1 (p1=normal, positionFeedback: 'required')--> b2 + * + * p2 is a `positionFeedback: 'none'` point no edge references — the live + * Westgate Hollow case, and the guard that this feature changes nothing there. + */ + +import { describe, it, expect } from 'vitest'; +import { + createScenarioHarness, + LAYOUT_ID, + POINT_CONFIRM_DELAY_MS, + POINT_CONFIRM_SWEEP_MS, + POINT_CONFIRM_TIMEOUT_MS, + POINT_FRESHNESS_TIMEOUT_MS, + POINT_REASSERT_MS, +} from './harness'; +import { PointRecord } from '../../src/ports/ILayoutRepository'; +import { GrantRequest } from '../../src/services/ReservationService'; + +const LOCO_3 = { + id: 'loco-3', + layoutId: LAYOUT_ID, + name: 'Loco 3', + address: 3, + type: 'diesel', + maxSpeed: 126, + brakingFactor: 0.5, +}; + +const BLOCKS = [ + { id: 'b1', layoutId: LAYOUT_ID, name: 'Block 1' }, + { id: 'b2', layoutId: LAYOUT_ID, name: 'Block 2' }, +]; + +function point(id: string, feedback: 'required' | 'none'): PointRecord { + return { + id, + layoutId: LAYOUT_ID, + name: `Point ${id}`, + dccAddress: 10, + blockId: 'b1', + positionFeedback: feedback, + }; +} + +const SENSORS = [ + { + id: 's1', + layoutId: LAYOUT_ID, + name: 'Sensor 1', + type: 'block_detection' as const, + blockId: 'b1', + mqttTopic: `layout/${LAYOUT_ID}/sensor/s1/reading`, + inService: true, + }, + { + id: 's2', + layoutId: LAYOUT_ID, + name: 'Sensor 2', + type: 'block_detection' as const, + blockId: 'b2', + mqttTopic: `layout/${LAYOUT_ID}/sensor/s2/reading`, + inService: true, + }, +]; + +async function seedAndStart(h: ReturnType) { + h.repo._setBlocks(BLOCKS); + h.repo._setPoints([point('p1', 'required'), point('p2', 'none')]); + h.repo._setSensors(SENSORS); + h.repo._setLocos([LOCO_3]); + await h.repo.createBlockEdge({ + layoutId: LAYOUT_ID, + fromBlockId: 'b1', + fromEnd: 'east', + toBlockId: 'b2', + toEnd: 'west', + pointConditions: [{ pointId: 'p1', requiredPosition: 'normal' }], + lengthMm: null, + }); + + await h.start(); + await h.sensorReports('s2', 'clear'); +} + +async function grantRoute(h: ReturnType) { + const edges = await h.repo.listBlockEdges(LAYOUT_ID); + const e1 = edges.find((e) => e.fromBlockId === 'b1' && e.toBlockId === 'b2'); + if (!e1) throw new Error('no edge b1 -> b2'); + + const request: GrantRequest = { + locoAddress: 3, + authority: 'manual', + startBlockId: 'b1', + path: { kind: 'edges', edgeIds: [e1.id] }, + }; + const grant = await h.service.requestRoute(request); + if (!grant.granted) throw new Error(`expected grant: ${JSON.stringify(grant.rejections)}`); + return grant.reservation.id; +} + +/** + * Runs the clock forward in freshness-window-sized steps while the simulated + * controller keeps re-asserting on the contract's 30 s interval — i.e. what a + * healthy layout does between commands. + */ +async function runHealthyFor(h: ReturnType, ms: number) { + let elapsed = 0; + while (elapsed < ms) { + const step = Math.min(POINT_REASSERT_MS, ms - elapsed); + await h.advance(step); + elapsed += step; + } +} + +describe('scenario: point controller liveness (#167)', () => { + it('1. a confirmed point whose controller goes quiet degrades to stale with confirmedPosition unknown — and latches NO point fault', async () => { + const h = createScenarioHarness(); + await seedAndStart(h); + await h.commandPoint('p1', 'normal'); + await h.advance(POINT_CONFIRM_DELAY_MS); + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('confirmed'); + + // The controller dies. Nothing announces this — that is the whole problem. + h.pointController.setDefaultMode('silent'); + await h.advance(POINT_FRESHNESS_TIMEOUT_MS + POINT_CONFIRM_SWEEP_MS); + + const p1 = h.service.getAllState().points.get('p1'); + expect(p1?.confirmation).toBe('stale'); + expect(p1?.confirmedPosition).toBe('unknown'); + + // A degrade, not a fault: silence is a device dying, not a device lying. + expect(h.service.getPointFaults()).toEqual([]); + // No route held it, so nothing else was affected either. + expect(h.service.getRouteFaults()).toEqual([]); + expect(h.service.getSystemStatus().status).toBe('online'); + + await h.service.stop(); + }); + + it('2. a healthy controller re-asserting on the contract interval never goes stale, however long it runs', async () => { + const h = createScenarioHarness(); + await seedAndStart(h); + await h.commandPoint('p1', 'normal'); + await h.advance(POINT_CONFIRM_DELAY_MS); + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('confirmed'); + + // Ten freshness windows of a controller doing what the contract obliges it to. + await runHealthyFor(h, POINT_FRESHNESS_TIMEOUT_MS * 10); + + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('confirmed'); + expect(h.service.getAllState().points.get('p1')?.confirmedPosition).toBe('normal'); + expect(h.service.getPointFaults()).toEqual([]); + + // The re-assert is NOT a licence to retain (docs/mqtt-contract.md D1, and + // the amended retention callout): re-assertion is necessary for retention + // and not sufficient for it, because a point's position can still change + // while its controller is offline. Checked on the wire, not at the call site. + expect(h.publishedRetained(`layout/${LAYOUT_ID}/point/p1/reading`)).toBe(false); + + await h.service.stop(); + }); + + it("3. a 'none' point never goes stale — nothing reports on it, and every live Westgate Hollow point is one", async () => { + const h = createScenarioHarness(); + await seedAndStart(h); + await h.commandPoint('p2', 'reverse'); + await h.advance(POINT_CONFIRM_DELAY_MS); + + h.pointController.setDefaultMode('silent'); + await h.advance(POINT_FRESHNESS_TIMEOUT_MS * 5); + + const p2 = h.service.getAllState().points.get('p2'); + expect(p2?.confirmation).not.toBe('stale'); + expect(p2?.commandedPosition).toBe('reverse'); + expect(h.service.getPointFaults()).toEqual([]); + expect(h.service.getSystemStatus().status).toBe('online'); + + await h.service.stop(); + }); + + it('4. a route holding the point when it goes stale is suspended with locks retained, its loco stopped, and a RouteFault latched — but still no PointFault', async () => { + const h = createScenarioHarness(); + await seedAndStart(h); + await h.sensorReports('s1', 'occupied'); + + const routeId = await grantRoute(h); + await h.advance(POINT_CONFIRM_DELAY_MS); + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('confirmed'); + + h.dcc.clearLog(); + h.pointController.setDefaultMode('silent'); + await h.advance(POINT_FRESHNESS_TIMEOUT_MS + POINT_CONFIRM_SWEEP_MS); + + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('stale'); + + // Suspended, not cancelled — the locks stay, because a train may be standing on it. + expect(h.service.listRoutes(['suspended']).map((r) => r.id)).toContain(routeId); + expect(h.service.getAllState().blocks.get('b1')?.lockedByRoute).toBe(routeId); + expect(h.service.getAllState().blocks.get('b2')?.lockedByRoute).toBe(routeId); + expect(h.service.getAllState().points.get('p1')?.lockedByRoute).toBe(routeId); + + // The loco was stopped on the wire, unconditionally — not gated on authority. + expect( + h.dcc.commandLog.some( + (c) => c.type === 'SET_SPEED' && c.data.speed === 0 && c.data.address === 3, + ), + ).toBe(true); + expect(h.service.getAllState().locos.get(3)?.speed).toBe(0); + + // One latch, not two: the route's road is in doubt, the point itself is not faulty. + expect(h.service.getPointFaults()).toEqual([]); + const routeFaults = h.service.getRouteFaults(); + expect(routeFaults).toHaveLength(1); + expect(routeFaults[0]).toMatchObject({ routeId, kind: 'point-not-confirmed', pointId: 'p1' }); + expect(routeFaults[0].reason).toContain('stale'); + + await h.service.stop(); + }); + + it('5. a stale point recovers on the next reading with nothing to acknowledge — it latched nothing to clear', async () => { + const h = createScenarioHarness(); + await seedAndStart(h); + await h.commandPoint('p1', 'normal'); + await h.advance(POINT_CONFIRM_DELAY_MS); + + h.pointController.setDefaultMode('silent'); + await h.advance(POINT_FRESHNESS_TIMEOUT_MS + POINT_CONFIRM_SWEEP_MS); + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('stale'); + + // The controller comes back and re-asserts on its own schedule. + h.pointController.setDefaultMode('confirm'); + await h.advance(POINT_REASSERT_MS); + + const p1 = h.service.getAllState().points.get('p1'); + expect(p1?.confirmation).toBe('confirmed'); + expect(p1?.confirmedPosition).toBe('normal'); + expect(h.service.getPointFaults()).toEqual([]); + + await h.service.stop(); + }); + + it('6. D12: resuming a route over a stale point is allowed, and the resume IS the probe — a recovered controller confirms it', async () => { + const h = createScenarioHarness(); + await seedAndStart(h); + await h.sensorReports('s1', 'occupied'); + + const routeId = await grantRoute(h); + await h.advance(POINT_CONFIRM_DELAY_MS); + + h.pointController.setDefaultMode('silent'); + await h.advance(POINT_FRESHNESS_TIMEOUT_MS + POINT_CONFIRM_SWEEP_MS); + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('stale'); + expect(h.service.listRoutes(['suspended']).map((r) => r.id)).toContain(routeId); + + // The controller was rebooting, not dead. Clear the route fault the degrade + // raised, then resume: no PointFault stands in the way, because none was + // ever latched. + h.pointController.setDefaultMode('confirm'); + await h.service.acknowledgeRouteFault(LAYOUT_ID, routeId); + + const resumed = await h.service.resumeRoute(routeId); + expect(resumed.resumed).toBe(true); + + // The resume re-commanded the point, which is what tests the staleness. + await h.advance(POINT_CONFIRM_DELAY_MS + POINT_CONFIRM_SWEEP_MS); + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('confirmed'); + expect(h.service.listRoutes(['active']).map((r) => r.id)).toContain(routeId); + + await h.service.stop(); + }); + + it('7. D12: the same resume against a genuinely dead controller times out into a real PointFault, and the route re-suspends', async () => { + const h = createScenarioHarness(); + await seedAndStart(h); + await h.sensorReports('s1', 'occupied'); + + const routeId = await grantRoute(h); + await h.advance(POINT_CONFIRM_DELAY_MS); + + h.pointController.setDefaultMode('silent'); + await h.advance(POINT_FRESHNESS_TIMEOUT_MS + POINT_CONFIRM_SWEEP_MS); + await h.service.acknowledgeRouteFault(LAYOUT_ID, routeId); + + // Still silent. The resume proceeds — and settles the question in 8 s + // rather than in another freshness window. + const resumed = await h.service.resumeRoute(routeId); + expect(resumed.resumed).toBe(true); + + await h.advance(POINT_CONFIRM_TIMEOUT_MS + POINT_CONFIRM_SWEEP_MS); + + expect(h.service.getAllState().points.get('p1')?.confirmation).toBe('timed-out'); + expect(h.service.getPointFaults()).toMatchObject([{ pointId: 'p1', kind: 'timeout' }]); + expect(h.service.getSystemStatus().status).toBe('safe-stop'); + expect(h.service.listRoutes(['suspended']).map((r) => r.id)).toContain(routeId); + + await h.service.stop(); + }); +}); diff --git a/packages/backend/tests/unit/domain/pointConfirmation.test.ts b/packages/backend/tests/unit/domain/pointConfirmation.test.ts index 23990f7..1429889 100644 --- a/packages/backend/tests/unit/domain/pointConfirmation.test.ts +++ b/packages/backend/tests/unit/domain/pointConfirmation.test.ts @@ -4,6 +4,7 @@ import { buildPointPositionMap, confirmationArms, effectivePosition, + evaluateStaleness, evaluateTimeout, initialPointState, onPointCommanded, @@ -14,7 +15,7 @@ import { PointReading, PointState } from '../../../src/domain/types'; const NOW = new Date('2026-08-14T00:00:00.000Z'); const LATER = new Date('2026-08-14T00:00:09.000Z'); // +9000ms -const POLICY: PointConfirmationPolicy = { timeoutMs: 8000 }; +const POLICY: PointConfirmationPolicy = { timeoutMs: 8000, freshnessTimeoutMs: 90_000 }; function reading(overrides: Partial = {}): PointReading { return { @@ -216,6 +217,105 @@ describe('evaluateTimeout', () => { }); }); + +// ─── evaluateStaleness (#167 D11) ─────────────────────────────────────────── + +/** A `required` point that has confirmed `normal` at `NOW` — the only state staleness can act on. */ +function confirmedAt(now: Date, feedback: 'required' | 'none' = 'required'): PointState { + const commanded = onPointCommanded(initialPointState('p1', feedback, now), 'normal', now); + return applyPointReading(commanded, reading({ position: 'normal' }), now); +} + +describe('evaluateStaleness (#167 D11)', () => { + it('goes stale past the window: stale, confirmedPosition unknown, awaitingSince cleared', () => { + const confirmed = confirmedAt(NOW); + const past = new Date(NOW.getTime() + 90_001); + const stale = evaluateStaleness(confirmed, past, POLICY); + expect(stale).not.toBeNull(); + expect(stale).toMatchObject({ + confirmation: 'stale', + confirmedPosition: 'unknown', + awaitingSince: null, + }); + assertAwaitingInvariant(stale!); + }); + + it('exactly at the window is still fresh — the boundary matches isSensorFresh', () => { + const confirmed = confirmedAt(NOW); + expect(evaluateStaleness(confirmed, new Date(NOW.getTime() + 90_000), POLICY)).toBeNull(); + expect(evaluateStaleness(confirmed, new Date(NOW.getTime() + 90_001), POLICY)).not.toBeNull(); + }); + + it('a re-assert inside the window keeps the point confirmed indefinitely', () => { + let p = confirmedAt(NOW); + // Three re-asserts at the contract's 30 s interval, checked just before each. + for (let i = 1; i <= 3; i += 1) { + const justBefore = new Date(NOW.getTime() + i * 30_000 - 1); + expect(evaluateStaleness(p, justBefore, POLICY)).toBeNull(); + p = applyPointReading(p, reading({ position: 'normal' }), new Date(NOW.getTime() + i * 30_000)); + expect(p.confirmation).toBe('confirmed'); + } + // ...and it is the RE-ASSERTS holding it up. The last one landed at + // NOW+90s, so the window runs out at NOW+180s and not before. + expect(evaluateStaleness(p, new Date(NOW.getTime() + 180_000), POLICY)).toBeNull(); + expect(evaluateStaleness(p, new Date(NOW.getTime() + 180_001), POLICY)).not.toBeNull(); + }); + + it("a 'none' point never goes stale — nothing reports on it (every live Westgate Hollow point today)", () => { + const confirmed = confirmedAt(NOW, 'none'); + expect(evaluateStaleness(confirmed, new Date(NOW.getTime() + 10_000_000), POLICY)).toBeNull(); + }); + + it("'unreported' stays unreported — D6's boot case is not a dead controller", () => { + const unreported = initialPointState('p1', 'required', NOW); + expect(unreported.lastReadingAt).toBeNull(); + expect(evaluateStaleness(unreported, new Date(NOW.getTime() + 10_000_000), POLICY)).toBeNull(); + }); + + it("'pending' is left to the 8 s confirmation deadline, which fires far sooner", () => { + const commanded = onPointCommanded(initialPointState('p1', 'required', NOW), 'reverse', NOW); + expect(evaluateStaleness(commanded, new Date(NOW.getTime() + 90_001), POLICY)).toBeNull(); + }); + + it('never re-labels an already-latched fault state — a sharp fact is not replaced by a vaguer one', () => { + const commanded = onPointCommanded(initialPointState('p1', 'required', NOW), 'normal', NOW); + const wayLater = new Date(NOW.getTime() + 90_001); + + const mismatch = applyPointReading(commanded, reading({ position: 'reverse' }), NOW); + expect(mismatch.confirmation).toBe('mismatch'); + expect(evaluateStaleness(mismatch, wayLater, POLICY)).toBeNull(); + + const indeterminate = applyPointReading(commanded, reading({ position: 'unknown' }), NOW); + expect(indeterminate.confirmation).toBe('indeterminate'); + expect(evaluateStaleness(indeterminate, wayLater, POLICY)).toBeNull(); + + const timedOut = evaluateTimeout(commanded, new Date(NOW.getTime() + 8000), POLICY)!; + expect(timedOut.confirmation).toBe('timed-out'); + expect(evaluateStaleness(timedOut, wayLater, POLICY)).toBeNull(); + }); + + it('a reading stamped in the future reads as fresh — a clock skew must not untrust a healthy point', () => { + const confirmed = confirmedAt(NOW); + expect(evaluateStaleness(confirmed, new Date(NOW.getTime() - 10_000_000), POLICY)).toBeNull(); + }); + + it('a stale point reads unknown through effectivePosition, which is what makes its edges untraversable', () => { + const stale = evaluateStaleness(confirmedAt(NOW), new Date(NOW.getTime() + 90_001), POLICY)!; + expect(effectivePosition(stale)).toBe('unknown'); + }); + + it('recovers on the next reading, with no acknowledgement — it latched nothing to clear', () => { + const stale = evaluateStaleness(confirmedAt(NOW), new Date(NOW.getTime() + 90_001), POLICY)!; + const recovered = applyPointReading( + stale, + reading({ position: 'normal' }), + new Date(NOW.getTime() + 91_000), + ); + expect(recovered.confirmation).toBe('confirmed'); + expect(effectivePosition(recovered)).toBe('normal'); + }); +}); + // ─── effectivePosition / buildPointPositionMap ───────────────────────────── describe('effectivePosition', () => { diff --git a/packages/backend/tests/unit/services/pointConfirmationService.test.ts b/packages/backend/tests/unit/services/pointConfirmationService.test.ts index a227432..1be89f3 100644 --- a/packages/backend/tests/unit/services/pointConfirmationService.test.ts +++ b/packages/backend/tests/unit/services/pointConfirmationService.test.ts @@ -5,7 +5,7 @@ import { PointConfirmationPolicy } from '../../../src/domain/pointConfirmation'; import { PointReading } from '../../../src/domain/types'; const NOW = new Date('2026-08-14T00:00:00.000Z'); -const POLICY: PointConfirmationPolicy = { timeoutMs: 8000 }; +const POLICY: PointConfirmationPolicy = { timeoutMs: 8000, freshnessTimeoutMs: 90_000 }; function reading(overrides: Partial = {}): PointReading { return { diff --git a/packages/frontend/src/components/LayoutPanel.tsx b/packages/frontend/src/components/LayoutPanel.tsx index a83f713..fb5763f 100644 --- a/packages/frontend/src/components/LayoutPanel.tsx +++ b/packages/frontend/src/components/LayoutPanel.tsx @@ -30,7 +30,9 @@ interface Props { * from the position itself, because the fact worth surfacing at a glance is * how far the position can be trusted — `mismatch`, `indeterminate` and * `timed-out` all read as fault-coloured but carry distinct glyphs and words, - * never colour alone. + * never colour alone. `stale` (#167) reads untrusted-yellow rather than + * fault-red on purpose: its controller went quiet, which degrades the position + * without latching anything. */ const UNSET = '#6c7086'; diff --git a/packages/frontend/src/diagram/encoding.ts b/packages/frontend/src/diagram/encoding.ts index a8f361d..fb86764 100644 --- a/packages/frontend/src/diagram/encoding.ts +++ b/packages/frontend/src/diagram/encoding.ts @@ -222,16 +222,25 @@ export const POINT_POSITION: Record<'normal' | 'reverse' | 'unknown', StateEncod * is drawn from can be trusted, and is what a point badge's colour/glyph are * keyed on rather than on the raw position (`LayoutPanel`, `PointKeyPanel`). * - * Six members, six glyphs — `mismatch`, `indeterminate` and `timed-out` all - * sit in "known bad" territory and must stay visibly distinct from one + * Seven members, seven glyphs — `mismatch`, `indeterminate` and `timed-out` + * all sit in "known bad" territory and must stay visibly distinct from one * another without relying on their (deliberately similar) fault colour, per * #81's rule that colour is never the sole carrier. `unreported` and * `pending` are both "no verdict yet" but for different reasons (never * commanded this session, vs. a deadline actively running) and get their own * glyphs for the same reason. + * + * `stale` (#167, docs/point-feedback.md D11) is the seventh, and it is + * deliberately NOT fault-coloured: a stale point is a **degrade**, not a + * fault — its controller went quiet, nothing latched, and it recovers on its + * own the moment a reading arrives. It takes the `unknown` yellow it shares + * with `indeterminate`, distinguished by its own glyph and word, because what + * the two have in common is exactly what the operator needs to read off the + * badge: the position cannot currently be trusted. Colouring it red would say + * "something is broken" about a node that may simply be rebooting. */ export const POINT_CONFIRMATION: Record< - 'unreported' | 'pending' | 'confirmed' | 'mismatch' | 'indeterminate' | 'timed-out', + 'unreported' | 'pending' | 'confirmed' | 'mismatch' | 'indeterminate' | 'timed-out' | 'stale', StateEncoding > = { unreported: { colour: '#6c7086', pattern: null, glyph: '–', label: 'unreported' }, @@ -240,6 +249,7 @@ export const POINT_CONFIRMATION: Record< mismatch: { colour: '#f38ba8', pattern: 'diag-occupied', glyph: '✗', label: 'mismatch' }, indeterminate: { colour: '#f9e2af', pattern: 'cross-unknown', glyph: '?', label: 'indeterminate' }, 'timed-out': { colour: '#f38ba8', pattern: 'cross-unknown', glyph: '⏱', label: 'timed-out' }, + stale: { colour: '#f9e2af', pattern: 'cross-unknown', glyph: '⌛', label: 'stale' }, }; /** diff --git a/packages/frontend/src/types.ts b/packages/frontend/src/types.ts index f4b1eb6..5ff3125 100644 --- a/packages/frontend/src/types.ts +++ b/packages/frontend/src/types.ts @@ -636,7 +636,9 @@ export interface BlockState { * commanded, `confirmedPosition` holding the reported value; `'indeterminate'` * — the last reading reported `'unknown'`, or was `'driver'`-sourced on a * `'required'` point; `'timed-out'` — the confirmation deadline elapsed with - * no reading. + * no reading; `'stale'` (#167) — the point HAD confirmed and then its + * controller stopped re-asserting inside the freshness window, which is a + * degrade rather than a fault and latches nothing. */ export type PointConfirmation = | 'unreported' @@ -644,7 +646,8 @@ export type PointConfirmation = | 'confirmed' | 'mismatch' | 'indeterminate' - | 'timed-out'; + | 'timed-out' + | 'stale'; /** * Mirrors `PointFeedbackMode` — whether a point is configured to require a