feat(#638): the overnight joint planner — pure packer + shadow mode (G2+G3) - #754
Open
traktore-org wants to merge 246 commits into
Open
feat(#638): the overnight joint planner — pure packer + shadow mode (G2+G3)#754traktore-org wants to merge 246 commits into
traktore-org wants to merge 246 commits into
Conversation
…G2+G3) One schedule for every floor: EV per-charger night targets (#634), load min-runtime deficits (#620), battery pre-charge — packed over one hourly price curve under one peak cap in one priority order (#576). coordinator/overnight_planner.py — plan_overnight(): a greedy priority- packer over price-sorted slots. Deliberately NOT an optimizer: pure, deterministic (ties break price->start->priority->id), ~170 lines, every allocation carries a one-line reason. Device floors are respected per slot: EV min-amps (a thin slot is skipped, a continuous battery takes it), switch loads are binary, the last slot is shortened rather than over-delivered. Floors stay guarantees — the packer moves WHEN/how-fast, never WHETHER; what doesn't fit under the cap is REPORTED as yielding (the 22:00 answer: "everything fits / X yields"). Shadow (G3): _shadow_overnight_plan runs after scheduler.evaluate — the same trigger cadence — building demands from the REAL models: build_night_target_map per charger (mode-gated; the #652 closure — the INFO line logs the scheduler's phantom EV model against the real map), load runtime deficits, the scheduler's own battery deficit_kwh. Hourly slots via tariff get_price_at; cap = peak_limit - expected overnight home. Logs summary at INFO, allocations at DEBUG, stashes _overnight_shadow_plan for diagnostics. Never actuates; any internal error degrades to a debug line (test-pinned with a hostile fake). test_battery_scheduler_rolling_horizon: the get_price_at assert_not_called pin loosened to what it meant — break-even rates come from the series (the 0.99 sentinel proves it); the shadow's curve reads are legitimate. Corpus: 13 packing scenarios incl. the #634 ensemble shape (EV + heater + battery under one cap — every floor met where reactive greed collides). Verification ladder: G1 baseline + G3 shadow accrue tonight on PROD from the same night; G4 (actuation flip) only on explicit call. Refs #638 #652 #634 #630 #576 #620
…ties + docs
Guido's review caught the modeling gap: WHERE a demand draws from changes
what constrains it. A Tier-2 "finish overnight from battery" load touches
no grid meter — no peak cap, no price — it spends the shared battery
budget (usable kWh above the reserve SOC). Demand.source ('grid'|'battery')
+ plan_overnight(battery_budget_kwh): battery demands pack time-ordered
(earliest, price-blind), cost 0, never consume slot cap; grid demands
(EV floors — never-from-battery standing rule — cheap-hours loads, the
battery pre-charge) compete for cap and pay the curve. 4 new corpus
scenarios pin it.
Priority sourcing corrected to the canonical one-list accessors (#576):
EV via _ev_priority_for(cid) (drag override wins immediately — same
accessor decide() uses; the cfg dict key was a weaker source), battery
via _device_registry.battery_surplus_priority() (the config key the
shadow read does not exist), loads unchanged (dev.priority). Expected
overnight home draw now uses _expected_night_home_w(energy) — the same
estimator the EV peak-managed rate uses — instead of a flat 300 W.
docs/OVERNIGHT_PLANNER.md: worked examples — with a peak cap (EV owns
the cheapest hours, a binary heater skips the thin remainder and still
fits, the battery soaks leftover cap), without one (everyone overlaps
freely, the report still catches deadline-unreachable floors), the
Tier-2 free-rider, and the 22:00 fits/yields answer. Priority = the one
drag list; accuracy = calibrated rated power + replan triggers.
Refs #638 #576 #652
…enabled Caught live on PROD within the first hour of the soak: zero OVERNIGHT-PLAN lines. Root cause — not log level (PROD logs SEM at debug) but placement: the shadow ran inside `if scheduler.enabled:` and battery_charge_scheduler_enabled defaults False (PROD: static tariff, summer, scheduler idle). An install without battery night charging still has EV floors and Tier-2 loads to plan every night. _run_battery_pipeline now carries its own once-per-evening trigger (_shadow_plan_date stamp, at the scheduler's trigger_hour) OUTSIDE the enabled branch; when the scheduler IS enabled its evaluate/replans still recompute on top. phantom=None (scheduler off) renders as "n/a" in the #652 evidence line instead of crashing the format. Pinned by test_shadow_fires_without_the_battery_scheduler (placement + None-phantom path). Refs #638
…review) One shared structure over existing accessors; four planners pack into it; explicit simplicity contract (non-goals). Red-team verified: 6/7 claims confirmed, discharge-limit refutation folded in (SEM's flow rules exist — the ledger reads them, doesn't reinvent). Refs #638
…iced slots honest + replan, level-consistency for cheap-hours loads
Guido's stress test ('will this survive Rien's tariff or just mine?'):
the ledger is provider-agnostic by construction, but three clauses were
implicit and one was a trap — packing a cheap_hours load into the
cheapest slot of a flat/static curve that execution's price_is_cheap
level-gate would never fire. Refs #638
… consciously build_night_ledger walks the night once: home draws the battery to the sunrise floor (power-bounded), the remainder lands on the meter, headroom follows — the takeover hour is DERIVED. Every battery mutation (Tier-2 draw, pre-charge arrival) re-walks the trajectory, so one Tier-2 kWh tonight honestly moves the takeover earlier and shrinks later headroom. pack_night packs demands by the one list against the ledger with the spec's tariff clauses: unpriced slots (day-ahead not published) are packed around and noted for the fingerprint replan; cheap-hours loads pack only into LEVEL-cheap slots (execution's price_is_cheap gate — a flat/static night with no cheap level yields honestly instead of scheduling a run execution would never fire); slots follow the market granularity (15-min native). #688 anti-cycle quantization: runs shorter than min-run are planned AT min-run (what can_deactivate would do), battery runs stay energy-capped (the reserve gate), and blocks of one load never sit closer than min-gap — contiguous slots are one continuous run. plan_overnight stays as a thin flat-slot adapter for the parked shadow hook. Corpus revision (conscious, per spec): the sunrise-floor scenario had Tier-2 outranking home — spec says home's claim on the battery is absolute, so 0.5 kWh above the floor goes to home's first hour and Tier-2 yields with zero; the min-gap scenario asserted a gap between contiguous blocks — contiguous is one run, a REAL gap (non-adjacent cheap slot inside the pause window) is now forced and pinned. Packer bug caught by corpus: battery constrains ENERGY/duration, not power — a binary 2 kW heater with 1 kWh above floor runs 2 kW for 30 min, not "average 1 kW below its floor". 31 tests green (13 flat-packing kept via the adapter + 4 source-axis + trajectory/takeover/sunrise/pre-charge + tariff robustness + quantization). Refs #638
…level_at The shadow hook now builds the ledger the spec describes instead of flat capped slots: live SOC → kWh, sunrise floor = max(reserve, the scheduler's public target_soc), home per slot from the weekday-aware hourly profile (flat _expected_night_home_w fallback), honest None prices for unpriced day-ahead tails, and per-slot cheap LEVELS from the new shared accessor. build_night_ledger derives the takeover hour; pack_night packs against it; the log now states "battery carries home until HH:MM — the grid takes over" and the stash carries the takeover for diagnostics. Load demands gain needs_cheap_level (cheap_hours) + the #688 min_run/min_gap windows. tariff: get_price_level_at(when) — base None (unknown = conservatively not-cheap), Static/Calendar by their NT rule, Dynamic from the classified series point covering the moment. One shared source of level truth so the plan packs cheap-hours loads exactly where execution's price_is_cheap would fire (spec §tariff clause 3); execution can adopt the same accessor at G4. Shadow test updated by design: while the battery carries home, home is NOT on the meter — the full peak is headroom (the old flat model subtracted home from every slot; the trajectory is exact) — and the no-takeover night is pinned. TestPriceLevelAt pins the accessor. Refs #638
…ked-state Tier-2 avail + predictor gap hours Reviewer HIGH (real): _walk recomputed headroom from peak − home_grid, WIPING in-place grid decrements — after a Tier-2 draw re-walked the trajectory, a later grid demand could over-subscribe a slot the EV already filled and the plan would log a confidently wrong "fits". LedgerSlot.grid_committed_w now records allocations and every walk derives headroom = peak − home_grid − committed (cap-override path too). Reviewer HIGH (refuted with evidence, pinned): higher priority number = first claim IS the one-list semantics (get_devices_sorted reverse=True; decide()'s reclaim gate; DEFAULT_BATTERY_SURPLUS_PRIORITY=100 outranks the default charger 3 by design — battery charges before the EV yields surplus, day and night). The negation is correct; a corpus test now pins the realistic values so the semantics cannot drift silently. Reviewer MEDIUM ×2 (real): Tier-2 avail now reads the WALKED state (soc + batt_in − floor − home's ACTUAL battery share − booked draws) so a second battery demand in the same slot is not under-scheduled; and _home_at treats the profile's 0.0 fallback hours as data gaps (a 0 W house is not a forecast) falling back to the flat estimate. 36 corpus/shadow tests green. Refs #638
… evening hour >= 21 went blind between midnight and morning — a restart at 00:48 (caught live on TEST) still owes the rest of the night a plan. Trigger = inside the window (>= trigger_hour or < 07), stamped by night-of date (now − 12h) so one night never double-fires across midnight. Refs #638
…ver gate Second placement lesson caught live on TEST (observer rig): the trigger sat inside _run_battery_pipeline, which is gated on 'not observer_mode' — the shadow, itself a log-only observer construct, was silenced exactly where shadow computation belongs. Moved above the gate; runs in both modes. Refs #638
Journals rotate (ha core logs caps at 100 lines — the one-shot nightly INFO burst scrolls off under 10s observer chatter); the stashed plan doesn't. diagnose now carries overnight_plan_shadow when a plan exists — the G3 verification surface and the 22:00 answer on demand. Refs #638
…NG on failure Three placement bugs were invisible because every degraded path was silent or debug-level in a 100-line journal window. 'Nothing needs the night' is itself a valid 22:00 answer (INFO + stashed record); an empty window says so; an internal failure logs WARNING during the shadow soak. Refs #638
The first refresh after a restart sees ZERO registered devices (delayed
rediscovery) — the shadow fired there, answered 'no demands' and stamped
the whole night (caught live via the self-explaining counts:
loads_seen=0, ev_targets={}). The hook now returns False on that
degenerate warm-up shape so the trigger retries next cycle and converges
the moment registration completes; real answers (incl. a genuine
nothing-needs-the-night with loads_seen>0) stamp as before, and a
failure stamps too (one WARNING per night, not per cycle).
Refs #638
…and gate Finding #2 (PROD 02:14): the SOC sensor read unavailable at the first post-deploy refresh — None became 0 kWh, a 77%-full battery planned as empty and the ledger derived a bogus 03:00 takeover (reality: home rode the battery all night, 98→64%, 0.48 kWh grid). Power readiness is the second warm-up dimension: a configured battery with no SOC reading is a not-ready world — the hook returns False and the trigger retries. Finding #1 (same plan): the off-mode heizband "YIELDS 3.1 kWh" — but compute_load_intent never night-runs an off/peak_only device; the demand builder now mirrors the intent gate (SURPLUS mode only). With both inputs corrected, the model's prediction for last night matches what the hardware actually did. Two pins added; 39 green. Refs #638
…k, not average Guido's review from the config card: the Battery total discharge limit (battery_max_discharge_power — the exact knob the ledger already reads) was applied per consumer, not as the shared inverter capability. And the corpus caught my first fix summing AVERAGE power — allocations start at slot start and OVERLAP, so the bound is instantaneous: 500 W home + a 1 kW pump for 30 min + a second 1 kW pump is 2.5 kW while they overlap. LedgerSlot.batt_out_w tracks concurrent committed power; a draw that would exceed the limit packs into a later slot or yields. Pinned. Refs #638
Guido: 'this kind of explication I like to get for the doc as well — good example and drawing, just to understand what SEM does at this stage.' The doc now leads with the four steps (ledger table → trajectory walk with the takeover drawing → drag-order packing with the coupling timeline → the 22:00 answer), the yield-is-a-report semantics, when it runs, where to see it, and the two battery limits — written for any user, worked example numbers, no vendor comparison. Refs #638
…r 18:30 The planner window is now→night_end, so slot count (and every energy figure derived from it) moved with the hour the suite ran at. Written just after midnight, where a 7-slot night made "400 W fits under the 5 kWh above the floor" true; from ~18:30 local the window is 14 slots, 5.6 kWh, and the battery hands over to the grid at 05:00 — a correct plan failing a wall-clock-shaped assertion. Freeze at 22:00, the hour the real trigger fires. Refs #638
… a night plan on it Live-caught on HA-TEST during the G3b scenario battery. Two restarts minutes apart produced ledgers 2.4 kWh apart (12.2 → 9.8 kWh usable) while the published fleet SOC never moved. The floor was 4.5 kWh in both runs (30% x 15.0 kWh), so capacity was stable and the swing had to be SOC — and 9.8/15.0 = 65.3% is battery 2's SOC to the decimal. The boot log confirms it: 10 s into the restart battery 1's power sensor was still unavailable, so the SOC average was taken over the ONE unit that could be read, and the shadow planner — which fires ~2 min after boot and stamps its answer for the whole night — built the ledger on a battery 1.7 kWh smaller than the real one. Consequence in that run: takeover moved 2 h earlier (01:00 vs 00:00 in the sibling run) and a tier-2 load was told it "YIELDS 0.4 kWh" it would in fact have received. This is the len>1 fleet-read class again (beta.11 ev_power, #616 ev_connected): an aggregate silently degrades to the subset of members that happen to be readable, and nothing downstream can tell. Two layers, because the reader lying is the root and the planner acting on the lie is the damage: - sensor_reader records SOC coverage (expected vs read) and warns once per gap instead of averaging a subset in silence. - PowerReadings carries battery_soc_partial / units_expected / units_read, so any consumer that must not act on a half-resolved fleet can ask. - _shadow_overnight_plan gains a THIRD readiness dimension: a partial fleet is a not-ready world, retry next cycle. Readiness was already "configured battery" (finding #1) and "SOC readable" (finding #2); "every configured unit readable" is the missing one. Tests: TestPartialFleetIsNotReady — partial retries, complete plans, and a single-battery install is never partial (flag absent reads as ready). Mutation-checked: disabling the gate fails the first test. Refs #638
… a warming one, and bound the wait Review of 5844e7c found the gate could starve the very thing it protects. Two ways: 1. BLOCKER — ``_soc_units_expected`` counted CONFIGURED units, but a unit whose SOC sensor auto-detect returns None is skipped every cycle. On an install where one battery's SOC sensor isn't discoverable (name carries no SOC keyword, no device-registry match, ambiguous global scan), the fleet reads 1-of-2 forever — permanently "not ready", so that install would never get a night plan at all. ``expected`` now counts units whose SOC sensor was DISCOVERED, and the undiscoverable case gets its own warning in its own words: no amount of waiting fixes a config gap, so it must not be phrased as "still warming". 2. HIGH — the wait was unbounded. A permanently offline unit (dead battery, integration removed) held the plan back every cycle, all night. No plan is worse than a plan on a visible subset, so the wait is now bounded by _SHADOW_PARTIAL_GRACE_S (10 min): a boot warm-up resolves in seconds and always wins the wait, while a unit silent past the window is treated as offline — plan on what reports, WARNING once, and carry the shortfall on the plan itself as ``battery_fleet_partial`` so a degraded plan can never read like a healthy one. That silence is what made this bug invisible. Also: reset the one-shot partial warning when nothing is readable, so partial → all-offline → partial is heard twice instead of once; declare the coverage/one-shot flags and the shadow's own state in __init__ instead of lazy getattr. Tests: +3 coordinator (bounded wait plans anyway with the note, the clock starts on the first short cycle, a whole fleet clears clock and note) and +4 reader (TestFleetSocCoverage: undiscoverable is not "expected", discovered-but-unavailable is, full coverage, one-shot reset). 20 in the file, 1088 in the SOC/battery/reader subset. Refs #638
… on whether it is reporting The live re-verify on TEST contradicted my own root cause, which is the useful part. After deploying the coverage fix the log said: No SOC sensor could be found for 1 of 2 configured battery units — at 23:31, nowhere near a boot. So battery 1's SOC was not "warming up"; it was *undiscoverable on that cycle*. And the earlier 19:46 plan had happily averaged both units (81.25% = mean of 97.5 and 65). Same rig, same config, two different answers minutes apart. Root cause, one level below the fleet average: every strategy in _auto_detect_battery_soc rejects a candidate whose state is currently unknown/unavailable — sound when CHOOSING between candidates, wrong as a permanent verdict, because detection runs every cycle and nothing is remembered. So the mapping battery-power-sensor → SOC-sensor is a function of whether the SOC sensor happens to be reporting right now. A blip on battery 1 therefore did not report "battery 1's SOC is unreadable" (a partial fleet — something you can wait for); it reported "battery 1 has no SOC sensor" (a whole fleet of one), and the average became battery 2's 65% with the coverage counters agreeing that 1-of-1 is complete. That also explains why the previous commit's careful discovered-vs-readable split couldn't work: both sides were computed from the same state-dependent function, so they always agreed. Fix: remember the mapping. Once resolved, a battery's SOC entity stays its SOC entity; only an entity that no longer exists at all (renamed, removed) earns a re-detect. A later unavailable state is then what it actually is — a read failure of a known sensor, i.e. the transient partial the planner waits for. Bonus: no full state-machine scan every cycle. Coverage is now three numbers, because there are three states wanting three reactions: CONFIGURED units, units whose SOC sensor is KNOWN, units that READ this cycle. read < known → wait (bounded). known < configured → a real config gap; waiting never fixes it, so plan now — but the plan says ``battery fleet partial: 1/2 units`` either way. A subset's answer must never be readable as the fleet's; that silence is the whole bug. Tests: TestSocEntityIdentityIsSticky (a blip keeps the mapping, a vanished entity is re-detected), the config-gap plan carries the label without waiting, and the configured count is recorded. 24 in the file; 1106 green in the SOC/battery/detection subset (incl. #250/#523/#529 autodetect suites). Refs #638
…isn't there The sticky map from the previous commit only helps once a sensor has been read at least once — and the night plan is stamped ~10 s after boot, before the modbus units have published anything. TEST proved it: at 23:44 the log still said "No SOC sensor could be found for 1 of 2 configured battery units", the plan went ahead on battery 2's 65% alone, and by the time the fleet read 75% (85/65) the stamp was already made. The classification was the bug. Detection needs a value before it will name a candidate, so at boot "no readable candidate" and "no candidate at all" give the same answer — and the three-count model read that as a configuration gap, which by design never waits. Existence is the honest discriminator, so ask it without looking at any value: _soc_candidate_exists() walks the same longest-stem-first candidate names as detection (shared _SOC_NAME_KEYWORDS so they cannot drift) plus a registry-only same-device scan, and counts a present-but-silent sensor as a KNOWN unit. Then read 1 < known 2 is partial, the shadow waits its bounded 600 s, and the plan lands on the whole fleet. A unit with no candidate at all still never gets waited for. No counterpart to detection's global last-resort scan on purpose: that one is a uniqueness guess, not this unit's sensor identity. Refs #638
Three findings from the HA-TEST scenario battery, all the same class: the plan answered a different question than the night loop does. **Finding #4 — an off-mode charger became a demand.** `build_night_target_map` answers "how much does this charger still NEED", not "will SEM give it any tonight". The night loop decides that separately with `_mode_allows_night_charging` (`off`, and `solar_only` without a per-charger "At least" floor). Live on TEST: EV mode Off, SEM's own state reading "Night charging disabled", and the shadow still planned 10 kWh of grid for it (cost 3.387). The demand builder now asks the same gate; an unevaluable gate plans the charger anyway (over-planning is visible in the summary, a demand deleted by a broken gate would not be). Opt-outs are named in the summary and in the no-demands `why` — "the plan has no EV line" read identically whether the charger opted out or the builder lost it. Sibling of finding #1 (the off-mode load). **Finding #5 — the plan ignored a cap execution enforces.** The packer read `config["peak_limit_w"]`, a key NOTHING in the repo writes: not the config flow, not a migration. So it was 0 on every install, the packer ran with infinite headroom ("headroom left inf W" in the live allocations), and the plan handed a 10 kW EV slot to a house on a 5 kW limit. Now reads the canonical execution authority `_get_peak_limit_w()` — load manager first, `config["target_peak_limit"]` (kW → W) behind it. A cap the plan ignores makes every `fits` verdict meaningless. **Review fixes (finding #3 follow-through).** The readiness block now sits ABOVE `if not demands:` — the "nothing needs the night" answer is stamped for the whole night exactly like a full ledger, so a half-read world must not produce one either. That answer carries `battery_fleet_partial` too. An empty night window clears `_shadow_partial_since` so no later night inherits a running clock. The SOC existence probe accepts a `battery` device-class sibling with no unit: the probe only decides whether waiting is worthwhile, not which sensor to read, so it is deliberately looser than detection's signature check. Tests: 40 in test_638_shadow_mode.py (was 29), including a registry-level existence-probe class — registered-but-stateless counts, disabled does not, another device's SOC is not a candidate (#250 class). Both new gates mutation-checked: reverting either fails exactly one named test, and dropping the fallback's `* 1000` fails the kW-read-as-W mutation. Full suite 5448 passed, 2 skipped, 2 xfailed. Refs #638
Finding #6, live on TEST. `LoadManager` enters SHEDDING at `peak >= target_peak_limit` on the 15-minute rolling average (features/load_management.py:639) and `_progressive_load_shedding` then trims down to `target - hysteresis` (:773). So an hour booked AT the cap is the one allocation execution is guaranteed to kill — and that is exactly what the packer wrote once finding #5 gave it a real cap: ev:ev_charger: 5000 W 02:00-03:00 @ 0.339 (slot #1 cheapest, headroom left 0 W) against a 5.0 kW limit, reported as `fits — 10.0 kWh planned`. Sustained for the hour that IS the shed trigger. The ledger now gets the level execution actually holds: cap minus this install's `peak_hysteresis` (0.2 kW default, read from config rather than hardcoded, because the shedder settles wherever this install's band puts it). Never allowed to reach 0 — that is the packer's "no limit at all" sentinel (`headroom_w = inf`), so a cap smaller than the hysteresis would flip the TIGHTEST houses on the fleet into planning as if unlimited. Live proof, same world before and after: `5000 W ×2 slots (headroom left 0 W)` became `4800 W ×2 + 4140 W 04:00-04:05 (headroom left 660 W)` — the 10 kWh honestly needs a third block once the margin is respected. 3 tests, each mutation-checked to fail alone: dropping the subtraction fails the threshold test and the configured-hysteresis test; hardcoding the 0.2 default fails the configured-hysteresis test; relaxing the sentinel guard to `max(0.0, ...)` fails the tight-cap test. Full suite 5451 passed, 2 skipped, 2 xfailed. Refs #638
…e 1.7 fixes The coexistence rule: the enhancement branch periodically absorbs the 1.7 fix lane and PROD redeploys from HERE, so PROD = latest fixes + the soaking planner with one writer to the hardware. The fix branch itself goes to develop separately (1.7.5-beta train) — the planner never does (1.8 line). Refs #638 #692 #693 #694 #695
…branch Lane reconciliation only — the fixes themselves were already here; this brings the manifest bump + CHANGELOG header so the next branch deploy reports the right version. Refs #638
…t closes evaluate() keeps every economic guarantee (deficit, break-even, anchored target SOC, charge power, negative-price override, replan triggers) and loses the window pick: the find_cheapest_hours call, _plan_night_schedule, _infer_slot_hours, NightChargeSchedule and TimeSlot are deleted, along with the phantom EV co-model (ev_kwh_needed/ev_max_power_w — #652's model dies here; the joint plan carries the real per-charger demands). Entity compat: battery_scheduler_schedule now derives from the stamped plan's battery blocks (schedule_view_from_plan — the exact as_dict shape, ev_w honestly 0). Reboot compat: the SCHEDULED verdict persists beside the plan (serialize/restore_battery_verdict) so a restart mid-block outside the evaluation window still actuates the restored night. The one-selector allowlist is EMPTY — the joint planner is the one selector left, and the ratchet is now the permanent pin. Refs #652
…ch dies
Comfort demands packed as comfort:{did} but the window collector only ever
asked the gate for load:{did} — a comfort block could never reach its
device, so banking runs never actuated. merge_load_gates (pure, in
overnight_actuation) now merges both demands per device: either in-block →
a run verdict; a hold only when EVERY covered demand holds; the earliest
block start wins the until.
PlanVerdict carries in_block — the run half. compute_load_intent (and its
imperative twin, which is what PROD runs) gains the ONE sanctioned place
the plan CREATES a run: a WILLING comfort band inside its planned block
banks from cheap_grid; forced rooms stay reactive; peak_freeze and
anti-cycle still gate the start.
The dead load_window helper and the legacy plan_window bool param are
deleted — the verdict is the only plan input left.
…e; the valve stays closed The advisor's discharge_blocks finally actuate — under three separated authorities: the PLAN says WHEN (arbitrage_sell_gate reads the stamped plan's sell blocks under the same trust rules as every gate), the LIVE economics say WHETHER (evaluate_arbitrage's verdict is now the in-block validity check, no longer a window authority), the user's modes say MAY (per-battery allow_arbitrage scan is real again — the v1.7.3 _any_allow_arb hardcode is gone — but every DEFAULT keeps the path dormant; #533 stands). Power discipline: the sell is capped at the block-implied watts (the advisor bounded delivery by the home's own draw — avoided import, never export-at-max) and fleet-split via effective_battery_count, resolving the multi-battery re-activation guard the old branch carried as a warning comment. Refs #533 Refs #523
…— ×16 languages The one gate concentrates every decision, so the card now concentrates the explanation. The plan payload carries structured not_scheduled rows (mode / disconnected — prose in summary is for logs, not rendering) and a live coverage map (the per-demand verdict, the user-facing twin of the '#638 coverage' log line, riding beside actuation so a mid-night fall to reactive changes the card next cycle, not next stamp). The Energy Plan card renders: a 'reactive — why' chip on any uncovered demand (the gate's named doubt translated — no internal jargon reaches a rendered surface), a 'Not scheduled tonight' list naming every deliberate exclusion with its why, and block prices on the row tooltips (why THIS window). 12 new keys ×16 languages; sem-localize regenerated; bundle rebuilt.
…documented With the private selectors retired, actuation-off would silently remove all cheap-window timing from a solar_plus_cheap install — the default flips ON and switch.sem_overnight_actuation remains the kill-switch. OVERNIGHT_PLANNER.md gains the one-gate section (no second window-picker anywhere, the per-family fail-open table, which reactive gates stay senior) and the user-facing 'How to read tonight's plan' walk. CHANGELOG entry is deliberately UNNUMBERED — the branch takes its version at merge time; the autopilot owns develop's number space (the beta-collision lesson).
…erage (missed hunk of 5fca35a)
…ne retires from the card
Guido's first live look at the one-gate card: 'ev_targets={...},
mode_opted_out=[]...' rendered verbatim read as an unfinished placeholder.
The idle payload now carries machine codes (ev_target_met /
no_load_needs_night / battery_no_deficit) plus the structured not_scheduled
rows; the card renders translated sentences (3 keys ×16 languages) with the
prose demoted to the hover tooltip. An unplugged car no longer counts as
'target met' — it appears under Not scheduled tonight with its real why.
…writing praun's report: the write sent raw watts with no unit validation — a kW-native setpoint got 3000 (its range clamp turned that into full tilt) and a current-native number would take watts as amperes. The sibling discharge-limit path already had the rule (_native_power_scale); the setpoint write now consults the SAME helper through the new public native_power_scale — one validation rule, not two. Non-power units and unreadable states REFUSE loudly (warning once per adapter, False to the caller so intent is never recorded — the #589 honesty contract). Watts stay the internal currency: the entity's native min/max scale INTO watts for the clamp and the de-dup, and the value converts to native only at the service-call boundary — so a kW entity's de-dup threshold stays 100 W, not 100 kW.
…lusion's other half, finally real Azlinon's live night: peak reached CRITICAL, the load manager shed his chest freezers, and the EVSE held 32 A — the single largest controllable load was exempt from peak management entirely. The actuation exclusion in load_management stays (#649: one writer per device); what was missing is the compensating control its docstring assumed: decide() never saw the peak state during the day. peak_state now resolves ONCE per cycle into FleetCycleState (the same effective_peak_state vocabulary the surplus controller gets) and rides the one-place thread into every charger's view. decide() applies a SENIOR clamp after mode dispatch — peak shed is a guarantee, senior even to always_max: SHEDDING clamps the commanded amps to the effective minimum (vehicle handshake floor respected), EMERGENCY idles the charger un-bridgeably. The EV throttles on the FIRST cycle of a shed level, before the LM's delayed progressive shed reaches anyone's freezer.
The #282 persistence was working — PROD's storage held the full session — but the restart's sensor warm-up published ev_connected=False before the charger integration had loaded, the end-detection took it as a real unplug, finalized the session into lifetime stats, and a fresh session started when charging resumed: 6.0 kWh became 1.6 kWh on the top chip and the session's cost and solar share were silently rewritten (the real session was mostly daytime solar; the amputated tail mostly evening grid). A disconnect is only real once CONFIRMED: never inside the two-minute boot warm-up window (a sensor that has not spoken yet is not a sensor saying no — the same class as the 23 kWh target blip logged the same evening), and only after three consecutive disconnected cycles, which also absorbs the KEBA UDP blip family (#35/#595). While unconfirmed the edge stays armed and the session keeps accumulating the moment the sensor returns.
… charging session (#753)
…he last private timeline retires Live 08.08: the Energy Plan card said WAITS·00:00 while the EV tab's mini-strip painted the static tariff's reactive prediction. When the gate covers the charger, compose_today_plan now takes the plan's own blocks: one charge-start row per future block (detail plan_ev_charge_joint, ×16 languages) and the min-reached row at the LAST block's end — the plan's promise, not a rate estimate. A malformed block distrusts the set (the gate's rule); no blocks keeps the legacy composition as the honest uncovered fallback. One data source, every surface.
…t pessimism — the 1.8 half The ledger's day_kwh was already the raw forecast (#598), so the packer sees the sky's own number. The real poisoning was the LEARNING: under an export limit the measured production is clamped to consumption, and the dampening tracker read that as 'the forecast over-promised' — sinking the factor and making every dampened consumer (fleet remaining-solar, the forecast night target) under-plan exactly the hidden kilowatts the probe reveals. Only a probe-CONFIRMED day (state harvest — production followed) skips the sample; a failed probe is plain clouds and still teaches.
Phases 1+2 built the drift models and the direction-aware ask; C5 gave banking its actuation. The missing depth: the ask sized energy only to TARGET, but pre-cooling's whole point is buying the band — cool to target − offset (heat: target + offset) in the cheap/free window so the room coasts through the expensive one. banking_energy_kwh gains offset_c (0 keeps the target-depth contract for offset-less callers) and comfort_plan_demand passes the band's own offset. The run's natural stop already agrees: crossing the edge flips the state to banked, which ends the C5 clause's hold. A cooling ClimateDevice end-to-end pre-cool ask is now pinned — the direction machinery was already in place.
…ws down to 1 A Two reporters, one ask: vehicle-side control (Tesla BLE, ESP32 numbers) charges below 6 A, but every current slider was hard-floored at min=6 so the config could never say what the hardware supports. The sliders now allow 1–N; enforcement lives where it always belonged — the current write clamps to the control entity's own min/max (#487) and brand adapters keep their own minima (KEBA's 6 A is its adapter's truth, untouched). Defaults stay 6 everywhere: nothing changes for existing installs. Bonus root-cause: effective_min_amps crashed with TypeError on a present-as-None ev_min_current (the classic config trap) — hardened with the or-chain fallback.
…es UI, multi-heat-pump path #688 (the remaining half): start_reserve_w — a fresh START needs margin on top of the threshold (600 W pump, 800 W ask) so the start plus the next cloud doesn't flip the surplus negative and cycle the device. Starts only: a running load keeps the plain threshold — reserving against a healthy run would CREATE the cycling this prevents. Settable through the same goal surface as every #620 knob; default 0 = today's behaviour. #751: the four power-strategy VALUES (active/idle/self-consume/off) get their config-card fields, shown when a strategy entity is configured — Sessy's defaults as placeholders, any integration's option names mappable (5 i18n keys ×16; the card gains its first generic text-option row). #685: multiple heat pumps documented as climate devices — each unit gets its own priority, comfort band, drift model and planner demands; SG-Ready stays the one-unit relay interface. Split metering → template-sum sensor.
|
Important Review skippedToo many files! This PR contains 207 files, which is 107 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (207)
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…k estimate no longer teaches the ratchet A 24 W Shelly PM load showed ~1.0 kW: the kWh-tick estimate (0.01 kWh over a short window reads ~1 kW instant) fed calibrate_rated_power's up-only ratchet, whose deriver cap (2x rated) re-based on every adoption — a geometric climb, persisted across restarts by the calibrated-ratings snapshot. An estimate must never teach the model (the #743/#753 class).
…d; the announcement reports the observation The taper latch (_declining_phase), session peak, SOC anchor with its timestamp and the stability give-up streak/backoff now ship in the diagnostics download — the promised half of #708 that doesn't need the reporter's car. And 'SOC anchored at 100%' asserted something SEM cannot know; the log now reports what was observed: the charge completed.
…honest announcement (#708)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One schedule for every floor: EV per-charger night targets (#634), load
min-runtime deficits (#620), battery pre-charge — packed over one hourly
price curve under one peak cap in one priority order (#576).
coordinator/overnight_planner.py — plan_overnight(): a greedy priority-
packer over price-sorted slots. Deliberately NOT an optimizer: pure,
deterministic (ties break price->start->priority->id), ~170 lines, every
allocation carries a one-line reason. Device floors are respected per
slot: EV min-amps (a thin slot is skipped, a continuous battery takes
it), switch loads are binary, the last slot is shortened rather than
over-delivered. Floors stay guarantees — the packer moves WHEN/how-fast,
never WHETHER; what doesn't fit under the cap is REPORTED as yielding
(the 22:00 answer: "everything fits / X yields").
Shadow (G3): _shadow_overnight_plan runs after scheduler.evaluate — the
same trigger cadence — building demands from the REAL models:
build_night_target_map per charger (mode-gated; the #652 closure — the
INFO line logs the scheduler's phantom EV model against the real map),
load runtime deficits, the scheduler's own battery deficit_kwh. Hourly
slots via tariff get_price_at; cap = peak_limit - expected overnight
home. Logs summary at INFO, allocations at DEBUG, stashes
_overnight_shadow_plan for diagnostics. Never actuates; any internal
error degrades to a debug line (test-pinned with a hostile fake).
test_battery_scheduler_rolling_horizon: the get_price_at assert_not_called
pin loosened to what it meant — break-even rates come from the series
(the 0.99 sentinel proves it); the shadow's curve reads are legitimate.
Corpus: 13 packing scenarios incl. the #634 ensemble shape (EV + heater +
battery under one cap — every floor met where reactive greed collides).
Verification ladder: G1 baseline + G3 shadow accrue tonight on PROD from
the same night; G4 (actuation flip) only on explicit call.
Refs #638 #652 #634 #630 #576 #620