Skip to content

Fix daily battery energy counters corrupting long-term statistics - #60

Open
FrancescoFdd5 wants to merge 5 commits into
wilds:masterfrom
FrancescoFdd5:master
Open

FrancescoFdd5 wants to merge 5 commits into
wilds:masterfrom
FrancescoFdd5:master

Conversation

@FrancescoFdd5

Copy link
Copy Markdown
Contributor

Closes #59.

Summary

Two independent bugs, each in its own commit, both affecting the energy data
this integration feeds into Home Assistant's long-term statistics.

  1. tot_pBatteria and tot_pBatteriaB are declared state_class: TOTAL but
    never publish a last_reset attribute, so their daily midnight reset is
    never recognised and their long-term statistics are corrupted every day.
  2. The get_energy.php request builds its date from the operating system's
    timezone rather than the one configured in Home Assistant.

1. Battery energy counters corrupt long-term statistics

tot_pBatteria ("Battery charged energy") and tot_pBatteriaB ("Battery
discharged energy") are daily counters: the endpoint is queried with
intervallo=d, and the plant resets them to 0 at midnight.

They are declared state_class=SensorStateClass.TOTAL and never set
last_reset. That combination is the problem. In
homeassistant/components/sensor/recorder.py, the reset branch for TOTAL
requires a last_reset that is both present and changed:

if (
    state_class != SensorStateClass.TOTAL_INCREASING
    and (last_reset := _last_reset_as_utc_isoformat(...)) != old_last_reset
    and last_reset is not None      # never true for these sensors
):

last_reset is always None, so None != None is false and the branch is never
taken. The TOTAL_INCREASING branch below it is not taken either. The reset is
therefore never detected, and the midnight drop back to 0 falls through to the
ordinary accumulation at the end of the loop:

_sum += new_state - old_state

So a full day's worth of energy is subtracted from the cumulative sum in one
step at midnight, and each day is booked as today's total minus yesterday's
total
instead of today's total.

Example from my own plant, running 1.0.9. The first column is what the plant
reports, the second is what the above arithmetic books into statistics:

Day Daily total reported by the plant Booked into statistics
20 Aug 10.2 kWh
21 Aug 5.8 kWh 5.8 - 10.2 = -4.4 kWh
22 Aug 12.7 kWh (reading at 18:00) 12.7 - 5.8 = +6.9 kWh

The same spurious midnight step is what produces the anomalous first hourly
bucket of each day reported in #59.

One clarification on that issue, now that the cause is understood: I originally
attributed it to the sensor resetting a couple of seconds after midnight rather
than exactly at midnight. That offset turns out to be irrelevant.
reset_detected() does not look at timing at all, so the reset would have gone
undetected regardless of when it happened.

Fix

Switch both sensors to TOTAL_INCREASING. reset_detected() then recognises
the drop on its own — any value below 90% of the previous reading — and opens a
new cycle at zero, with no last_reset required.

This also makes the two sensors consistent with every other daily counter in the
integration, all of which already use TOTAL_INCREASING: tot_pReteIn,
tot_pReteOut, ePannelli, eBatteria and eConsumed.

Both sensors have behaved this way since they were introduced in e45cf80 (#55).


2. Energy endpoint queried with the OS timezone

The get_energy.php URL was built from three separate datetime.now() calls:

year=datetime.now().year,
month=datetime.now().month,
day=datetime.now().day,

Two problems. datetime.now() returns the operating system's local time, not
the timezone configured in Home Assistant, so on a host whose clock is set to a
different zone the integration asks for the wrong day around midnight. And
because the three calls are independent, they can straddle midnight and build a
date whose year, month and day come from different days — on 31 December that
yields a date a full year off.

Fix

Read the clock once, through homeassistant.util.dt.now(), which returns the
current time in Home Assistant's configured timezone.


Note for users upgrading

The fix only corrects statistics going forward. Statistics already recorded
under TOTAL stay wrong and have to be corrected by hand in
Developer Tools -> Statistics, oldest day first, since adjustments propagate
forward through the cumulative sum. The affected days are easy to spot: they are
negative or implausibly low.

Testing

Running on my production Home Assistant instance since 22 Aug 2026. The daily
counters now reset cleanly at midnight, the anomalous first-hour bucket from #59
is gone, and the recorded statistics match the figures reported by the plant.

Note on the version bump

The third commit bumps manifest.json to 1.0.10 for easy test with HA. Happy to drop it if you would
rather handle versioning and releases yourself.

FrancescoFdd5 and others added 5 commits August 22, 2026 16:33
tot_pBatteria and tot_pBatteriaB are daily counters that the plant resets
to 0 at midnight, but they were declared with state_class TOTAL and never
publish a last_reset attribute. In that combination the statistics
compiler never takes the reset branch, so the midnight drop is booked as
an ordinary delta: each day's recorded energy becomes
(today's total - yesterday's total) instead of today's total, which also
yields negative days whenever today's total is lower than yesterday's.

TOTAL_INCREASING lets reset_detected() spot the drop (value below 90% of
the previous reading) and open a new cycle at zero. It also aligns these
two sensors with every other daily counter in the integration:
tot_pReteIn, tot_pReteOut, ePannelli, eBatteria and eConsumed.

Regression from e45cf80, which introduced both sensors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The get_energy.php request built its date from three separate
datetime.now() calls. Those used the operating system's timezone rather
than the one configured in Home Assistant, and three independent reads
can straddle midnight and produce a date whose year, month and day come
from different days.

Read the clock once, via homeassistant.util.dt.now().

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Unauthorized and empty-body guards in the controller compared bytes to
str and could never match, and a reply that was not JSON was only logged as
a warning. An expired PHP session therefore left the integration serving
frozen values, reported as successful updates, until Home Assistant was
restarted. Responses are now validated properly and refresh retries once
through a fresh login.

Other fixes:

- url-encode the login form by passing a dict to httpx, so passwords
  containing & = + or non-ascii characters work
- detect rejected credentials through the missing plant id instead of
  crashing with AttributeError, and raise ConfigEntryAuthFailed so Home
  Assistant asks for the password again instead of retrying forever
- use (x & 0xF0) >> 4 for the EV state nibble: >> binds tighter than & in
  Python, so all four EV binary sensors were reading the low nibble
- compute self sufficiency in float math, int() truncation reported a flat
  100% for every day below 1 kWh of consumption
- honour the scan interval chosen during setup, and reload the entry when
  the options change
- give the coordinator a timeout that is not tighter than the requests it
  wraps
- guard get_raw_data and native_value against payload keys that are missing,
  for instance the EV fields on a plant without a wallbox
- publish the monitor and energy payloads together, so a partial refresh
  cannot mix fresh and stale data
- give each config entry its own httpx client and cookie jar
- let the options flow change the sensor selection, not just the interval
- validate the scan interval and re-enable the duplicate plant check
- drop the unused IntegrationSensor along with the integration dependency
- align strings.json with translations/en.json and use literal strings, the
  %key% references are only resolved for core integrations

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
get_energy.php was decoded against two days of real data. Every tot_* is the
integral of its matching 20-minute kW series, and tot_pBatteria + tot_pReteIn
equals tot_pUtenze on both days. tot_pBatteria is therefore self-consumed
energy, not battery charge: in the series it tracks pUtenze whenever the house
is not drawing from the grid, and drops to zero the moment the battery empties
and pReteIn takes over.

consumed_energy now reads tot_pUtenze, which the portal publishes directly.
The old bought + discharged formula ignored the solar consumed straight by the
house and under-reported by roughly a factor of three: 3.35 against a real
9.99 kWh on 2026-08-29. Self sufficiency divides by it and moves from 46.27%
to 81.98% on the same day. Expect a step in the statistics of Daily consumed
energy; the earlier values were simply wrong.

New sensors, all from fields the payloads already carried:

- Battery level BMS / inverter / Ah, the three state of charge readings the
  plant reports side by side
- Active alarms, a count carrying the raised alarm names as an attribute,
  mapped positionally onto the 128 entry getAlarmDesc table. That table is
  static, so it is fetched once and cached; the fetch runs after the payloads
  are published and restores the session if it fails, so an optional extra can
  never invalidate a session the monitor and energy calls just used.
- Last alarm and Board time
- Extended firmware, and Export limit as a bare number: the plant reports
  60000 on a 4 kW system, so the scale is not confirmed and labelling it watts
  would be a guess
- Phase 1/2/3 power and Grid phase 1/2/3 power, zero on a single phase plant
- WiFi and Alarm binary sensors

Existing entities are deliberately untouched: no rename, no removal, no unit
change. That includes Battery charged energy, which reads tot_pBatteria and is
therefore a duplicate of Daily self consumed energy under a name that says the
opposite. Renaming it would split its long-term statistics.

Because monitored_variables is stored per config entry, the new sensors have to
be ticked in the integration options before they appear. The two new binary
sensors follow the existing BINARY SENSORS selection.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reset giornaliero del sensore "Battery charged energy" genera un picco anomalo nelle statistiche orarie (Energy Dashboard)

1 participant