Skip to content

Configurable meter drivers - #1

Open
tescalada wants to merge 9 commits into
tariff-modal-formfrom
configurable-meter-drivers
Open

Configurable meter drivers#1
tescalada wants to merge 9 commits into
tariff-modal-formfrom
configurable-meter-drivers

Conversation

@tescalada

@tescalada tescalada commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Introduces configurable meter drivers and per-provider metering settings, and migrates the metering stack off the vendored OpenAPI/sparknet-http client onto the generic meter-driver-spec package (consumed as a released wheel) with runtime driver discovery.

Changes

  • Configurable meter drivers + provider settings — add a metering provider settings model and a meter.provider_id column, surfaced through meter_view (DB migration 0.84).
  • meter-driver-spec migration — migrate metering to meter-driver-spec models with runtime driver discovery; consume it as a released wheel; bump to v1.3.0 and update the gRPC import path.
  • sparknet-http client — replace the vendored OpenAPI metering client with the sparknet-http package; add a sparknet-http gRPC/SSE runtime client; point imports at the grpc/http subpackages.
  • Tariffs — add a tariff modal form and shared tariff form fields.
  • Readings — support skipping legacy meter-scalar scaling on readings.
  • Cleanup — remove the dead legacy IC meter scalars and the fake-reading generator command.

Notes

  • The 0.84 migration adds meter.provider_id and rebuilds meter_view; its downgrade drops meter_view before removing the column and no longer re-runs meterschema.sql in-migration (the migration runner rebuilds views after migrations).
  • Adds sparkmeter/config/tests/test_provider_settings.py.

@tescalada
tescalada force-pushed the configurable-meter-drivers branch 3 times, most recently from 8bf6a10 to cb2f374 Compare July 19, 2026 15:56
@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown

Coverage

Coverage report
FileStmtsMissCoverMissing
sparkmeter
   __version__.py16288%32, 38
   _wsgi_mount_smoketest.py97970%19–152
   app.py3151197%19–20, 111, 368–375, 552–554, 675, 682–683
   asgi.py792766%45, 63–66, 86, 151–176, 183, 187
   cli.py40685%57–68
   cli_app.py880%4–20
   cli_prompts.py151220%8–12, 17–21, 26–28
   conftest.py181995%51–52, 85–86, 108–120, 209
   controller.py2434283%181, 205–206, 529, 583–640
   debug_memory.py32320%6–63
   periodic.py563145%60–66, 71–75, 80–84, 93–98, 103–111, 124–128
   sentry_proxy.py342235%38, 44–47, 56–65, 73–82
   servercommand.py1127236%47, 56–60, 64–69, 73–78, 82–85, 89–90, 101–103, 112–181
sparkmeter/alembic/versions
   0.32_merge_sms_tables.py46393%61–67
   0.77_snapshot_support.py128298%340, 401
sparkmeter/api
   customerviews0.py283199%301
   historyviews0.py1149219%49–55, 69–72, 81–131, 144–175, 185–216, 230–257
sparkmeter/config
   configparameter.py58297%80, 111
   configviews.py100397%156–158
sparkmeter/database
   alchemy.py54493%61–63, 74
   database.py232399%497, 579–580
   ormobject.py55198%65
   session.py21210%12–62
   types.py54394%91–92, 125
sparkmeter/ground
   groundcommand.py201050%20–35
sparkmeter/history
   historyview.py351654%30–31, 41–54, 65–72
sparkmeter/meter
   meterdomain.py7732797%895, 1793–1794, 1805–1856, 1899
sparkmeter/metering
   api.py27967%66–76
   events.py164498%91, 291–292, 325
   reconcile.py137497%142–143, 279, 286
sparkmeter/system
   systemcommand.py67593%35, 130–132, 134–135
sparkmeter/tariff
   tariffcommand.py821582%27–44
sparkmeter/transaction
   transactionview.py167199%71
sparkmeter/user
   userdomain.py1871294%416–434
sparkmeter/web
   unittestutils.py1951194%73–81, 330, 339, 405, 407
   views.py72199%71
TOTAL1381962196% 

Tests Skipped Failures Errors Time
1559 0 💤 0 ❌ 0 🔥 3m 48s ⏱️

@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown

Diff Coverage

Diff: origin/main...HEAD, staged and unstaged changes

  • sparkmeter/alembic/versions/0.84_add_meter_driver_selection.py (100%)
  • sparkmeter/asgi.py (92.3%): Missing lines 45
  • sparkmeter/config/configdict.py (100%)
  • sparkmeter/config/configparameter.py (100%)
  • sparkmeter/config/configparametertypes.py (100%)
  • sparkmeter/config/configviews.py (96.1%): Missing lines 156-158
  • sparkmeter/config/provider_settings.py (100%)
  • sparkmeter/config/providerform.py (100%)
  • sparkmeter/event/eventspecs.py (100%)
  • sparkmeter/meter/meterdomain.py (100%)
  • sparkmeter/meter/meterform.py (100%)
  • sparkmeter/meter/meterstate.py (100%)
  • sparkmeter/metering/cli.py (100%)
  • sparkmeter/metering/dispatch.py (100%)
  • sparkmeter/metering/events.py (100%)
  • sparkmeter/metering/http_sse.py (100%)
  • sparkmeter/metering/lifespan.py (100%)
  • sparkmeter/metering/provider_config.py (100%)
  • sparkmeter/metering/reconcile.py (100%)
  • sparkmeter/metering/runtime_client.py (100%)
  • sparkmeter/metering/runtime_registry.py (100%)
  • sparkmeter/reading/readingdomain.py (100%)
  • sparkmeter/reading/readingview.py (100%)
  • sparkmeter/tariff/tariffview.py (100%)
  • sparkmeter/web/forms.py (100%)
  • sparkmeter/web/unittestutils.py (100%)

Summary

  • Total: 1774 lines
  • Missing: 4 lines
  • Coverage: 99%

sparkmeter/asgi.py

Lines 41-49

  41 # When this file is launched via `python -m sparkmeter.asgi`, Python executes it
  42 # as `__main__`. Register the canonical module name as an alias to the running
  43 # module so in-process helpers don't accidentally import a second copy.
  44 if __name__ == "__main__":
! 45     sys.modules.setdefault("sparkmeter.asgi", sys.modules[__name__])
  46 
  47 
  48 @asynccontextmanager
  49 async def app_lifespan(app: FastAPI):

sparkmeter/config/configviews.py

Lines 152-162

  152         provider=provider,
  153     )
  154 
  155     if request.method == "POST" and form.validate():
! 156         form.save()
! 157         flash(form.notification_message(), "success")
! 158         return redirect(url_for("config.meter_driver"))
  159 
  160     return form.render(provider=provider)
  161 

@tescalada
tescalada force-pushed the configurable-meter-drivers branch 2 times, most recently from 0733849 to c6f08d9 Compare July 21, 2026 03:28
@tescalada
tescalada changed the base branch from main to tariff-modal-form July 21, 2026 03:28
@tescalada
tescalada force-pushed the configurable-meter-drivers branch from c6f08d9 to b2bf3eb Compare July 21, 2026 03:50
@tescalada
tescalada force-pushed the configurable-meter-drivers branch from b2bf3eb to 77f73f5 Compare July 31, 2026 12:13
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown

Coverage

Coverage report
FileStmtsMissCoverMissing
sparkmeter
   __version__.py16288%32, 38
   _wsgi_mount_smoketest.py97970%19–152
   app.py3151197%19–20, 111, 368–375, 552–554, 675, 682–683
   asgi.py792766%45, 63–66, 86, 151–176, 183, 187
   cli.py40685%57–68
   cli_app.py880%4–20
   cli_prompts.py151220%8–12, 17–21, 26–28
   conftest.py181995%51–52, 85–86, 108–120, 209
   controller.py2434283%181, 205–206, 529, 583–640
   debug_memory.py32320%6–63
   periodic.py563145%60–66, 71–75, 80–84, 93–98, 103–111, 124–128
   sentry_proxy.py342235%38, 44–47, 56–65, 73–82
   servercommand.py1127236%47, 56–60, 64–69, 73–78, 82–85, 89–90, 101–103, 112–181
sparkmeter/alembic/versions
   0.32_merge_sms_tables.py46393%61–67
   0.77_snapshot_support.py128298%340, 401
sparkmeter/api
   customerviews0.py283199%301
   historyviews0.py1149219%49–55, 69–72, 81–131, 144–175, 185–216, 230–257
sparkmeter/config
   configparameter.py58297%80, 111
   configviews.py100397%156–158
sparkmeter/database
   alchemy.py54493%61–63, 74
   database.py232399%497, 579–580
   ormobject.py55198%65
   session.py21210%12–62
   types.py54394%91–92, 125
sparkmeter/ground
   groundcommand.py201050%20–35
sparkmeter/history
   historyview.py351654%30–31, 41–54, 65–72
sparkmeter/meter
   meterdomain.py7732797%895, 1793–1794, 1805–1856, 1899
sparkmeter/metering
   api.py27967%66–76
   events.py164498%91, 291–292, 325
   reconcile.py137497%142–143, 279, 286
sparkmeter/system
   systemcommand.py67593%35, 130–132, 134–135
sparkmeter/tariff
   tariffcommand.py821582%27–44
sparkmeter/transaction
   transactionview.py167199%71
sparkmeter/user
   userdomain.py1871294%416–434
sparkmeter/web
   unittestutils.py1951194%73–81, 330, 339, 405, 407
   views.py72199%71
TOTAL1381962196% 

Tests Skipped Failures Errors Time
1559 0 💤 0 ❌ 0 🔥 2m 32s ⏱️

Add the metering subsystem — runtime client, command dispatch, event stream,
lifespan and reconciliation — talking to a driver over gRPC and HTTP+SSE using
the released meter-driver-spec wheel.

Delete the vendored sparkmeter/metering/_generated OpenAPI client and the
regen-metering-wire.sh script that produced it; the local codegen workflow and
its pre-commit and setup.cfg entries go with them.
Delete the `reading create-fake` command and its ReadingGenerator, which
synthesized random readings for demo/testing. Drop the now-empty reading
CLI group registration and its coverage in the CLI-registration and
meter-command tests.
No source path scales readings any longer: slipstream is gone, and the
fake-reading generator (its only remaining caller) was just removed.
Provider readings arrive in engineering units and are persisted as-is.

Drop the apply_meter_scalars parameter from add_reading / save_raw_reading
and delete Meter.apply_scalars. Update the reading controller tests to
supply realistic engineering-unit values directly instead of raw counts
that were scaled down at persist time, and remove the apply_scalars unit
test.
…hing

Add sparkmeter/metering/runtime_registry.py (set_running_app/get_running_app)
as the single source for the running public ASGI app, replacing the
sys.modules/__dict__ reflection that located it for the sync-to-async metering
bridge. Remove all globals()[...] app-publishing from asgi.py: the registry
holds the public app and a private _internal_app caches the internal app, both
resolved via __getattr__. Inline the redundant _get_running_public_app wrapper
into its one call site.
Drop the _is_ground() wrapper and its SPARKMETER_MODE env fallback, now
that main derives ground/cloud from the HEROKU config key. The two
lifespan guards call config.is_cloud() directly, and the lifespan tests
drive the branch by setting config["HEROKU"] rather than stubbing the
removed helper.
The _metering_enabled() helper re-read and re-normalized SM_OFFLINE in an
except-fallback, duplicating the SM_-prefix env normalization ConfigDict
already does at load time. Replace it with a config.is_offline() method
that reads the normalized OFFLINE value, alongside is_ground()/is_cloud().
The two metering guards call config.is_offline() directly, and the tests
drive the branch by setting config["OFFLINE"] rather than stubbing the
removed helper.
@tescalada
tescalada force-pushed the configurable-meter-drivers branch from 77f73f5 to 1f5758a Compare July 31, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant