Skip to content

[19.0][MIG] ni_observation: Migration to 19.0 - #158

Merged
piruin merged 89 commits into
nirun-life:19.0from
wasud:19.0-mig-p1-observation
Oct 21, 2026
Merged

piruin merged 89 commits into
nirun-life:19.0from
wasud:19.0-mig-p1-observation

Conversation

@wasud

@wasud wasud commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Decisions to review

  • D4, data (UoM):
    • data/uom_uom_data.xml is rewritten for the 19 UoM model: no categories, relative_uom_id + relative_factor, same xmlids. Each relative_factor is 1/factor of the 16.0 data, so every conversion gives the same result (1 kg/m³ = 100 mg/dl).
    • OpenUpgrade 19 converts only units without an xmlid ("left to the module providing them"), and Nirun's units are noupdate. So migrations/19.0.1.0.0/post-migration.py converts them from OpenUpgrade's legacy columns and drops the old category xmlids. It does nothing without an OpenUpgrade run.
  • D4, OpenUpgrade bug:
    • Upstream uom/19.0.1.0/post-migration.py sets relative_factor = factor for "bigger" units without an xmlid. 18.0 stored factor < 1 for those (dozen = 1/12), so their ratio is inverted: a user-created "Box of 24" becomes 1/24 of a unit.
    • Reproduced on the rehearsal. upgrade_path/uom/19.0.1.0/post-nirun_uom_bigger.py (separate [FIX] commit) recomputes them as 1/factor and runs after OpenUpgrade's script.
    • Reported and fixed upstream: [19.0][OU-FIX] uom: relative_factor of bigger units is 1 / factor OCA/OpenUpgrade#6054. Drop the workaround once it is merged.
  • D4, pre-existing data issue: /sec and /hour are defined like time units (1 /min = 60 /sec), the inverse of a rate. Conversions are kept the same on purpose (no data change in a migration); noted in the README. Fixing it is a 16.0 data change (P0-4 [MIG 19.0][P0-4][FIX] Pre-existing bugs found during the 19.0 migration scan (16.0) #135?).
  • D5, security: a res.groups.privilege "Observation" (res_groups_privilege_observation) holds Viewer/User/Manager. The groups get sequence and privilege_id, users → user_ids. Same pattern as P1-2 ([MIG 19.0][P1-2] ni_patient #137).
  • D7, list widget: kept the 18 port. Core account 19 adds section menus, subsections and price totals, which observations do not use; the parts the widget relies on (getColumns, getSectionColumns, getCellClass, focusCell, this.columns) are unchanged.

What changed

  • [MIG] ni_observation: Migration to 19.0 on top of the 18.0 PR, then the [FIX] upgrade_path commit.
  • Migrator: _sql_constraints → models.Constraint, _cr/_context → env.cr/env.context.
  • Manual work:
    • D4: data, post-migration, views/uom_uom_views.xml removed (uom_type is gone).
    • D5: groups.
    • self.env._(...) everywhere.
    • SQL in the migration with psycopg2.sql.
    • Inheritable lambda default.
    • f-string display name.
  • Bug fixed: safe_eval(..., nocopy=True) raises TypeError on 19 (the argument is gone; safe_eval now updates the context dict by itself). The bare except hid it, so computed observations (BP, BMI) silently stopped; the existing tests caught it.
  • New test tests/test_uom.py: reference units, relative units, conversions identical to 16.0, observation types keep their unit.
  • th.po regenerated, 18.0 translations merged.

Why

Closes #139 (P1-4, epic #131).

Migration notes

  • Effort: L (D4).
  • What the migrator did: constraints, _cr/_context.
  • Manual work:
    • UoM data, migration script, the OpenUpgrade workaround;
    • privilege;
    • self.env._;
    • safe_eval.
  • Surprises:
    • OpenUpgrade leaves module units to the module.
    • OpenUpgrade inverts "bigger" units.
    • safe_eval lost nocopy, silently.
    • 19 _compute_quantity no longer checks categories.

How tested

  • Unit/integration tests: 29 tests pass on 19 (25 existing + 4 UoM); fresh install and -u log have no WARNING lines.

  • Real OpenUpgrade 18→19 (OpenUpgrade 19 @ 4783645, second --upgrade-path), on an 18 DB without demo, holding:

    • a patient and a sheet with BP and glucose;
    • two user-created units in a Nirun category (one smaller, one bigger).

    Results:

    • all 15 Nirun units converted, conversions equal to 18;
    • readings keep value and unit;
    • category xmlids gone;
    • privilege groups present;
    • with the workaround, the user-created bigger unit converts like on 18 (without it: 0.1 instead of 10).
  • Manual check (browser, Odoo 19):

    • sheet with sections and value columns, BP computes "150 / 95";
    • sheet kanban card;
    • Units page lists the Nirun units with ratio and reference;
    • encounter Observation tab → wizard with the default category → code value → Save;
    • no JS console errors.
  • pre-commit clean on ni_observation and upgrade_path.

Documentation

  • Module README.md (19.0, units of measure, upgrade, privilege) and upgrade_path/README.md

Notes for reviewer

@piruin piruin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Blocking (mechanical): the branch conflicts with 19.0. It is stacked on #155, so rebase once that lands.

Fix: copyright headers were lost. The 18→19 step removed the <!-- Copyright … NSTDA --> header from 8 XML files:

  • report/ni_encounter_observation_view.xml, report/ni_patient_observation_view.xml
  • views/ni_observation_views.xml, views/ni_observation_sheet_views.xml, views/ni_observation_type_views.xml, views/ni_observation_interpretation_views.xml, views/ni_observation_reference_range_views.xml, views/ni_observation_value_code_views.xml

The cause is probably the migrator rewriting the search-view <group> tags with lxml, which drops the comment at the top of the file. Please restore the headers.

Recommended: log errors in the BP/BMI compute. ni_observation_sheet.py still wraps the safe_eval call in except Exception: continue. That is why the nocopy breakage silently stopped BP/BMI from computing. Log the exception (e.g. _logger.warning(..., exc_info=True)) so the next break is visible.

Minor: there is no [IMP] ni_observation: pre-commit auto fixes commit before [MIG] ni_observation: Migration to 19.0. The series goes straight from the 18.0 [MIG] to the 19.0 one.

Checked, OK: UoM. I checked this against the OpenUpgrade 19 source (uom/19.0.1.0/post-migration.py):

  • OpenUpgrade converts only units without an xmlid (__export__ counts as none), and for bigger units it sets relative_factor = factor, which inverts the ratio. The bug is confirmed.
  • post-nirun_uom_bigger.py selects the same set of units and skips the old reference units (relative_uom_id IS NOT NULL). The upstream fix, OCA/OpenUpgrade#6054, is still open, so the workaround is still needed.
  • In migrations/19.0.1.0.0/post-migration.py and data/uom_uom_data.xml, relative_factor = 1/factor gives the same conversions as 16–18 (1 mg/dl = 0.01 kg/m³, 1 ×10⁶/µL = 1000 ×10³/µL, /hour = 60 /min as in the old data).

Checked, OK: other changes. The privilege, user_ids, models.Constraint, self.env._, the lambda default and the safe_eval nocopy removal are all right. CI is green.

@piruin piruin mentioned this pull request Oct 19, 2026
5 tasks done
@wasud
wasud force-pushed the 19.0-mig-p1-observation branch 2 times, most recently from 7814e46 to adeb71d Compare October 19, 2026 05:22
@wasud

wasud commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Pushed (head adeb71d); the questionnaire stack (#161) is rebased on top.

  • Conflict with 19.0: already gone after the last rebase onto 19.0; the branch contains the current 19.0 head and merges cleanly.
  • Copyright headers: restored in the 8 XML files, inside [MIG] ni_observation: Migration to 19.0. You were right about the cause: odoo-module-migrate 18→19 drops the top comment when it rewrites the search-view <group>. data/ni_observation_reference_range_data.xml had no header on 18.0 either, so I left it as is.
  • Missing pre-commit commit: added [IMP] ni_observation: pre-commit auto fixes (61bc296) before the 19.0 [MIG], with the migrator's automatic changes (version, models.Constraint, self.env.cr, self.env.context, <group>). The [MIG] commit now holds only the manual changes. The final tree is unchanged apart from the headers.
  • Logging in the BP/BMI compute: new commit [IMP] ni_observation: log why a computed observation type fails (adeb71d). It logs _logger.warning(..., exc_info=True) with the type code and sheet id, and the existing exception test asserts the warning. The new warning turned up a broken test: test_code_not_assigning_result_does_not_create_obs used the type code test-noassign-inp, which is cut to 16 characters, so its compute code failed with a NameError instead of testing "no result assigned". I shortened the code, and the test now asserts that nothing is logged. The seeded BP/BMI code already guards against missing inputs, so normal entry doesn't log.

Tests on a fresh DB: ni_observation 31/31, and ni_questionnaire + ni_observation 64/64. pre-commit is clean on the changed files.

Noonmomo and others added 26 commits October 21, 2026 08:26
* [IMP] ni_device: Update required field

* [IMP] ni_device: add devie_usage_log

* [IMP] ni_community_care: update usage log views

* [IMP] ni_community_care: update usage log views

* [IMP] ni_careplan: multi company rule

* [IMP] ni_community_care: update approval careplan report

* [IMP] ni_community_care: update cron id for Create Backfill Approvals

* [IMP] ni_community_care: improve approval report template to avoid printing reports with no data

* [IMP] ni_observation: Add Observation wizard for better UX

* [IMP] ni_observation: remove ni_observation_value_code.type_id use only type_ids

* [IMP] ni_community_care: update approval_category_report_template

* [FIX] ni_careplan: not deleted on form

* [FIX] ni_careplan: resource created by careplan not check period_start

* [IMP] ni_community_care: add pivot view

* [IMP] ni_observation: add parent_id for observation category

* [IMP] ni_observation: observation wizard action from patient

* [FIX] ni_observation: error on select blood_abo

* [FIX] ni_observation: invalid domain for  blood_rh

Took 16 minutes

* [IMP] ni_community_care_attendance: optimize get_employee_dashboard

* [IMP] ni_device: update manifest

* [IMP] ni_device,ni_observation: update device logs

* [IMP] ni_device: update device logs

* [IMP] ni_observation,ni_device: update device logs

* [IMP] ni_observation,ni_device: update device logs

* [IMP] ni_device,ni_observation: update device usage

* [IMP] ni_device,ni_observation: update device usage

* [IMP] ni_device: update ni_device_usage_view_kanban re-direct to res_model form

* [IMP] ni_device: add ni_device_definition

* [IMP] ni_device: update ni_device_definition

* [IMP] ni_device: update device_label_layout_template

* [IMP] ni_device: add compute fields and update views

* [IMP] ni_device : add tracking important fields, add report device lost workflow

* [IMP] ni_device : add employee search by device name, identifier, serial_number

* [FIX] ni_device : fix holder not updated when submitting request_hold

* [IMP] ni_device: All text has been translated into English.

* [IMP] ni_device: All text has been translated into English.

* [IMP] ni_device: translated into Thai and add active_repair_state

* [IMP] ni_device: translated into Thai

* [IMP] ni_device: Update required field

* [IMP] ni_device: add devie_usage_log

* [IMP] ni_community_care: update usage log views

* [IMP] ni_community_care: update usage log views

* [IMP] ni_device: update manifest

* [IMP] ni_observation,ni_device: update device logs

* [IMP] ni_device,ni_observation: update device usage

* [IMP] ni_device,ni_observation: update device usage

* [IMP] ni_device: add ni_device_definition

* [IMP] ni_device: update ni_device_definition

* [IMP] ni_device: update ni_observation_sheet_view_form_inherit

* [IMP] ni_device: rebased 16.0-device branch

* [IMP] ni_device: Update from Thai to English.

* [IMP] ni_device: update device usage log

* [ADD] ni_community_care_device: add commu device to update ni_patient_view_form_simp

* [IMP] ni_device: Update from Thai to English.

* [IMP] ni_community_care: update device kanban

* [IMP] ni_community_care: polish device views

* [IMP] ni_device: Update from Thai to English.

* [IMP] ni_device: Translate thai

* [IMP] ni_device: update ni_device_usage_view_kanban

* [IMP] ni_community_care: fix pre-commit failed

* [IMP] ni_community_care_device: remove init models

---------

Co-authored-by: Piruin P. <piruin.p@gmail.com>
Co-authored-by: BesTinG <79839187+bestiing@users.noreply.github.com>
Use int(float(value)) so that strings like "42.5" are accepted and
truncated to 42, instead of raising ValueError. Add unit tests for
_inverse_value covering all value types.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… social history categories with reference ranges

Adds 22 new observation types across lipid panel (CHOL, LDL, HDL, TG), CBC
(WBC, RBC, Hb, Hct, PLT), kidney function (Cr, BUN, eGFR, UA), liver enzymes
(ALT, AST, ALP), electrolytes (Na, K, Cl), GCS (exam), and smoking/alcohol
status (social history). Includes 96 reference ranges with gender-specific
splits where clinically relevant, plus 6 new UOM categories and units.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… unit tests

- Set body-weight and body-height as children of type_bmi so _eval_compute_types
  resolves them via child_ids (consistent with the BP pattern)
- Apply textwrap.dedent + strip before safe_eval so multi-line / indented code
  stored in XML text nodes evaluates without IndentationError
- Add TestComputeCode: 11 tests covering float/char results, update-on-write,
  no-duplicate idempotency, empty-input skip, None result, missing assignment,
  exception safety, and seeded BP/BMI integration tests

Adds a compute_code Text field on ni.observation.type that lets users write
Python expressions (via ace editor) to derive a type's value from its child
component observations — analogous to ir.cron's code field.

Evaluation logic lives in ObservationSheet._eval_compute_types(), triggered
after ni.observation create/write on value fields. The skip_compute_eval
context flag prevents recursion when writing back the computed result.

Seeds compute_code on type_bp ("SYS / DIA") and type_bmi as reference examples.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…de tests; link input observations as child_ids on computed observation

Replace fragile parent_id mutations on shared fixtures with child_ids in the
create call, and give each edge-case test its own isolated input type.

_eval_compute_types now sets child_ids on the computed ni.observation record
to the component observations used in the calculation. This lets callers verify
which inputs drove a given computed value (e.g. bp_s/bp_d for BP, weight/height
for BMI).

Also fixes three bugs in the partial ni.observation parent/child field definitions:
- ondelete="" → ondelete="set null"
- child_ids pointed to wrong model "ni.patient.observation" → "ni.observation"
- @api.depends("child_count") → @api.depends("child_ids")
- Missing _ in imports

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
type_body_weight and type_body_height referenced type_bmi via parent_id
before it was defined, causing a forward-reference error on module load.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
parent_id/parent_path follow the same pattern already used by
ni.observation.category and ni.condition.category, enabling real
child_of/parent_of matching against coded observation values (e.g.
ni_goal targets) instead of a flat set.

Took 1 hour 49 minutes

Took 15 minutes
…'s event/request id

WorkflowMixin._to_workflow() wrote the parent record's own id into
ni.workflow.event.parent_id / ni.workflow.request.parent_id, but those
fields reference other ni.workflow.event/request rows, not the source
model's own id. This either raised a foreign key violation or silently
linked to an unrelated workflow row, depending on whether the id happened
to collide with an existing workflow row.

Surfaced via ni_observation: an observation type with a computed parent
(e.g. Systolic/Diastolic Blood Pressure -> Blood Pressure) hit this as
soon as both child values were saved, since ni.observation.sheet links
the children to the newly computed parent via child_ids, which re-writes
each child's parent_id and re-triggers the workflow mirror write.

Fixes nirun-life#109. Also fixes ni_device's
test_requires_supported_observation_type, which tried to clear a
device's observation_type_ids via its definition + recompute, but
_compute_from_definition only ever fills a device's observation_type_ids
from a non-empty definition and never clears it.
`default=fields.Datetime.now()` (and, in ni_reception, the stdlib
`fields.datetime.now()`) is evaluated once when the model class is
defined, freezing every record's default to that one moment instead
of the actual creation time. Pass the bare callable so it evaluates
per record.

Affected: ni_observation (observation wizard), ni_document_ref,
ni_medication (suggest wizard), ni_reception, and ni_patient's
ni.workflow.event.mixin (shared by every clinical event model:
procedures, immunizations, medication dispenses, etc).
`ref_range__hba1c__HU` spanned 6.4-100.0 while `ref_range__hba1c__HH`
spanned 7.0-100.0, so any HbA1c >= 7.0 matched both "Significantly
High" and "Critical High" at once. Cap HU at 7.0 so it hands off to
HH, matching the boundary-continuity pattern every other multi-tier
range in this file follows.
The observation_ids editable tree relied on field-level context
(default_sheet_id, default_patient_id, default_encounter_id,
default_occurrence) to fill a manually-added line, but the web client
only applies context defaults for fields declared in that tree's own
arch. Declare them as invisible fields so a new line stays linked to
the sheet/patient being edited.
The form had three inner groups inside a two-column outer group, so the
third one wrapped onto a second row and left the cell beside it empty - a
hole in the middle of the form with nothing to put in it.

Two columns now: what was measured on the left, who it was measured on -
and, for the layers that add them, with which device and where - on the
right.

The value row printed the number twice ("128.0  128.00  mmHg"): `value`
is the Char computed from the typed field beside it and had no modifier,
so both showed. It stays in the arch for the case where value_type is
empty, and for the inverse that writes the typed field back.

Occurrence joins the form, readonly once the reading belongs to a sheet -
_check_occurrence() puts the sheet's own back, so an edit there would be
silently undone. user_id is read from the sheet, so a reading opened on
its own still says who took it.
On small screens the x2many falls back to kanban, and the sheet only
had an inline tree, so readings were cramped on phones. Inline kanban
and form give each reading a card: numbers stack value over unit,
text and choices get a full-width line clamped to 3 lines, and the
type icon falls back per value type when none is set.

Took 3 hours 23 minutes
OpenUpgrade 19 converts units without an xmlid with relative_factor =
factor for "bigger" units, which inverts their ratio (a Box of 24 becomes
1/24 of a unit). Recompute them as 1 / factor from OpenUpgrade's legacy
columns.
The latest-observation views keep one row per patient (or encounter) and type
with DISTINCT ON, ordered by occurrence only. When two observations of a type
share an occurrence (created in the same transaction, for example by two
questionnaires mapped to the same type), PostgreSQL may return either row. The
patient and encounter forms then read a row that the next query no longer
returns, and fail with "records ... cannot be found". Break the tie by id.

Found by the Phase 1 data migration pilot (nirun-life#141); 16.0 has the same views.
The sheet skips a computed type when its compute code raises, without a
trace. When safe_eval stopped accepting nocopy, BP and BMI silently stopped
computing and nothing in the log said so. Log the exception as a warning
with the type and sheet, and keep skipping it.

The warning showed that the "code does not assign result" test never ran
its code: the input type code was cut to 16 characters, so the code failed
with a NameError instead. Shorten the code and assert that nothing is logged.
@wasud
wasud force-pushed the 19.0-mig-p1-observation branch from adeb71d to 57567fc Compare October 21, 2026 01:40
@piruin
piruin merged commit cedfbd3 into nirun-life:19.0 Oct 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants