ACE-073: CrossDatasourceMetric schema + deployment validation

[ashwin-agami]

Spec: ACE-073

Stacked on #156 (ACE-072). Base is the ACE-072-bridge-as-model-type branch, so this PR shows only the ACE-073 delta. Retarget to main once #156 merges. (Both branches carry a merge of main, incl. Sandeep's mcp<2 pin #159 + the sql-guard fix #153.)

Summary

Second slice of F16 — cross-datasource metrics: the metric itself. Adds a first-class CrossDatasourceMetric (+ SubMeasure) — per-datasource "pieces" (native aggregate bindings) reconciled on a shared key via a {alias} combine formula — plus deployment-level validation. Representation + validation only; execution is ACE-074.

Changes

• models.py — SubMeasure (datasource/binding/grain/alias) + CrossDatasourceMetric. Model validator enforces: executable == "federated", ≥2 pieces across ≥2 datasources, non-empty reconcile_on, non-empty + unique aliases, each grain non-empty with arity == reconcile_on, non-empty combine. Adds "federated" to the Executable literal and rejects it on the join types. cross_datasource_metrics bucket on OrgRecord.
• org_record.py — load/merge the metrics bucket (inline + optional sidecar), dedup by name, per-entry + corrupt-file leniency (mirrors the bridge loader).
• validator.py — metric checks folded into validate_deployment, all fail-closed: unattached datasource / unresolved grain column / no bridge connecting the pieces on reconcile_on (connected-component check) / combine not referencing exactly the pieces / single-source. Plus the fail-open fix: run metric checks even when there are zero bridges.
• Tests — tests/test_cross_datasource_metric.py (30): model rejections, every deployment reject path, the disconnected-datasources repro, a composite-key happy path, federated-rejected-on-joins, dedup, leniency.

Test plan

• uv run dev.py check green (1636 passed); 100% patch coverage on touched lines.
• Reviewed by the Agami panel (correctness + security, high lane) + a fix pass; all findings resolved (bridge connectivity, combine must reference exactly the pieces, grain non-empty/arity).

Checklist

• Spec-linked (Spec: ACE-073), scope-contained — no execution (ACE-074), no proposal (ACE-075), no UI (ACE-076); model_store.py untouched (persistence rode ACE-072's lossless fix)
• Public-safe: synthetic acme/account_key fixtures only
• Fail-closed validation; no SQL executed (binding/combine are inert strings)

[Copilot]

Pull request overview

This PR implements the ACE-073 “cross-datasource metric” representation + validation slice of F16 by adding a first-class CrossDatasourceMetric (with SubMeasure) to the semantic model, loading/merging it into the deployment OrgRecord, and enforcing deployment-level fail-closed validation rules (bridge connectivity, piece resolution, and combine-formula correctness).

Changes:

• Adds SubMeasure + CrossDatasourceMetric models, introduces executable="federated" as a metric-only value, and rejects it on join edge types.
• Extends OrgRecord loading to merge cross_datasource_metrics from inline + optional sidecar YAML, with leniency and dedup-by-name.
• Extends validate_deployment() to validate cross-datasource metrics (pieces, bridge connectivity via connected-component, and combine placeholder refs) even when there are zero bridges.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
packages/agami-core/src/semantic_model/models.py	Adds the new metric schema (SubMeasure/CrossDatasourceMetric), updates Executable, and extends OrgRecord to include metrics.
packages/agami-core/src/semantic_model/org_record.py	Loads/merges cross-datasource metrics from a new sidecar file, with leniency + dedup behavior.
packages/agami-core/src/semantic_model/validator.py	Adds deployment-level validation for cross-datasource metrics and adjusts early-return behavior to ensure metrics are always validated when present.
tests/test_cross_datasource_metric.py	Adds model + deployment validation coverage, including loader leniency and dedup scenarios.

Comments suppressed due to low confidence (1)

packages/agami-core/src/semantic_model/models.py:816

• CrossDatasourceMetric._shape() checks grain is non-empty and has the same arity as reconcile_on, but it doesn’t reject duplicate grain columns (e.g. ["account_key","account_key"]). Since grain defines the grouping key for each piece, duplicates are almost certainly a modeling error and will later complicate reconciliation/execution.

        # Each piece groups by its LOCAL version of the composite key, so its grain must be non-empty
        # and carry exactly one column per reconcile_on component — a 2-part key needs a 2-column grain.
        for sm in self.sub_measures:
            if not sm.grain:
                raise ValueError(
                    f"cross-datasource metric {self.name!r}: sub_measure {sm.alias!r} has an empty grain "
                    "— each piece must group by its local key column(s)"
                )
            if len(sm.grain) != len(self.reconcile_on):
                raise ValueError(
                    f"cross-datasource metric {self.name!r}: sub_measure {sm.alias!r} grain {sm.grain} "
                    f"has {len(sm.grain)} column(s) but reconcile_on {self.reconcile_on} has "
                    f"{len(self.reconcile_on)} — each piece groups by its local version of the composite key"
                )

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ashwin-agami]

Thanks @copilot — both valid, fixed in 5a7cfaa.

1. _bridge_entries now returns [] when the value under the key isn't a list (e.g. metrics: <scalar>), instead of letting the caller iterate a string character-by-character.
2. CrossDatasourceMetric._shape now rejects duplicate columns in reconcile_on and in each sub-measure's grain — a dup would set-match a shorter bridge key (_bridge_matches_key uses set equality) and skew the grain-arity check.

Both locked by tests: test_model_rejects_duplicate_reconcile_on, test_model_rejects_duplicate_grain (and the existing scalar-sidecar test still passes via the clean [] path).