fix(aio): stop mixing arrayJoin and duplicated JSON-property reads with aggregates in personal spend

[posthog]

Problem

Two error tracking issues fired within a minute of each other on the AI observability personal spend endpoint (products/ai_observability/backend/api/personal_spend.py), both surfacing as ClickHouse Not found column in block errors — with two distinct causes in the same file.

• _fetch_by_tool puts arrayJoin(splitByChar(',', properties.$ai_tools_called)) in the select list alongside sum(toFloat(properties.$ai_total_cost_usd)). ClickHouse evaluates the array join before aggregation, and once the cost property resolves to a materialized column, that projection doesn't survive into the post-array-join block.
• _fetch_summary inlines the same product_filter property comparison twice — once in countIf(...), once in sumIf(..., ...). Duplicating that read across sibling aggregate clauses hits the same class of ClickHouse failure once the two textually-identical occurrences get optimized differently.

Tests didn't catch either: the test environment reads these properties via JSONExtractRaw, not the materialized-column path that triggers the bug in production.

Changes

Both queries now read the property once, in an inner subquery, and aggregate over the resulting plain column outside:

• _fetch_by_tool: the arrayJoin split and the raw cost_usd / input_tokens columns are projected in an inner SELECT; count()/sum()/avg() run in the outer query.
• _fetch_summary: the product_filter comparison is read once into an is_scoped boolean in the inner SELECT; countIf/sumIf reference that column instead of re-embedding the filter.

No response shape or behavior change — same fields, same values, just restructured queries.

Investigation notes

I looked for a wider fix in the shared HogQL printer (visit_json_subcolumn_access in posthog/hogql/printer/clickhouse.py), since the underlying "Not found column" pattern isn't unique to this endpoint. I didn't make that change: it's exercised by an extensive existing snapshot test suite and by an active JSON-events-schema migration, and I don't have ClickHouse available in this sandbox to validate a printer-level change against real query execution or to regenerate the affected snapshots. A broader printer fix should be picked up separately, with a live ClickHouse to verify against.

I also found #73367, an open PR that expands _fetch_by_tool with new attribution fields but doesn't touch this query bug — this PR is a separate, additive bugfix that #73367 will need to rebase around.

How did you test this code?

I couldn't run the ClickHouse-backed test suite in this sandbox (no ClickHouse/Postgres available). I:

• Confirmed both restructured queries parse as valid HogQL (parse_select).
• Ran ruff check/ruff format (clean) and mypy on the changed file (clean).
• Hand-updated test_personal_spend.ambr to match the new query shape, based on the existing snapshot's join/settings structure — this is unverified against a live ClickHouse run and should be checked/regenerated in CI (@snapshot_clickhouse_queries needs the real driver to capture the printed SQL).

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

No user-facing docs cover this internal endpoint.

🤖 Agent context

Autonomy: Fully autonomous

I (Claude Sonnet 5, via PostHog Code) investigated and fixed this from an inbox report describing two production exceptions from this endpoint. No skills were invoked beyond reading the product's CLAUDE.md. I considered and rejected a global fix in the shared HogQL printer (routing JSON-subcolumn path access through getSubcolumn uniformly) — plausible per the report's diagnosis, but too high-blast-radius to make blind in an environment without ClickHouse to validate it or run the existing printer snapshot suite. Scoped the fix to the two concrete queries that actually produced the observed exceptions instead.

---

Created with PostHog Desktop from this inbox report.

[trunk-io]

Merging to master in this repository is managed by Trunk.

• To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

[scheduled-actions-posthog]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.

[scheduled-actions-posthog]

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.