Skip to content

feature - implement core scalar functions and operators (#32)#43

Merged
dannymeijer merged 1 commit into
mainfrom
feature/32-implement-rfc-015-core-scalar-functions
May 25, 2026
Merged

feature - implement core scalar functions and operators (#32)#43
dannymeijer merged 1 commit into
mainfrom
feature/32-implement-rfc-015-core-scalar-functions

Conversation

@dannymeijer
Copy link
Copy Markdown
Collaborator

Summary

This PR implements InQL RFC 015 / issue #32 by moving scalar helpers onto a registry-backed ColumnExpr application model, adding the core scalar/operator helper slice, and closing RFC 015 as Implemented. It also removes DataFusion-specific semantics from the function/lowering path by keeping Substrait as the IR boundary and routing Session execution through a backend adapter dispatch layer.

Type of change

  • Bug fix
  • New feature
  • Refactor / maintenance
  • Documentation
  • CI / tooling
  • RFC (adds/updates docs/rfcs/*)

Area(s)

Select the primary areas touched (labels sync from checked lines when the triage workflow runs):

  • Package & tests
  • Specification (RFCs)
  • Documentation
  • Automation & repo config
  • Other

Key details

  • User-facing behavior: Adds the RFC 015 core scalar surface: casts, comparisons, boolean logic, null/NaN predicates, arithmetic, conditionals, membership/range predicates, and ordering helpers. Ordering helpers are valid through order_by([...]) and lower to real Substrait sort fields.
  • Internals: Replaces bespoke scalar function/operator expression variants with registry-backed ScalarFunctionApplicationExpr, keeps structural column/literal nodes, splits helper modules by logical family, and extends Substrait lowering through registry metadata, built-in Rex shapes, and structural sort-field lowering. Session execution now dispatches through a backend adapter boundary over Substrait plans; DataFusion remains the first adapter.
  • Risks: Broad public helper and lowering surface. Existing helper compatibility is covered, but downstream callers may notice that scalar function metadata and order_by now use the registry-backed/explicit-sort-key model.

Testing / verification

  • make ci (or make fmt-check, make build, make test)
  • Manual verification described below

Manual verification notes:

  • make fmt-check
  • make test-style
  • make registry-metadata (passes; still emits the known stale-lock warning)
  • make build
  • make build-locked
  • make test (108 passed)
  • make smoke-consumer
  • git diff --check

Docs impact

  • No docs changes needed
  • Docs updated
  • Docs follow Divio intent (tutorial/how-to/reference/explanation) where applicable

If docs updated:

  • Link(s): docs/language/reference/functions/index.md, docs/language/reference/dataset_methods.md, docs/language/reference/execution_context.md, docs/release_notes/v0_1.md, docs/rfcs/015_core_scalar_functions.md, docs/rfcs/014_function_registry.md, docs/rfcs/README.md

Checklist

  • I kept public docs user-focused and moved internals to contributing docs when appropriate
  • I avoided duplicating canonical install/run instructions in multiple places
  • I added/updated tests where it materially reduces regressions

Closes #32

@incan-triage-bot incan-triage-bot Bot added documentation Improvements or additions to documentation package Library source, tests, incan.toml specification docs/rfcs/ normative RFCs labels May 25, 2026
@dannymeijer dannymeijer self-assigned this May 25, 2026
@dannymeijer dannymeijer added this to the InQL-v0.1 milestone May 25, 2026
@dannymeijer dannymeijer force-pushed the feature/32-implement-rfc-015-core-scalar-functions branch 4 times, most recently from f34fc33 to b221264 Compare May 25, 2026 18:29
@dannymeijer dannymeijer force-pushed the feature/32-implement-rfc-015-core-scalar-functions branch from b221264 to 126f825 Compare May 25, 2026 18:55
@dannymeijer dannymeijer marked this pull request as ready for review May 25, 2026 19:10
@dannymeijer dannymeijer merged commit 233e441 into main May 25, 2026
4 checks passed
@dannymeijer dannymeijer deleted the feature/32-implement-rfc-015-core-scalar-functions branch May 25, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation package Library source, tests, incan.toml specification docs/rfcs/ normative RFCs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFC 015: Core scalar functions and operators

1 participant