feat: Support FHIRPath trace() function#2584
Open
piotrszul wants to merge 10 commits intorelease/9.6.0from
Open
feat: Support FHIRPath trace() function#2584piotrszul wants to merge 10 commits intorelease/9.6.0from
piotrszul wants to merge 10 commits intorelease/9.6.0from
Conversation
Add the trace(name) function which logs a JSON representation of each evaluated value via SLF4J, then returns the input collection unchanged. This uses a custom Spark Catalyst expression (TraceExpression) to produce the logging side effect during query evaluation. The optional projection parameter is not yet supported. Two YAML reference test exclusions are added for the projection parameter and a known limitation with primitive .id extension access after trace. Closes #2580 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce TraceCollector interface and ListTraceCollector for capturing trace() output during FHIRPath evaluation. The collector is threaded through EvaluationContext into TraceExpression, which now carries FHIR type metadata alongside the trace label. A TraceCollectorProxy (serializable, static registry) wraps the list collector to survive Spark plan serialization. SingleInstance Evaluator creates a collector per evaluation, expands array values into individual trace entries, and includes them in the result DTO. Python bindings return trace data in the result dict under a 'traces' key with label, type, and values. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
piotrszul
added
fhirpath
…race collection Cover proxy delegation, close semantics, buildTraceResults grouping and value expansion, and end-to-end evaluate() with trace() expressions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sanitiseRow() stripped null/synthetic fields from nested Rows but kept the parent StructField pointing at the original schema. Spark's Row.json() uses the parent's dataType to positionally map nested values, causing field-value misalignment in JSON output. Fixes #2583 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Guard TraceExpression hot-path conversion behind log level check, unify Optional usage for trace collector across builders, deduplicate value conversion logic, and extract shared typed-value helper in Python API. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document that TraceExpression intentionally skips null values via nullSafeEval. Remove redundant fhirType field from TraceResult — the per-value type in TypedValue is sufficient. Document the literal-only limitation for the trace name argument. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The FHIRPath trace() function now accepts an optional second argument (projection expression) that is evaluated against the input and logged, while the input collection is still returned unchanged. Also marks TraceExpression as Nondeterministic to prevent Catalyst from optimizing away logging side effects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ble evaluation
Chained traces like trace('inner').trace('outer') caused the inner
expression to fire exponentially due to the binary form evaluating both
children independently with Nondeterministic preventing caching.
TraceExpression is now unary (no projection) and TraceProjectionExpression
is binary (with projection), with shared logic in TraceHelper.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move the shared SLF4J logger into TraceHelper so both TraceExpression and TraceProjectionExpression use the same instance, and remove the redundant log parameter from logAndCollect. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…eter The trace(name, projection) feature is now implemented, so the exclusion that expected this test to error is no longer needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
trace()function with both unarytrace(name)and binarytrace(name, projection)forms, logging a string representation of the input collection (or projected values) at TRACE level via SLF4JTraceCollectorinterface, threading trace entries throughSingleInstanceEvaluationResultandPathlingContext.evaluateFhirPath()evaluate_fhirpath()returns atraceslist withlabeland typedvalues)sanitiseRow()schema misalignment for nested structs — parent StructField dataType now reflects the sanitised nested schema, preventing field-value shifting inRow.json()outputCloses #2580
Fixes #2583
Test plan
TraceFunctionTest— pass-through semantics, SLF4J logging at TRACE level, collector capture, error casesListTraceCollectorTest— trace collector unit testsTraceExpressionTest— unary and binary trace expression evaluationtest_evaluate_fhirpath.py— Python API tests covering basic results, return types, context expressions, variables, trace output structure, and error handlingSingleInstanceEvaluatorTest— nested struct schema alignment and JSON output correctness🤖 Generated with Claude Code