[Backport 1.13.latest] Relay v2 parser output via otel log format for output parity - #16373
Merged
Merged
Conversation
* fix(v2-parser): relay fusion parser output via otel log format for v2 parity
The subprocess relay for `--use-v2-parser` shelled out with `--log-format
json` and forwarded every line as a flat, unstructured Note event,
producing duplicate banners, per-node span noise, and losing the
original fusion error/warning codes.
Switch to `--log-format otel` and parse the JSONL envelope (SpanStart/
SpanEnd/LogRecord). Spans are dropped unconditionally, which structurally
eliminates the span-derived noise without a hand-maintained denylist.
Only an allowlist of LogRecord event types is relayed: LogMessage and
UserLogMessage relay their body (with the fusion error/warning code
composed back into the message text), StdoutMessage/StderrMessage relay
as-is, and ProgressMessage is rendered v1-side to reproduce fusion's
"{action} {target} ({description})" formatting. Severity is mapped from
the OTLP severity_number onto dbt-core's EventLevel.
Unparseable or unrecognized lines now relay at INFO instead of WARN,
fixing a latent bug where a stray fusion stderr line could get promoted
to a raised EventCompilationError under --warn-error and abort the run.
Also forward --log-level-file off. The v2 parser defaults its file log to
{--project-dir}/logs/dbt.log, which is the same path dbt-core writes its
own file log to when --log-path isn't redirected, so both processes were
appending the same relayed events to one file. Everything the subprocess
emits already reaches dbt-core over stdout and lands in dbt.log through
dbt-core's own logger.
* Narrow severity_number type instead of silencing mypy
The `# type: ignore[arg-type]` only covered one of the codes mypy emits for
int(object) depending on version, so it failed under mantle's mypy with
call-overload. An isinstance narrow needs no ignore and keeps the relay
identical between dbt-core and mantle.
* Relay the v2 parser's end-of-run status line
The status line ('Finished 'parse' with N warnings and M errors') has no
LogRecord of its own: the v2 parser renders it in its console formatter
out of Invocation span-end attributes. The relay dropped every span
wholesale, so the line went missing from --use-v2-parser output.
Read the aggregate counts off the Invocation span end instead, and
re-render the line with dbt-core's own ui color helpers. Every other span
is still dropped.
* Drop StdoutMessage/StderrMessage from the relay allowlist
These event types exist for console output only and are not intended to
reach the otel stream, so allowlisting them for forward-compatibility was
speculative. Restore the TODO about switching to the Python OTel decoding
library once it is released.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 594ee2a)
tauhid621
approved these changes
Sep 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport 594ee2a from #16325.