Skip to content

(prefect-dbt)Fix source freshness logs routing to flow instead of enclosing task - #22989

Open
vyagubov wants to merge 2 commits into
PrefectHQ:mainfrom
vyagubov:dbt_freshness
Open

(prefect-dbt)Fix source freshness logs routing to flow instead of enclosing task#22989
vyagubov wants to merge 2 commits into
PrefectHQ:mainfrom
vyagubov:dbt_freshness

Conversation

@vyagubov

@vyagubov vyagubov commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Checklist

  • This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • If this is a complex change, a maintainer has confirmed the proposed approach on the linked issue.
  • If this pull request adds or changes functionality, it includes tests or explains why tests are not needed.
  • If this pull request changes user-facing behavior, it updates documentation or explains why documentation is not needed.
  • If this pull request removes docs files, it includes redirect settings in mint.json.
  • If this pull request adds functions or classes, it includes helpful docstrings.

closes #22990

When PrefectDbtRunner.invoke(['source', 'freshness', ...]) is called inside a Prefect task, logs from source node events were routed to the flow run instead of the enclosing task.

Root cause:

_process_logging_sync always called task_state.get_task_logger(node_id, ...) for any event with node_info. For dbt build this works fine — a Prefect task is registered per model node via _process_node_started_sync. But _get_manifest_node_and_config only looks in manifest.nodes, not manifest.sources, so source nodes during source freshness never get a task registered. get_task_run_id(node_id) returns None, the logger is created with task_run_id=None, and Prefect routes those logs to the flow.

Meanwhile events without node_info (e.g. dbt startup lines) already fell back to hydrated_context(context) + get_run_logger(), which correctly restores the caller's task run context — that's why dbt deps logs appeared in the task but source freshness per-source logs did not.

@github-actions github-actions Bot added the bug Something isn't working label Aug 31, 2026
@vyagubov

vyagubov commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

Tested in local environment manually. I ran dbt run, dbt build, and dbt source freshness. All finished successfully, logs are as expected.

One CI test failed — it appears unrelated to this change (a google-protobuf import error).

@vyagubov
vyagubov marked this pull request as ready for review August 31, 2026 13:58

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dbt source freshness logs appear only in flow run instead of the enclosing task

1 participant