Skip to content

feat: add build orchestrator#322

Draft
Blackgaurd wants to merge 1 commit into04-17-feat_add_build_worker_that_executes_a_single_jobfrom
04-17-feat_add_build_orchestrator
Draft

feat: add build orchestrator#322
Blackgaurd wants to merge 1 commit into04-17-feat_add_build_worker_that_executes_a_single_jobfrom
04-17-feat_add_build_orchestrator

Conversation

@Blackgaurd
Copy link
Copy Markdown
Collaborator

@Blackgaurd Blackgaurd commented Apr 17, 2026

Adds service/orchestrator.py with run_build() that coordinates level-by-level execution of a topological build plan.

What changed

  • New service/orchestrator.py with run_build(dataset_name, version, start, end)
    • Calls schedule_build() to get a BuildPlan (Kahn's algorithm)
    • Iterates levels sequentially (barrier model)
    • Executes each job via execute_job() from the worker module
    • On failure: sets cancelled event and raises RuntimeError
    • Structured logging for level start/complete and plan summary
  • New tests/service/test_orchestrator.py with 6 tests:
    • Single root works, level-by-level order correct (C -> B -> A)
    • Failure at level N prevents level N+1
    • NoValidTimestampsError propagates as RuntimeError
    • Diamond graph executes correctly (D once, {B,C} parallel-safe, A last)
    • Cancelled event is set on failure

Why

The orchestrator ties the scheduler and worker together. It owns the "execute the plan" concern, separate from "compute the plan" (scheduler) and "build one dataset" (worker).

Benefit

run_build() is the drop-in replacement for _build_recursive(). PR 6 will wire it into build_dataset() and delete the old code.

🤖 Generated with Claude Code

Copy link
Copy Markdown
Collaborator Author

Blackgaurd commented Apr 17, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@Blackgaurd Blackgaurd changed the base branch from 04-17-feat_add_build_worker_that_executes_a_single_job to graphite-base/322 April 18, 2026 00:06
@Blackgaurd Blackgaurd force-pushed the 04-17-feat_add_build_orchestrator branch from 0a97d64 to 6fb344e Compare April 18, 2026 00:06
@Blackgaurd Blackgaurd changed the base branch from graphite-base/322 to 04-17-feat_add_build_worker_that_executes_a_single_job April 18, 2026 00:07
@Blackgaurd Blackgaurd force-pushed the 04-17-feat_add_build_worker_that_executes_a_single_job branch from 4a2ca80 to f0f679d Compare April 25, 2026 04:18
@Blackgaurd Blackgaurd force-pushed the 04-17-feat_add_build_orchestrator branch from 6fb344e to 461b6ea Compare April 25, 2026 04:18
@Blackgaurd Blackgaurd force-pushed the 04-17-feat_add_build_worker_that_executes_a_single_job branch from f0f679d to d50c783 Compare April 25, 2026 04:24
@Blackgaurd Blackgaurd force-pushed the 04-17-feat_add_build_orchestrator branch from 461b6ea to 3820536 Compare April 25, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant