Skip to content

refactor(core): add shared atomic_write_text helper, adopt in querygen planning-summary export#237

Merged
saschagobel merged 3 commits into
mainfrom
feat/querygen-resumable/02-atomic-write-helper
Jun 4, 2026
Merged

refactor(core): add shared atomic_write_text helper, adopt in querygen planning-summary export#237
saschagobel merged 3 commits into
mainfrom
feat/querygen-resumable/02-atomic-write-helper

Conversation

@henrycgbaker

@henrycgbaker henrycgbaker commented May 29, 2026

Copy link
Copy Markdown
Collaborator

Stack (6 open PRs): #237#238#239#240#241#242

Chain goal: Add per-batch checkpoint/resume to querygen so a re-run with the same run_id skips already-completed work. Nests resumable state under <run_dir>/checkpoints/, writes a frozen Stage 1 result (planning + dedup) plus per-batch Stage 1 and Stage 2 artifacts, and orchestrates resume behind a fail-fast config-drift gate (--force overrides) so checkpoints are never silently reused under changed settings.

This PR (1/6): Extracts the tmpfile + atomic-rename idiom into a shared core/atomic_io.py helper and adopts it for querygen's planning-summary write — the atomic-write primitive every later checkpoint artifact (#239–242) writes through.


What

  • core/atomic_io.pyatomic_write_text context manager: PID+uuid-uniquified tempfile, Path.replace for the atomic rename, no fsync (matches house style).
  • Querygen export.py adopts it for export_planning_summary (indented JSON for human-scannable persisted artifacts).
  • Note: annotation record_builder/export_runner should adopt atomic_write_text in a follow-up — kept out here to keep this PR's dependency surface to querygen only.

Why

Extracts the tmpfile + atomic-rename idiom (previously duplicated inline in two annotation sites — record_builder.write_partition_manifest and export_runner.write_export_csv) into one shared core helper, then adopts it for querygen's planning-summary write. Reaches the rule-of-3 with the upcoming querygen artifact writes in #239–242.

Test plan

  • tests/unit/core/test_atomic_io.py — round-trip + temp-cleanup-on-exception.

@github-actions github-actions Bot added feature Adds or expands user-facing functionality size: S 50-199 LOC labels May 29, 2026
@henrycgbaker henrycgbaker added the querygen Changes affecting the synthetic query generation tool label May 29, 2026
@saschagobel saschagobel added refactor Improves internal structure without changing user-facing behavior and removed feature Adds or expands user-facing functionality labels Jun 4, 2026
@saschagobel saschagobel self-requested a review June 4, 2026 07:29
@saschagobel saschagobel changed the title refactor(core): add shared atomic_write_text helper, adopt in querygen export refactor(core): add shared atomic_write_text helper, adopt in querygen planning-summary export Jun 4, 2026
@github-actions github-actions Bot added the feature Adds or expands user-facing functionality label Jun 4, 2026
@saschagobel saschagobel removed the feature Adds or expands user-facing functionality label Jun 4, 2026

@saschagobel saschagobel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a helpful refactor and tests cover the core behavior well. I left two non-blocking comments: one on simplifying the dense atomic_write_text docstring, and one on avoiding a one-off local JSON helper / applying atomic JSON writes consistently.

Comment thread src/pragmata/core/atomic_io.py

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’d avoid adding a private _atomic_write_json() helper in querygen/export.py. As currently used, it adds an extra layer for one call site; if this stays local, I think the write is clearer inline in export_planning_summary().

More importantly, this creates an inconsistency in the same module: export_planning_summary() uses atomic JSON writing, while export_queries() still writes meta_path directly with Path.write_text(). Since eval and later querygen artifacts will likely need the same behavior, I’d prefer to either:

  1. inline the atomic_write_text() block here and keep the PR narrowly scoped, or
  2. promote this to a shared atomic_write_json() helper in core/atomic_io.py and use it for both export_planning_summary() and the query metadata JSON.

My preference is option 2, with a small helper that wraps json.dump(..., indent=2) plus a trailing newline. That avoids a local one-off abstraction and gives the upcoming eval/querygen artifact writes one canonical JSON path.

@saschagobel saschagobel marked this pull request as ready for review June 4, 2026 08:08
Base automatically changed from feat/querygen-resumable/01-llm-default-timeout to main June 4, 2026 08:10
@github-actions github-actions Bot added the feature Adds or expands user-facing functionality label Jun 4, 2026
…n export

Extracts the tmpfile+rename atomic-write idiom into core/atomic_io.py and routes
querygen's planning-summary export through it (indented JSON). Annotation
(record_builder/export_runner) should adopt atomic_write_text later; left out
here to keep this PR's dependency surface to querygen only.
@saschagobel saschagobel merged commit 0caf6d2 into main Jun 4, 2026
6 checks passed
@saschagobel saschagobel deleted the feat/querygen-resumable/02-atomic-write-helper branch June 4, 2026 10:28
@saschagobel saschagobel removed the feature Adds or expands user-facing functionality label Jun 4, 2026
@henrycgbaker henrycgbaker restored the feat/querygen-resumable/02-atomic-write-helper branch June 4, 2026 21:03
@henrycgbaker henrycgbaker deleted the feat/querygen-resumable/02-atomic-write-helper branch June 24, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

querygen Changes affecting the synthetic query generation tool refactor Improves internal structure without changing user-facing behavior size: S 50-199 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants