Skip to content

refactor(operators): use semantic backfill, estimator, catalog, and content identifiers - #966

Draft
seonghobae wants to merge 28 commits into
mainfrom
refactor/post-keyman-backfill-semantic-identifiers
Draft

refactor(operators): use semantic backfill, estimator, catalog, and content identifiers#966
seonghobae wants to merge 28 commits into
mainfrom
refactor/post-keyman-backfill-semantic-identifiers

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Problem

Seven bounded operator commands on protected main@83eba56149eb802cd63642c507c324c9976ec78e used generic organization-owned identifiers that obscured database, record, result, sampling, estimation-run, batch, catalog-payload, post-content, and configuration responsibilities:

  • post-Keyman backfill
  • thread-group-key backfill
  • deterministic channel-weight estimation
  • queued LLM channel-weight estimation
  • ADR 0250 occupational construct catalog synchronization
  • post-content queue backfill
  • synchronous post-content backfill

No other open PR owned these naming paths.

Completed test-first repairs

  • post-Keyman RED: 577ebda05f8995cc3289c7305d0116f5b65014e2
  • thread-group-key RED: f0825c7cd5655bcee57e5c48b49bcc2ae1520b9c
  • deterministic estimator RED: c098fe623c96d264c9b2a5f79bc96765ea73cc62
  • queued LLM estimator RED: cdeaf039551899c77e68e28b6ef79e74ffee4f80
  • occupational catalog synchronizer RED: 4acd000164597b3ddfdecf2d69fc088df17955e3
  • occupational catalog synchronizer GREEN: 56e5d3a21524411b20cb6028b1a4cecc2e423c26
  • post-content queue RED: 6982746bd1be37b2599b9fbba92dcffef566dd30, 8f03f6d4b082562807e2a67d26d92527f1f9bdb2
  • post-content queue GREEN: 78e8d85b7c93f001d3b7ab74a954f2d168b2913c
  • synchronous post-content RED: 52443f8fdce26054c9c3783adfcff933e342bf0b
  • synchronous post-content GREEN: 7cefc9690b6515c63cc7a86af9263bb09d3f8373
  • production and every repository-local private caller were updated in ordinary non-force history

Representative names include _run_post_keyman_backfill, _run_thread_group_key_backfill, _run_channel_weight_estimation, _submit_batch_estimation, _collect_batch_estimation, synchronize_occupational_construct_catalog, database_connection, catalog_payload, runtime_settings, source_post_rows, candidate_window, estimation_run_record, pair_judgment_records, and channel_weight_estimate. The queue slice adds _queue_backfill_parser, valkey_client, source_post_records, post_content_job_request, and backfill_summary. The synchronous content slice adds _post_content_backfill_parser, selected_post_records, source_post_record, normalized_post_content, described_image_count, and semantic SQL aliases.

Compatibility

Regression contracts preserve existing CLI flags, aggregate JSON result keys, provider request/response fields, pair-sampling geometry, weight fitting, fixed O*NET URL/release/output fields, source-digest validation, SQL selection behavior, PostgreSQL schema, transaction rollback, bounded selection, timeout, incomplete-judgment behavior, persistence, resource close, event publication, and persisted provenance. No HTTP or public product API changes and no database migration.

Verification

  • Current synchronous post-content slice: 90 passed across naming, normalization, persistence, queue, worker, and static-SQL coverage
  • Prior six-slice exact-head verification: 91 passed, 2 skipped
  • Ruff check (excluding each script's pre-existing path-bootstrap E402 and executable-bit EXE001), Ruff format, Python compileall, and git diff --check passed
  • Remote four-file GREEN tree matches the locally verified tree byte-for-byte
  • Protected main still pins the older fast-mlsirm@d025b7d237d8db7ca97a5611606c6285d5870895; the dedicated consumer cutover is now PR deps(fast-mlsirm): consume immutable v0.9.1 owner release target #970 at exact head 50807d1e7484fd0aa65bdbf6af015bbe8d739d80, where repository-local Tests/Ontology/PROV-O/SAST are GREEN against the immutable owner v0.9.1 target. deps(fast-mlsirm): consume immutable v0.9.1 owner release target #970 remains unmerged because central Security/CodeQL terminal evidence and qualifying independent approval are still missing. This naming PR therefore does not copy owner source, add a fallback kernel, weaken the backend gate, or absorb that dependency bump before the canonical consumer path lands.

Current exact head: 7cefc9690b6515c63cc7a86af9263bb09d3f8373. Proposed until fresh exact-head checks and independent review complete.

@coderabbitai

coderabbitai Bot commented Sep 7, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae changed the title refactor(keyman-backfill): use semantic operator identifiers refactor(backfills): use semantic operator identifiers Sep 7, 2026

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current-head review finding: the semantic-identifier refactor itself is bounded and I do not see a behavioral/DDD regression in the reviewed diff, but the verification text is now factually stale at the canonical dependency boundary. LineageWeave still pins fast-mlsirm to d025b7d237d8db7ca97a5611606c6285d5870895, while the owner has an immutable v0.9.1 release targeting 09f762ded35786dd1078222a4577ff09d649816f; the current pin predates that release. I opened #967 as the dedicated consumer cutover path. Keep this PR narrow: do not copy fast-mlsirm source, add a fallback kernel, weaken the full backend gate, or fold the dependency bump into this naming-only delta. The PR should describe the full-suite limitation as the current stale consumer pin/toolchain boundary and reference #967 rather than calling fast-mlsirm unreleased.

@seonghobae seonghobae changed the title refactor(backfills): use semantic operator identifiers refactor(operators): use semantic backfill and estimator identifiers Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Current-head repair note (965668b65f5d91a0e53196de5191394d114592e6): the review finding about the dependency boundary is addressed in the PR body and Gap baseline. fast-mlsirm is now described as immutable v0.9.1, with the stale consumer pin and full-suite cutover owned by #967; this naming PR does not copy owner source, add a fallback, weaken the gate, or absorb the dependency bump. The added channel-weight slice was test-first (c098fe623c96d264c9b2a5f79bc96765ea73cc62) and preserves CLI, JSON, pair geometry, fitting, SQL, and persisted provenance contracts. Local focused verification: 38 passed, Ruff clean, compileall and diff check clean. Fresh exact-head security workflows remain queued, so the PR stays Draft.

@seonghobae seonghobae changed the title refactor(operators): use semantic backfill and estimator identifiers refactor(operators): use semantic backfill, estimator, and catalog identifiers Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Exact-head repair note (72f91c3e7297d92a186fb429641496dfdff1d0b8): ADR 0250 catalog synchronization now has a test-first semantic-identifier slice (4acd000164597b3ddfdecf2d69fc088df17955e3 RED → 56e5d3a21524411b20cb6028b1a4cecc2e423c26 GREEN). The private operator vocabulary now names the catalog parser, payload, database connection, runtime settings, command arguments, and synchronized construct count. The fixed O*NET URL/release, --target-dsn, output keys, digest validation, transactional UPSERT, and connection close remain unchanged. Fresh local current-tree verification: 55 passed, 2 skipped; Ruff check (with only the scripts' pre-existing path-bootstrap E402 excluded), Ruff format, compileall, and diff check passed. The PR remains Draft pending fresh exact-head GitHub checks and independent review.

Copy link
Copy Markdown
Contributor Author

Exact-head repair note (872a46831bf499e880d97ee2e423485e088b2add): the post-content queue backfill now has test-first semantic identifier coverage (6982746bd1be37b2599b9fbba92dcffef566dd30 and 8f03f6d4b082562807e2a67d26d92527f1f9bdb2 RED → 78e8d85b7c93f001d3b7ab74a954f2d168b2913c GREEN). Private command, database, Valkey, source-record, completion, job-request, aggregate-result, and SQL-alias vocabulary is now bounded to post-content queue language. Existing CLI flags, JSON keys, PostgreSQL schema, selection semantics, transaction boundaries, Valkey publication, and close behavior remain unchanged. Fresh local current-tree verification: 91 passed, 2 skipped; Ruff check/format, compileall, and diff check passed. The PR remains Draft pending fresh exact-head GitHub checks and independent review.

@seonghobae seonghobae changed the title refactor(operators): use semantic backfill, estimator, and catalog identifiers refactor(operators): use semantic backfill, estimator, catalog, and content identifiers Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant