feat(email): deterministic dedupe provenance — gate strong fingerprints on genuine Date (naruon#1086) - #1195
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe email pipeline now tracks Date and Message-ID provenance, persists Date provenance, preserves source bytes, gates strong fingerprints on trusted dates, and classifies duplicates as ChangesEmail deduplication
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant RFC822Message
participant email_parser
participant import_worker
participant email_dedupe_service
participant Email
RFC822Message->>email_parser: provide Date and Message-ID headers
email_parser-->>import_worker: return parsed fields and provenance
import_worker->>email_dedupe_service: provide candidate and source bytes
email_dedupe_service->>Email: compare fingerprints and identity
Email-->>email_dedupe_service: return stored row data
email_dedupe_service-->>import_worker: return dedupe decision
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
seonghobae
changed the title
seonghobae
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
PR governance metadata gate is not ready for
|
|
Closing after Loop drain: permanently blocked — branch was updated onto develop for mergeability, which cleared prior APPROVED robot evidence; re-review (CodeRabbit/OpenCode) and/or central gate jobs (metadata-only gate evaluation, coverage-evidence) remained pending/stuck without a re-runnable workflow handle. Not force-merging (merge-gate policy). Re-open a focused PR when robot capacity is available. Related product security fixes that reimplemented cleanly remain on branch goal/carddav-path-traversal-decode (#1206) for relaunch. |
Pull request was closed
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Current-head provenance and deduplication revalidation completed: parser evidence classification, import and IMAP persistence, conservative migration backfill, strong-fingerprint gating, and review-required decisions are covered. Refresh central review evidence for this exact head.
|
@coderabbitai review Please re-review the current head |
|
✅ Action performedReview finished.
|
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Current head c14fcae24b99a65d7755a2d4db7eaf93bf431f6e has all repository-local required workflows successful and every current review thread resolved. Re-triggering central current-head OpenCode review and merge scheduling after its RFC 5322 prerequisite PR.
Pull request was converted to draft
|
Temporarily sequencing this provenance/deduplication PR behind #1192 because both modify RFC 5322 date parsing and email-ingest contracts. Keeping this PR in draft prevents an overlapping merge race. After #1192 lands, update onto current |
|
@jules Refresh this PR onto the current Resolve the overlap in
Use test-first conflict resolution. Preserve all current #1192 tests and add the provenance regressions in a separate focused test module if that avoids destructive edits. Rebase or rebuild the branch so it is zero commits behind |
c14fcae to
d948528
Compare
All actionable findings were addressed or explicitly withdrawn in resolved threads. The branch is now rebuilt as one current-develop commit at d948528, preserving later address-display hardening and isolating provenance tests in a focused module.
|
@coderabbitai review Please review exact head |
|
✅ Action performedReview finished.
|
|
@coderabbitai review |
|
…ce-qyxe67 Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
|
@coderabbitai review Please review the unchanged exact current head |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
expected='f98c6fe4f7b452c988f8fae3bb963c97aa8da43b'
printf '%s\n' '== Exact requested commit =='
printf 'checked_out='; git rev-parse HEAD
git cat-file -t "$expected"
git show -s --format='commit=%H%nparents=%P%nsubject=%s' "$expected"
printf '%s\n' '== Merge parents and changed scope =='
git diff-tree --no-commit-id --name-status -r "$expected"
git diff --check "${expected}^1" "$expected"
printf '%s\n' '-- Delta from first parent --'
git diff --stat "${expected}^1" "$expected"
printf '%s\n' '-- Delta from second parent --'
git diff --stat "${expected}^2" "$expected"
printf '%s\n' '== Aggregate scope from current develop merge base =='
base="$(git merge-base "$expected" origin/develop 2>/dev/null || git merge-base "$expected" develop)"
printf 'merge_base=%s\n' "$base"
git diff --name-status "$base" "$expected"
git diff --check "$base" "$expected"
printf '%s\n' '== Focused source identity and provenance symbols =='
rg -n -C 4 --glob '*.py' \
'(def (_extract_date|_extract_date_with_provenance|_email_fingerprint|source_email_fingerprint|canonical_email_source_content|process_fetched_email|_message_bytes)|date_provenance|header_date|message_id_provenance|source_content|strong_email_fingerprint|generate_email_fingerprint|resolve_candidate_disposition|classify_dedupe_decision|process_fetched_email\s*\()' \
backend/services backend/db backend/tests || true
printf '%s\n' '== Relevant file outlines =='
for f in \
backend/services/email_parser.py \
backend/services/email_dedupe_service.py \
backend/services/email_import_service.py \
backend/services/imap_worker.py \
backend/services/pop3_worker.py \
backend/db/models.py \
backend/alembic/versions/0018_email_date_provenance.py \
backend/tests/test_email_parser_provenance.py \
backend/tests/test_email_dedupe_service.py \
backend/tests/test_email_import_service.py \
backend/tests/test_imap_worker.py \
backend/tests/test_pop3_worker.py \
backend/tests/test_source_bound_email_dedupe.py
do
if [ -f "$f" ]; then
echo "--- $f"
ast-grep outline "$f" --items all || true
fi
done
printf '%s\n' '== Migration graph declarations =='
rg -n -C 1 --glob '*.py' \
'^(revision|down_revision|branch_labels|depends_on)\s*=' \
backend/alembic/versions || true
printf '%s\n' '== Workflow changes in aggregate diff =='
git diff --name-status "$base" "$expected" -- '.github/workflows/**' || true
printf '%s\n' '== Current PR state and checks =='
gh pr view 1195 --repo ContextualWisdomLab/naruon \
--json state,headRefOid,baseRefOid,mergeStateStatus,reviewDecision,statusCheckRollup 2>&1 || true
gh pr checks 1195 --repo ContextualWisdomLab/naruon \
--json name,state,conclusion,workflow,link,completedAt 2>&1 || trueLength of output: 156685 🏁 Script executed: #!/bin/bash
set -euo pipefail
show_range() {
local file="$1" start="$2" end="$3"
printf '\n--- %s, Lines %s-%s ---\n' "$file" "$start" "$end"
nl -ba "$file" | sed -n "${start},${end}p"
}
show_range backend/services/email_dedupe_service.py 49 320
show_range backend/services/email_import_service.py 173 365
show_range backend/services/email_import_service.py 805 925
show_range backend/services/imap_worker.py 27 115
show_range backend/services/imap_worker.py 315 365
show_range backend/services/pop3_worker.py 85 210
show_range backend/services/email_parser.py 166 280
show_range backend/alembic/versions/0018_email_date_provenance.py 1 100
show_range backend/tests/test_source_bound_email_dedupe.py 1 230
show_range backend/tests/test_imap_worker.py 50 140
show_range backend/tests/test_pop3_worker.py 85 195
printf '%s\n' '== Every direct process_fetched_email call =='
rg -n -C 4 --glob '*.py' '\bprocess_fetched_email\s*\(' backend || true
printf '%s\n' '== Email fingerprint lookup predicates =='
rg -n -C 5 --glob '*.py' \
'(select\(Email\)|Email\.fingerprint|Email\.message_id|fingerprint\s*==|message_id\s*==)' \
backend/services/email_import_service.py backend/services/imap_worker.py || true
printf '%s\n' '== Parser compatibility wrapper and references =='
rg -n -C 3 --glob '*.py' '\b_extract_date\s*\(' backend || true
printf '%s\n' '== Current checks, supported fields =='
gh pr checks 1195 --repo ContextualWisdomLab/naruon \
--json name,state,workflow,link,completedAt,description 2>&1 || true
printf '%s\n' '== Coverage-evidence status, if present =='
gh pr view 1195 --repo ContextualWisdomLab/naruon \
--json statusCheckRollup --jq '.statusCheckRollup[] | select(.name == "coverage-evidence" or .name == "backend (Python 3.14)") | {name, status, conclusion, detailsUrl, startedAt, completedAt}' \
2>&1 || trueLength of output: 2261 🏁 Script executed: #!/bin/bash
set -euo pipefail
show_range() {
local file="$1" start="$2" end="$3"
printf '\n--- %s, Lines %s-%s ---\n' "$file" "$start" "$end"
awk -v start="$start" -v end="$end" \
'NR >= start && NR <= end { printf "%d\t%s\n", NR, $0 }' "$file"
}
show_range backend/services/email_dedupe_service.py 49 320
show_range backend/services/email_import_service.py 173 365
show_range backend/services/email_import_service.py 805 925
show_range backend/services/imap_worker.py 27 115
show_range backend/services/imap_worker.py 315 365
show_range backend/services/pop3_worker.py 85 210
show_range backend/services/email_parser.py 166 280
show_range backend/alembic/versions/0018_email_date_provenance.py 1 100
show_range backend/tests/test_source_bound_email_dedupe.py 1 230
show_range backend/tests/test_imap_worker.py 50 140
show_range backend/tests/test_pop3_worker.py 85 195
printf '%s\n' '== Every direct process_fetched_email call =='
rg -n -C 4 --glob '*.py' '\bprocess_fetched_email\s*\(' backend || true
printf '%s\n' '== Email fingerprint lookup predicates =='
rg -n -C 5 --glob '*.py' \
'(select\(Email\)|Email\.fingerprint|Email\.message_id|fingerprint\s*==|message_id\s*==)' \
backend/services/email_import_service.py backend/services/imap_worker.py || true
printf '%s\n' '== Parser compatibility wrapper and references =='
rg -n -C 3 --glob '*.py' '\b_extract_date\s*\(' backend || true
printf '%s\n' '== Current checks, supported fields =='
gh pr checks 1195 --repo ContextualWisdomLab/naruon \
--json name,state,workflow,link,completedAt,description 2>&1 || true
printf '%s\n' '== Coverage-evidence status, if present =='
gh pr view 1195 --repo ContextualWisdomLab/naruon \
--json statusCheckRollup --jq '.statusCheckRollup[] | select(.name == "coverage-evidence" or .name == "backend (Python 3.14)") | {name, status, conclusion, detailsUrl, startedAt, completedAt}' \
2>&1 || trueLength of output: 73545
I found no new code blocker in the requested scope.
I verified that the aggregate PR delta contains no workflow changes.
I did not emit an approval action. You are interacting with an AI system. |
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headf98c6fe4f7b452c988f8fae3bb963c97aa8da43b. -
Head SHA:
f98c6fe4f7b452c988f8fae3bb963c97aa8da43b -
Workflow run: 32068450034
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Backend (13 files)"]
S1 --> I1["API and service runtime"]
I1 --> R1["Review risk: Backend (13 files)"]
R1 --> V1["backend tests"]
Evidence --> S2["Docs: email-source-identity-provenance.md"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs: email-source-identity-provenance.md"]
R2 --> V2["docs review"]
|
@opencode-agent @cwl-noema-review please independently re-review exact current head |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current headc7aedc6a6a09bc91156e9c62e44f18cf8b4d3846. -
Head SHA:
c7aedc6a6a09bc91156e9c62e44f18cf8b4d3846 -
Workflow run: 32523127357
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Backend (13 files)"]
S1 --> I1["API and service runtime"]
I1 --> R1["Review risk: Backend (13 files)"]
R1 --> V1["backend tests"]
Evidence --> S2["Docs: email-source-identity-provenance.md"]
S2 --> I2["operator or user guidance"]
I2 --> R2["Review risk: Docs: email-source-identity-provenance.md"]
R2 --> V2["docs review"]
Verified: coverage-evidence and other non-metadata-gate required checks on the current head now report pass (gh pr checks). The coverage-evidence failure this review cited has since cleared on rerun. Dismissing as superseded.
opencode-agent
Bot
added
area: ci-cd
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Pull request overview: inspected changed-file evidence for backend/alembic/versions/0018_email_date_provenance.py, backend/db/models.py, backend/services/email_dedupe_service.py, backend/tests/test_imap_worker.py, and backend/tests/test_pop3_worker.py from the bounded current-head packet. Approval sufficiency: current-head source/diff evidence plus Coverage Decision PASS support the conservative backfill, explicit date_provenance model field, source-bound ingest handoff, and no remaining review/check blockers. Verification posture: review used bounded current-head evidence because direct reads under /home/runner/work/_temp/opencode-pr-head were blocked by external-directory policy; conclusions rely on the authoritative current-head Changed files, Focused changed hunks, CodeGraph evidence, and Coverage execution evidence. Linter/static: no completed failed GitHub Checks were present when evidence was collected, and no separate current-head lint failure evidence was supplied. TDD/regression: targeted regression surfaces are present in backend/tests/test_imap_worker.py and backend/tests/test_pop3_worker.py, and Coverage Decision PASS says supported repository test suites passed. Coverage: Coverage execution evidence reports Result: PASS and says supported repository test suites passed. Docstring coverage: Coverage execution evidence says configured repository docstring gates passed or docstring coverage was advisory. DAG: base-to-head focused flow below, derived from current-head Changed files plus CodeGraph changed-scope exploration.
flowchart LR
A["backend/services/email_parser.py"] --> B["backend/services/email_import_service.py"]
A --> C["backend/services/imap_worker.py"]
A --> D["backend/services/pop3_worker.py"]
B --> E["backend/services/email_dedupe_service.py"]
C --> E
D --> E
G["backend/alembic/versions/0018_email_date_provenance.py"] --> F["backend/db/models.py"]
E --> F
H["backend/tests/test_imap_worker.py + backend/tests/test_pop3_worker.py"] --> C
H --> D
PoC/execution: no trusted browser/E2E receipt applied; runtime evidence is bounded repository test coverage PASS plus source-trace/current-head test assertions. DDD/domain: this change preserves the email identity boundary by separating sender-authored Date provenance from collection-time observation. CDD/context: reviewed the parser/import/IMAP/POP3/dedupe/model/migration context named in current-head Changed files and CodeGraph changed-scope exploration. Similar issues: Changed file history shows earlier source-bound dedupe and canonical-value validation fixes already folded into backend/services/email_dedupe_service.py. Claim/concept check: the PR claim that synthetic collection timestamps should not become strong duplicate evidence is consistent with the conservative unknown backfill and raw-message handoff assertions in the current-head tests. Standards search: source limitation—no separate bounded RFC 5322 or Fellegi-Sunter reference excerpt was supplied, but the inspected current-head hunks did not expose new numeric thresholds or formula changes requiring external re-derivation before approval. Compatibility/convention: inspected changed names such as date_provenance, source_content, and DedupeDecision; the visible schema/code identifiers are descriptive two-word names and the PR body says the effective date contract is preserved. Breaking-change/backcompat: backcompat is preserved by adding date_provenance with non-null unknown defaults instead of reinterpreting historical rows as parsed sender dates. Implementation completeness: the inspected runtime hunks are concrete implementations, not pass/.../NotImplementedError placeholders. Performance: the added hashing/canonicalization work is linear in message/source size, and no new unbounded fan-out was evident in the inspected hunks. Developer experience: DX surface is the backend test/coverage contract; Review execution contracts list cd backend && python3 -m pytest tests and coverage commands, and Coverage Decision PASS confirms the repository-native suite passed. User experience: UX surface is duplicate disposition in backend email ingest rather than web UI; the change is conservative because ambiguous provenance is held out of strong auto-linking instead of silently manufacturing duplicates. Visual/DOM: non-web interaction surface reviewed—migration, parser/import/worker flow, test assertions, and documentation path; no changed DOM surface was in scope. Accessibility/i18n: non-web backend/docs changes only; no new UI interaction or localized string surface was exposed in the inspected hunks. Supply-chain/license: Changed files contain no dependency or lockfile edits, and no current-head failed security/check evidence was supplied. Packaging: despite the repo-wide unpackaged_source_surfaces signal, current-head Review execution contracts also list backend/pyproject.toml, frontend/package.json, repo-native test/coverage/security commands, and Coverage Decision PASS, which is sufficient evidence that the changed backend surface is under an existing package/test contract. Security/privacy: inspected evidence shows IMAP/POP3 worker tests preserve validated bounded import flow and raw-source handoff; no new client-visible sequential identifiers, secret logging, or unresolved security check failures were surfaced.
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including backend/alembic/versions/0018_email_date_provenance.py, backend/db/models.py, backend/services/email_dedupe_service.py, backend/services/email_import_service.py, backend/services/email_parser.py, and 9 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects backend/alembic/versions/0018_email_date_provenance.py to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source claims require trusted bounded source evidence prepared outside the isolated model process; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: deterministic repair does not infer browser runtime execution; source-backed DOM/UI evidence and trusted workflow receipts were reviewed when present, and non-web surfaces used API/CLI/log/docs/workflow evidence instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
Adversarial validation
{"status":"passed","probes":[{"path":"backend/alembic/versions/0018_email_date_provenance.py","line":49,"hypothesis":"Historical `email_records` rows could be backfilled as if they had genuine sender `Date` metadata and begin auto-linking on synthetic timestamps.","attack_or_counterexample":"Apply the migration to a database containing legacy rows with no provenance metadata and check whether the new column silently marks them parsed.","evidence":"Trusted current-head diff at backend/alembic/versions/0018_email_date_provenance.py:49 observed the added non-null column use `server_default=\"unknown\"` after the column-presence guard, so existing rows backfill to `unknown` rather than `parsed`; Trusted current-head source binding at backend/alembic/versions/0018_email_date_provenance.py:49; source-line-sha256=d65ae56094134c58bc31960d954ee5924e99eea77eb05df2053c1c78799ecbe7","outcome":"falsified"},{"path":"backend/tests/test_imap_worker.py","line":129,"hypothesis":"The IMAP ingest path could drop immutable RFC822 bytes before handing an email to the import/dedupe pipeline, forcing fallback identity back onto collection-time fields.","attack_or_counterexample":"Challenge `_sync_tenant` with a fetched IMAP message that has explicit `raw_message` bytes and verify the downstream handoff preserves those bytes.","evidence":"Coverage execution evidence at bounded-review-evidence.md:694-698 reports `Result: PASS` with supported repository test suites passed, and trusted current-head test source at backend/tests/test_imap_worker.py:129 asserts `kwargs[\"source_content\"] == raw_message` after `await worker._sync_tenant(config)`; Trusted current-head source binding at backend/tests/test_imap_worker.py:129; source-line-sha256=73a7b47ddc06dd9d9f5feb937f5d6c82559143e3c7b347fdba88dab17c4b92bb","outcome":"falsified"},{"path":"backend/tests/test_pop3_worker.py","line":183,"hypothesis":"The POP3 ingest path could likewise discard raw transport bytes and regress fallback dedupe identity to mutable parsed fields.","attack_or_counterexample":"Challenge `_sync_tenant` with a retrieved POP3 message and verify the worker passes the exact `raw_message` bytes into `process_fetched_email`.","evidence":"Coverage execution evidence at bounded-review-evidence.md:694-698 reports `Result: PASS` with supported repository test suites passed, and trusted current-head test source at backend/tests/test_pop3_worker.py:183 asserts `imported[0][\"source_content\"] == raw_message` after `await worker._sync_tenant(config, asyncio.Semaphore(1))`; Trusted current-head source binding at backend/tests/test_pop3_worker.py:183; source-line-sha256=abf1a0751d9551cdce533f9314abe51df1494b825d698692c2607331dbc9d969","outcome":"falsified"}],"residual_risk":"Direct reads under `/home/runner/work/_temp/opencode-pr-head` were blocked by external-directory policy, so approval relies on the authoritative bounded current-head packet rather than fresh full-file reads for `backend/services/email_import_service.py`, `backend/services/email_parser.py`, and `docs/doctoring/email-source-identity-provenance.md`. Within that limit, current-head evidence still shows no unresolved review threads, no completed failed checks, and PASS coverage for the repository-native test surface."}- Result: APPROVE
- Reason: Current-head evidence supports the provenance gate: the migration backfills
date_provenanceconservatively, dedupe code adds source-bound identity primitives, IMAP/POP3 tests preserve raw RFC822 bytes, coverage passed, and no unresolved threads or failed checks remain. - Head SHA:
c7aedc6a6a09bc91156e9c62e44f18cf8b4d3846 - Workflow run: 32924628799
- Workflow attempt: 1
Superseded automated OpenCode approval whose explicit review evidence does not match exact current head 0b74bef; a fresh current-head review is required.
seonghobae
added
the
enhancement
ChatGPT Codex Connector
2 participants
Summary
Prevent synthetic collection timestamps from becoming strong duplicate evidence.
date_provenance,header_date, andmessage_id_provenancefrom the email parser while preserving the existing effectivedatecontract;-0000dates to timezone-aware UTC;date_provenanceonemail_recordsthrough Alembic revision0018_email_date_provenance, conservatively backfilling existing rows tounknown;auto_link,review_required, anddistinctclassification plus 1:N disposition resolution.Data and compatibility boundary
date_provenance(two-wordsnake_case).unknown, which can widen clerical review but cannot manufacture an automatic duplicate.datebehavior remains parsed-header-or-fallback.Dateheader was genuinely parsed.Current exact candidate
f6e60ef85e6877df7e978a6f86f019c2c32cf795.develop):f98c6fe4f7b452c988f8fae3bb963c97aa8da43b.develop:dd8d15191338b841f9e6f3a06507c6a5643b95d0(includes feat(calendar): prevent status-weighted double booking #1367).f6e60ef85e6877df7e978a6f86f019c2c32cf795+dd8d15191338b841f9e6f3a06507c6a5643b95d0.0018_email_date_provenance(develop still ended at0017_merge_newsdom_carddav_heads; no retarget required).ddd05c5a: none. The merge was conflict-free and did not change dedupe-provenance product code.f98c6fe4f7b452c988f8fae3bb963c97aa8da43b: 101 passed, 0 failed in 0.36s. NoTimeout/Fatal/Warn/Deniedoutput.The exact current product diff remains fourteen durable files:
backend/alembic/versions/0018_email_date_provenance.pybackend/db/models.pybackend/services/email_dedupe_service.pybackend/services/email_import_service.pybackend/services/email_parser.pybackend/services/imap_worker.pybackend/services/pop3_worker.pybackend/tests/test_email_dedupe_service.pybackend/tests/test_email_import_service.pybackend/tests/test_email_parser_provenance.pybackend/tests/test_imap_worker.pybackend/tests/test_pop3_worker.pybackend/tests/test_source_bound_email_dedupe.pydocs/doctoring/email-source-identity-provenance.mdVerification boundary
Predecessor-head test, check, and review results do not transfer. This exact head must establish its own complete repository CI, migration/single-head, coverage, security, container, and current-head review evidence. Queued, skipped, stale, predecessor-head, author-only, or model-only evidence is non-passing.
Focused contracts cover parsed/missing/invalid Date provenance,
-0000normalization, source-bound identities, collection-time independence, import/IMAP/POP3 parity, conservative migration/backfill behavior, deterministic 1:N disposition, and distinct raw messages collected at the same instant.Customer next action
Do not merge from this reconcile. Wait for exact-head product CI on
f98c6fe4f7b452c988f8fae3bb963c97aa8da43b, then obtain an independent non-author APPROVE. Author, merge-bot, and predecessor-head approvals are not sufficient.Reference
Fellegi, I. P., & Sunter, A. B. (1969). A theory for record linkage. Journal of the American Statistical Association, 64(328), 1183–1210. https://doi.org/10.1080/01621459.1969.10501049
Refs #1086
Merge gate
Merge only after every required unchanged exact-head check is terminal-success, all actionable current-head review threads are resolved, a qualifying independent non-author APPROVE exists where live organization policy requires it, and normal protected-branch rules accept the head without bypass. Predecessor evidence does not transfer. The central trusted-
uvmaterializer repair remains an external control-plane prerequisite for freshcoverage-evidence; an infrastructure failure must not be represented as a source-code finding.