Skip to content

fix(comments): persist threads_comment_id, dedup re-fetches, in-place attribution backfill (#261) - #262

Merged
ajianaz merged 2 commits into
developfrom
fix/comments-pipeline
Sep 10, 2026
Merged

ajianaz merged 2 commits into
developfrom
fix/comments-pipeline

Conversation

@ajianaz

@ajianaz ajianaz commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What

  • Store::insert_comment now accepts and persists threads_comment_id — the column existed since migration 001, CommentData carried it, and the reply route requires it, but the insert never wrote it. Every API reply was guaranteed to fail with NO_THREADS_ID.
  • Dedup on insert:
    • exact threads_comment_id match reuses the existing row;
    • when Meta omits ids (standard access, third-party commenters), id-less rows dedup by (post_id, text).
  • Attribution backfill in place: when an id-less legacy row later arrives with id + author, the row is updated via COALESCE (row id, sentiment, reply state preserved) instead of inserting a twin.
  • Empty-string Meta ids normalize to None on both the API and MCP fetch paths.

Why

The first real inbound comment (canary carousel post, 2026-09-10) exposed the pipeline broken end-to-end: text stored but author_username/threads_comment_id null, reply-via-API guaranteed dead, and re-fetches duplicating rows. Verified across two accounts (#261).

Behavior validated against the official Threads docs: username/permalink fields are documented as only working for public users and your own user, so third-party identity can legitimately be absent → the text-fallback dedup is the durable design (consequence of the platform contract, not a workaround).

Testing

  • 5 new integration tests (store_comments.rs): id persistence, id dedup, text-fallback dedup, backfill-in-place, distinct-text separation.
  • Full workspace: 245 passed / 0 failed.
  • clippy -D warnings clean; cargo fmt clean.

Known limitations (conscious trade-offs, follow-up candidates)

  • Cora review flagged a check-then-insert TOCTOU race under concurrent fetches. Not reachable today (comment fetch is a single sequential API path; SQLite serializes writes). Airtight fix = unique index migration once ids are commonly present.
  • Text-fallback dedup collapses distinct id-less comments with identical text — only key available when Meta omits ids. Frequency unknown; revisit after App Review restores identity fields.

… attribution backfill (#261)

## What

- Store::insert_comment now accepts and persists threads_comment_id
  (column existed since migration 001 but the insert never wrote it,
  which made POST /api/comments/{id}/reply permanently fail with
  NO_THREADS_ID).
- Dedup on insert: exact threads_comment_id match reuses the row;
  Meta's standard-access /replies may omit id/from for third-party
  commenters (observed + consistent with the docs note that username/
  permalink only work for public users and your own user), so id-less
  rows dedup by (post_id, text).
- Attribution backfill: when an id-less row later arrives with id and
  author, the legacy row is updated in place via COALESCE (original row
  id, sentiment and reply state preserved) instead of inserting a twin.
- Empty-string Meta ids ("" when the field is absent) normalize to None
  on both the API and MCP fetch paths.
- 5 new integration tests (store_comments.rs): id persistence, id dedup,
  text-fallback dedup, backfill-in-place, distinct-text separation.

## Why

First real inbound comment exposed the pipeline end-to-end broken:
text stored but author/id null, reply-via-API guaranteed to fail, and
re-fetches duplicating rows. Verified live on v0.9.3 across two accounts
(#261). Identity redaction behavior validated against the official
Threads docs (username/permalink are conditional), so the text-fallback
dedup is the durable design, not a workaround.

## Testing

- 5 new integration tests, all green.
- Full workspace: 245 passed / 0 failed.
- clippy -D warnings clean, cargo fmt clean.
Comment thread crates/titen-core/src/store.rs Fixed
@github-actions

github-actions Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

🔍 Cora AI Code Review

❌ Blocked — critical issues found.

🔴 Error (1)

  • crates/titen-core/src/store.rs:1539 — insert_comment runs the existence SELECT (WHERE post_id = ?1 AND threads_comment_id = ?2) and the later INSERT as separate statements with no transaction and no visible unique constraint. Both the axum /comments fetch route and the MCP fetch_comments tool call this concurrently; two interleaved calls can both find no existing row and both INSERT, recreating the duplicate rows this change is meant to eliminate.

Review powered by cora-code · BYOK · MIT

…ng findings

- Id-less dedup now matches author too: two different users posting
  identical text stay distinct rows.
- Attribution backfill restricted to same-author or still-anonymous
  rows (most recent candidate wins): an unrelated commenter's row can
  never absorb the incoming threads_comment_id.
- +2 regression tests covering both scenarios (7 total in suite).

Workspace: 247 passed / 0 failed; clippy -D warnings + fmt clean.
@ajianaz
ajianaz merged commit 4d3483b into develop Sep 10, 2026
13 checks passed
ajianaz added a commit that referenced this pull request Sep 11, 2026
…in (#268)

Legacy fetches before #262 stored comment rows without
threads_comment_id. The #261 backfill can never reconcile rows that
already have an attributed twin: once the twin exists, every fetch
takes the exact-id match branch, so the id-less row lives forever and
double-counts the comment in sentiment and reply workflows.

Add Store::cleanup_superseded_comments() and run it at the end of
migrate(): delete id-less rows that have an attributed twin (same
post_id + text) with a matching or absent author. Rows carrying reply
workflow state (reply_status != 'new', replied_at, reply_text) are
never deleted. Idempotent; purge count logged at startup.

Closes #266

Co-authored-by: ajianaz <ajianaz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants