From cd8ff413d4ed8a5f2855c47a21a31db5661cd487 Mon Sep 17 00:00:00 2001
From: Seongho Bae <me@seonghobae.me>
Date: Sat, 5 Sep 2026 16:55:09 +0900
Subject: [PATCH] fix(search): preserve full-document trigram storage

Replace four overflowing GiST leaf representations with full-content GIN in a forward migration; leave revision 0010 unchanged. Preserve exact ranking SQL and rollback data, and keep the proposal Draft until representative search and migration costs are verified.

Real migrated PostgreSQL: four regression failures before repair; 131 passing tests after repair. Record the owner boundary, rejected options, diagnostic receipts, and ADR-0020 numbering inventory.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Seongho Bae <me@seonghobae.me>
---
 ARCHITECTURE.md                               |  11 +
 CHANGELOG.md                                  |   3 +
 .../versions/0020_search_trigram_storage.py   |  55 +++++
 .../hybrid_retrieval/retrieval_channels.py    |  10 +-
 ...est_email_read_state_migration_postgres.py | 220 +++++++++++++++++-
 .../adr/0020-full-document-trigram-storage.md |  56 +++++
 docs/adr/README.md                            |   1 +
 docs/doctoring/search_trigram_storage.md      |  99 ++++++++
 .../language-agnostic-hybrid-retrieval.md     |  42 ++--
 9 files changed, 472 insertions(+), 25 deletions(-)
 create mode 100644 backend/alembic/versions/0020_search_trigram_storage.py
 create mode 100644 docs/adr/0020-full-document-trigram-storage.md
 create mode 100644 docs/doctoring/search_trigram_storage.md

diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 9d2cbba18..71ae553af 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -135,6 +135,17 @@ boundaries in the pure access-policy evaluator for platform operations, but it
 does not bypass data-region or consent denies; see
 `docs/operations/auth-key-management.md`.
 
+## Search storage repair boundary (Proposed)
+
+Naruon owns its four normalized PostgreSQL search expressions. The forward
+`0020_search_trigram_storage` candidate replaces whole-document GiST indexes
+with GIN without changing stored content, scope, score, or ranking SQL. GIN
+does not supply distance-only kNN acceleration; representative query latency
+and migration lock/build cost remain release gates, not assumed equivalence.
+RankWeave continues to own fusion and query normalization. See
+[ADR-0020](docs/adr/0020-full-document-trigram-storage.md) and the
+[PostgreSQL reproduction](docs/doctoring/search_trigram_storage.md).
+
 ## Local deployment boundary
 
 `docker-compose.yml` provides the blessed local stack: Postgres with pgvector,
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b0f3cf8c3..3abcb557e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,7 @@
 ## [Unreleased]
+- Proposed: repair storage failures for long email and document content while
+  retaining complete text and search scores. Search latency and deployment
+  validation remain required before this change is released.
 - Starlette `TestClient`의 기존 `httpx2==2.5.0` pin을 core 개발·테스트 의존성으로 승격하고, deprecated `httpx` fallback 경고 억제를 제거했습니다.
 - 긴 이메일·첨부 본문을 의미 단위 청크로 임베딩한 뒤 기존 email/attachment 벡터 계약으로 평균화하고, 청크 요청·벡터 누적을 제한된 창으로 처리합니다. OpenAI `text-embedding-3-*`에는 저장 차원(`1536`)을 직접 요청하도록 보강했습니다. 합성 메일 fixture 5건(70청크)과 provider 요청 계약으로 1,536차원 벡터 경로를 검증했으며, 실행 시 선택한 임베딩 제공자에 본문·파싱된 첨부 텍스트를 전송할 수 있습니다. 회사 기밀 데이터는 fixture·commit·PR·log에 포함하지 않습니다.
 - EmailDetail 테스트가 지원하지 않는 스레드 병합/분리 버튼을 `textContent`뿐 아니라 `aria-label`과 `title` 접근 가능 이름으로도 검출하도록 바꿔, 아이콘 전용 버튼 회귀를 놓치지 않습니다.
diff --git a/backend/alembic/versions/0020_search_trigram_storage.py b/backend/alembic/versions/0020_search_trigram_storage.py
new file mode 100644
index 000000000..6e32c2f0e
--- /dev/null
+++ b/backend/alembic/versions/0020_search_trigram_storage.py
@@ -0,0 +1,55 @@
+"""Replace whole-document GiST leaf arrays with full-content GIN indexes.
+
+Keep migration 0010 immutable and preserve its normalization and index names.
+GIN supports trigram predicates, not distance-only kNN acceleration; unchanged
+ranking queries require measured performance evidence before this proposal lands.
+"""
+
+from alembic import op
+import sqlalchemy as sa
+
+revision = "0020_search_trigram_storage"
+down_revision = "0019_email_read_state_repair"
+
+_SEARCH_INDEX_DEFINITIONS = (
+    (
+        "ix_email_records_search_document_trgm",
+        "email_records",
+        "search_normalized_text(coalesce(subject, '') || ' ' || body)",
+    ),
+    (
+        "ix_email_attachments_content_trgm",
+        "email_attachments",
+        "search_normalized_text(content)",
+    ),
+    (
+        "ix_content_segments_safe_text_trgm",
+        "content_segments",
+        "search_normalized_text(safe_text_content)",
+    ),
+    (
+        "ix_project_graph_objects_search_document_trgm",
+        "project_graph_objects",
+        "search_normalized_text(title || ' ' || summary)",
+    ),
+)
+
+
+def upgrade() -> None:
+    """Rebuild the four owner indexes atomically without rewriting documents."""
+    for index_name, table_name, document_expression in _SEARCH_INDEX_DEFINITIONS:
+        op.drop_index(index_name, table_name=table_name, if_exists=True)
+        op.create_index(
+            index_name,
+            table_name,
+            [sa.literal_column(document_expression).label("search_document")],
+            postgresql_using="gin",
+            postgresql_ops={"search_document": "gin_trgm_ops"},
+        )
+
+
+def downgrade() -> None:
+    """Keep corrected indexes so application rollback preserves large records."""
+    # Reinstating GiST can fail once valid high-entropy documents are stored.
+    # A different index strategy needs its own forward, data-preserving repair.
+    return None
diff --git a/backend/services/hybrid_retrieval/retrieval_channels.py b/backend/services/hybrid_retrieval/retrieval_channels.py
index 27b01b375..2ff0066c9 100644
--- a/backend/services/hybrid_retrieval/retrieval_channels.py
+++ b/backend/services/hybrid_retrieval/retrieval_channels.py
@@ -7,10 +7,10 @@
 
 Lexical channels rank by pg_trgm word-similarity distance (``<->>``)
 over the SQL expression ``search_normalized_text(<document text>)``,
-which migration 0010_language_agnostic_search indexes with GiST
-trigram indexes. The expressions built here MUST stay textually
-identical to the indexed expressions, or PostgreSQL will not use the
-indexes. Character trigrams are language-agnostic: no per-language
+which migration 0020_search_trigram_storage indexes with full-content GIN
+trigram indexes. The expressions built here preserve the indexed normalization,
+but GIN does not accelerate this distance-only ordering: query performance is
+a separate rollout gate. Character trigrams are language-agnostic: no per-language
 tokenizer or ``to_tsvector`` configuration is involved (G6).
 
 Dense channels rank by pgvector cosine distance over the stored
@@ -106,7 +106,7 @@ def _lexical_scored_statement(
         normalized_query_expression, document_expression
     )
     # ``document <->> query`` = 1 - word_similarity(query, document);
-    # kNN-ordering form served by the GiST trigram indexes.
+    # Preserve exact ranking; GIN does not provide GiST kNN acceleration.
     lexical_distance = document_expression.op("<->>")(
         normalized_query_expression
     )
diff --git a/backend/tests/test_email_read_state_migration_postgres.py b/backend/tests/test_email_read_state_migration_postgres.py
index d6294e69c..782ee026c 100644
--- a/backend/tests/test_email_read_state_migration_postgres.py
+++ b/backend/tests/test_email_read_state_migration_postgres.py
@@ -1,4 +1,4 @@
-"""PostgreSQL regression coverage for 0011_email_read_state.
+"""PostgreSQL regression coverage for read-state and search-storage repairs.
 
 String-matching the revision file's source (test_alembic_migrations.py)
 cannot detect a destructive downgrade or prove the upgrade is actually
@@ -6,20 +6,29 @@
 database in each of the shapes it must handle.
 """
 
-import subprocess
+import hashlib
 import secrets
+import subprocess
 import sys
 import uuid
+from datetime import datetime, timezone
 from pathlib import Path
 
 import asyncpg
 import pytest
 from asyncpg.exceptions import InvalidAuthorizationSpecificationError, InvalidPasswordError
-from sqlalchemy import text
+from sqlalchemy import func, inspect, select, text, update
 from sqlalchemy.engine import make_url
-from sqlalchemy.ext.asyncio import create_async_engine
+from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
 
 from core.config import settings
+from db.models import (
+    Attachment,
+    ContentNodeRecord,
+    ContentSegmentRecord,
+    Email,
+    ProjectGraphObjectRecord,
+)
 
 pytestmark = pytest.mark.postgres
 
@@ -225,3 +234,206 @@ async def test_upgrade_head_repairs_a_database_already_stamped_past_0011(
     _run_migrations(fresh_database_url)
 
     assert await _column_exists(fresh_database_url, "email_records", "is_read") is True
+
+
+def _search_storage_rows(record_suffix: str):
+    """Build related historical records using the production mapper dependencies."""
+    message_id = f"<search-storage-{record_suffix}@example.com>"
+    content_hash = hashlib.sha256(b"historical quartz").hexdigest()
+    email = Email(
+        user_id="search-storage-user",
+        organization_id="search-storage-org",
+        message_id=message_id,
+        sender="sender@example.com",
+        subject="historical",
+        body="quartz",
+        date=datetime(2026, 9, 5, tzinfo=timezone.utc),
+    )
+    attachment = Attachment(
+        email=email,
+        filename=f"search-storage-{record_suffix}.txt",
+        content="historical quartz",
+    )
+    node = ContentNodeRecord(
+        email=email,
+        content_node_uid=f"search-node-{record_suffix}",
+        source_kind="email_body",
+        source_record_uid=message_id,
+        node_kind="document",
+        node_path="/document[1]",
+        ordinal_index=0,
+        safe_text_content="historical quartz",
+        content_hash=content_hash,
+    )
+    segment = ContentSegmentRecord(
+        email=email,
+        content_node=node,
+        content_segment_uid=f"search-segment-{record_suffix}",
+        source_kind="email_body",
+        source_record_uid=message_id,
+        segment_kind="paragraph",
+        segment_path="/document[1]/paragraph[1]",
+        ordinal_index=0,
+        safe_text_content="historical quartz",
+        content_hash=content_hash,
+        word_count=2,
+    )
+    project_object = ProjectGraphObjectRecord(
+        email=email,
+        primary_content_segment=segment,
+        object_uid=f"search-project-{record_suffix}",
+        user_id=email.user_id,
+        organization_id=email.organization_id,
+        workspace_id="workspace-search-storage-org",
+        object_type="requirement",
+        title="historical",
+        summary="quartz",
+        confidence=0.9,
+        source_segment_uids=[segment.content_segment_uid],
+        extractor_name="deterministic_reference",
+        extractor_version="test",
+    )
+    return {
+        "email_records": email,
+        "email_attachments": attachment,
+        "content_segments": segment,
+        "project_graph_objects": project_object,
+    }
+
+
+def _search_document_values(column_names, document: str):
+    """Split a complete test document across a surface's real storage columns."""
+    # The first SHA-256 word fits the project's 240-character title limit.
+    parts = document.split(" ", 1) if len(column_names) == 2 else [document]
+    values = dict(zip(column_names, parts, strict=True))
+    if "safe_text_content" in values:
+        values["content_hash"] = hashlib.sha256(document.encode()).hexdigest()
+        values["word_count"] = len(document.split())
+    return values
+
+
+@pytest.mark.parametrize(
+    ("surface", "column_names"),
+    [
+        ("email_records", ("subject", "body")),
+        ("email_attachments", ("content",)),
+        ("content_segments", ("safe_text_content",)),
+        ("project_graph_objects", ("title", "summary")),
+    ],
+    ids=["email", "attachment", "segment", "project"],
+)
+@pytest.mark.asyncio
+async def test_search_trigram_storage_forward_repair_preserves_large_documents(
+    fresh_database_url,
+    surface,
+    column_names,
+):
+    """Catch whole-document index overflow and destructive index rollback."""
+    _run_migrations(fresh_database_url, revision="0019_email_read_state_repair")
+    engine = create_async_engine(fresh_database_url)
+    session_factory = async_sessionmaker(engine, expire_on_commit=False)
+    historical_rows = _search_storage_rows("historical")
+    model = type(historical_rows[surface])
+    primary_key = inspect(model).primary_key[0]
+    columns = [getattr(model, name) for name in column_names]
+    search_document = columns[0]
+    if len(columns) == 2:
+        search_document = columns[0] + " " + columns[1]
+
+    async def assert_document(row_id, expected: str, tail_query: str):
+        """Verify complete stored bytes and the literal perfect tail-word score."""
+        async with engine.connect() as connection:
+            row = (
+                await connection.execute(
+                    select(
+                        *columns,
+                        func.word_similarity(
+                            func.search_normalized_text(tail_query),
+                            func.search_normalized_text(search_document),
+                        ),
+                    ).where(primary_key == row_id)
+                )
+            ).one()
+        assert " ".join(row[:-1]) == expected
+        # A complete final word has exactly the query's trigrams, so score = 1.
+        assert row[-1] == 1.0
+
+    try:
+        async with session_factory.begin() as session:
+            session.add_all(historical_rows.values())
+            await session.flush()
+            historical_id = inspect(historical_rows[surface]).identity[0]
+
+        _run_migrations(fresh_database_url)
+        await assert_document(historical_id, "historical quartz", "quartz")
+        async with engine.connect() as connection:
+            index_methods = list(
+                await connection.scalars(
+                    text(
+                        "SELECT access_method.amname FROM pg_index AS index_entry "
+                        "JOIN pg_class AS index_object ON index_object.oid = index_entry.indexrelid "
+                        "JOIN pg_class AS table_object ON table_object.oid = index_entry.indrelid "
+                        "JOIN pg_namespace AS table_schema ON table_schema.oid = table_object.relnamespace "
+                        "JOIN pg_am AS access_method ON access_method.oid = index_object.relam "
+                        "WHERE table_schema.nspname = 'public' AND table_object.relname = :table_name "
+                        "AND index_object.relname LIKE '%_trgm' AND index_entry.indisvalid"
+                    ),
+                    {"table_name": surface},
+                )
+            )
+        assert index_methods == ["gin"], "storage repair must retain a valid trigram index"
+
+        documents = {}
+        for tail_query in ("quartz", "zircon", "topaz"):
+            # Distinct digests exercise diverse trigram keys, not repeated text;
+            # the non-hex tail word occurs only beyond the 32 KiB boundary.
+            prefix = " ".join(
+                hashlib.sha256(f"{surface}:{tail_query}:{index}".encode()).hexdigest()
+                for index in range(1024)
+            )
+            assert len(prefix.encode()) > 32 * 1024
+            documents[tail_query] = f"{prefix} {tail_query}"
+
+        inserted_rows = _search_storage_rows("inserted")
+        for name, value in _search_document_values(
+            column_names, documents["quartz"]
+        ).items():
+            setattr(inserted_rows[surface], name, value)
+        async with session_factory.begin() as session:
+            session.add_all(inserted_rows.values())
+            await session.flush()
+            inserted_id = inspect(inserted_rows[surface]).identity[0]
+        await assert_document(inserted_id, documents["quartz"], "quartz")
+
+        async with engine.begin() as connection:
+            await connection.execute(
+                update(model)
+                .where(primary_key.in_([historical_id, inserted_id]))
+                .values(_search_document_values(column_names, documents["zircon"]))
+            )
+        for row_id in (historical_id, inserted_id):
+            await assert_document(row_id, documents["zircon"], "zircon")
+
+        _run_migrations(fresh_database_url)
+        for row_id in (historical_id, inserted_id):
+            await assert_document(row_id, documents["zircon"], "zircon")
+
+        _run_downgrade(fresh_database_url, "0019_email_read_state_repair")
+        for row_id in (historical_id, inserted_id):
+            await assert_document(row_id, documents["zircon"], "zircon")
+        # Downgrade must retain the corrected indexes and their ability to accept
+        # new large values, not reinstall the known failing GiST representation.
+        async with engine.begin() as connection:
+            await connection.execute(
+                update(model)
+                .where(primary_key.in_([historical_id, inserted_id]))
+                .values(_search_document_values(column_names, documents["topaz"]))
+            )
+        for row_id in (historical_id, inserted_id):
+            await assert_document(row_id, documents["topaz"], "topaz")
+
+        _run_migrations(fresh_database_url)
+        for row_id in (historical_id, inserted_id):
+            await assert_document(row_id, documents["topaz"], "topaz")
+    finally:
+        await engine.dispose()
diff --git a/docs/adr/0020-full-document-trigram-storage.md b/docs/adr/0020-full-document-trigram-storage.md
new file mode 100644
index 000000000..3200823a8
--- /dev/null
+++ b/docs/adr/0020-full-document-trigram-storage.md
@@ -0,0 +1,56 @@
+# ADR-0020: Full-document trigram storage repair
+
+**Status:** Proposed — storage correctness candidate; performance and protected integration remain unverified.
+**Date:** 2026-09-05
+**Decision owner:** Naruon maintainers
+**Scope:** Naruon-owned PostgreSQL search expressions and migration history. RankWeave retains fusion and query-normalization ownership; no provider, external source, or domain truth is transferred.
+
+## Context and constraints
+
+A user can import or restore a long document whose content is valid under the archive contract yet fail before export or search. In the pending #1497 integration, the unchanged large cited-segment test failed when PostgreSQL updated `content_segments.safe_text_content`. The real migrated schema failed with SQLSTATE 54000, while an ORM-only schema had hidden the index limit. This is a storage finding, not evidence that provenance serialization is wrong.
+
+The four whole-document indexes introduced by `0010_language_agnostic_search` cover email subject/body, attachment content, content segments, and project title/summary. A PostgreSQL 16.15 isolated probe with 32,768 bytes and 4,097 distinct trigrams failed for both GiST signature lengths 256 and 2024; the same complete value persisted under GIN. Upstream leaf compression retains a trigram array, so increasing the internal signature length does not cap the leaf value.
+
+Requirements remain full-content persistence, unchanged normalization and exact similarity ordering, tenant scope, reversible application rollout without record loss, and measured page p95 at or below 20 ms. A successful storage test alone does not satisfy the latency requirement.
+
+## Proposed decision
+
+In the context of complete document import, restore, and search,
+facing whole-document GiST leaf-size failures,
+we decided for a forward GIN storage-repair candidate using installed PostgreSQL capabilities
+and against truncation, larger GiST signatures, and an unvalidated similarity threshold,
+to achieve full-content persistence without changing candidate scores or ownership,
+accepting index rebuild cost and loss of GiST distance-order acceleration as unresolved rollout risks.
+
+- Add `0020_search_trigram_storage` after `0019_email_read_state_repair`; retain the published `0010` source and revision identity.
+- Replace only the four canonical search indexes with GIN using the same complete normalized expressions and existing names. Use structured Alembic index operations in one transaction; do not copy or update product records.
+- Keep the SQL distance ordering, limit, score, joins, and owner predicates unchanged. GIN does not accelerate distance-only top-k queries. Do not describe this candidate as indexed kNN or declare equivalent performance.
+- Downgrade retains the corrected indexes and all data. Recreating the known failing GiST indexes could prevent rollback after valid large records have been stored. Retirement or an index-strategy replacement needs its own reviewed forward migration.
+- Schema/migration glue remains in the existing Alembic boundary. No Python computational core, new dependency, or duplicated RankWeave implementation is introduced.
+
+## Alternatives and rejection reasons
+
+| Alternative | Assessment |
+|---|---|
+| Increase `siglen` | Rejected: both tested values fail on the same input; internal signatures do not bound leaf arrays. |
+| Truncate, hash, exclude, or shrink content | Rejected: loses searchable content, changes the contract, or conceals the failure. |
+| Remove search indexes without replacement | Rejected: abandons existing index-supported operators. |
+| Add a fixed similarity threshold for GIN | Rejected without a recall-preserving derivation; it can omit candidates and alter fusion ranks. |
+| Chunked GiST or a separate search runtime | Deferred for measured design comparison: storage/chunk identity, cross-boundary matches, global exact ranking, and tenant isolation require explicit contracts. A prefix-only index is insufficient. |
+| Full-content GIN, unchanged ranking SQL | Selected as a Draft correctness candidate, not approval to ship an unmeasured slow search path. |
+
+## Risks, verification, and follow-up
+
+The transactional rebuild can hold locks and require substantial storage. The deployment owner must estimate index size, profile build/lock duration on representative data, serialize the exact-head migration, and verify rollback before production application. Do not weaken locking or cancel a live migration to accelerate review.
+
+Run real fresh and historical migrations, full-content inserts/updates across all four surfaces, tail-query score assertions, repeat upgrade, and retained-record downgrade/re-upgrade. Then integrate the prerequisite into the existing #1468 → #1427 → #1497 stack without force and rerun the original unchanged archive-size regression.
+
+Before acceptance or release, compare query plans and end-to-end p95/CPU/memory with representative permitted data across all affected pages and locales. Include cold and steady-state requests, failures, all size classes, and simultaneous writers. Do not downsample away expensive cases or claim synthetic unit records as production performance evidence. If ranking is the bottleneck, develop the proven hot-path contract at its canonical owner, Rust-first, and consume an immutable release. This PR remains Draft while that gate or hosted checks/review are incomplete.
+
+## References (APA 7th)
+
+PostgreSQL Global Development Group. (n.d.). *pg_trgm—Support for similarity of text using trigram matching (PostgreSQL 16).* Retrieved September 5, 2026, from https://www.postgresql.org/docs/16/pgtrgm.html
+
+PostgreSQL Global Development Group. (n.d.). *trgm_gist.c* [Source code, commit ad6ffe6a1ffddf19603b13633f054f3d66ef4277]. https://github.com/postgres/postgres/blob/ad6ffe6a1ffddf19603b13633f054f3d66ef4277/contrib/pg_trgm/trgm_gist.c#L107-L143
+
+The live experiment and reproduction receipt belong in [doctoring](../doctoring/search_trigram_storage.md). This Proposed ADR neither changes external-owner maturity nor proves a protected merge or deployed behavior.
diff --git a/docs/adr/README.md b/docs/adr/README.md
index 4d461fff6..868f94839 100644
--- a/docs/adr/README.md
+++ b/docs/adr/README.md
@@ -13,6 +13,7 @@ govern implementation.
 | [ADR-0002](0002-fitted-topic-artifact-consumption.md) | Conditionally consume only a versioned fitted topic artifact through a fail-closed adapter | Proposed | Target `PLANNED`; runtime `BLOCKED-UPSTREAM` |
 | [ADR-0003](0003-separate-topic-measurement-from-agenda-generation.md) | Keep statistical measurement separate from agenda generation | Proposed | Target and future capability `PLANNED`; no implementation authorization |
 | [ADR-0004](0004-status-weighted-calendar-conflicts.md) | Evaluate CalDAV VEVENT overlaps by occupying status; cancelled does not occupy | Accepted | `ACCEPTED-NARUON-POLICY`; advisory evaluate API only |
+| [ADR-0020](0020-full-document-trigram-storage.md) | Repair whole-document trigram persistence without changing ranking; measure latency before rollout | Proposed | Storage candidate; no protected integration or performance acceptance |
 
 The complete topic-intelligence requirements, architecture, contract, UML,
 conceptual ERD, security, test, and operability graph is indexed at
diff --git a/docs/doctoring/search_trigram_storage.md b/docs/doctoring/search_trigram_storage.md
new file mode 100644
index 000000000..c2734da8a
--- /dev/null
+++ b/docs/doctoring/search_trigram_storage.md
@@ -0,0 +1,99 @@
+# Whole-document search storage failure
+
+## Evidence boundary
+
+Observed 2026-09-05. Proposed Naruon storage repair; no protected merge,
+deployment, latency acceptance, or signed-browser restore is claimed.
+The owner branch starts at #1503
+`19d5860bc27e860acba940390f5792721cd99e5e`. The implementing PR records the
+tested source head/tree; the published `0010_language_agnostic_search` file
+is unchanged. [ADR-0020](../adr/0020-full-document-trigram-storage.md) records
+the decision, rejected alternatives, lock cost, and performance prerequisite.
+
+## Failure and causal probe
+
+The pending #1497 integration of `705d8ece2c97edc8575ea59766fd8f68bf4cdb82`
+with #1427 `02366791b2a449b8b23b527dcc550996361c0f96` completed fresh and
+repeated migrations after a local revision-graph repair. Its broader test run
+then reported 1 failed / 288 passed: the unchanged
+`test_export_counts_cited_segment_bytes_once` failed before export when storing
+full segment content. That uncommitted integration result is not a PR-head pass.
+
+An isolated PostgreSQL 16.15 probe confirmed all four migration-created search
+indexes used GiST. A 32,768-byte document containing 4,097 distinct trigrams
+produced the following results:
+
+| Probe | SQLSTATE | Outcome |
+|---|---|---|
+| GiST, `siglen=256` | `54000` | Index row requires 12,304 bytes; maximum 8,191 |
+| GiST, `siglen=2024` | `54000` | Same failure |
+| GIN, complete text | `00000` | Insert succeeds; complete stored text equals input |
+
+This rules out signature enlargement as a repair. The upstream leaf compression
+path materializes the complete trigram array. The original 8 MiB-class archive
+regression remains unchanged; the smaller diagnostic is additional causal
+evidence, not a replacement acceptance case.
+
+The new owner regression first failed on all four surfaces (4 failed,
+4 deselected) at the expected index-size boundary. It exercises historical
+small records, new high-entropy values, updates, exact tail-word similarity
+`1.0`, full-value equality, repeat upgrade, and writes after downgrade followed
+by re-upgrade. The corrected path must also retain a valid trigram index;
+dropping all indexes is not an acceptable pass.
+
+## Reproduction
+
+Use a task-owned disposable PostgreSQL database with pgvector, pg_trgm, and
+unaccent, a free loopback port, random test-only credentials, and isolated
+resources. Never point this fixture at customer data: it creates and removes
+separate test databases. From `backend/`, supply the test database URL and
+bootstrap secret without printing them:
+
+```sh
+uv sync --locked
+uv run --frozen python scripts/migrate_db.py
+uv run --frozen python scripts/migrate_db.py
+uv run --frozen python -m pytest -q -W error -ra --tb=short \
+  tests/test_email_read_state_migration_postgres.py \
+  tests/test_alembic_migrations.py tests/test_bootstrap_db.py \
+  tests/test_data_api.py tests/test_legacy_document_scope_postgres.py \
+  tests/test_workspace_document_migration.py \
+  tests/test_container_dependency_pin_contract.py \
+  tests/test_search.py tests/test_search_postgres.py \
+  tests/test_search_answer.py tests/test_hybrid_retrieval_fusion.py
+```
+
+The first complete repaired run returned 131 passed, zero failed/skipped.
+The final run with the actual-index-presence assertion also returned 131
+passed, zero failed/skipped, in 28.03 seconds. Tests using in-process API overrides are not proof
+of browser cookie authentication, external embeddings, or deployed endpoints.
+The high-entropy records are unit/regression inputs, not production performance
+data. The unchanged ranking query has no GIN distance-order acceleration.
+
+The pinned local image is
+`pgvector/pgvector@sha256:ccc6e83d6e35e931dc7c5def2022729d5a6c370318d099181995567ff1fb4d6b`.
+Each run uses read-only root, no-new-privileges, 256 MiB shared memory and
+temporary database storage. Its exit trap removes only its exact Compose
+project; other containers and persistent volumes remain untouched.
+
+## Retained diagnostic receipts
+
+Local files under `/private/tmp/naruon-search-index-rca.VRmUrq` are not hosted
+artifacts or deployment evidence. Their hashes permit checking the local record:
+
+| Artifact | SHA-256 |
+|---|---|
+| `index_probe.sql` | `c3374657723b58bdb2bb92bc14cb6d86a0bc871541c4e644df9090499c0ad69f` |
+| `index_probe.log` | `e49ec2d43fb1686ae242eebefc869ad243527fa3ef7cc48c793e6e9dcf1c4ed1` |
+| `owner_red.xml` | `acf98ced8c477d8ee2c44914319161ad1605435640ece86d0ae0f69bbc8e6a89` |
+| `owner_final.xml` | `27e2773571bd25cbd23f9b93038191c0cf2d6c63f70bad949d6f8773e60c4164` |
+| `owner_final_migration.log` | `a91543007102a19ab62a85f51dc9b3e027bf58c6c6b1101e82e93b5f82b19b3f` |
+
+Follow-up: measure representative search and migration costs, address the
+resulting performance findings at the canonical owner, propagate without force
+through #1468 → #1427 → #1497, and rerun the complete migrated restore suite.
+Do not close a predecessor or promote any owner proposal to released evidence.
+APA 7th primary-source citations are retained in ADR-0020. The number was
+selected after checking all 160 open PR file lists on 2026-09-05, including
+the second file page of #1287. ADR-0008 already belongs to #1418 and ADR-0019
+appears in #1419; the abandoned local 0008 filename was never committed.
diff --git a/docs/engineering/language-agnostic-hybrid-retrieval.md b/docs/engineering/language-agnostic-hybrid-retrieval.md
index 6553bbf6e..f2790683e 100644
--- a/docs/engineering/language-agnostic-hybrid-retrieval.md
+++ b/docs/engineering/language-agnostic-hybrid-retrieval.md
@@ -6,6 +6,17 @@ bullet of ContextualWisdomLab/naruon#975. Disciplines: **G6**
 (KG is the product), **CP-2** (surface evidence + calibrated
 confidence).
 
+## Proposed storage repair (2026-09-05)
+
+[ADR-0020](../adr/0020-full-document-trigram-storage.md) records a forward
+full-content GIN replacement for the four whole-document GiST indexes. Large,
+diverse inputs can exceed GiST's leaf-size limit even at maximum `siglen`.
+The candidate preserves normalized text, scores, ordering, and scope, but GIN
+does not accelerate distance-only top-k queries. Keep this change Draft until
+representative latency, lock/build cost, and current-head hosted gates are
+verified; unit data is not production performance evidence. See the
+[real PostgreSQL receipt](../doctoring/search_trigram_storage.md).
+
 ## What changed
 
 Context Search (`POST /api/search`) previously scored with
@@ -18,7 +29,7 @@ It now runs two channels per query and fuses them per candidate:
 
 | Channel | Mechanism | Language handling |
 |---|---|---|
-| Lexical | `pg_trgm` character-trigram `word_similarity` over `search_normalized_text(document)` with GiST `gist_trgm_ops(siglen=256)` kNN (`<->>`) | Character n-grams — no tokenizer, no per-language config; NFC + `unaccent` + `lower` fold both sides |
+| Lexical | Exact `pg_trgm` `word_similarity` over `search_normalized_text(document)`, ordered by `<->>`; the Proposed GIN storage repair does not provide indexed kNN | Character n-grams — no tokenizer, no per-language config; NFC + `unaccent` + `lower` fold both sides |
 | Dense | pgvector cosine over stored multilingual embeddings | Multilingual embedding space (provider-routed) |
 
 Search surfaces (naruon#975): `email_records` (subject+body),
@@ -37,7 +48,8 @@ as an `IMMUTABLE` SQL wrapper (the documented pattern for making
 Korean input (macOS file names, some webmail clients emit NFD) match
 composed storage; `unaccent` makes `hop ban nhac` match `họp ban
 nhạc`. The Python query path applies the identical NFC step
-(`services/hybrid_retrieval/query_normalization.py`).
+through the pinned `rankweave` dependency re-exported by
+`services/hybrid_retrieval/__init__.py`.
 
 Degradation: with no LLM provider (or embedding failure) search runs
 lexical-only instead of failing — the previous behavior returned HTTP
@@ -118,23 +130,16 @@ is intentionally not used.
   `sparsevec`) can be added without touching fusion.
 - **SQL-side fusion (UNION + ORDER BY)** — the old approach; fusing in
   Python keeps the fusion function pure, unit-tested, and
-  strategy-swappable, and lets each channel use its own index-served
-  ordering.
+  strategy-swappable. Index-served lexical ordering does not hold for the
+  Proposed GIN storage repair; the measured performance gate remains open.
 
 ## OSMU spin-off assessment (one source, multi use)
 
-`services/hybrid_retrieval/score_fusion.py` +
-`query_normalization.py` are deliberately naruon-free (no model /
-framework imports) and could ship as a standalone "Postgres hybrid
-retrieval fusion" micro-library; `retrieval_channels.py` is the only
-schema-coupled file. Assessment this round: **below the extraction
-threshold** — ~175 lines of generic code with exactly one consumer.
-Extraction (own repo + submodule import per the 따로-또-같이 rule)
-becomes justified when a second consumer materializes
-(semantic-data-portal / scopeweave hybrid search, or the pg_bigm /
-SPLADE `sparsevec` channel). Revisit then, including product naming
-and domain availability; the package boundary already makes the move
-mechanical.
+The earlier extraction assessment is historical. Current source imports fusion
+and query normalization from the pinned `rankweave` package and re-exports them
+through `services/hybrid_retrieval/__init__.py`. Naruon owns the schema-bound
+`retrieval_channels.py` statements and migrations, not a copied fusion runtime.
+This storage repair changes no RankWeave contract or model-routing authority.
 
 ## Operational notes
 
@@ -143,6 +148,11 @@ mechanical.
   the deploy image), the `search_normalized_text` function, and four
   GiST expression indexes. Downgrade drops indexes + function, keeps
   extensions.
+- The Proposed `0020_search_trigram_storage` revision replaces those four
+  indexes with GIN in the existing migration transaction. Its downgrade keeps
+  the corrected indexes so newly valid large records do not make rollback fail.
+  Published revision 0010 remains unchanged; plan storage and lock time before
+  any production migration.
 - The channel SQL expressions in
   `services/hybrid_retrieval/retrieval_channels.py` must stay
   textually identical to the indexed expressions.