feat: add knowledge RAG with hybrid retrieval and source citations

[Lling0000]

Summary

• Adds a project-private/account-shared knowledge library for reports, PDFs, DOCX, Markdown, text and public web pages. Sources are versioned, parsed into recursive parent/child passages, and retrieved through Qdrant dense search, BM25 and title/entity recall with RRF fusion and API reranking.
• Chat, report sections and content generation share retrieval and persistent source citations. Access is rechecked on retrieval and source viewing; outbound content uses only explicitly permitted sources. Indexing supports leases, retries, cancellation, rebuilds and orphan cleanup.
• Adds the knowledge UI in five languages, an optional Qdrant Compose profile, a 120-question evaluation corpus, and operational/evaluation documentation.

Verification

• Commands run: full backend suite (700 passed, 3 legacy-route skips), Ruff, frontend TypeScript/Vite build, and the RAG suite with a real Qdrant server (37 passed).
• Manual checks: headless browser checks for desktop/mobile, upload, search, configuration, citation original text, centered dialogs and Unicode/emoji offsets.
• Capacity: 10,000 synthetic documents / 500,000 passages / 1,024-dimensional vectors, 5 concurrent retrievals, real Qdrant 1.19.1 + SQLite. P50 62 ms, P95 2,828 ms; excludes model API latency.
• Follow-up: hosted SDK tracing is disabled for RAG-backed generation; 123 targeted tests passed. Existing Trustabl findings are tracked in the validation notes.
• Model quality: automated/browser models are fixtures. Real Embedding/Rerank credentials were unavailable, so actual Recall/nDCG acceptance remains unverified; live evaluation has an explicit quality gate.
• Setup and APIs · Validation scope and results

Checklist

• The change is focused and scoped to one topic
• Relevant tests and builds were run
• Docs were updated if behavior or configuration changed
• My commits use an email linked to my GitHub account or my GitHub noreply email
• If I used AI or automation, human attribution is preserved via the commit author or Co-authored-by

[github-actions]

Trustabl scan

Lling0000/OpenCMO · codex/rag · 44 findings

Readiness goes from 95 → 100 (+5)

Readiness now   🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩   95 / 100

Projected if all findings resolved   🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩   100 / 100   +5

Findings by severity

Severity	Count
critical	0	▱▱▱▱▱▱▱▱
high	3	▰▱▱▱▱▱▱▱
medium	5	▰▱▱▱▱▱▱▱
low	35	▰▰▰▰▰▰▰▰
info	1	▱▱▱▱▱▱▱▱

Projected headroom — estimate, not a re-scan

Fix scope	Readiness	Δ
Fix critical	95 → 95	+0
+ high	95 → 97	+2
+ medium	97 → 98	+1
+ low	98 → 100	+2
+ info (all)	100 → 100	+0

Projected by re-applying trustabl's own scoring with the listed findings resolved (nothing new introduced). Treat as guidance, not a guarantee.

Metric	Value
Repository	Lling0000/OpenCMO
Branch	codex/rag
Readiness score	95
Risk score	5
Findings	44
Max severity	high
Native exit	1
Rules version	184118273e833a55b83be9a77da993b616cd0b1588c66610f55c328b82243503

❌ Failed

Gate	Threshold	Detected	Result
trustabl native	exit 0	exit 1	❌

Failed due to: trustabl gated (medium+ or --strict)