Skip to content
BranchesTags

Repository files navigation

Agentic RAG

Production agentic retrieval-augmented generation system.

React chat UI → FastAPI → LangGraph (memory → retrieve → rerank → augment) → GPT-5.6 Luna token stream. Optional OpenAI web search. Embeddings: Alibaba tongyi-embedding-vision-flash (768-d). Rerank: qwen3-rerank. State in PostgreSQL. Vectors in Qdrant. Deploy on AWS ECS Fargate.

Table of contents

  1. High-level architecture
  2. Component map
  3. Request path
  4. LangGraph agent
  5. RAG pipeline
  6. Memory model
  7. Context window
  8. Auth
  9. Database design
  10. Vector store
  11. Ingest
  12. Streaming contract
  13. Frontend UX
  14. Local vs AWS
  15. Harness
  16. API
  17. Repo layout
  18. Run it
  19. Env

1. High-level architecture

flowchart LR
  U[User] --> FE[React UI<br/>Vite + Nginx]
  FE -->|HTTPS / JWT| ALB[ALB]
  ALB -->|/ | FE
  ALB -->|/api /health /ready| API[FastAPI]
  API --> PG[(PostgreSQL<br/>users chats messages<br/>documents memories)]
  API --> QD[(Qdrant<br/>768-d cosine)]
  API --> OA[OpenAI<br/>gpt-5.6 + web_search]
  API --> DS[Alibaba Model Studio<br/>embed + rerank]
  API --> S3[(S3 / local disk)]
Loading

Two data planes:

Plane Store What lives there
Relational PostgreSQL Identity, chat history (STM source), document metadata, extracted LTM rows
Vector Qdrant Chunk embeddings, memory embeddings, ANN + similar-passage search

2. Component map

flowchart TB
  subgraph Client
    SB[Sidebar<br/>new chat + history + files]
    PB[Prompt box<br/>+ file/image + web search]
    CV[Chat view<br/>markdown + SSE cursor]
    PS[Passages panel<br/>ranked + similar]
  end

  subgraph API["backend/app"]
    AUTH[auth routes]
    CHAT[chats + SSE]
    DOC[documents]
    SRCH[semantic / ranked / similar]
    MW[ASGI middleware<br/>request-id · rate limit · headers]
  end

  subgraph Agent["LangGraph"]
    M[memory]
    R[retrieve]
    A[augment]
    G[generate / stream]
  end

  subgraph RAG
    P[parser]
    C[chunker]
    E[embed]
    RK[rerank]
    CTX[context packer]
  end

  SB --> AUTH
  PB --> CHAT
  PB --> DOC
  CV --> CHAT
  PS --> SRCH
  CHAT --> Agent
  DOC --> P --> C --> E
  M --> PG[(Postgres)]
  M --> QD[(Qdrant)]
  R --> E
  R --> QD
  R --> RK
  A --> CTX
  G --> OA[GPT-5.6 Luna]
Loading

3. Request path

Chat turn from the browser to tokens on screen.

sequenceDiagram
  autonumber
  actor User
  participant UI as React
  participant API as FastAPI
  participant PG as PostgreSQL
  participant LG as LangGraph
  participant QD as Qdrant
  participant DS as DashScope
  participant OA as OpenAI Luna

  User->>UI: type + optional files + web search
  UI->>API: POST /api/documents (if files)
  API->>DS: embed chunks / image
  API->>QD: upsert points
  UI->>API: POST /api/chats/{id}/stream  JWT
  API->>PG: insert user message
  API-->>UI: SSE status=retrieving
  API->>LG: memory → retrieve → augment
  LG->>PG: last N messages (STM)
  LG->>DS: query embedding
  LG->>QD: ANN search user_id filter
  LG->>DS: qwen3-rerank
  LG-->>API: passages + packed context
  API-->>UI: SSE passages
  API->>OA: Responses stream + optional web_search
  OA-->>UI: SSE token / tool / citation
  API->>PG: insert assistant message
  API->>OA: extract durable facts
  API->>QD: upsert LTM vectors
  API-->>UI: SSE done
Loading

4. LangGraph agent

Graph is compiled per request (closes over the DB session).

stateDiagram-v2
  [*] --> memory
  memory --> retrieve
  retrieve --> augment
  augment --> [*]
Loading
Node Job
memory STM = last STM_MESSAGE_LIMIT messages. LTM = ANN over memories collection
retrieve Query embed → Qdrant top-k → qwen3-rerank top-n. Fail-open if embed/rerank is down
augment Pack LTM + STM + passages + question into CONTEXT_WINDOW_TOKENS

Generation is outside the graph so FastAPI can stream SSE without buffering the whole answer.

flowchart LR
  Q[User question] --> SAN[sanitize / injection guard]
  SAN --> MEM[memory node]
  MEM --> RET[retrieve + rank]
  RET --> AUG[augment / budget]
  AUG --> GEN[GPT-5.6 stream]
  GEN --> PERSIST[save message]
  PERSIST --> LTM[maybe extract LTM]
Loading

5. RAG pipeline

Classic parse → chunk → embed → retrieve → rank → augment → generate.

flowchart TB
  F[PDF / DOCX / text / image] --> PARSE[parser]
  PARSE -->|text| CHUNK[recursive chunker<br/>size 800 · overlap 120]
  PARSE -->|image| VEMB[vision embed<br/>tongyi-embedding-vision-flash]
  CHUNK --> TEMB[document embed<br/>text_type=document]
  TEMB --> QD[(Qdrant documents)]
  VEMB --> QD

  UQ[User query] --> QEMB[query embed<br/>text_type=query]
  QEMB --> ANN[cosine ANN top-k=20]
  QD --> ANN
  ANN --> RR[qwen3-rerank top-n=6]
  RR --> PACK[context window packer]
  STM[short-term messages] --> PACK
  LTM[long-term facts] --> PACK
  PACK --> LLM[GPT-5.6 Luna]
  WEB{web_search on?} -->|yes| TOOL[hosted web_search]
  TOOL --> LLM
  LLM --> OUT[SSE tokens + citations]
Loading
Step Model / code Notes
Parse rag/parser.py PDF pages, DOCX, text, images
Chunk rag/chunker.py Headings / paragraphs, then windows
Embed tongyi-embedding-vision-flash 768-d, independent multimodal vectors
Retrieve Qdrant cosine Filter user_id (+ optional document_id)
Rank qwen3-rerank Compatible /reranks API
Augment rag/augmenter.py Priority pack into token budget
Generate gpt-5.6 Responses API, optional web_search

Semantic search vs similarity:

flowchart LR
  subgraph Semantic
    Q1[question text] --> E1[query vector] --> S1[ANN vs all chunks]
  end
  subgraph Similarity
    P[passage id] --> V[stored vector] --> S2[ANN neighbors]
  end
Loading

6. Memory model

flowchart TB
  subgraph STM["Short-term · working context"]
    MSG[messages table<br/>last 16 in this chat]
  end

  subgraph LTM["Long-term · user-scoped"]
    ROW[memories table<br/>kind + content]
    VEC[Qdrant memories<br/>same 768-d space]
  end

  TURN[finished turn] --> X[LLM extract 0-3 facts]
  X --> ROW
  X --> VEC
  NEXT[next question] --> STM
  NEXT --> ANN2[ANN over memories]
  ANN2 --> VEC
Loading
Kind Lifetime Scope Store
STM This chat chat_id PostgreSQL messages
LTM Across chats user_id PostgreSQL memories + Qdrant memories

Extracted kinds: fact | preference | entity.

7. Context window

Packer is greedy by priority, then token estimate (chars / 4).

flowchart TB
  P100["100 · User question"] --> BUDGET[remaining tokens]
  P80["80 · Retrieved passages 1..n"] --> BUDGET
  P40["40 · Conversation STM"] --> BUDGET
  P30["30 · Long-term memory"] --> BUDGET
  BUDGET --> CTX[single context string]
  CTX --> LLM
Loading

Default budget: CONTEXT_WINDOW_TOKENS=24000 (retrieved + memory, not the full model window).

8. Auth

sequenceDiagram
  participant U as User
  participant UI as React
  participant API as FastAPI
  participant PG as users

  U->>UI: register / login
  UI->>API: POST /api/auth/register|login
  API->>PG: bcrypt hash / verify
  API-->>UI: access JWT + refresh JWT
  UI->>API: Authorization Bearer access
  Note over API: HS256 · type=access|refresh
  API-->>UI: 401
  UI->>API: POST /api/auth/refresh
  API-->>UI: new pair
Loading

Passwords are bcrypt (72-byte cap). Production refuses default SECRET_KEY.

9. Database design

PostgreSQL via SQLAlchemy + Alembic (001_initial, 002_indexes).

erDiagram
  users ||--o{ chats : owns
  users ||--o{ documents : uploads
  users ||--o{ memories : remembers
  chats ||--o{ messages : contains

  users {
    uuid id PK
    varchar email UK
    varchar name
    varchar password_hash
    timestamptz created_at
    timestamptz updated_at
  }

  chats {
    uuid id PK
    uuid user_id FK
    varchar title
    timestamptz created_at
    timestamptz updated_at
  }

  messages {
    uuid id PK
    uuid chat_id FK
    varchar role
    text content
    jsonb citations
    jsonb meta
    timestamptz created_at
  }

  documents {
    uuid id PK
    uuid user_id FK
    varchar filename
    varchar content_type
    varchar storage_path
    varchar status
    int chunk_count
    text error
    timestamptz created_at
  }

  memories {
    uuid id PK
    uuid user_id FK
    varchar kind
    text content
    uuid source_chat_id
    bool active
    timestamptz created_at
  }
Loading

Tables

users - account.

Column Type Notes
id UUID PK
email varchar(320) UK lowercased
name varchar(120)
password_hash varchar(255) bcrypt
created_at / updated_at timestamptz

chats - sidebar history.

Column Type Notes
id UUID PK
user_id UUID FK → users ON DELETE CASCADE indexed
title varchar(240) first prompt if still “New chat”
created_at / updated_at timestamptz index (user_id, updated_at)

messages - STM source + transcript.

Column Type Notes
id UUID PK
chat_id UUID FK → chats CASCADE
role varchar(20) user | assistant
content text streamed answer stored after complete
citations JSONB { passages, web }
meta JSONB context token stats
created_at timestamptz index (chat_id, created_at)

documents - ingest metadata (bytes live on disk or S3).

Column Type Notes
id UUID PK also Qdrant document_id payload
user_id UUID FK
filename varchar(512) sanitized basename
content_type varchar(128)
storage_path varchar(1024) local path or s3://bucket/key
status varchar(32) pending → processing → ready | failed
chunk_count int
error text truncated on failure

memories - durable facts (vector twin in Qdrant, id aligned).

Column Type Notes
id UUID PK Qdrant point id
user_id UUID FK
kind varchar(32) fact / preference / entity
content text
source_chat_id UUID optional
active bool

Isolation

flowchart LR
  U1[user A] --> C1[A chats]
  U1 --> D1[A documents]
  U1 --> V1[Qdrant filter user_id=A]
  U2[user B] --> C2[B chats]
  U2 --> D2[B documents]
  U2 --> V2[Qdrant filter user_id=B]
Loading

Every list/get/delete is user_id == current_user. Vectors are never queried without that payload filter.

Indexes

Name Table Columns
ix_users_email users email unique
ix_chats_user_id chats user_id
ix_chats_user_updated chats user_id, updated_at
ix_messages_chat_id messages chat_id
ix_messages_chat_created messages chat_id, created_at
ix_documents_user_id documents user_id
ix_documents_user_created documents user_id, created_at
ix_memories_user_id memories user_id

10. Vector store

Two collections, same dimension, cosine.

flowchart TB
  subgraph documents
    P1[point uuid]
    P1 --> VEC1[vector 768]
    P1 --> PAY1["user_id · document_id · filename<br/>chunk_index · text · modality"]
  end
  subgraph memories
    P2[point = memory.id]
    P2 --> VEC2[vector 768]
    P2 --> PAY2["user_id · text · kind"]
  end
Loading

Payload indexes: user_id, document_id (keyword).

11. Ingest

stateDiagram-v2
  [*] --> processing: upload
  processing --> ready: chunks upserted
  processing --> failed: parse / embed error
  ready --> [*]: delete file + Qdrant points
  failed --> [*]: delete
Loading

Allowed: .pdf .docx .txt .md .csv .json .png .jpg .jpeg .bmp .webp. Path traversal stripped. Size cap MAX_UPLOAD_MB.

12. Streaming contract

POST /api/chats/{id}/stream - text/event-stream. ASGI middleware (not BaseHTTPMiddleware) so the body is not buffered.

type Meaning
status retrieving / uploading / generating / searching the web
passages reranked chunks for the side panel
token next text delta
tool Luna web_search running / done
citation URL annotation from web search
error fail-open message
message persisted assistant id
done stream closed 
sequenceDiagram
  UI->>API: POST stream
  API-->>UI: status retrieving
  API-->>UI: passages
  API-->>UI: status generating
  loop tokens
    API-->>UI: token
  end
  API-->>UI: message id
  API-->>UI: done
Loading

13. Frontend UX

flowchart LR
  AUTH[Register / login] --> APP
  subgraph APP[Main shell]
    SIDE[New chat · history · files]
    THREAD[Messages + composer]
    RIGHT[Retrieved + similar passages]
  end
  THREAD --> PLUS[+ file or image]
  THREAD --> WEB[Web search chip → Luna]
Loading

Mobile: hamburger opens the sidebar over a scrim.

14. Local vs AWS

Local (Compose)

flowchart LR
  B[Browser :80 / :5173] --> FE[frontend nginx]
  FE --> BE[backend :8000]
  BE --> PG[(postgres :5432)]
  BE --> QD[(qdrant :6333)]
  BE --> DISK[uploads volume]
Loading

AWS (Terraform infra/aws)

flowchart TB
  IN[Internet] --> ALB[Application Load Balancer]
  ALB -->|/ | FE[frontend :80]
  ALB -->|/api /health /ready| BE[backend :8000]
  subgraph Fargate["ECS Fargate task"]
    FE
    BE
    QD[qdrant sidecar :6333]
  end
  BE --> RDS[(RDS PostgreSQL 16)]
  BE --> S3[(S3 uploads SSE)]
  BE --> SM[Secrets Manager]
  BE --> CW[CloudWatch logs]
  AS[CPU autoscaling 1-4] --> Fargate
Loading

ALB path rules: default → frontend; /api/*, /health, /ready → backend.

15. Harness

Reliability around every model/tool call (agent/harness.py).

flowchart LR
  CALL[embed / rerank / LLM] --> CB{circuit open?}
  CB -->|yes| FAIL[fail-open or 5xx]
  CB -->|no| RETRY[tenacity 3x<br/>HTTP / timeout]
  RETRY --> TRACE[RunTrace]
Loading

Also: token budget, prompt-injection markers, production config gate (APP_ENV=production).

16. API

Method Path Auth Notes
POST /api/auth/register no returns token pair
POST /api/auth/login no
POST /api/auth/refresh refresh JWT
GET /api/auth/me access
GET/POST /api/chats access list / create
GET/PATCH/DELETE /api/chats/{id} access
POST /api/chats/{id}/stream access SSE
GET/POST/DELETE /api/documents access
POST /api/search/semantic access ANN only
POST /api/search/ranked access ANN + rerank
GET /api/search/similar/{id} access neighbors
GET /health no liveness
GET /ready no Postgres + Qdrant

17. Repo layout

agentic-rag/
├── frontend/                 React (Vite) + nginx
├── backend/
│   ├── app/
│   │   ├── api/routes/       auth, chats, documents, search
│   │   ├── agent/            LangGraph, harness, memory, prompts
│   │   ├── rag/              parse, chunk, retrieve, augment, ingest
│   │   ├── services/         OpenAI, DashScope, Qdrant, S3
│   │   ├── db/               SQLAlchemy models + session
│   │   └── core/             config, JWT, middleware
│   ├── alembic/versions/
│   └── tests/
├── infra/aws/                Terraform ECS + ALB + RDS + S3
├── docker-compose.yml
└── .github/workflows/ci.yml

18. Run it

cp .env.example .env
# set SECRET_KEY, OPENAI_API_KEY, DASHSCOPE_API_KEY
docker compose up --build

http://localhost - register, New chat, attach a file, toggle Web search.

Without Compose: Postgres + Qdrant up, then

cd backend && alembic upgrade head && uvicorn app.main:app --reload
cd frontend && npm install && npm run dev
cd backend && pytest -q

AWS

  1. Push images to ECR.
  2. cd infra/aws && cp terraform.tfvars.example terraform.tfvars
  3. terraform init && terraform apply
  4. Open the ALB DNS.

19. Env

Variable Default Role
SECRET_KEY - JWT signing
OPENAI_API_KEY - GPT-5.6 Luna
OPENAI_MODEL gpt-5.6 generation + web_search
DASHSCOPE_API_KEY - embed + rerank
EMBEDDING_MODEL tongyi-embedding-vision-flash 768-d
RERANK_MODEL qwen3-rerank
DATABASE_URL local compose asyncpg
QDRANT_URL http://qdrant:6333
CONTEXT_WINDOW_TOKENS 24000 pack budget
STM_MESSAGE_LIMIT 16
LTM_RECALL_LIMIT 8
AWS_S3_BUCKET empty local disk if unset
APP_ENV development production validates secrets

Do not commit .env or *.tfvars.

About

Agentic RAG: FastAPI + React + LangGraph + Qdrant on AWS

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages