AI Debug Assistant is a backend copilot for investigating failures in distributed systems. It combines recent and historical logs, technical documentation, and an LLM so a developer can ask questions such as:
Why is
payment-apitiming out for tracea1b2c3d4-e5f6-7890-abcd-ef1234567890?
Follow-up questions can reuse the same conversation session, so the service and trace ID do not need to be repeated.
- Accepts structured application logs through an HTTP API.
- Buffers fresh logs in Redis so they are searchable before indexing finishes.
- Sends logs and uploaded documents through Kafka for asynchronous processing.
- Parses, chunks, embeds, and stores documentation in Qdrant.
- Filters, groups, fingerprints, and embeds critical/warning/error logs in Qdrant.
- Infers service, trace ID, environment, and log level from natural language.
- Retrieves from documentation, indexed logs, and the recent-log buffer.
- Uses OpenAI to generate an answer grounded in retrieved context.
- Remembers the active
serviceandtrace_idin Redis across follow-up requests.
+----------------------+
| OpenAI |
| embeddings + answers |
+----------^-----------+
|
Client -> FastAPI -> auth -> rate limit -> query orchestration
| |
| +-> Redis response cache
| +-> Redis session memory
| +-> Redis recent logs
| +-> Qdrant docs + logs
|
+-> Kafka topics -> worker -> parse/process -> embed -> Qdrant
^ ^
| |
logs documents
POST /api/ask
-> create or reuse session_id
-> load remembered service and trace_id from Redis
-> extract identifiers from the current query
-> merge context (request context > current query > session memory)
-> check the response cache using query + resolved context
-> select docs, logs, or hybrid retrieval
-> search Qdrant and, for logs, the recent Redis buffer
-> filter and rank results
-> build a grounded prompt and call the LLM
-> cache the answer and refresh session memory TTL
-> return answer + confidence + source + session_id
Only service and trace_id are persisted as conversation memory. The system does not store the full chat transcript. Session data is scoped by authenticated API key and expires after SESSION_TTL seconds (24 hours by default). If Redis is unavailable, requests continue without caching, rate limiting, recent-log retrieval, ingestion status, or persistent conversation memory.
POST /api/logs
-> validate and normalize log
-> add to the Redis recent-log window
-> publish to Kafka logs-topic
-> worker batches logs
-> retry transient processing failures
-> send exhausted failures to ingestion-dlq
-> update ingestion status through queued / processing / completed / failed
-> keep WARNING, ERROR, and CRITICAL entries
-> aggregate by trace and select the highest-severity event
-> fingerprint and deduplicate recurring errors
-> detect incidents and frequency spikes
-> embed summaries and upsert them into Qdrant
The Redis recent-log window closes the gap between receiving a log and making its Qdrant vector available. Hybrid retrieval merges both stores by error fingerprint and boosts fresh, intent-matching results.
POST /api/upload-doc
-> validate type and size
-> save to shared temporary storage
-> publish metadata to Kafka documents-topic
-> worker parses and chunks the file
-> embed each chunk
-> upsert chunks into Qdrant
-> delete the temporary file, or move it to failed_ingestion on error
Supported files: .pdf, .txt, .md, and .log.
| Concern | Technology |
|---|---|
| API and validation | FastAPI, Pydantic |
| LLM and embeddings | OpenAI |
| Vector search | Qdrant |
| Cache, session memory, fresh logs | Redis |
| Asynchronous ingestion | Kafka, aiokafka |
| Document parsing | Docling, PyMuPDF |
| Local deployment | Docker Compose |
Requirements: Docker with Compose and an OpenAI API key.
- Copy the provided environment template, then add your secrets:
cp .env.example .env- Start the API, worker, Qdrant, Redis, and Kafka:
docker compose up --build- Confirm the API is healthy:
curl http://localhost:8000/Interactive OpenAPI documentation is available at http://localhost:8000/docs.
Python 3.11 is recommended because it matches the Docker image.
python3 -m venv venv
source venv/bin/activate
pip install -r requirements-ingestion.txt
pip install -r requirements-dev.txt
docker compose up -d qdrant redis kafka kafka-init
uvicorn app.main:app --reloadRun the ingestion worker in a second terminal:
source venv/bin/activate
python -m app.workersWhen the API and worker run on the host, use localhost for Qdrant, Redis, and Kafka. Docker Compose overrides those values with container hostnames for containerized services.
All /api/* endpoints require this header:
x-api-key: value-of-APP_API_KEY
POST /api/ask
Start a session:
curl -X POST http://localhost:8000/api/ask \
-H 'Content-Type: application/json' \
-H 'x-api-key: replace-with-a-client-api-key' \
-d '{
"query": "Why is payment-api failing for trace a1b2c3d4-e5f6-7890-abcd-ef1234567890?"
}'Response:
{
"answer": "...",
"confidence": "high",
"retrieval_source": "logs",
"session_id": "5b9ec524-43ee-42b9-86ba-982e8e6f42ea",
"bug_location": {
"service": "payment-api",
"file": "/app/payments/gateway.py",
"function": "charge_card",
"line": 142,
"release_version": "2026.07.07-1",
"commit_sha": "abc123"
},
"citations": [
{
"source_type": "log",
"source": "payment-api",
"score": 0.91,
"excerpt": "Service: payment-api Level: ERROR Message: Timed out while calling the bank gateway",
"document_id": null,
"trace_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"retrieval_source": "stream",
"bug_location": {
"service": "payment-api",
"file": "/app/payments/gateway.py",
"function": "charge_card",
"line": 142
}
}
]
}Send a follow-up by returning the session_id:
{
"query": "What should I check next?",
"session_id": "5b9ec524-43ee-42b9-86ba-982e8e6f42ea"
}The follow-up automatically reuses debugging coordinates such as service, trace_id, request_id, endpoint, exception_type, and release_version. New values in the query replace remembered values. Integrations can override all inferred values with explicit context:
{
"query": "What caused this failure?",
"session_id": "5b9ec524-43ee-42b9-86ba-982e8e6f42ea",
"context": {
"service": "checkout-api",
"trace_id": "4f0966ad-dc3e-40ef-b31b-08876874728f",
"environment": "prod",
"level": "ERROR"
}
}retrieval_source is llm_only, rag_docs, logs, or hybrid.
POST /api/logs
curl -X POST http://localhost:8000/api/logs \
-H 'Content-Type: application/json' \
-H 'x-api-key: replace-with-a-client-api-key' \
-d '{
"service": "payment-api",
"level": "ERROR",
"message": "Timed out while calling the bank gateway",
"timestamp": "2026-07-07T10:00:00Z",
"environment": "prod",
"exception_type": "TimeoutError",
"stack_trace": "File \"/app/payments/gateway.py\", line 142, in charge_card\nTimeoutError",
"trace_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"release_version": "2026.07.07-1",
"commit_sha": "abc123",
"endpoint": "/checkout",
"downstream_service": "bank-gateway"
}'The endpoint returns 202 Accepted with an ingestion_id; durable processing happens in the worker. If file, function, and line are omitted, the system attempts to extract them from Python-style stack traces.
POST /api/upload-doc
curl -X POST http://localhost:8000/api/upload-doc \
-H 'x-api-key: replace-with-a-client-api-key' \
-F 'file=@./runbook.md'The endpoint returns a document_id and 202 Accepted.
GET /api/ingestion-status/{ingestion_id}
Status values are queued, processing, completed, and failed. Failed statuses include error and error_type when available.
GET /preserves the original lightweight health response.GET /health/livereturns process liveness.GET /health/readychecks Redis, Qdrant, Kafka producer initialization, and OpenAI configuration.GET /metricsexposes Prometheus metrics for HTTP requests, retrieval latency, and ingestion stages.
The router uses a small tested classifier with weighted log/document signals:
- Log-heavy query: search logs.
- Documentation-heavy query: search documents.
- Tie or ambiguous query: search both.
Intent extraction recognizes:
| Signal | Example | Use |
|---|---|---|
| UUID trace ID | a1b2...7890 |
Exact Qdrant and Redis filter |
| Request ID | req-abc-123 |
Exact retrieval filter |
| Non-UUID trace | trace-abc-123 |
Semantic hint and score boost |
| Service | payment-api |
Semantic hint and score boost |
| Environment | prod, staging |
Semantic hint and score boost |
| Level | error, critical |
Semantic hint and score boost |
| Endpoint | /checkout |
Score boost for matching production route |
| Status code | status 500 |
Score boost for matching failure class |
| Exception | TimeoutError |
Score boost for exact exception evidence |
| Release | release 2026.07.07-1 |
Score boost for matching deploy version |
| Commit | commit abc1234 |
Score boost for matching source version |
Results below SIMILARITY_THRESHOLD_MEDIUM are removed. The highest remaining score determines low, medium, or high confidence.
For log-backed answers, the retriever also selects the highest-scoring bug_location from the evidence. This lets /api/ask return a developer-facing service/file/function/line when logs include explicit location fields or parseable stack frames.
Retrieval ranking is telemetry-aware: exact matches on trace/request/endpoint/release/exception outrank generic semantic matches, and logs with bug-location, stack trace, 5xx status, release, commit, or endpoint evidence receive a small causal-evidence boost.
Pydantic loads configuration from environment variables and .env.
See app/core/config.py for the complete list.
app/
├── api/routes/ HTTP endpoints
├── cache/ response cache and session memory
├── core/ configuration, prompts, clients, logging
├── ingestion/
│ ├── chunkers/ document chunking strategies
│ ├── kafka/ producer and worker consumer
│ ├── logs/ fresh buffer, processing, deduplication
│ ├── parsers/ PDF, Markdown, incident, fallback parsers
│ └── pipelines/ log/document transformation pipelines
├── middleware/ API-key auth and rate limiting
├── retrieval/ routing, query intent, hybrid retrieval
├── schemas/ API and event models
├── services/ answer orchestration
├── vector/ embeddings and Qdrant stores
└── workers/ worker entry point
- Redis usage on request paths uses
redis.asyncio. - Kafka log batches flush by batch size or elapsed time.
- Kafka offsets are committed only after successful processing or DLQ handoff.
- Ingestion failures are retried with exponential backoff, then published to
ingestion-dlq. - Ingestion status is queryable by ID.
- Conversation sessions are scoped by authenticated API key.
- Answers include retrieved evidence citations.
- Health, readiness, HTTP metrics, retrieval latency metrics, and ingestion-stage tracing are exposed.
- Routing uses a tested weighted classifier rather than raw keyword counts.
The current implementation provides an end-to-end Retrieval-Augmented Generation (RAG) pipeline with document ingestion, semantic search, log ingestion, Azure OpenAI integration, Redis caching, Kafka-based asynchronous processing, and hybrid retrieval. The following enhancements are planned before considering the system production-ready:
- Ensure responses are strictly grounded in retrieved context.
- Prevent unsupported or hallucinated answers.
- Return "Insufficient information" when evidence is unavailable.
- Improve hybrid retrieval by combining semantic search with keyword/BM25 search.
- Introduce reranking to prioritize the most relevant documents and logs.
- Improve section-level retrieval precision for technical documentation.
- Support metadata-aware retrieval using structured fields (service, level, endpoint, trace ID, etc.).
- Route analytical queries to structured filtering instead of semantic search when appropriate.
- Enable incident summaries and root-cause analysis across related logs.
- The current implementation stores only session metadata for debugging.
- Full conversational memory can be added later using summarization and context management while maintaining privacy and token efficiency.