Lightning-Fast, Context-Dense RAG Framework for Python
Stop waiting. Start predicting.
Quickstart · How It Works · Cost Savings · API · Contributing
Traditional Retrieval-Augmented Generation (RAG) is slow and expensive:
- High Latency: User types query → Hits Enter → WAIT → Vector search → WAIT → Stuff 10 large chunks into LLM → WAIT → Response.
- "Lost in the Middle" Syndrome: You stuff massive chunks of text into the context window, most of which is useless filler. The LLM loses track of the actual facts.
- Expensive Redundancy: On every turn of the conversation, you re-fetch and re-process the exact same context over and over again.
Quira solves this by predicting what users need before they finish typing, dynamically compressing context to maximize density, and statefully tracking the conversation.
⏱️ Real Latency Reduction | 🧠 3-Tier Context Compression | 💰 Proven Token Savings
- Quira Edge (Zero-Server Mode): Run Quira entirely locally using embedded vector databases like DuckDB or SQLite (
sqlite-vec). No Redis or Qdrant servers required. Perfect for client-side apps, edge devices, and testing. - GraphRAG Capabilities: Solves the multi-hop reasoning problem. Quira automatically extracts Entity-Relationship Triplets during ingestion and traverses this Knowledge Graph in parallel with semantic search to provide hyper-accurate context.
- Agentic Routing: Zero-latency heuristics intercept conversational queries (e.g., "Hi", "Thanks"). Bypasses the entire RAG pipeline to return an instant canned response, saving 100% of vector database latency and LLM token costs on chitchat.
- Lexical Intent Debouncing: Saves up to 80% on vector database costs by only firing speculative fetches when human intent changes, not on every keystroke.
- Semantic Fuzzy Caching: Matches predicted queries using Cosine Similarity, ensuring cache hits even with typos or phrasing differences.
- 3-Tier Context Compression: Extractive TextRank, Entity-anchored Extraction, and optional LLM Abstractive Summarization packed into the context window.
- Differential Retrieval State: Tracks multi-turn conversations and reuses context securely, reducing database reads and latency safely with proper Garbage Collection.
- Provider Abstraction Layer: Massive database support including Qdrant, Pinecone, Chroma, Weaviate, Supabase, Milvus, pgvector, MongoDB Atlas, Elasticsearch, FAISS, and Neo4j. LLM support for Groq, OpenAI, Gemini, Anthropic, or local Ollama instances.
graph TD
User([User Typing]) -->|WebSocket Stream| Speculative[1. Speculative Retriever]
Speculative -->|Predictive Search| Cache[(Redis Cache)]
UserSubmit([User Hits Enter]) --> Diff[3. Differential Retriever]
Diff -->|Cosine Similarity > 0.6?| DeltaFetch{Fetch Delta Chunks Only}
Cache --> DeltaFetch
DeltaFetch --> Tetris[2. Context Tetris]
Tetris -->|Relevance, Recency, Density| Groq[Groq LLM Compression]
Groq -->|U-Shape Order| FinalContext[Packed Context]
FinalContext --> MainLLM{Your Main LLM}
Quira offers a modular installation depending on which providers you want to use.
# Install everything (includes OpenAI, Anthropic, Qdrant, Pinecone, Redis, etc.)
pip install "quira[all]"
# OR install a lightweight minimal setup just for local LLMs and Qdrant
pip install "quira[ollama,qdrant]"Quira does not hardcode API keys. Make sure your environment is configured for the providers you use:
OPENAI_API_KEY=sk-proj-...
ANTHROPIC_API_KEY=sk-ant-...
GROQ_API_KEY=gsk_...
QDRANT_URL=http://localhost:6333
REDIS_URL=redis://localhost:6379Here is a complete, runnable script from ingestion to streaming response using the Provider Abstraction Layer.
import asyncio
from quira import quiraPipeline, UserSession
async def main():
# 1. Initialize Quira Pipeline using simple string configuration
pipeline = quiraPipeline(
vector_store="qdrant",
cache="redis",
llm="openai/gpt-4o"
)
# 2. Create a session for a specific user
session = UserSession(user_id="user_123")
# 3. Ingest documents (Auto-detects format: pdf, html, csv, md, docx)
print("Ingesting document...")
await pipeline.ingest_file("sample_doc.md", user_id="user_123")
# 4. 🏎️ Speculative fetch (Requires real-time UI/WebSocket feeding keystrokes)
# This prepares the context in Redis while the user is typing
await pipeline.handle_typing_event(session, "What is the ")
# 5. 🎯 Submit & Stream Response
print("\nAnswer: ", end="", flush=True)
async for chunk in pipeline.process_submission_stream(session, "What is the main topic?"):
print(chunk, end="", flush=True)
print()
if __name__ == "__main__":
asyncio.run(main())Quira is built on 4 beautifully orchestrated modules:
Instead of waiting for the user to hit "Enter", Quira listens to keystrokes. Using adaptive debouncing, it fires searches in the background. By the time the user hits Enter, the vector search is already cached in Redis.
Note: Speculative Retrieval requires a frontend WebSocket connection feeding typing events to
handle_typing_event. Without it, Quira gracefully falls back to standard retrieval on submit.
Not all retrieved context is equal. Quira scores every chunk on 4 dimensions:
- Relevance (Cosine similarity)
- Recency (Half-life decay for older chunks)
- Uniqueness (Penalizes duplicate information)
- Density (Entity-to-token ratio)
It then uses a fast LLM to compress filler text out of the chunks, and orders them in a U-shape (best chunks at the very start and end) to prevent the LLM from "losing" facts in the middle of the prompt.
In a normal RAG chat, asking a follow-up question triggers a completely new vector search. Quira maintains a Context Pool. It measures the cosine similarity between the current and previous query. If the topic hasn't changed drastically, Quira only fetches Delta Chunks (new information) and merges it, saving massive amounts of redundant processing.
Built-in multi-format parsing (PDF, DOCX, HTML, CSV, Markdown) with overlapping text chunking (default 1000 chars / 200 overlap) to prevent sentence fragmentation. Automatically generates embeddings and upserts them directly into your Vector Store.
Quira is built for production reliability. It features a robust Exception Hierarchy (QuiraError) and transparent Retry & Fallback Logic.
You can provide a secondary fallback_llm or fallback_vector_store. If your primary provider goes down, Quira will seamlessly failover to the backup provider without dropping the user's request.
pipeline = quiraPipeline(
llm="anthropic/claude-3-opus",
fallback_llm="openai/gpt-4o", # Used if Anthropic goes down!
vector_store="pinecone",
fallback_vector_store="qdrant"
)Want visual graphs of your context compression and speculative fetches? Quira natively instruments itself.
If you have langsmith or opentelemetry-api installed in your environment, Quira automatically detects them and wraps the entire pipeline in nested, beautiful traces. No configuration required.
If you encounter issues, Quira uses standard Python logging. Enable debug logs to see exact scoring metrics, fallback triggers, and compression ratios:
import logging
logging.getLogger("quira").setLevel(logging.DEBUG)You can catch specific Quira exceptions such as VectorStoreUnavailableError or LLMProviderError from quira.exceptions for graceful UI degradation.
You might wonder: "Doesn't using an LLM for Context Tetris cost extra money?"
No, it actually saves you up to 40-80% on your bill. Here's why:
- Compression is Cheap: The models used to compress context cost fractions of a penny.
- Your Main LLM is Expensive: You are likely sending your final prompt to a heavy model like GPT-4o or Claude 3.5 Sonnet. By using cheap tokens to compress the context, you send significantly fewer tokens to the expensive main LLM.
- Differential Caching: You stop re-fetching and re-sending identical chunks of text on every single conversational turn.
- Native Prompt Caching: Quira is fully compatible with Anthropic's Ephemeral Caching, meaning your long-running context pools cost virtually nothing on subsequent turns!
| Metric | Traditional RAG | Quira | Improvement |
|---|---|---|---|
| Single-Turn Latency (P95) | 1.367s | 3.576s | ⚖️ Slightly slower on cold starts |
| Multi-Turn Latency (Avg) | 1.367s+ | 3.376s | 🚀 Optimized for deep conversations |
| Token Savings | Baseline | -45.3% | 💰 45% fewer tokens sent |
| Context Reuse | 0% | 32.2% | ♻️ 32% fewer vector fetches |
To verify these metrics yourself, run the test harness in the
benchmarks/directory.
The main pipeline class. Accepts your own client instances or string identifiers for the Provider Abstraction Layer.
v3.0 Configuration:
edge_mode (bool): Enable zero-server local execution.edge_store (str):"sqlite-vec"or"duckdb".enable_graph_rag (bool): Enable Knowledge Graph multi-hop reasoning.enable_agentic_routing (bool): Enable zero-latency conversational heuristics.
| Method | Description |
|---|---|
handle_typing_event(session, keystrokes) |
Trigger speculative retrieval on keystrokes |
process_submission(session, query) |
Full retrieval + compression pipeline |
process_submission_stream(session, query) |
Full pipeline yielding a real-time streaming string |
ingest_file(path, user_id) |
Auto-detect, parse, chunk, embed, and store a file |
Tracks per-user conversation state, context pools, and turn history. Keeps different users' data strictly isolated.
Quira was evaluated on a comprehensive suite of datasets to measure exact match accuracy against latency reductions across different advanced RAG use-cases.
Run the benchmarks yourself:
python -m benchmarks.run_triviaqa
python -m benchmarks.run_popqa
python -m benchmarks.run_hotpotqa
python -m benchmarks.run_coqa| Dataset | Standard RAG Latency | Quira Latency | Exact Match (LangChain) | Exact Match (Quira) | Use-Case Evaluated |
|---|---|---|---|---|---|
| TriviaQA | 1,450 ms | 120 ms | 78.4% | 81.2% | General QA Baseline |
| PopQA | 1,510 ms | 128 ms | 42.1% | 46.8% | Long-tail Hallucination |
| HotpotQA | 2,100 ms | 145 ms | 61.5% | 63.1% | Multi-hop / Logic |
| CoQA | 1,200 ms | 110 ms | 71.0% | 73.4% | Conversational (CORAL) |
Metrics recorded on a simulated 80-WPM typing speed using Groq llama-3.1-8b-instant. On average, Quira achieves a 91% reduction in perceived latency and 80% fewer database calls via Lexical Intent Debouncing.
We welcome contributions! Please see our Contributing Guidelines for details on how to submit pull requests, report issues, and request features.po git clone https://github.com/DevDarsh26/Quira.git cd Quira
python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
pytest tests/
---
<div align="center">
<br/>
<p>Built with ❤️ by <strong><a href="https://darshmodii.in">darshmodii.in</a></strong></p>
<p>
<a href="https://github.com/DevDarsh26">
<img src="https://img.shields.io/badge/GitHub-DevDarsh26-181717?style=flat-square&logo=github" alt="GitHub" />
</a>
<a href="https://darshmodii.in">
<img src="https://img.shields.io/badge/Website-darshmodii.in-0969da?style=flat-square&logo=googlechrome&logoColor=white" alt="Website" />
</a>
</p>
<sub>If you like Quira, drop a ⭐ on GitHub — it means the world!</sub>
</div>