Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,17 @@

Dated list of meaningful guide updates. Roughly [Keep a Changelog](https://keepachangelog.com) flavored.

## 2026-05-27 — LightRAG model setup refresh

### Changed
- Updated the LightRAG setup examples in `part3-lightrag-setup.md` and the
combined `README.md` to use current model endpoints: Kimi K2.6 via the
international Moonshot API (`https://api.moonshot.ai/v1`) for quality,
Cerebras `gpt-oss-120b` for speed, Fireworks Qwen3-Embedding-8B for
embeddings, and local Ollama as the free/private option. This replaces the
stale PR #1 examples that referenced `kimi-2.5`, the China-only
`api.moonshot.cn` endpoint, and deprecated Cerebras `qwen-3-32b`.

## 2026-05-27 — Ecosystem Directory Update

### Added
Expand Down
58 changes: 43 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -787,8 +787,8 @@ LightRAG does vector DB + knowledge graph **in parallel** during ingestion. One
### Prerequisites

- Python 3.11+
- An LLM API key (for entity extraction during ingestion — OpenAI, Anthropic, or any OpenAI-compatible provider)
- An embedding API key (Fireworks recommended for high-quality 4096-dim embeddings, or use local Ollama)
- An LLM API key for entity extraction during ingestion — **Kimi K2.6** (quality), **Cerebras GPT OSS 120B** (speed), or any OpenAI-compatible provider
- An embedding API key — **Fireworks + Qwen3-Embedding-8B** for high-quality 4096-dim embeddings, or local **Ollama + nomic-embed-text** for free

### Install LightRAG

Expand All @@ -809,37 +809,65 @@ pip install -e ".[api]"

Create `~/.hermes/lightrag/.env`:

**Option A — Kimi K2.6 + Fireworks (quality default):**

```bash
# LLM for entity extraction (during ingestion)
LLM_BINDING=openai
LLM_MODEL=kimi-k2.6
LLM_BINDING_HOST=https://api.moonshot.ai/v1
LLM_BINDING_API_KEY=<your-moonshot-api-key>

# Embedding model (for vector storage)
EMBEDDING_BINDING=fireworks
EMBEDDING_MODEL=accounts/fireworks/models/qwen3-embedding-8b
EMBEDDING_API_KEY=<your-fireworks-api-key>
```

**Option B — Cerebras GPT OSS 120B + Fireworks (speed default):**

```bash
# LLM for entity extraction (during ingestion)
LLM_BINDING=openai
LLM_MODEL=kimi-2.5 # What we actually use — great quality/cost ratio
LLM_BINDING_API_KEY=<your-api-key>
LLM_MODEL=gpt-oss-120b
LLM_BINDING_HOST=https://api.cerebras.ai/v1
LLM_BINDING_API_KEY=<your-cerebras-api-key>

# Embedding model (for vector storage)
EMBEDDING_BINDING=fireworks
EMBEDDING_MODEL=accounts/fireworks/models/qwen3-embedding-8b
EMBEDDING_API_KEY=<your-fireworks-api-key>
```

**Option C — local Ollama (free, quality varies):**

# Or use local Ollama (free, no API key needed):
# EMBEDDING_BINDING=ollama
# EMBEDDING_MODEL=nomic-embed-text
```bash
# LLM for entity extraction
LLM_BINDING=ollama
LLM_MODEL=qwen3:32b
LLM_BINDING_HOST=http://localhost:11434

# Embedding model
EMBEDDING_BINDING=ollama
EMBEDDING_BINDING_HOST=http://localhost:11434
EMBEDDING_MODEL=nomic-embed-text
```

> **Security tip:** Set restrictive permissions on this file: `chmod 600 ~/.hermes/lightrag/.env`

> **Where to get API keys:** Kimi/Moonshot uses [platform.kimi.ai](https://platform.kimi.ai) and the international base URL `https://api.moonshot.ai/v1`; Cerebras uses [cloud.cerebras.ai](https://cloud.cerebras.ai); Fireworks uses [fireworks.ai](https://fireworks.ai).

### Entity Extraction Model — What to Use

This is the LLM that reads your documents and pulls out entities and relationships during ingestion. Quality here directly determines how good your knowledge graph is.

| Model | Speed | Quality | Cost | Recommendation |
|-------|-------|---------|------|----------------|
| **Kimi 2.5** | Fast | Excellent | Cheap | **What we use.** Great balance of quality, speed, and cost for entity extraction |
| **Cerebras + Qwen 3** | Blazing fast | Very good | Very cheap | **Fastest option in the world.** Cerebras inference at 2000+ tok/s makes bulk ingestion fly |
| Gemini 3.1 Flash | Fast | Good | Cheap | Solid fallback, huge context |
| Claude Sonnet 4 | Medium | Excellent | Mid-range | Overkill for ingestion but works great |
| **Ollama local** | Depends on GPU | Unpredictable | Free | Untested for this use case — might mess up entity extraction quality. Use at your own risk |

> **Our recommendation:** Kimi 2.5 for quality, Cerebras + Qwen 3 if you're ingesting a lot of documents and speed matters. Both are cheap and reliable. We haven't tested local Ollama for entity extraction — it's free but the extraction quality is unverified and you might get a messy graph.
| **Kimi K2.6** | Fast | Excellent | Cheap | Best quality/cost default for entity extraction via Moonshot's OpenAI-compatible API |
| **Cerebras GPT OSS 120B** | Blazing fast | Very good | Very cheap | Fastest current Cerebras production default; use when bulk ingestion speed matters most |
| Gemini 3.1 Flash | Fast | Good | Cheap | Solid fallback with huge context |
| Claude Sonnet 5 | Medium | Excellent | Mid/high | Overkill for ingestion but useful for very messy documents |
| **Ollama local** | Depends on GPU | Unpredictable | Free | Viable for private/local ingestion; validate graph quality before trusting it |

> **Embedding quality matters.** If you have a GPU with 8GB+ VRAM, run `nomic-embed-text` locally via Ollama for free. If you want the best quality, use Fireworks' Qwen3-Embedding-8B (4096 dimensions) — the search accuracy difference is dramatic.

Expand Down Expand Up @@ -1148,7 +1176,7 @@ cd ~/.hermes/lightrag/LightRAG && lightrag-server --port 9623
### Slow ingestion

Entity extraction is LLM-bound. Speed it up:
- Use a faster model for ingestion (Cerebras + Qwen 3 is the fastest option, or Kimi 2.5)
- Use a faster model for ingestion (Cerebras GPT OSS 120B for speed, Kimi K2.6 for quality, Gemini 3.1 Flash as a cheap fallback)
- Process documents in parallel batches
- Use a local model if you have GPU capacity

Expand Down
58 changes: 49 additions & 9 deletions part3-lightrag-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ LightRAG does vector DB + knowledge graph **in parallel** during ingestion. One
### Prerequisites

- Python 3.11+
- An LLM API key (for entity extraction during ingestion — OpenAI, Anthropic, or any OpenAI-compatible provider)
- An embedding API key (Fireworks recommended for high-quality 4096-dim embeddings, or use local Ollama)
- An LLM API key for entity extraction during ingestion — **Kimi K2.6** (quality), **Cerebras GPT OSS 120B** (speed), or any OpenAI-compatible provider
- An embedding API key — **Fireworks + Qwen3-Embedding-8B** for high-quality 4096-dim embeddings, or local **Ollama + nomic-embed-text** for free

### Install LightRAG

Expand All @@ -70,25 +70,65 @@ pip install -e ".[api]"

Create `~/.hermes/lightrag/.env`:

**Option A — Kimi K2.6 + Fireworks (quality default):**

```bash
# LLM for entity extraction (during ingestion)
LLM_BINDING=openai
LLM_MODEL=kimi-k2.6
LLM_BINDING_HOST=https://api.moonshot.ai/v1
LLM_BINDING_API_KEY=<your-moonshot-api-key>

# Embedding model (for vector storage)
EMBEDDING_BINDING=fireworks
EMBEDDING_MODEL=accounts/fireworks/models/qwen3-embedding-8b
EMBEDDING_API_KEY=<your-fireworks-api-key>
```

**Option B — Cerebras GPT OSS 120B + Fireworks (speed default):**

```bash
# LLM for entity extraction (during ingestion)
LLM_BINDING=openai
LLM_MODEL=google/gemini-3.1-flash
LLM_BINDING_API_KEY=<your-gemini-api-key-or-oauth-token>
LLM_MODEL=gpt-oss-120b
LLM_BINDING_HOST=https://api.cerebras.ai/v1
LLM_BINDING_API_KEY=<your-cerebras-api-key>

# Embedding model (for vector storage)
EMBEDDING_BINDING=fireworks
EMBEDDING_MODEL=accounts/fireworks/models/qwen3-embedding-8b
EMBEDDING_API_KEY=<your-fireworks-api-key>
```

**Option C — local Ollama (free, quality varies):**

# Or use local Ollama (free, no API key needed):
# EMBEDDING_BINDING=ollama
# EMBEDDING_MODEL=nomic-embed-text
```bash
# LLM for entity extraction
LLM_BINDING=ollama
LLM_MODEL=qwen3:32b
LLM_BINDING_HOST=http://localhost:11434

# Embedding model
EMBEDDING_BINDING=ollama
EMBEDDING_BINDING_HOST=http://localhost:11434
EMBEDDING_MODEL=nomic-embed-text
```

> **Security tip:** Set restrictive permissions on this file: `chmod 600 ~/.hermes/lightrag/.env`

> **Tip:** Use a cheap GPT-5.5-mini/Gemini Flash-class model for entity extraction. It doesn't need to be your smartest model — it just needs to reliably identify entities and relationships. Cheaper models save money on ingestion.
> **Where to get API keys:** Kimi/Moonshot uses [platform.kimi.ai](https://platform.kimi.ai) and the international base URL `https://api.moonshot.ai/v1`; Cerebras uses [cloud.cerebras.ai](https://cloud.cerebras.ai); Fireworks uses [fireworks.ai](https://fireworks.ai).

### Entity Extraction Model — What to Use

This is the LLM that reads your documents and pulls out entities and relationships during ingestion. Quality here directly determines how good your knowledge graph is.

| Model | Speed | Quality | Cost | Recommendation |
|-------|-------|---------|------|----------------|
| **Kimi K2.6** | Fast | Excellent | Cheap | Best quality/cost default for entity extraction via Moonshot's OpenAI-compatible API |
| **Cerebras GPT OSS 120B** | Blazing fast | Very good | Very cheap | Fastest current Cerebras production default; use when bulk ingestion speed matters most |
| Gemini 3.1 Flash | Fast | Good | Cheap | Solid fallback with huge context |
| Claude Sonnet 5 | Medium | Excellent | Mid/high | Overkill for ingestion but useful for very messy documents |
| **Ollama local** | Depends on GPU | Unpredictable | Free | Viable for private/local ingestion; validate graph quality before trusting it |

> **Embedding quality matters.** If you have a GPU with 8GB+ VRAM, run `nomic-embed-text` locally via Ollama for free. If you want the best quality, use Fireworks' Qwen3-Embedding-8B (4096 dimensions) — the search accuracy difference is dramatic.

Expand Down Expand Up @@ -391,7 +431,7 @@ cd ~/.hermes/lightrag/LightRAG && lightrag-server --port 9623
### Slow ingestion

Entity extraction is LLM-bound. Speed it up:
- Use a faster model for ingestion (Gemini 3.1 Flash, Kimi K2.6, or Claude Haiku)
- Use a faster model for ingestion (Cerebras GPT OSS 120B for speed, Kimi K2.6 for quality, Gemini 3.1 Flash as a cheap fallback)
- Process documents in parallel batches
- Use a local model if you have GPU capacity

Expand Down
Loading