Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .agents/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,11 @@ notebrain-cli/
│ ├── client.go
│ └── *_test.go
└── cmd/
├── root.go
├── cli.go
├── ingest.go
├── search.go
├── backlinks.go
├── refs.go
├── connections.go
├── hidden.go
├── tags.go
Expand All @@ -75,7 +76,7 @@ notebrain-cli/
- Name test files `*_test.go` alongside the source file.
- **Go Vendoring:** This repository uses Go vendoring (`vendor/`). Whenever dependencies in `go.mod` or `go.sum` are added, removed, or updated, you MUST run `go mod vendor` before running tests or builds.
- **Strict Non-Regression Guardrails:** When refactoring or removing features, always add explicit assertion tests across `internal/configfile/` and `internal/store/` to verify that existing core functions, default settings, TOML key resolution, and database initialization do not regress or depend on removed parameters.
- **CLI Testing & Flag Standards:** When executing CLI commands or writing automated tests/scripts for NoteBrain, strictly use the exact flag names `--vault-path` and `--chroma-path` (never `--vault` or `--db`). For graph and note commands (`backlinks`, `connections`, `hidden`, `tags`, `get`), pass exactly one positional argument: the note slug (`<note>`). For `boosted` search, always provide the required `--seed=<slug>` flag. When testing `hidden` connection discovery where already-linked notes should be included, pass `--include-linked`. Note that `backlinks` and `connections` canonicalize link targets by stripping `#heading` anchors and matching exact vault subfolders. Use `--show-tags` to show tags in CLI output. Debug logging is enabled via `--debug`. When testing `reset` in automated scripts, pipe confirmation via stdin (`echo yes | ./notebrain reset`). To avoid contextual empty-result hints in automated scripts, always request machine formats (`--format=json`, `tsv`, or `--jsonpath`). When testing LLM-based PDF ingestion, use `--llm-model` and provide the required API key via environment variables (`DEEPSEEK_API_KEY`, or `OPENROUTER_API_KEY`).
- **CLI Testing & Flag Standards:** When executing CLI commands or writing automated tests/scripts for NoteBrain, strictly use the exact flag names `--vault-path` and `--chroma-path` (never `--vault` or `--db`). For graph and note commands (`backlinks`, `connections`, `hidden`, `tags`, `get`, `refs`), pass exactly one positional argument: the note slug (`<note>`). For `refs`, use `--only-images`/`--only-pdf`/`--only-other`/`--only-external-links` to limit by kind (no flags = all kinds; old `--images`/`--pdf`/`--other`/`--external-links` still parse but are deprecated) and `--include-missing` to surface broken attachment links. For `boosted` search, always provide the required `--seed=<slug>` flag. When testing `hidden` connection discovery where already-linked notes should be included, pass `--include-linked`. Use `--candidate-chunks` (not the removed `--top-k`) to control deep hidden-analysis depth. Note that `backlinks` and `connections` canonicalize link targets by stripping `#heading` anchors and matching exact vault subfolders. Use `--show-tags` to show tags in CLI output. Debug logging is enabled via `--debug`. When testing `reset` in automated scripts, pipe confirmation via stdin (`echo yes | ./notebrain reset`). To avoid contextual empty-result hints in automated scripts, always request machine formats (`--format=json`, `tsv`, or `--jsonpath`). When testing LLM-based PDF ingestion, use `--with-pdf` (deprecated alias: `--enable-pdf`) and `--llm-model`, and provide the required API key via environment variables (`DEEPSEEK_API_KEY`, or `OPENROUTER_API_KEY`). Use `--exclude-notes` (not the deprecated `--exclude-note`) to exclude notes from search.

## Coding Conventions

Expand Down
275 changes: 275 additions & 0 deletions .agents/plans/Plan.md

Large diffs are not rendered by default.

13 changes: 9 additions & 4 deletions .agents/skills/notebrain/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: notebrain-assistant
description: Search and explore an Obsidian vault through the NoteBrain CLI (semantic search, tags, backlinks, connections, hidden links, boosted retrieval). Use it whenever the user mentions their notes, knowledge base, Obsidian vault, semantic search, finding connections or unlinked notes, or asks exploratory questions like "what do I know about X", "find notes related to Y", "what connects to Z", or "summarize my notes on W" — even when they never say NoteBrain, vector search, or ChromaDB.
description: Search and explore an Obsidian vault through the NoteBrain CLI (semantic search, tags, backlinks, connections, hidden links, boosted retrieval). Use it whenever the user mentions their notes, knowledge base, Obsidian vault, semantic search, finding connections or unlinked notes, or asks exploratory questions like "what do I know about X", "find notes related to Y", "what connects to Z", "summarize my notes on W", "what does this note reference or embed", or "are any links broken" — even when they never say NoteBrain, vector search, or ChromaDB.
license: MIT
compatibility: Requires the `notebrain` binary on PATH.
allowed-tools: Bash(notebrain:*), Bash(./notebrain:*)
Expand All @@ -11,7 +11,7 @@ allowed-tools: Bash(notebrain:*), Bash(./notebrain:*)
NoteBrain indexes an Obsidian vault into local ChromaDB and answers read-only questions about it: semantic search, tag queries, graph structure, and note retrieval. It never mutates the vault — for writes, use standard file tools or obsidian-cli and keep NoteBrain for the discovery step.

References, read on demand:
- [references/example.md](references/example.md) — 16 worked scenarios, exact commands, verified pitfalls.
- [references/example.md](references/example.md) — 18 worked scenarios, exact commands, verified pitfalls.
- [references/flags.md](references/flags.md) — every flag, default, and config override.
- [references/schema.md](references/schema.md) — JSON/TSV output shape, `--jsonpath` use.

Expand Down Expand Up @@ -46,7 +46,7 @@ notebrain search "<topic>" --format json --include-text --context-window 1 --lim
**Lean shapes:**
- Top candidates/slugs only: drop `--context-window`, use `--jsonpath="$.results[*].note_slug"`.
- Note-level (not chunk-level) list: `--group-by-note` to collapse to the best chunk per note; `dedupe` via `--jsonpath="$.results[*].note_slug" | sort -u`.
- Weak matches above the `--min-score 0.5` floor, or `--tag`, `--section`, `--has-tasks`, `--has-code`, `--exclude-note`.
- Weak matches above the `--min-score 0.5` floor, or `--tag`, `--section`, `--has-tasks`, `--has-code`, `--exclude-notes`.
- Multi-topic at once — boost by adding positional queries: `search "redis pubsub" "kafka brokers"`.
- A show-tags + `--jsonpath="$.results[0].tags"` reveals real note tags in one call.

Expand All @@ -60,6 +60,7 @@ Only when the task needs **graph structure** or **related-but-unlinked** notes,
| ------ | ------- | ------------- |
| Reading / metadata only | `get` | `--meta` (header, no body) or `--head N` (first N chunks) — full `get` only on direct demand |
| What links to a note | `backlinks` | exactly the slug |
| What a note references / embeds | `refs` | kind filters; `--include-missing` for broken links |
| What's graph-neighbour | `connections` | `--hops 1–2` (exponential blow-up beyond) |
| Meaning-related but NOT linked | `hidden` | `--deep` for section-level matches |
| Related **including** linked | `hidden` | `--include-linked` |
Expand All @@ -73,9 +74,13 @@ Only when the task needs **graph structure** or **related-but-unlinked** notes,

Semantic search returns zero results or nothing above `--min-score`, so `search` automatically falls back to a token scan over titles/paths/tags/text. Rows arrive `"lexical": true`, `score: 0`; the header prints `Lexical Search (no semantic matches)`. So a short word like `Lecture` can still hit. When even that returns nothing, lengthen the query into a descriptive phrase or switch to a `tags` query if the word is a heading/tag keyword. No fallback for `boosted` or `hidden`.

### Refs: what a note references

`refs` lists the note's attachments (images, PDFs, other) and external http(s) links, in first-occurrence order, read fresh from the note file on disk — no index staleness. It does NOT list links to other notes (that is `backlinks`/`connections`). Broken references are hidden by default; `--include-missing` surfaces them as `"missing": true`. External links are never missing and never touched over the network. No kind flags = every kind; the filters are pure kind selectors, no scores.

## Slug discipline

Slugs are the handle; titles are not. For graph and `get` commands, pass the exact `note_slug` returned by a prior `search`/`tags` — never a bare title, titles are ambiguous. Since the deterministic-resolution fix, a missing note is an **error** (`note not found: "<input>" ...`), not a silently guessed phantom slug. A "no indexed chunks" / "note not found" failure is normally a breadth-resolution problem, not a missing note. Slugs also go stale mid-conversation on schedule (cron re-ingest): if a slug that worked earlier now 404s, re-resolve via `search` before retrying.
Slugs are the handle; titles are not. For graph, `get`, and `refs` commands, pass the exact `note_slug` returned by a prior `search`/`tags` — never a bare title, titles are ambiguous. Since the deterministic-resolution fix, a missing note is an **error** (`note not found: "<input>" ...`), not a silently guessed phantom slug. A "no indexed chunks" / "note not found" failure is normally a breadth-resolution problem, not a missing note. Slugs also go stale mid-conversation on schedule (cron re-ingest): if a slug that worked earlier now 404s, re-resolve via `search` before retrying.

## Tag discovery

Expand Down
8 changes: 7 additions & 1 deletion .agents/skills/notebrain/references/example.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Quick reference: the major scenarios with the proven command sequence. Pair with
| 5 | List all notes tagged X | `notebrain tags "X" --children --limit 50 --format tsv` |
| 6 | Semantic search | `search "<q>" --format=json --include-text --limit 3`; escalate: `--top-k 2 --context-window 1`; stop when top score ≥ 0.75 |
| 7 | Multi-topic comparison | `notebrain search "redis pubsub" "kafka brokers" --limit 5 --top-k 2 --format json` |
| 8 | Filtered search | add `--tag "kubernetes"`, `--section "Architecture > Components"`, `--has-tasks`, `--has-code`, `--exclude-note "<slug>"`, `--min-score 0.3` |
| 8 | Filtered search | add `--tag "kubernetes"`, `--section "Architecture > Components"`, `--has-tasks`, `--has-code`, `--exclude-notes "<slug>"`, `--min-score 0.3` |
| 9 | Zero-result handling | short common words now fall back to a lexical token scan (`"lexical": true`, `score: 0`); if still nothing → longer descriptive phrase or `tags` query; never grep the vault |
| 10 | Backlinks | `notebrain backlinks "<slug>" --format json --limit 10` |
| 11 | Connections | `notebrain connections "<slug>" --hops 2 --format tsv` |
Expand All @@ -24,6 +24,8 @@ Quick reference: the major scenarios with the proven command sequence. Pair with
| 14 | Metadata-only extraction | `--jsonpath`, `--format tsv`, `--show-file-path=false` (cuts ~40–50% of tokens) |
| 15 | Context vs full `get` | context: `--context-window 1 --include-text`; full note only on explicit demand: `get "<slug>"` |
| 16 | Stale-index recovery | a slug that 404s mid-conversation → re-resolve: `search "<title>" --limit 3 --jsonpath="$.results[*].note_slug"` |
| 17 | Reference inventory | `notebrain refs "<slug>" --format json` (all kinds); kind filters: `--only-images` / `--only-pdf` / `--only-other` / `--only-external-links` |
| 18 | Broken-link audit | `notebrain refs "<slug>" --include-missing --format tsv` → rows with `missing` = `true` are broken; omit `--include-missing` to see only existing files |

## Semantics (verified)

Expand All @@ -32,6 +34,7 @@ Quick reference: the major scenarios with the proven command sequence. Pair with
- **`--section` is exact-match**: it compares against the stored `heading_path` string verbatim. Partial or parent paths return 0 results silently — copy the full `heading_path` from a search result.
- **`--jsonpath`**: dotted paths, `[*]`, and `[0]` only — no jq-style pipe expressions, filters, or object construction. Multi-field extraction → `--format tsv` or two `--jsonpath` calls.
- **Config overrides defaults**: `~/.notebrain/config/config.toml` can enable `include-text`/`context-window` (and set `min-score`/`limit`/`top-k`) — output then carries `text`/`context` even without flags. Pass `--include-text=false`/`--context-window=0` explicitly for lean output.
- **`refs` reads the file, not the index**: results come from a fresh parse of the note on disk — never stale, but only reflect what the current file contains. Order is first occurrence in the note; rows dedupe by resolved path (or exact URL).

## Pitfalls (verified)

Expand All @@ -41,6 +44,7 @@ Quick reference: the major scenarios with the proven command sequence. Pair with
- **Weak matches**: add `--min-score 0.3` (or `0.5` for precision); results below ~0.30 are noise. Note: config may already set a `min-score` floor, so low-score results can be absent by design.
- **`get`**: `--meta` (header only: title, path, tags, chunk count) or `--head N` (first N chunks, `Chunks` still shows the total) cover most needs for cheap reads — reach for the full note only on demand. For metadata see also scenarios 3/14.
- **Stale index**: scheduled re-ingest can invalidate cached slugs mid-conversation; re-verify via `search` before `--deep`/`backlinks` after any 404.
- **`refs` scope**: attachments (image/pdf/other) and external http(s) links only — links to other notes never appear (use `backlinks`/`connections`). So `refs --include-missing` catches broken attachments, not broken `[[wikilinks]]` to notes. `refs` is markdown-notes only; PDF extractions error out.

## Phrase → Scenario Map

Expand All @@ -56,4 +60,6 @@ Quick reference: the major scenarios with the proven command sequence. Pair with
| "unlinked / hidden concepts near Y" | 12 |
| "concepts about X around note Y" | 13 |
| "everything on topic X" | 5 or 6 |
| "what images / attachments does this note use" | 17 |
| "are any links / attachments broken" | 18 |
| "why did that search return nothing" | 9 |
Loading