Skip to content

fix: surface doc parse failures instead of empty success - #31

Merged
fpt merged 1 commit into
mainfrom
fix-doc-parse-robustness
Jul 2, 2026
Merged

fpt merged 1 commit into
mainfrom
fix-doc-parse-robustness

Conversation

@fpt

@fpt fpt commented Jun 29, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes the two robustness gaps found while verifying the dq-based parsers against live pkg.go.dev. The same pattern existed in all three doc readers (godoc, rustdoc, pydoc).

The problems

In Read*Paged and SearchWithin*:

  1. Silent empty on parse failure — the parser's matched flag was discarded (_, document = parse...). When all parsers missed (e.g. the site renamed a CSS class), the reader returned "" with a nil error, indistinguishable from "this package has no docs."
  2. Caching failures — docCache.Set ran even for an empty document, poisoning the cache for the 30-minute TTL on a transient miss or markup drift.

These matter because the parsers are CSS-class-coupled and the golden tests run against saved HTML, so they can't catch live drift — the failure mode was silent empty output.

The fix

Read*Paged / SearchWithin* now check the parse result and return ErrNotFound when nothing matched (or the parsed document is blank), and only cache non-empty parses. Applied consistently across godoc.go, rustdoc.go, pydoc.go (6 call sites).

matched, parsed := parseDocsRsDocument(doc)
if !matched || strings.TrimSpace(parsed) == "" {
    return "", 0, false, ErrNotFound
}
document = parsed
docCache.Set(cacheKey, document, cache.DefaultExpiration)

(godoc keeps its parseDocument → parseReadme fallback; the guard runs after both.)

Verification

  • go build, go vet, full go test ✅
  • make lint → 0 issues
  • Golden tests unaffected (they exercise parse* directly, not the readers)

🤖 Generated with Claude Code

The godoc/rustdoc/pydoc readers discarded the parser's `matched` flag and
cached + returned the (possibly empty) document with a nil error. If a site
changed its markup so nothing matched, callers got empty output that looked
like "no docs" rather than a failure, and the empty result was cached for the
30-minute TTL — poisoning the cache until expiry.

Now Read*Paged and SearchWithin* check the parse result and return ErrNotFound
when nothing matched (or the document is blank), and only cache non-empty
parses. Applied consistently across godoc, rustdoc, and pydoc.

This makes pkg.go.dev/docs.rs/docs.python.org markup drift surface as an error
instead of silently empty output (golden tests run against saved HTML and
cannot catch live drift).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@fpt
fpt merged commit 80ed155 into main Jul 2, 2026
1 check passed
@fpt
fpt deleted the fix-doc-parse-robustness branch July 2, 2026 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant