Repository navigation
Fix audit engine hang and real-site crawling; expand rule set to 35 - #3
Merged
Merged
Conversation
…ules Fixes - Crawler never closed its queue, so `rankcore audit` hung until interrupted. - Manual Accept-Encoding disabled Go's gzip handling; compressed pages parsed as garbage. - CLI flags after the target were ignored; --json mixed progress into stdout. - Absolute same-origin links were never crawled; protocol-relative links were treated as internal. - Link graph keyed raw hrefs against absolute URLs, so every page was an orphan. - :443 port stripping left a trailing colon. - Broken-link and canonical-target rules were stubs; now real site-wide checks. - verify marked unreached pages as fixed and broke when the origin port changed. - Redirect chains are recorded hop by hop; SSRF guard dials the validated IP. - Agent detection checked the wrong directory; install.ps1 closed the shell on failure. Additions - Sitemap discovery (robots.txt, indexes, gzip) seeds the crawl. - New rules: sitemap entries, orphans, redirecting links, multiple canonicals, duplicate titles/descriptions, hreflang return links, structured-data required properties, search-crawler blocks (35 rules total). - Exit codes and --fail-on gate, local directory targets, new crawl flags. - Deterministic sorted findings with catalog category/automation, per-page report. - x/net v0.61.0, CI matrix with race detector, versioned release builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the deterministic engine actually work end to end, then fills in the parts of the master plan that were stubbed.
Fixes
rankcore audithung until Ctrl+C. The old integration test only passed via its 20s timeout; it now completes in ~1s.Accept-Encodingmanually disabled Go's transparent gzip.rankcore audit <url> --out x --json, the form the skill uses), and--jsonmixed progress text into stdout.//cdn…links were treated as internal; the link graph compared raw hrefs to absolute URLs, so every page was an orphan.:443stripping leftexample.com:.verifymarked unreached pages "fixed" and lost track of findings when the dev-server port changed.install.ps1closed the PowerShell session on failure.Additions
--fail-on <severity>;audit ./distserves a local build;--max-depth,--ignore-robots,--timeout,--max-render.x/netv0.61.0; CI on Linux/Windows/macOS using thego.modversion, race detector on Linux; release builds stamp the version.Behaviour changes to note
Testing
gofmt,go vet,go test ./...pass locally (Windows).🤖 Generated with Claude Code