Skip to content

Monthly Pathfinder gap-analysis automation#96

Merged
jpr5 merged 3 commits into
mainfrom
blitz/pathfinder-gaps/integration-automation
Jun 8, 2026
Merged

Monthly Pathfinder gap-analysis automation#96
jpr5 merged 3 commits into
mainfrom
blitz/pathfinder-gaps/integration-automation

Conversation

@jpr5

@jpr5 jpr5 commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a monthly GitHub Actions workflow that pulls Pathfinder analytics, clusters unanswered / empty-result queries into actionable documentation gaps (deterministic clustering + optional LLM classification), and reports high-severity gaps to Slack with a Notion report.
  • Durable month-over-month state via workflow artifacts (not cache, which evicts at 7 days idle) so the high-severity dedup survives the monthly cadence; persist-before-alert ordering prevents repeat-alert storms.

Required configuration (before enabling)

  • New repo secrets: PATHFINDER_ANALYTICS_TOKEN (analytics API bearer), ANTHROPIC_API_KEY (optional — LLM classification; falls back to deterministic clustering if unset), NOTION_TOKEN (report publishing).
  • Reuses the existing SLACK_WEBHOOK_OSS_ALERTS org secret.
  • Workflow grants actions: read (cross-run artifact download for the prior-state baseline).

Notes

  • scripts/gap-analysis/ is excluded from the npm tarball (.npmignore); @anthropic-ai/sdk is a devDependency (only this CI script uses it, dynamically imported).
  • Typechecked via tsconfig.scripts.json; 93 unit tests cover clustering, LLM-output recovery, dedup/cap, the dry-run contract, and prompt-injection escaping of untrusted query text.
  • Deferred (non-blocking) robustness follow-ups tracked for a follow-up PR: cold-start state-chain log nuance, additional LLM-output recovery edge-cases, Notion >100-block batching, CJK query normalization.

Test plan

  • CI green (typecheck-scripts, prettier, vitest).
  • Provision the 3 secrets, then a manual workflow_dispatch dry-run to verify analytics fetch + report generation without alerting.

@jpr5 jpr5 force-pushed the blitz/pathfinder-gaps/integration-automation branch 2 times, most recently from 4b495e6 to 044fcd9 Compare June 8, 2026 23:23
@jpr5 jpr5 force-pushed the blitz/pathfinder-gaps/integration-automation branch from 044fcd9 to 43318ba Compare June 8, 2026 23:26
@jpr5 jpr5 merged commit 7afc681 into main Jun 8, 2026
7 checks passed
@jpr5 jpr5 deleted the blitz/pathfinder-gaps/integration-automation branch June 8, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant