feat: add staff search intelligence dashboard and admin CLI

[Patrick-Erichsen]

Serve staff search insights through dashboard, API, and CLI

Problem

Give maintainers a consistent, privacy-preserving view of search demand and official gaps.

Change

Add bounded daily aggregation and retention with one staff-authorized report shared by the dashboard, HTTP API, and admin CLI. Current package metadata and advisory intent classifications remain separate from historical official-gap counts.

Validation

Real API/CLI parity verified for 32 latest-week searches and source totals 20/12. Anonymous and ordinary-user requests returned 401/403. Inspected 24 desktop/mobile states covering loading, empty, populated, filtered, partial, historical, error, and recovery views.

Review status

Draft for human review. This is a dedicated search-intelligence stack, independent of the plugin-page/category work. No merge or production deployment is authorized.

The full integrated stack at ca843a963a passed bun run ci:unit (6,582 passed, 3 skipped), bun run ci:types-build, package/HTTP/browser checks, and the non-audit static constituents. bun run ci:static remains red on seven dependency advisories in the unchanged baseline lockfile; that repair is deliberately outside this stack. These results are from the saved reviewed head, not a claim that CI on this PR has passed.

Dashboard proof used http://127.0.0.1:3250/management/search-insights, a real local Convex backend, the local staff persona, and explicitly synthetic 32-search data. The UI does not imply these are production searches. Source filters, official gaps, advisory company intent, and Featured leads share the API/CLI owner.

Real browser proof

Related draft PRs

Review ClawHub bottom-to-top: capture → web attribution → dashboard/API/CLI → weekly producer.

Separate-repository companions: OpenClaw Control UI and Hermit receiver. The producer requires the Hermit receiver; the Control UI source marker requires the capture boundary. No plugin-page/category stack branches are included.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
clawhub	Ready	Preview	Sep 8, 2026 11:14pm UTC

[clawsweeper]

Codex review: blocked before merge. Reviewed September 8, 2026, 7:18 PM ET / 23:18 UTC.

ClawSweeper review

What this changes

Adds staff-only search-demand reports shared by a management dashboard, HTTP endpoint, and admin CLI, backed by daily aggregates and advisory classifications.

Merge readiness

⛔ Blocked before merge - 3 items remain

This remains distinct, useful work absent from current main. One dashboard correctness defect needs repair; the contributor’s explicit human-review hold also remains in effect.

Priority: P2
Reviewed head: 78165570347bd17feb55211912dcf058031cb46b

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	Useful shared implementation and concrete browser evidence, with one report-state defect and an unresolved deployment-continuity check.
Proof confidence	🐚 platinum hermit (4/6)	Not applicable: The collaborator-authored PR is exempt from the ordinary contributor proof gate. Inspected real-browser images show the local staff report with 32 searches and 20/12 source totals; they do not independently establish the stated HTTP denials or persisted-state upgrade behavior.
Patch quality	🦐 gold shrimp (3/6)	1 actionable review finding remain.

Verification

Check	Result	Evidence
Real behavior	Not applicable	Not applicable: The collaborator-authored PR is exempt from the ordinary contributor proof gate. Inspected real-browser images show the local staff report with 32 searches and 20/12 source totals; they do not independently establish the stated HTTP denials or persisted-state upgrade behavior.
Evidence reviewed	10 items	Pinned scope and live state: Read the complete introduced changes across 23 files. GitHub REST confirms the reviewed head remains open and draft against the web-capture branch; the captured body explicitly reserves human review and authorizes neither merge nor production deployment. Still necessary: Current-main searches for search intelligence, search insights, search demand, and official gaps found no equivalent report. Tree checks on main and v0.23.3 found no search-insights implementation. The related capture, web-attribution, and weekly-producer PRs remain separate open layers. Failed filter requests retain old results: The effect clears loading/error state but retains the previous report. Its rejection handler sets only an error, and completion makes the old report visible again under the newly selected controls.
Findings	1 actionable finding	[P2] Clear or explicitly bind the report when filter loading fails
Security	None	None.

How this fits together

ClawHub collects bounded manual plugin-search observations. This layer aggregates them into staff reports and enriches them with current public package metadata for curation.

flowchart TD
  A[Manual search observations] --> B[Daily aggregation and retention]
  B --> C[Shared search report]
  D[Weekly advisory classifications] --> C
  E[Public package metadata] --> C
  F[Staff authorization] --> C
  C --> G[Dashboard and admin CLI]

Loading

Before merge

• Clear or explicitly bind the report when filter loading fails (P2) - After a successful load, change Source or View and let the next request fail. The rejection handler retains report, and finally clears loading, so the old rows and totals render beneath the newly selected filters. For example, selecting company opportunities can show the previous unfiltered rows as that view. Clear the report when its request parameters change, or retain its successful parameters and visibly identify stale results; cover failure followed by successful retry.
• Resolve merge risk (P1) - First-deploy and redeploy behavior with existing observations is unproven because the supplied fixture resets aggregate and cursor state.
• Complete next step (P2) - Repair stale results after failed filter changes, provide deployment-continuity evidence, and obtain explicit clearance of the draft human-review hold after the prerequisite stack is ready.

Findings

• [P2] Clear or explicitly bind the report when filter loading fails — src/routes/-management/SearchInsightsPage.tsx:36-40

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Patch scope	+2185/-1 across 23 files	The change spans storage, scheduled jobs, HTTP, CLI, and staff UI.
Production and test delta	Production +1548/-1; tests +531; docs +98; generated +8	Production growth implements the stated shared reporting layer and includes 181 lines of local fixtures.

Merge-risk options

Maintainer options:

1. Verify deployment continuity (recommended)
   Demonstrate first aggregation over existing captured observations and redeployment with a persisted cursor, confirming no duplicated or skipped counts and no unrelated data changes.

Technical review

Best possible solution:

Keep one shared staff report, bind displayed results to the successful filter request, and preserve aggregate counts and ingestion progress across deployment.

Do we have a high-confidence way to reproduce the issue?

Yes, by source inspection: load a report, change Source or View, and reject the next request; the previous report becomes visible under the new controls. This path was not executed during the read-only review.

Is this the best way to solve the issue?

The shared backend is the right layer and avoids separate dashboard/CLI calculations. The implementation needs request-bound rendering and deployment-continuity validation before it is ready.

Full review comments:

• [P2] Clear or explicitly bind the report when filter loading fails — src/routes/-management/SearchInsightsPage.tsx:36-40
  After a successful load, change Source or View and let the next request fail. The rejection handler retains report, and finally clears loading, so the old rows and totals render beneath the newly selected filters. For example, selecting company opportunities can show the previous unfiltered rows as that view. Clear the report when its request parameters change, or retain its successful parameters and visibly identify stale results; cover failure followed by successful retry.
  Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.93

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 651a7641a204.

Labels

Label changes:

• add P2: This is a bounded staff reporting feature with a recoverable filter-error defect.
• add merge-risk: 🚨 compatibility: New durable aggregates and an automatically scheduled ingestion cursor need deployment-continuity evidence beyond reset-and-seed fixtures.
• add proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. The collaborator-authored PR is exempt from the ordinary contributor proof gate. Inspected real-browser images show the local staff report with 32 searches and 20/12 source totals; they do not independently establish the stated HTTP denials or persisted-state upgrade behavior.
• add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
• add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: The collaborator-authored PR is exempt from the ordinary contributor proof gate. Inspected real-browser images show the local staff report with 32 searches and 20/12 source totals; they do not independently establish the stated HTTP denials or persisted-state upgrade behavior.

Label justifications:

• P2: This is a bounded staff reporting feature with a recoverable filter-error defect.
• merge-risk: 🚨 compatibility: New durable aggregates and an automatically scheduled ingestion cursor need deployment-continuity evidence beyond reset-and-seed fixtures.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
• status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: The collaborator-authored PR is exempt from the ordinary contributor proof gate. Inspected real-browser images show the local staff report with 32 searches and 20/12 source totals; they do not independently establish the stated HTTP denials or persisted-state upgrade behavior.
• proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. The collaborator-authored PR is exempt from the ordinary contributor proof gate. Inspected real-browser images show the local staff report with 32 searches and 20/12 source totals; they do not independently establish the stated HTTP denials or persisted-state upgrade behavior.

Evidence

What I checked:

• Pinned scope and live state: Read the complete introduced changes across 23 files. GitHub REST confirms the reviewed head remains open and draft against the web-capture branch; the captured body explicitly reserves human review and authorizes neither merge nor production deployment. (78165570347b)
• Still necessary: Current-main searches for search intelligence, search insights, search demand, and official gaps found no equivalent report. Tree checks on main and v0.23.3 found no search-insights implementation. The related capture, web-attribution, and weekly-producer PRs remain separate open layers. (651a7641a204)
• Failed filter requests retain old results: The effect clears loading/error state but retains the previous report. Its rejection handler sets only an error, and completion makes the old report visible again under the newly selected controls. (src/routes/-management/SearchInsightsPage.tsx:36, 78165570347b)
• Shared report and authorization boundary: The dashboard action requires an active admin/moderator; HTTP uses existing API-token and moderator checks. Catalog enrichment omits viewer privileges, preserves public visibility, and applies release-security gates to Featured candidates. No credentials, permissions, dependencies, or installation hooks are changed. (convex/searchInsights.ts:29, 78165570347b)
• Persistent-state rollout coverage: Four new persistent tables and hourly aggregation are introduced. The local fixture deletes observations, aggregates, cursor state, and classifications before seeding, so its successful run does not demonstrate upgrading an existing capture deployment or resuming a persisted cursor after redeployment. (convex/searchInsightsFixtures.ts:65, 78165570347b)
• Inspected browser proof: Inspected both prepared images linked from the captured PR body. The after image visibly shows the staff dashboard with 32 searches, source totals 20/12, four queries, classifications, and a Featured candidate. The body identifies a real local Convex setup and synthetic fixture. Screenshots establish the rendered result, not independently the claimed HTTP denials.

Likely related people:

• unknown: The claimed source-line change could not be verified from bounded local history. (role: source history unknown; confidence: low)
• jesse-merhi: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Repair failed-filter rendering and cover rejection followed by retry.
• Provide deployment-continuity evidence using existing observations and persisted aggregate/cursor state.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.