feat: include scoped skill demand in search intelligence

[Patrick-Erichsen]

What Problem This Solves

Staff search intelligence currently sees plugin demand but misses manual skill searches and cannot distinguish a filtered shelf from the full catalog.

User Impact

Staff can filter the dashboard, API and CLI by plugins or skills and by catalog, shelf or legacy scope. Existing observations and aggregation state remain intact; historical rows keep unknown scope and do not establish catalog-wide company opportunities. Skill coverage begins when collection actually starts. Homepage skill shelf collection remains a separate follow-up.

Why This Change Was Made

The existing privacy-preserving observation store and aggregation cursor now handle both artifact kinds. Header, full search and native Skills searches consume one ephemeral intent per catalog; current metadata uses shared public visibility, security, installability and Featured ownership checks. The weekly plugin classifier only classifies catalog gaps. Suggestions remain advisory.

Connected regressions repaired: footer navigation could count the same intent again; a failed skill search could also display “No matches”; GitHub skills lacking an installable source could appear eligible.

Evidence

• Real Chromium before/after: skill responses were previously unobserved; now one header skill intent and one native Skills intent each record once, with response count1. Whitespace, navigation, repeated submit and reload add zero observations.
• Real additive Convex deployment preserved75 raw rows,20 aggregate rows, the exact state/cursor and unrelated stats; new arrivals advance that cursor and replay adds zero. Artifact/scope HTTP reports equal the real CLI; anonymous401, invalidkind400, private,no-store retained.
• Final rebased tree: bun run ci:static passed with audit0; bun run ci:unit6845 passed,3 skipped; bun run ci:types-build passed. Package verification passed before the mechanical rebase; the four feature commits are identical by range-diff.
• Sol high autoreview: no findings, confidence0.84. Positive production delta484 lines provides shared skill capture/report/eligibility ownership; tests +447 net; generated +2.

Real screenshots and the detailed snapshot are attached in the UI proof comment. The proof uses disposable fixtures, not production demand.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
clawhub	Ready	Preview	Sep 15, 2026 8:03pm UTC

[Patrick-Erichsen]

ClawHub UI Proof

Status: passed
Mode: before-after
Scenario: manual skill demand capture and additive report upgrade
Provider: isolated local Convex and real Chromium
Baseline: d747991d3c
Candidate: 023602c8 (range-diff identical to reviewed2bad26f7)
Trigger: authorized-local-proof

Baseline header settled search	Candidate header settled search

Baseline native settled search	Candidate native settled search

Raw proof files: https://github.com/openclaw/clawhub/tree/qa-artifacts/clawhub-ui-proof/pr-3714/scoped-skill-demand

[clawsweeper]

Codex review: blocked before merge. Reviewed September 15, 2026, 4:10 PM ET / 20:10 UTC.

ClawSweeper review

What this changes

Adds manual skill-search observations and catalog-versus-filtered-shelf reporting to staff dashboards, HTTP responses, and the admin CLI.

Regression provenance

Possible regression — suspected (reviewed change). No predecessor PR is attributed.

Merge readiness

⛔ Blocked before merge - 5 items remain

This remains a useful extension beyond current main, with credible browser and upgrade evidence. The weekly digest still loses scope information, and the durable collection specifications need updating.

Priority: P2
Reviewed head: 023602c8dcacc69a201cebc82e9da7efc3760c8d
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	Useful implementation with credible runtime and upgrade proof, but weekly scope loss prevents a clean correctness verdict.
Proof confidence	🐚 platinum hermit (4/6) ✨ media proof bonus	Sufficient (linked_artifact): The linked real Chromium/local Convex report exercises header and native Skills searches through HTTP capture into persisted observations, showing two independent skill intents, replay suppression, populated-store preservation, and HTTP/CLI parity. It supports capture and upgrade behavior; mixed-scope weekly serialization remains a separate correctness finding.
Patch quality	🦐 gold shrimp (3/6)	2 actionable review findings remain.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (linked_artifact): The linked real Chromium/local Convex report exercises header and native Skills searches through HTTP capture into persisted observations, showing two independent skill intents, replay suppression, populated-store preservation, and HTTP/CLI parity. It supports capture and upgrade behavior; mixed-scope weekly serialization remains a separate correctness finding.
Evidence reviewed	9 items	Introduced change boundary: The verified merge-base-to-head comparison contains 37 files, with 1,308 additions and 375 deletions. The live REST response confirms the reviewed head remains open against codex/claw-724-weekly-digest. Current main does not implement skill capture: Current main's observation mutation accepts only artifactKind plugin. GitHub verifies #3630 as merged at this SHA; it implements the underlying plugin capture, not this extension. Release boundary: The observation module is absent from the supplied latest release, v0.23.3. Neither that release nor inspected main establishes implementation of scoped skill demand.
Findings	2 actionable findings	[P2] Preserve scope when constructing weekly digest rows [P3] Update the durable search collection specifications
Security	None	None.

How this fits together

ClawHub records bounded, anonymous search-result facts and aggregates them into staff demand reports. Those reports also feed advisory Featured suggestions and a weekly digest delivered through Hermit.

flowchart TD
  A[Manual skill or plugin search] --> B[Public search response]
  B --> C[Bounded anonymous observation]
  C --> D[Daily aggregates by catalog and scope]
  D --> E[Staff dashboard and API and CLI]
  D --> F[Weekly digest]
  G[Current public catalog eligibility] --> E
  F --> H[Hermit delivery]

Loading

Decision needed

Question	Recommendation
Should mixed-scope weekly rows ship through a coordinated Hermit contract update, or remain outside the digest until that representation is supported?	Coordinate scoped digest support: Extend producer and receiver compatibly so scope survives validation and rendering, including existing frozen payloads.

Why: The current receiver cannot represent scope, and silently collapsing it changes the meaning of reported demand; choosing a temporary narrower digest changes operator-visible coverage.

Before merge

• Preserve scope when constructing weekly digest rows (P2) - The new scope/query key retains multiple rows for the same query, but buildSearchDigest drops scope and emits the same unfiltered search URL for each. With threshold-qualified catalog and shelf observations for notion, the weekly payload contains indistinguishable notion entries with different counts, potentially occupying multiple top-five slots. Featured and mover cohorts have the same problem. Preserve the distinction through a compatible rendered contract or deliberately construct an unambiguous cohort; Hermit's strict row allowlist means a producer-only field addition is insufficient.
• Update the durable search collection specifications (P3) - This introduces persisted skill observations and scope, while specs/plugin-search-intelligence.md explicitly says Skills remain unobserved and observations have only artifactKind plugin. The web and staff-report specifications also omit the new collection and legacy-scope semantics. Update those contracts with supported controls, exclusions, scope meaning, and preserved privacy/retention rules, as required by AGENTS.md, so future work does not follow contradictory collection instructions.
• Resolve merge risk (P1) - Mixed-scope weekly reporting needs a receiver-compatible representation; adding scope only in ClawHub would fail Hermit's strict payload validation.
• Complete next step (P2) - Resolve the digest representation, repair scope loss with compatible validation and rendering, and update the durable search specifications.
• Resolve maintainer decision - Resolve the maintainer decision shown above before merge.

Findings

• [P2] Preserve scope when constructing weekly digest rows — convex/searchWeeklyDigest.ts:275-282
• [P3] Update the durable search collection specifications — convex/pluginSearchObservations.ts:24-25

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Diff size	+1,308 / -375 across 37 files	The change spans capture, persistent aggregates, report consumers, and weekly delivery.
Net line growth	production +484, tests +447, generated +2	Production growth has a stated purpose: shared skill capture, scoped reports, and current eligibility resolution.

Merge-risk options

Maintainer options:

1. Preserve scope across the digest boundary (recommended)
   Coordinate compatible producer/receiver support and verify one query appearing in catalog, shelf, and legacy cohorts renders unambiguously.
2. Hold the digest extension
   Pause the mixed-scope digest behavior until its representation is agreed without discarding the useful capture and report work.

Technical review

Best possible solution:

Preserve scope through the weekly digest's validated, rendered contract while retaining legacy data, anonymous capture, and advisory-only recommendations.

Do we have a high-confidence way to reproduce the issue?

Yes, source establishes a deterministic PR-head case: give one query threshold-qualified catalog and shelf aggregates, then build the weekly digest; both rows survive but their scope does not. This review did not execute target code.

Is this the best way to solve the issue?

The shared capture and aggregation approach is appropriate and avoids a competing analytics store. The digest adaptation is incomplete because its serialized contract cannot express the new row identity.

Full review comments:

• [P2] Preserve scope when constructing weekly digest rows — convex/searchWeeklyDigest.ts:275-282
  The new scope/query key retains multiple rows for the same query, but buildSearchDigest drops scope and emits the same unfiltered search URL for each. With threshold-qualified catalog and shelf observations for notion, the weekly payload contains indistinguishable notion entries with different counts, potentially occupying multiple top-five slots. Featured and mover cohorts have the same problem. Preserve the distinction through a compatible rendered contract or deliberately construct an unambiguous cohort; Hermit's strict row allowlist means a producer-only field addition is insufficient.
  Confidence: 0.98
• [P3] Update the durable search collection specifications — convex/pluginSearchObservations.ts:24-25
  This introduces persisted skill observations and scope, while specs/plugin-search-intelligence.md explicitly says Skills remain unobserved and observations have only artifactKind plugin. The web and staff-report specifications also omit the new collection and legacy-scope semantics. Update those contracts with supported controls, exclusions, scope meaning, and preserved privacy/retention rules, as required by AGENTS.md, so future work does not follow contradictory collection instructions.
  Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.95

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against b5729e702ea8.

Labels

Label changes:

• add P2: This is a bounded staff-intelligence improvement with a concrete reporting correctness defect.
• add merge-risk: 🚨 other: Dropping scope can produce misleading weekly rankings and indistinguishable repeated query entries.
• add proof: sufficient: Contributor real behavior proof is sufficient. The linked real Chromium/local Convex report exercises header and native Skills searches through HTTP capture into persisted observations, showing two independent skill intents, replay suppression, populated-store preservation, and HTTP/CLI parity. It supports capture and upgrade behavior; mixed-scope weekly serialization remains a separate correctness finding.
• add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
• add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (linked_artifact): The linked real Chromium/local Convex report exercises header and native Skills searches through HTTP capture into persisted observations, showing two independent skill intents, replay suppression, populated-store preservation, and HTTP/CLI parity. It supports capture and upgrade behavior; mixed-scope weekly serialization remains a separate correctness finding.

Label justifications:

• P2: This is a bounded staff-intelligence improvement with a concrete reporting correctness defect.
• merge-risk: 🚨 other: Dropping scope can produce misleading weekly rankings and indistinguishable repeated query entries.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
• status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (linked_artifact): The linked real Chromium/local Convex report exercises header and native Skills searches through HTTP capture into persisted observations, showing two independent skill intents, replay suppression, populated-store preservation, and HTTP/CLI parity. It supports capture and upgrade behavior; mixed-scope weekly serialization remains a separate correctness finding.
• proof: sufficient: Contributor real behavior proof is sufficient. The linked real Chromium/local Convex report exercises header and native Skills searches through HTTP capture into persisted observations, showing two independent skill intents, replay suppression, populated-store preservation, and HTTP/CLI parity. It supports capture and upgrade behavior; mixed-scope weekly serialization remains a separate correctness finding.

Evidence

What I checked:

• Introduced change boundary: The verified merge-base-to-head comparison contains 37 files, with 1,308 additions and 375 deletions. The live REST response confirms the reviewed head remains open against codex/claw-724-weekly-digest. (023602c8dcac)
• Current main does not implement skill capture: Current main's observation mutation accepts only artifactKind plugin. GitHub verifies feat(search): capture bounded manual plugin-search observations #3630 as merged at this SHA; it implements the underlying plugin capture, not this extension. (convex/pluginSearchObservations.ts:24, b5729e702ea8)
• Release boundary: The observation module is absent from the supplied latest release, v0.23.3. Neither that release nor inspected main establishes implementation of scoped skill demand. (87ca030c30f3)
• Scope is lost at digest serialization: The producer now retains separate scope/query rows, but buildSearchDigest serializes only query, counts, and the unfiltered search URL. Mixed catalog/shelf/legacy observations can therefore produce indistinguishable entries and consume multiple shortlist positions. Existing weekly tests only add catalog fixtures. (convex/searchWeeklyDigest.ts:277, 023602c8dcac)
• Affirmative Hermit dependency: The changed weekly producer passes the resulting payload to deliverSearchDigest; specs/search-weekly-digest.md identifies Hermit as the authoritative receiver and renderer. This makes its payload contract relevant to repairing scope loss. (convex/searchWeeklyDigest.ts:314, 023602c8dcac)
• Receiver contract requires coordinated repair: Repository ownership was verified through GitHub. The receiver uses exact recursive field allowlists, accepts no scope field in digest rows, and renders query/counts without scope. Adding a producer-only scope field would be rejected. (src/clawhubSearchIntelligence/api.ts, 4593a1863cd3)

Likely related people:

• Patrick-Erichsen: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Resolve the digest representation and add a regression covering the same query in catalog, shelf, and legacy scopes.
• Update the durable capture, web attribution, and reporting specifications.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[Patrick-Erichsen]

The scope-loss finding and both Rank-up moves are resolved in the coordinated atomic stack through ClawHub #3718, with Hermit #44 already merged and deployed as receiver version 81. There will be no intermediate production deployment of this collection layer with the old live v1 builder.

At combined head b5a6462c96519589d4bc8c5e4ea847375be4191d, convex/lib/searchEvidenceDigest.ts retains scope on cohort rows and recommendation queries; convex/searchWeeklyDigest.ts emits separate plugin/skill catalogs and classifies only catalog gaps. Hermit validates and renders the closed scope values. Frozen legacy payloads are preserved unchanged.

The requested regression at convex/lib/searchEvidenceDigest.test.ts:240 projects and JSON-round-trips the same notion query in catalog/shelf/legacy scopes with counts 6/5/4, proving distinct gap, mover, and candidate evidence plus catalog-only company opportunities. All five digest-builder tests and the full 6,856-test combined suite pass. The existing four capture/web/report/weekly specifications now document skill coverage, scopes, exclusions, privacy/retention, and native cancellation semantics.

The real persistent Convex upgrade proof additionally verifies legacy weeks, new v2 production, restart continuity, and unchanged replay; its nine relevant source blobs match the current combined head: producer proof. Actual Discord posting stays unverified and is intentionally excluded from Patrick's first read-only production dry run.