fix(self-driving): reconcile promoted tools with the real ask, restore the leak test - #1031
Merged
gewenyu99 merged 1 commit intoJul 29, 2026
Conversation
…e the leak test Two follow-ups to #1028, found by running the flow and reading the skill the ask actually comes from. The promoted-tools list was hand-written from a PR description rather than computed, so it was wrong both ways. Recomputed as the intersection of the options step 5 offers and the kinds this repo can detect: adds Shortcut, Honeybadger, Raygun, Kustomer, Plain, Canny, Productboard, Snyk and GoogleSearchConsole, all offered and all detectable, so all previously dropped on the floor; removes Intercom, which the ask never lists, so promoting it pointed the agent at a source it cannot create. The regression test guarding the frameworkContext fix is also missing from this branch — #1028 was merged at its first commit, so the test pushed afterwards was orphaned. Restored verbatim, and re-verified it fails when the shared-key write is reintroduced. Generated-By: PostHog Code Task-Id: df870021-4bdf-465b-ba4f-d486c37b97d3
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
gewenyu99
marked this pull request as ready for review
July 29, 2026 22:10
gewenyu99
merged commit Jul 29, 2026
85701c4
into
tom/self-driving-detected-sources
14 checks passed
gewenyu99
added a commit
that referenced
this pull request
Jul 29, 2026
Seven multi-line comment blocks from #1028 and #1031 collapsed to a single line each. Net -71 lines, no behaviour change. The rationale that was in them belongs in the PR descriptions, which already carry it: why the key is self-driving's own rather than the warehouse program's, how the promoted list was computed, and why the count tag fires at zero and at -1. Generated-By: PostHog Code Task-Id: df870021-4bdf-465b-ba4f-d486c37b97d3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two things found by actually running the flow and reading the skill the ask comes from, rather than inferring.
1. The promoted-tools list was wrong in both directions. I wrote it by hand from #1022's description instead of computing it. Against the real catalogue — the
optionsarray in context-mill'sself-driving/references/5-connected-tools.md— it was missing nine tools that are both offered and detectable, and it included one the ask never offers.2. The regression test guarding #1028's fix isn't on this branch. #1028 was merged at its first commit; the test commit pushed shortly after was orphaned. So the frameworkContext fix is live with nothing holding it in place.
Why
The list decides what gets promoted to the top of the connected-tools ask. Missing entries silently never surface, so a repo using Shortcut or Snyk gets no benefit from the scan at all. The wrong entry is worse than useless: promoting a tool the ask doesn't list points the agent at a source it can't create.
And a fix with no test is one refactor away from regressing — this one is invisible when it breaks, because the symptom is a paragraph quietly appearing in a different program's prompt.
Changes
SOURCE_DETECTORScan detect. Adds Shortcut, Honeybadger, Raygun, Kustomer, Plain, Canny, Productboard, Snyk, GoogleSearchConsole; drops Intercom. The comment now records which half came from where, and that the guard test only covers the detector half — the skill's catalogue lives in another repo and nothing here can see it change.Test plan
pnpm build && pnpm test && pnpm fix— 1671 tests pass, 0 lint errors. The guard test confirms all 21 kinds exist in the source registry. Re-verified the restored leak test fails when the shared-key write is reintroduced, so it is actually load-bearing.Notes for review
5-connected-tools.mdcontains no reference to the detected block — the ordering is hardcoded in the skill, so the wizard-side change stays inert until the companion context-mill PR lands. Worth confirming that's tracked; "safe to land in either order" doesn't hold.Created with PostHog Code