Repository navigation
✨ knowledge: seed hosted spokes with the primary repo's knowledge dir and an 8 KB primer default - #17146
Conversation
… and an 8 KB primer default A hive with no knowledge.git_sources now gets one entry at boot pointing at its primary repo's knowledge directory (bead_synthesizer.publish.path, else docs/knowledge/ then src/docs/knowledge/, probed via the GitHub contents API on the default branch), named <repo>-knowledge at the project layer. In-memory, idempotent, operator lists untouched. knowledge.primer.max_bytes default rises from 4096 to 8192. Fixes #17145 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. @clubanderson this PR is held at Failing checks:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. Review findings from Correctness review at 747624a (read the diff plus 1. The updated parity test still asserts the old default — CI 2. Seeding is not gated on --- a/src/cmd/hive/boot_knowledge_seed.go
+++ b/src/cmd/hive/boot_knowledge_seed.go
@@ func seedKnowledgeGitSource(
if cfg == nil {
return false
}
+ if !cfg.Knowledge.Enabled {
+ logger.Debug("knowledge: git source seed skipped", "reason", "knowledge is disabled")
+ return false
+ }
if cfg.Knowledge.GitSources != nil {(and set 3. "In-memory only; never writes Review findings from Conclusion: the seed's "absent key" gate does not survive the first config save, so the feature is likely to fire once at most — or never on a spoke whose 1.
|
There was a problem hiding this comment.
Security perspective (read at head 747624a).
Seeded clone URL can point at a different host than the one probed (GHE hives with blank base_url)
src/cmd/hive/boot_knowledge_seed.go:82-85 builds the git-source URL from raw cfg.GitHub.BaseURL, falling back to config.DefaultGitHubBaseURL (https://github.com) when blank. But GitHubConfig.ResolvedBaseURL() (src/pkg/config/config.go:4047-4064) exists precisely because "pooled/placeholder GHE hives legitimately carry base_url: "" with api_url: https://github.ibm.com/api/v3", and nothing in config loading fills BaseURL from APIURL.
On such a hive the probe (b.ghClient, built from ResolvedAPIURL()) confirms owner/repo has docs/knowledge on the GHE instance, then the seed appends URL: https://github.com/owner/repo. The auto-connect loop (src/cmd/hive/main.go:3023-3029) hands that to ConnectGitSource, which does an unauthenticated git clone (src/pkg/knowledge/gitsource.go:200-212, GIT_TERMINAL_PROMPT=0, no token). Whoever holds the same owner/repo namespace on public github.com — internal GHE org names are frequently unregistered publicly — gets their docs/knowledge/ indexed at the project layer and injected into every kick's # Relevant Knowledge primer. The probe and the clone must agree on host.
base := strings.TrimRight(cfg.GitHub.ResolvedBaseURL(), "/")
(replacing the base := strings.TrimRight(cfg.GitHub.BaseURL, "/") / if base == "" { base = config.DefaultGitHubBaseURL } block). Please add a table case with APIURL = "https://github.example.com/api/v3", BaseURL = "" expecting https://github.example.com/acme/widgets.
Advisory (not security): for a private primary repo the same unauthenticated clone will fail with a WARN at every boot since the seeded source carries no credentials; consider checking repo visibility in the probe or documenting it.
— hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
There was a problem hiding this comment.
Read this against the tree at 8269ed9. The seed logic itself is sound and the referenced symbols all exist with the signatures the probe interface needs (pkg/github/defaultbranch.go:40, pkg/github/client.go:4415; ProjectConfig.ForgeKind at pkg/config/config.go:1179; DefaultGitHubBaseURL at config.go:3720). One finding that contradicts the stated design:
The seeded entry is not in-memory-only — it is persisted to hive.yaml by the next unrelated save (medium).
seedKnowledgeGitSourceappends tocfg.Knowledge.GitSourceson the live*config.Config(src/cmd/hive/boot_knowledge_seed.go, theappendjust before the INFO log; wired atmain.goinbootKnowledgeWith).Config.Save→saveLockedmarshals the whole in-memory struct:data, err := yaml.Marshal(c.redactedForPersist())atsrc/pkg/config/config.go:7396, andredactedForPersistdoes not touchKnowledge.GitSources.Save()is reached from ~12 call sites that have nothing to do with knowledge — e.g.SetAgentPausedAndSave(config.go:7357), and every dashboards.saveConfig()(src/pkg/dashboard/api.go:747) on any settings change.
Consequence: after the first pause/resume or dashboard save, hive.yaml gains knowledge.git_sources: [{name: <repo>-knowledge, …}]. From then on the key is non-nil, so the seed's "absent key" check treats it as operator-set: the probe never re-runs (a moved knowledge dir or changed default branch is never picked up), and the PR body / agent-knowledge-and-memory.md sentence "Nothing is written to hive.yaml" is false in practice. Not data-destroying — the persisted entry is the correct one today — but it silently converts a default into operator config, which is exactly what the body says it avoids.
Two ways to make the claim true, either is small:
- Don't mutate
cfg: haveseedKnowledgeGitSourcereturn theGitSourceConfigYAMLand havebootKnowledgeWithconnect it alongside (not appended to)b.cfg.Knowledge.GitSources. - Or keep the append but strip seeded entries in
redactedForPersist(would need a marker on the entry).
Option 1 also keeps the dashboard's handleGitSourcesList/Disconnect semantics unchanged, since those read s.deps.Config.Knowledge.GitSources (api_knowledge.go:1076, :1125).
The max_bytes 4096→8192 half is consistent across knowledge_config.go, primer_render.go, the parity test and the docs table.
— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Fixes #17145
Hosted spokes had
knowledge.git_sourcesempty and a 4 KiB primer cap, so the repo's curated knowledge entries never reached a kick and 15 of 25 primed facts were dropped (verified on hive's own spoke).Seam: spoke-side boot.
bootKnowledgeruns afterbootGitHub, soseedKnowledgeGitSource(src/cmd/hive/boot_knowledge_seed.go) runs just before the existing git-source auto-connect loop, using the hive's own*github.Clientfor the probe (DefaultBranch+PathExistsAtRef, 404 → skip).knowledge.git_sourcesis absent, seed<repo>-knowledge(layerproject, branch = repo default branch) at the first existing path amongbead_synthesizer.publish.path,docs/knowledge/,src/docs/knowledge/. In-memory only; never writeshive.yaml.git_sources: [](what a dashboard disconnect saves) opts out. Non-GitHub forge, no client, lookup/probe error, or no dir → one DEBUG line. Seeding → one INFO line.knowledge.primer.max_bytesdefault 4096 → 8192 (floor 512 unchanged); docs and parity test updated.