Skip to content

✨ knowledge: edit an existing git source (PUT /api/knowledge/git-sources) - #17596

Merged
hivecommons-hive[bot] merged 17 commits into
v6from
scanner/fix-17588
Nov 11, 2026
Merged

hivecommons-hive[bot] merged 17 commits into
v6from
scanner/fix-17588

Conversation

@hivecommons-hive

@hivecommons-hive hivecommons-hive Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Right now the only way to fix a mistyped URL, name, subpath, branch or layer on a knowledge git source is Remove + re-add. That throws away the clone and re-indexes everything, even when only the display name was wrong. This PR lets the operator edit the source in place and only re-clones when the change requires it.

Why it is shaped this way:

  • Identity stays {url, subpath}, the same key connect and disconnect use. Edits go through PUT /api/knowledge/git-sources and are owner-gated like the other mutations, so the auth model is unchanged. The source can be moved with new_url / new_subpath.
  • Name/layer edits never touch the clone. Fixing a label should not cost a full clone and re-index. The live source and its hive.yaml entry are rewritten in place.
  • URL/branch/subpath edits re-clone in one request. The old source keeps serving until the new clone is ready, so a bad edit cannot leave the operator without the source they had. A branch-only change wants the same clone directory, so the new branch is cloned into a sibling staging directory and renamed over the old clone only after it has been indexed; the original clone is never deleted on a failed clone. If the source was removed while the clone ran, the Remove wins and the request returns 404 rather than re-publishing it.
  • One knowledge_git_source_update audit row per edit, not a disconnect+connect pair, because from the operator's side it is a single action.
  • UI: each row gets Edit, which loads the row into the Add Git Source form; the button becomes Save and a Cancel control resets the form. Missing required fields use the inline-error helper.

Note on the prerequisite: #17589 (inline field errors, #17587) merged into v5, not v6, so kbFieldError / kbClearFieldError were not on v6. This PR carries that change over unchanged (static-src JS/CSS, its node test and its changelog fragment) so the edit form can reuse it here. The contents are byte-identical, so the next v5→v6 top-up should merge cleanly.

Docs: the OpenAPI spec, the api-reference route tables (adding the PUT row shifted later api.go line citations by one), the contribute API page and knowledge-curator.md are updated. static/index.html is regenerated from static-src/manifest.txt.

Closes #17588
Closes #17583
Refs #17610 (busy-state/default-branch fix merged to v5 as #17614; this PR carries its changelog fragment byte-identical)
Addendum (#17583 maintainer repro)

  • Busy state: the handler looked the button up with a stale button[onclick="kbAddGitSource()"] selector, so Clone & Index / Save never disabled or showed "Cloning…". It now targets #kb-subs-content [data-action="kbAddGitSource"].
  • Blank branch: the client guessed main, which fails for repos with another default branch. A blank branch is now sent blank; the server resolves it with git ls-remote --symref <url> HEAD (falling back to main only if the remote reports no HEAD symref; ls-remote failures surface) and records the resolved branch in the saved config. Placeholder is now "Branch (blank = repo default)".
  • Failures invisible on return: a failed connect/save is kept in client state and rendered as an error: <reason> row with Retry (reloads the form, reusing Edit/Save) and Dismiss. No server persistence added.

Tests: Go table test for default-branch resolution plus a ConnectGitSource blank-branch test; JS assertions for the selector and error row. static/index.html regenerated.

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@clubanderson

…ces)

A fat-fingered URL, name, subpath, branch or layer on a knowledge git
source could only be corrected by Remove + re-add, which throws away the
clone and re-indexes even when only the display name was wrong.

Add PUT /api/knowledge/git-sources, owner-gated like connect/disconnect,
taking {url, subpath} as the identity plus the new fields. A name/layer
change rewrites the live source and its config entry in place without
touching the clone; a url/branch/subpath change swaps in a fresh clone in
the same request, keeping the old source if the new clone fails. Either
way one knowledge_git_source_update audit row is written. The dashboard
gets an Edit button per row that loads it into the Add Git Source form
(Save / Cancel).

The inline required-field helper from #17589 only landed on v5, so it is
carried here unchanged to let the edit form reuse it on v6.

Closes #17588
Part of #17583

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 10, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 10, 2026
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v6

Conflicts are checked with every PR merged onto the current tip of v6, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive

hivecommons-hive Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

hive/awaiting-ci: pushed 919acb1 — docs-only, applies the reviewer-2 follow-up on 4dcc6fb (the YAML example comment at knowledge-curator.md:33 no longer calls name the lookup key). #17599 has landed, so the advisory mutation job no longer blocks the gate. CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Review findings from reviewer (folded here because this head already carries its hive review):

Read the tree at 038c83e (not just the diff): src/pkg/knowledge/api.go, gitsource.go, filestore.go, the dashboard handler and the JS edit flow. This looks correct to me.

What I verified:

  • Sync-loop ownership. bindSyncContext (gitsource.go:168) binds the cancel func before the goroutine starts, and StopSyncLoop (:186) waits on syncDone, so by the time swapGitSource (api.go) runs promote() the old loop's git pull has returned. The branch-only edit renaming the old clone aside is therefore race-free against the pull, as the comment claims.
  • Failed clone leaves the old source intact. initGitSourceForConnect on the staged gs runs before any mutation of k.gitSources; promoteStagedClone renames old → trash → staging → final, and restores on failure. defer os.RemoveAll(staging) is a no-op after a successful promote (dir already renamed). Covered by TestGitSourcesUpdateRecloneFailureKeepsOriginal and TestReplaceGitSourceStagesWhenCloneDirUnchanged.
  • Remove-wins. swapGitSource re-matches identity under the write lock, so a source disconnected during the clone returns ErrGitSourceNotFound → 404 and the new gs is never published (TestReplaceGitSourceDoesNotResurrectRemovedSource).
  • withConfig copy. Shares store/stopSync/syncDone, so StopSyncLoop on the copy still stops the goroutine that holds the original pointer. FileStore.SetName/Relocate now take the store lock, and reindex snapshots rootDir once — consistent.
  • Handler. Owner-gated; validateGitSourceFields is behaviourally identical to the three inline checks it replaces; ValidateGitSourceURLContext runs on a changed URL before ResolveGitSourceDefaultBranch shells out to git ls-remote, so SSRF posture is unchanged. The UI (knowledge.js) always sends branch: branch || '', which the handler resolves and compares against current.Branch, so a Save with an unchanged branch does not re-clone.
  • Body matches diff; the v5 carry-overs (kbFieldError, 🐛 knowledge: Clone & Index shows busy state and resolves a blank branch to the repo default #17614 fragment) are explained and byte-identical as stated.

One pre-existing limitation this feature makes a little easier to hit (not a defect of this PR, no action required):

  • ensureCloned (gitsource.go:312-318) pulls an existing checkout rather than re-cloning, and DisconnectGitSource does not delete the clone dir. So editing a URL/subpath onto a slug that still holds a stale clone from a previously removed source will reuse that checkout's branch while hive.yaml records next.Branch. The staging path only covers the same-slug case. Connect has the same behaviour today, so this is an observation for a follow-up, not a blocker.

I have not approved or merged anything; a maintainer's signature is still needed.

stage 080-awaiting-merge-admission → 070-fixing: check-cancelled

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Read the full backend diff against the v6 tree (knowledge/api.go, gitsource.go, filestore.go, dashboard/api_knowledge.go) plus the sync loop and Init they interact with. Looks correct to me; nothing here blocks landing. Two low-severity notes a maintainer may want to weigh, neither a defect in the stated behaviour.

What I verified

  • SSRF ordering: the new git ls-remote in ResolveGitSourceDefaultBranch only ever runs after ValidateGitSourceURLContext. ConnectGitSource validates at src/pkg/knowledge/api.go:1074 before resolving at :1088; ReplaceGitSource validates at :1208; the handler validates a changed new_url before resolving (api_knowledge.go, handleGitSourcesUpdate) and an unchanged URL was validated at connect. ls-remote goes through gitCmdArgs (http.followRedirects=false) and gitSourceEnv (GIT_ALLOW_PROTOCOL), same as clone/pull.
  • Branch-only edit: sameCloneDir is decided on CloneDir() not URL, so a .git-suffix or scheme edit also takes the staging path. promoteStagedClone renames the old clone aside before the staged one in, and restores it on failure; the deferred os.RemoveAll(staging) is a no-op after a successful rename. The old loop is stopped (swapGitSource, api.go:1329) before the rename, so no git pull writes into the directory being moved.
  • withConfig snapshots store/ready from the old pointer the loop still runs on; Sync only reads g.store and Init is the only writer of store/ready, so the snapshot cannot go stale. The copy inherits stopSync/syncDone, so stopping the copy stops the shared loop.
  • ReplaceGitSource → swapGitSource re-matches identity under the lock, so a Remove during the clone wins (TestReplaceGitSourceDoesNotResurrectRemovedSource covers it).
  • Handler validation is shared via validateGitSourceFields, equivalent to the removed inline checks.

Low-severity notes (not blockers)

  1. api.go:1215-1224 → :1267 — the conflict check for the new identity runs under RLock and is released before the clone; swapGitSource only re-matches the old identity. A POST connecting config.URL+Subpath during the (long) clone window would leave two sources with the same identity after the swap. Narrow race, operator-only surface; a re-check of the new identity inside swapGitSource would close it.
  2. api.go:1161 and :1329 — StopSyncLoop waits for the loop goroutine while holding k.mu.Lock(). Cancelling the context kills a running git pull, but a store.Reindex() already in progress is not cancellable, so every knowledge read blocks until it finishes. Bounded, but worth knowing on a large clone.

The PR body matches the diff; the carried-over kbFieldError helper and changelog fragments are byte-identical as stated.

Review findings from ci-maintainer (folded here because this head already carries its hive review):

Style/maintainability review (head 919acb1) — no in-scope blocker; two advisory notes and one adjacent bug for backlog.

  1. Whole-clone RemoveAll runs under the KnowledgeAPI lock. swapGitSource (src/pkg/knowledge/api.go) calls promote() while holding k.mu.Lock(), and promoteStagedClone ends with os.RemoveAll(trash) on the old clone. The renames are cheap, but deleting a full repo checkout under the API-wide mutex blocks every GitSources()/search reader for the duration. Consider returning the trash path from promote and deleting it after the lock is released (or via the existing defer os.RemoveAll(staging) pattern in ReplaceGitSource).

  2. Blank-branch resolution is now duplicated three times: ConnectGitSource (api.go, the new if strings.TrimSpace(config.Branch) == "" block), ReplaceGitSource (same block verbatim), and handleGitSourcesUpdate (src/pkg/dashboard/api_knowledge.go, the if next.Branch == "" block). The PR rightly deduplicated the field validation into validateGitSourceFields "so the two handlers cannot drift"; the same argument applies here — a small resolveBranchIfBlank(ctx, &cfg) helper would keep the fallback/error-wrapping identical.

  3. (out-of-scope, pre-existing) The stale button[onclick=...] selector this PR fixes for Clone & Index (static-src/js/knowledge.js, kbAddGitSource) is still present for Import Document (kbImportDocFromModal, button[onclick="kbImportDocFromModal()"]) and kbReimportDoc ([onclick="kbReimportDoc('…')"]), so those busy states never render either. Worth a follow-up.

Advisory: audit event knowledge_git_source_update breaks the git_source_connect / git_source_disconnect sibling naming in the same file, but it is the name #17588 specifies, so no change requested.

Review findings from adjudicator (folded here because this head already carries its hive review):

Correctness review (head 9f49eb5)

  1. In-place element writes race with lock-free slice readers — UpdateGitSourceMetadata (src/pkg/knowledge/api.go:1185, k.gitSources[i] = gs.withConfig(cfg)) and swapGitSource (api.go:1329, k.gitSources[i] = gs) mutate elements of the existing backing array under k.mu. But SearchAllWithVaults takes only the slice header under RLock (api.go:934, sources := k.gitSources) and iterates it after unlocking (api.go:947), and Stats reads k.gitSources with no lock at all (api.go:274). Before this PR the only mutations were append and Disconnect's fresh slice, so those readers were safe; now a concurrent search during a PUT is a write/read race on the element (-race will flag it). Fix: copy-on-write — build a new slice in both functions as DisconnectGitSource already does (api.go:1146-1157).

  2. Name edit leaves the sync loop on the stale config — withConfig returns a copy (src/pkg/knowledge/gitsource.go:144-162), but the loop goroutine started by startGitSourceSync (api.go:1120-1126) still holds the original *GitSource. Every tick StartSyncLoop → Sync → injectRepoConventions reads g.config of the old object (gitsource.go:223-250), and repoNameFromGitSource derives the repo from cfg.Name when it is owner/repo-shaped (gitsource.go:253-255). So after correcting a mistyped display name, the periodic reindex keeps injecting the derived-conventions page under the old repo name until restart (and logs the old name). Simplest fix: mutate config/store.SetName in place under g.mu instead of copying, or have withConfig also re-point the loop (e.g. loop reads g.Config() under lock).

Out of scope (pre-existing, noting for backlog): for a URL/subpath change whose new slug directory already exists on disk (old clones are never deleted), ensureCloned pulls the existing checkout instead of cloning config.Branch (gitsource.go:311-317), so the requested branch is silently ignored.

Tests for metadata-vs-reclone, staged promotion, mid-clone disconnect and blank-branch resolution are present and look adequate.

Review findings from adjudicator-2 (folded here because this head already carries its hive review):

Style review at head c554b21 — no in-scope blockers.

One small gap between the body and the diff, outside the #17588 contract (belongs to the #17583 addendum):

  • Network/parse failures bypass the new failure row — src/pkg/dashboard/static-src/js/knowledge.js:2173-2179 sets _kbGitSourceFailed only on the !r.ok branch; the catch at knowledge.js:2184-2185 (fetch rejected, or await r.json() throwing on a non-JSON error body) still just calls hiveAlert('Error: …'), so those failures still "disappear on return". Consistent with the file's other catch blocks, so low severity; either populate _kbGitSourceFailed there too or narrow the body's claim.

Otherwise: shared validateGitSourceFields keeps connect/update from drifting, ErrGitSourceNotFound/ErrGitSourceConflict sentinels map cleanly to 404/409, and the Edit button reuses the existing data-action/data-argN idiom of the Remove button. Tests cover the metadata-only vs re-clone matrix required by #17588.

Evidence bundle `#17596c554b21` sha256:db14bc88e5830557de087b6c1764b119996066e7ebff53827674537492078711 — download from the dashboard or `hivectl review evidence #17596`.

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 10, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the v6 tree and the callers. This looks correct to me.

What I verified:

  • knowledge/api.go ReplaceGitSource follows ConnectGitSource exactly (validate → RLock conflict scan → newGitSourceForConnect → initGitSourceForConnect → swap under Lock → go startGitSourceSyncLoopForConnect), so the test seams keep working and the old source stays in k.gitSources until the new clone has succeeded.
  • gitsource.go withConfig shares the *FileStore pointer with the original GitSource. Sync() (gitsource.go:171-177) reindexes that same store in place, so the sync loop still running on the old pointer keeps the renamed source's index fresh. Layer is read live via gs.Config().Layer (api.go:959, api.go:1012), so a layer-only edit takes effect without re-indexing.
  • dashboard/api_knowledge.go handleGitSourcesUpdate applies the same name/branch/subpath/URL checks as handleGitSourcesConnect (api_knowledge.go:1167-1181) and is owner-gated. The hive.yaml rewrite drops any stale entry that already carries the new {url, subpath}, which is the right call since ReplaceGitSource has already refused the case where a live source owns it.
  • knowledge.js sends {url, subpath, new_url, new_subpath, name, branch, layer} on Save and POST on Add; the submit label/reset go through kbGitSourceSubmitLabel().

One note, not a defect in this PR: on a URL/subpath move (sameIdentity == false) the old clone directory is left on disk and the old source's StartSyncLoop goroutine keeps pulling it until s.deps.Ctx is cancelled. DisconnectGitSource (api.go:1121-1140) has the same behaviour today, so this PR is consistent with the existing pattern — just flagging it in case a follow-up wants a per-source cancel.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 10, 2026
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 10, 2026
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Mutation testing (advisory)

Head c554b2136.

No surviving mutants reported in the packages that finished.

⏱ Time budget (1020s) exhausted: 2 of 2 changed package(s) were not fully mutated, so this report is partial:

  • src/pkg/dashboard — unfinished
  • src/pkg/knowledge — not started

Advisory only; this job never blocks the PR (#17349).

@hivecommons-hive hivecommons-hive Bot added the stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) label Nov 10, 2026
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) labels Nov 10, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Test gap: TestGitSourcesUpdateRecloneFailureKeepsOriginal never exercises the restore path it is named for.

In the same directory branch change case the sequence in ReplaceGitSource is: os.RemoveAll(old clone dir) → fresh clone fails → restoreGitSource → os.RemoveAll again → re-clone of the old branch. But installHermeticGit makes every git clone exit HIVE_FAKE_GIT_CLONE_EXIT while the env is set, so the restore clone fails as well, swapGitSource is never reached, and the original *GitSource object simply stays in k.gitSources. The test then asserts only deps.Knowledge.GitSources() metadata and the config slice, both of which are untouched on that path regardless — so it passes without ever showing that an operator "is left with the source they had before the edit" (the on-disk clone was wiped and never came back).

Two cheap additions would pin the behaviour the PR actually promises:

  1. Let the fake fail only the first clone (e.g. HIVE_FAKE_GIT_CLONE_FAIL_ONCE pointing at a marker file: fail if the marker is absent, create it, succeed afterwards). Then assert on HIVE_FAKE_GIT_LOG that a second clone ran with --branch main and that the clone dir contains the seeded README.md afterwards.
  2. Keep the current always-fail case as an explicit "restore also failed" case and assert the one observable it has — the git source restore after failed edit error log — rather than metadata that could not have changed.

Not blocking on its own; flagging because the test comment makes a stronger claim than the assertions verify, and the restore branch is the only part of this change with no coverage at all.


🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs-currency (read at head 75a506c)

Docs cover the new endpoint well (OpenAPI, route tables, contribute page, curator guide, changelog). Two advisory accuracy points, neither blocking:

  1. "Old source keeps serving if the new clone fails" is unconditional in the docs but conditional in the code. dashboard/openapi.json:14586 and src/docs/knowledge-curator.md:132-136 say the old source stays in place on a failed edit. That holds for a url/subpath move, but for a branch-only edit ReplaceGitSource first deletes the shared clone directory (src/pkg/knowledge/api.go:1201) and on failure only attempts a re-clone (api.go:1207, restoreGitSource api.go:1222-1228); if that re-clone also fails the error is just logged (api.go:1227) and the operator is left with a source whose clone is gone. Suggest qualifying the sentence in both places (e.g. "a branch-only edit clears the clone first; the original is re-cloned if the new clone fails, and must be re-added if that also fails") and the 500 description.

  2. Stale source citation on the line this PR edited. src/docs/knowledge-curator.md:120-121 now reads GET/POST/PUT/DELETE … (pkg/dashboard/api_knowledge.go:980-986,988-1070,1072-1114) but at head the handlers are at api_knowledge.go:1131 (List), :1139 (Connect), :1223 (Disconnect), :1273 (Update) — the new PUT handler is not covered by the cited range. Since the PR refreshed the shifted api.go citations elsewhere, this one should be refreshed too.

Out of scope (pre-existing): knowledge-curator.md:45 still says name is "the key used to look up/remove the source via the API", which the new bullet at :132 (identity = url+subpath) now directly contradicts.

— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

hivecommons-hive Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Review follow-ups (below review.backlog.min_severity: P2)

Non-blocking findings below the backlog filing line are listed here instead of as issues, one comment per PR, updated in place. Tick them off as they are fixed; set review.backlog.min_severity: P3 to file every finding.

  • P3 info docs-currency — Stale api_knowledge.go line citations kept in rewritten knowledge-curator.md sentences (src/docs/knowledge-curator.md:121): knowledge-curator.md:121 cites api_knowledge.go:980-986,988-1070,1072-1114 for the GET/POST/PUT/DELETE handlers and :50 cites api_knowledge.go:1032-1034 for the layer default, but at head handleG…
  • P3 info docs-currency — Stale cmd/hive/main.go startup-loop citation in knowledge-curator.md (src/docs/knowledge-curator.md:149): The unchanged context line 'main.go's startup loop at cmd/hive/main.go:2421-2467' points at a range that no longer contains the git_sources boot loop; at head c554b21 the loop is `for _, gsc := ran…
  • P3 info intent-alignment — 🐛 [v5] Knowledge: Clone & Index shows no busy state (stale onclick selector) and silently assumes branch main when blank #17610 addendum (busy state, ls-remote default branch, failed-attempt row) is outside [scanner] ✨ [v5] Knowledge: Edit an existing git source (PUT /api/knowledge/git-sources) #17588 (src/pkg/dashboard/static-src/js/knowledge.js:2151): kbAddGitSource now also carries the 🐛 [v5] Knowledge: Clone & Index shows no busy state (stale onclick selector) and silently assumes branch main when blank #17610 fixes (selector change to [data-action="kbAddGitSource"], blank branch sent as '' and resolved server-side via git ls-remote in ConnectGitSource/ReplaceGitS…
  • P3 info intent-alignment — PR bundles v5-to-v6 ports of 🐛 knowledge: inline required-field errors on Add Git Source, Add Sub and Import forms #17589 and 🐛 knowledge: Clone & Index shows busy state and resolves a blank branch to the repo default #17614 alongside the [scanner] ✨ [v5] Knowledge: Edit an existing git source (PUT /api/knowledge/git-sources) #17588 feature (src/pkg/dashboard/static-src/js/knowledge.js:2071): Beyond the [scanner] ✨ [v5] Knowledge: Edit an existing git source (PUT /api/knowledge/git-sources) #17588 contract, the diff carries kbFieldError/kbClearFieldError ([scanner] 🐛 [v5] Knowledge: inline required-field errors for Add Git Source / Add Sub / Import forms #17587, merged to v5 as 🐛 knowledge: inline required-field errors on Add Git Source, Add Sub and Import forms #17589), the busy-state selector fix, server-side default-branch resolution (ResolveGitSourceDefau…
  • P3 info intent-alignment — Ported 🐛 [v5] Knowledge: Clone & Index shows no busy state (stale onclick selector) and silently assumes branch main when blank #17610 fix diverges from v5 (🐛 knowledge: Clone & Index shows busy state and resolves a blank branch to the repo default #17614): helper exported on v6, unexported on v5 (src/pkg/knowledge/gitsource.go:372): The PR body says it carries 🐛 knowledge: Clone & Index shows busy state and resolves a blank branch to the repo default #17614's changelog fragment byte-identical, but it also re-implements the whole 🐛 [v5] Knowledge: Clone & Index shows no busy state (stale onclick selector) and silently assumes branch main when blank #17610 fix on v6 with a different surface: ResolveGitSourceDefaultBranch is exported here…
  • P3 info style — Audit action prefix differs from sibling git-source mutations (src/pkg/dashboard/api_knowledge.go:1437): The new handler writes audit action knowledge_git_source_update (as [scanner] ✨ [v5] Knowledge: Edit an existing git source (PUT /api/knowledge/git-sources) #17588 specifies), while the adjacent connect/disconnect handlers write git_source_connect (api_knowledge.go:1229) and git_source_d…
  • P3 low correctness — KnowledgeAPI.GitSources() reads k.gitSources without k.mu while new locked writers replace slice elements (src/pkg/knowledge/api.go:1330): src/pkg/knowledge/api.go:1330-1336 iterates k.gitSources with no lock; this PR adds UpdateGitSourceMetadata (api.go:1182) and swapGitSource (api.go:1322) which assign slice elements under k.mu, and h…
  • P3 low correctness — KnowledgeAPI.GitSources() reads k.gitSources without k.mu while the new in-place element writes hold it (src/pkg/knowledge/api.go:1311): GitSources() (and the layer-stats loop at api.go:274) iterate k.gitSources with no lock. The PR adds in-place writes k.gitSources[i] = gs under k.mu in UpdateGitSourceMetadata and swapGitSource, so a…
  • P3 low correctness — Re-clone to an identity whose clone dir already exists reuses the stale checkout and ignores the requested branch (src/pkg/knowledge/gitsource.go:312): ReplaceGitSource (src/pkg/knowledge/api.go:1237-1253) only stages when the new slug equals the old clone dir. When the new identity differs but its slug directory already exists on disk — which happe…
  • P3 low correctness — Replace into a pre-existing slug directory ignores the requested branch (src/pkg/knowledge/gitsource.go:311): Old clone dirs are never deleted on disconnect/replace, and ensureCloned (gitsource.go:311-317) pulls an existing checkout rather than cloning config.Branch. A URL/subpath edit (or re-add) whose new …
  • P3 low correctness — ReplaceGitSource leaves the fresh clone on disk when the old source was removed mid-clone (identity changed) (src/pkg/knowledge/api.go:1263): When config.URL/Subpath differ from the old identity, newGitSourceForConnect clones directly into finalDir (api.go:1230-1247) and only the sameIdentity staging dir has a deferred os.RemoveAll (api.go…
  • P3 low correctness — Replaced/edited git sources leak their StartSyncLoop goroutine (src/pkg/knowledge/gitsource.go:228): Pre-existing: DisconnectGitSource never stops a source's sync loop, and the new ReplaceGitSource/UpdateGitSourceMetadata inherit this. After a branch-only edit two loops git-pull the same clone dir e…
  • P3 low correctness — promoteStagedClone can leave a .replaced-* dir behind if the old source's git pull is in flight (src/pkg/knowledge/api.go:1290): promoteStagedClone renames finalDir aside and RemoveAll(trash) errors are discarded; the old source's StopSyncLoop runs only after promote inside swapGitSource, so an in-flight Sync's git pull may st…
  • P3 low docs-currency — knowledge-curator.md api_knowledge.go handler citations are stale and PUT has none (src/docs/knowledge-curator.md:120): The 'Static config vs. the runtime API' paragraph cites api_knowledge.go:980-986,988-1070,1072-1114 (and 1100-1110, 1032-1034 elsewhere) for the GET/POST/DELETE handlers; at merge-base these al…
  • P3 low docs-currency — knowledge-curator.md says name is the API lookup/remove key; identity is url+subpath (src/docs/knowledge-curator.md:45): Pre-existing: line 45 says name is 'the key used to look up/remove the source via the API', but DELETE and the new PUT match on url+subpath (src/pkg/dashboard/api_knowledge.go:1223ff, 1273ff). The …
  • P3 low intent-alignment — ReplaceGitSource leaves old sync loop running on shared clone dir (src/pkg/knowledge/api.go:1228): swapGitSource replaces the slice entry (api.go:1228) but nothing cancels the replaced source's StartSyncLoop goroutine (gitsource.go:208), which runs on the server-lifetime ctx. For a same-identity b…
  • P3 low security — Git source layer value accepted unvalidated (connect and update) (src/pkg/dashboard/api_knowledge.go:1352): handleGitSourcesUpdate casts the request's free-form layer string straight to knowledge.LayerType (api_knowledge.go:1352), mirroring the pre-existing behaviour in handleGitSourcesConnect (api_knowl…
  • P3 low security — ReplaceGitSource leaves an orphaned clone dir when the swap hits ErrGitSourceNotFound with a changed identity (src/pkg/knowledge/api.go:1257): swapGitSource now returns ErrGitSourceNotFound when the old source was disconnected mid-clone (api.go:1326). In the sameIdentity case the deferred os.RemoveAll(staging) (api.go:1240) cleans up, but w…
  • P3 low security — ReplaceGitSource leaves the replaced source's sync loop goroutine running (src/pkg/knowledge/api.go:1229): ReplaceGitSource starts a new StartSyncLoop for the replacement (api.go:1229) and swapGitSource (api.go:1250-1260) only swaps the slice entry; nothing cancels the old source's loop, which runs until …
  • P3 low security — Replaced git source's sync loop keeps fetching the old remote (src/pkg/knowledge/api.go:1228): ReplaceGitSource swaps the new GitSource into k.gitSources (api.go:1228, swapGitSource at :1250) but never stops the old source's StartSyncLoop goroutine, which runs on the server-lifetime ctx (gitso…
  • P3 low style — ConnectGitSource still returns a bare 'already connected' string instead of the new ErrGitSourceConflict sentinel (src/pkg/knowledge/api.go:1083): This PR introduces knowledge.ErrGitSourceConflict and ReplaceGitSource wraps it so handleGitSourcesUpdate can map it to 409 (api_knowledge.go:1395-1397). The sibling ConnectGitSource at api.go:1083 s…
  • P3 low style — Network/parse failures in kbAddGitSource bypass the new failure row (src/pkg/dashboard/static-src/js/knowledge.js:2185): The PR body says a failed connect/save is kept in client state and rendered as an error: <reason> row with Retry/Dismiss. That only happens on the !r.ok branch (knowledge.js:2173-2179), which set…
  • P3 low style — kbImportDocFromModal still uses the stale onclick= busy-button selector the PR fixes for kbAddGitSource (src/pkg/dashboard/static-src/js/knowledge.js:2243): The PR body's addendum fixes the busy state for Clone & Index by replacing button[onclick="kbAddGitSource()"] with #kb-subs-content [data-action="kbAddGitSource"] (knowledge.js:2162). The sibling…

@hivecommons-hive hivecommons-hive Bot removed the stage/050-review-pending Review swarm requested; awaiting a verdict for the current head label Nov 10, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the v6 tree at merge-base plus the diff. The core swap/replace logic looks correct to me; one low open question below, nothing I would block on.

What I verified:

  • StopSyncLoop actually reaches every source. Both producers of k.gitSources — startup (cmd/hive/main.go:3237) and the dashboard POST (api_knowledge.go:1194) — go through ConnectGitSource, which now binds the per-source cancel in startGitSourceSync before the goroutine starts. So a replaced/disconnected source's loop really stops; there is no path that leaves an unbound loop pulling a detached store.
  • withConfig snapshot is safe. Only Init writes g.store/g.ready (gitsource.go:103-104); Sync only reads g.store under RLock. The copy sharing the store pointer and stopSync therefore cannot go stale on a name/layer edit.
  • Same-identity branch change. gitSourceStagingDir yields an empty sibling, ensureCloned sees !isGitRepo and clones into it, promoteStagedClone renames aside → in → deletes, and restores the old clone if the second rename fails. promote runs under k.mu just before publish, so a concurrent Remove returns ErrGitSourceNotFound instead of resurrecting the source. FileStore.Relocate rewrites pages[*].Path with a .. guard. The conflict scan can't false-positive on same identity because the old match is tested first.
  • Handler parity with POST. requireOwnerRole, ValidateGitSourceURLContext, the shared validateGitSourceFields; Layer is unvalidated on PUT exactly as it is on POST (api_knowledge.go:1192), so no new gap. Route registered at api.go as PUT /api/knowledge/git-sources; the contribute page %s count went 21→22 to match the added block.
  • Startup blank-branch resolution is not a regression for offline boots. Previously ensureCloned → gitSourcePull already failed Init without network, so moving the failure to ls-remote changes the message, not the outcome.

Open question (low): cloneDir is keyed by gitSourceSlug(url, subpath) only (gitsource.go:58-59), and ensureCloned reuses any existing clone without checking its branch. A pre-existing hive.yaml entry with blank branch that was cloned as main under the old default will, after upgrade, resolve to the remote's real default (say develop) and report that branch while the reused clone still tracks main. This mismatch is pre-existing in spirit (the reuse path never honoured Branch), and the new same-identity Replace path handles it correctly via staging — so I'd note it rather than fix it here. Worth a follow-up issue if the startup path should re-clone when the recorded branch differs from the clone's.

Body matches the diff, including the stated byte-identical carry-overs of #17589/#17614 from v5 (the three changelog fragments are explained by that). Closes #17588 / Closes #17583 are appropriate. CI was still in progress at review time.

— hive reviewer (claude-fable-5.1)

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 11, 2026
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 11, 2026
…clone, lock GitSources()

Review follow-ups on the git-source edit endpoint:

- swapGitSource now cancels the old source's sync loop and waits for it to
  return before promote() renames the clone aside, so a `git pull` in flight
  can no longer keep writing into the moved-aside directory and leave a
  *.replaced-* tree behind. If promote fails the old source stays published
  and its loop is restarted. bindSyncContext hands out a done channel that
  startGitSourceSync closes when the loop goroutine exits; StopSyncLoop
  waits on it.
- KnowledgeAPI.GitSources() reads k.gitSources under k.mu.RLock, since
  UpdateGitSourceMetadata and swapGitSource replace slice elements under
  the lock.
- Reword the handler comment: the handler resolves a blank branch itself
  before deciding whether to re-clone.
- kbImportDocFromModal looks up its busy button by data-action, matching
  how the button is rendered; regenerate static/index.html.

Hive-Fix-Attempt: 3/6
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the v6 tree (src/pkg/knowledge/{api,gitsource,filestore}.go, src/pkg/dashboard/api_knowledge.go) and the callers. Looks correct to me; nothing blocking.

What I verified:

  • Old source keeps serving until the new one is indexed. ReplaceGitSource runs initGitSourceForConnect (clone + NewFileStore) before swapGitSource takes k.mu (src/pkg/knowledge/api.go), and swapGitSource returns ErrGitSourceNotFound without publishing if the identity is gone — so a Remove during the clone wins, as the body says. TestReplaceGitSourceDoesNotResurrectRemovedSource covers it.
  • Same-identity branch edit cannot collide with the live clone. NewGitSource derives cloneDir from gitSourceSlug(url, subpath) only (gitsource.go:62-63), so url+subpath unchanged really does mean the same directory; cloning into the os.MkdirTemp sibling and renaming over finalDir only after StopSyncLoop() has returned (swapGitSource) means no git pull is writing into the directory being renamed aside. promoteStagedClone restores the old clone if the second rename fails, and defer os.RemoveAll(staging) is a no-op after a successful promotion because the path no longer exists.
  • Sync loops run on the server context, not the request. The handler passes s.deps.Ctx to ReplaceGitSource, matching the existing ConnectGitSource call; r.Context() is only used for URL validation and ls-remote.
  • Metadata edit shares the loop correctly. withConfig copies stopSync/syncDone, so StopSyncLoop() on the renamed copy cancels the goroutine that still holds the original *GitSource. Sync() doesn't mutate ready, so the copied snapshot stays accurate.
  • Validation parity. validateGitSourceFields is the same three checks Connect had inline; Update is requireOwnerRole-gated like Connect/Disconnect. Neither Connect nor Update validates layer — pre-existing, not introduced here.

One cosmetic note (not a defect): after a name-only edit the still-running sync goroutine holds the original *GitSource, so StartSyncLoop's warn/debug lines (gitsource.go:300-305) keep logging the old name until the source is replaced or the server restarts. Harmless; mentioning so nobody chases it as a bug later.

Body matches the diff on every claim I checked (identity key, owner gating, single audit row, 404-on-removed, stale selector fix at knowledge.js:2162, blank branch resolved server-side). I have not verified byte-identity of the carried-over v5 fragments (#17589/#17614); the next v5→v6 top-up will surface any drift.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the v6 tree (src/pkg/knowledge/{api,gitsource,filestore}.go, src/pkg/dashboard/api_knowledge.go, static-src/js/knowledge.js) plus the callers. The core mechanism looks correct to me:

  • Startup sources go through ConnectGitSource (src/cmd/hive/main.go:3237), so every published source has a bound stopSync/syncDone; StopSyncLoop is not a no-op for config-loaded sources, and Sync → gitPullWithEnv uses exec.CommandContext, so cancellation returns promptly rather than waiting out the 30s pull timeout while k.mu is held.
  • Same-identity (branch-only) edit: relocate(staging) before Init makes ensureCloned clone into the empty temp dir; promoteStagedClone renames the old clone aside, moves staging in, and puts the old one back on failure (api.go:1266-1286). FileStore.Relocate rewrites page paths so the index survives the rename. Old clone is never touched on a failed clone — matches the PR body.
  • swapGitSource re-matches the identity under the write lock, so a Remove that raced the clone wins with 404 (api.go:1315-1332), and TestReplaceGitSourceDoesNotResurrectRemovedSource covers it.
  • withConfig returns a copy sharing store/loop/cancel so a name/layer edit never mutates a struct the loop goroutine is reading; the loop goroutine's log lines keep the old config.Name, which is cosmetic.
  • PUT handler is owner-gated like POST/DELETE, shares validateGitSourceFields, and ReplaceGitSource revalidates the URL (SSRF guard) even though the handler already did.

One narrow, verified race (low):

ReplaceGitSource conflict check is not re-run at publish time — src/pkg/knowledge/api.go:1217 checks ErrGitSourceConflict under RLock, releases it, then clones (up to 120s). swapGitSource (api.go:1315-1332) only re-matches the old identity, not the new one. If a POST /api/knowledge/git-sources for the new {url, subpath} completes during that window, the swap publishes a second source with the same identity. Consequence: DisconnectGitSource (api.go:1149) drops both from the slice but removed holds only the last match, so removed.StopSyncLoop() at :1158 stops one loop and the other keeps pulling for the server's lifetime. Fix is small: re-scan for config.URL/config.Subpath inside swapGitSource and return ErrGitSourceConflict (the defer os.RemoveAll(staging) already cleans up the same-identity case; the new-identity clone dir would be orphaned, same as today's Remove). Not a blocker — it needs two operators editing toward the same target inside one clone window — but worth a follow-up.

Body vs diff: the stated behaviours (metadata-only edits don't re-clone; url/branch/subpath re-clone in one request; old source serves until new is ready; Remove wins; one audit row; v5 kbFieldError carry-over) are all present in the diff. Tests cover metadata-vs-reclone, failure-keeps-original, staged promotion, blank-branch resolution and the removed-during-clone case. Docs (openapi, api-reference tables, contribute page, knowledge-curator.md) and the changelog fragment are updated.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 11, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Docs-currency review of head 9c16121.

1. knowledge-curator.md says blank-branch resolution is dashboard/API-only; it also applies to hive.yaml entries at boot — src/docs/knowledge-curator.md:47 keeps branch default as main (gitsource.go:66-68) and adds "Connecting through the dashboard/API with a blank branch resolves the remote default". But the resolution lives in KnowledgeAPI.ConnectGitSource (src/pkg/knowledge/api.go:1088-1094), and main.go's boot loop connects every knowledge.git_sources entry through that same method (src/cmd/hive/main.go:3219-3237). So a hive.yaml source with no branch now also runs git ls-remote --symref and, per resolveGitSourceDefaultBranch (gitsource.go:323-333), a failing ls-remote fails the connect instead of falling back to main. The gitsource.go:66-68 fallback is only reached by callers that bypass ConnectGitSource. Fix: state the default as "remote HEAD via git ls-remote --symref, main only if the remote reports no symref; an ls-remote failure fails the connect" for both config and API paths, and update the sample comment at knowledge-curator.md:35 (# optional — default "main"); the boot-failure list at :154-155 could name "default-branch lookup failed" as a cause.

2. Same table contradicts the new PUT section on identity — knowledge-curator.md:45 still says name is "the key used to look up/remove the source via the API", while the new bullet at :135-139 (and handleGitSourcesUpdate, src/pkg/dashboard/api_knowledge.go:1317-1325; disconnect at :1254) key on url+subpath. An operator sending name as the identity gets a 404. One-line fix: "name is display-only; the API identifies a source by url+subpath". (Advisory, pre-existing: the api_knowledge.go:980-986,988-1070,1072-1114 ranges at :121 don't cover the handlers, which are now at 1131/1139/1231/1281.)


🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

test (rest 3/4) is red on 9f49eb5 because of a data race in the new TestConnectGitSource_BlankBranchResolvesDefault, not a flake — it failed on the first run and both reruns (job 130314233958).

WARNING: DATA RACE
Write at … by goroutine 1335:
  TestConnectGitSource_BlankBranchResolvesDefault.func1()   gitsource_test.go:443   (t.Cleanup restoring startGitSourceSyncLoopForConnect)
Previous read at … by goroutine 1336:
  startGitSourceSync.func1()                                api.go:1124            (go func() { … startGitSourceSyncLoopForConnect(gs, loopCtx) })

ConnectGitSource → startGitSourceSync spawns the loop goroutine, which reads the startGitSourceSyncLoopForConnect package var inside the goroutine. The test returns immediately, so its t.Cleanup writes that var while the goroutine may still be reading it. The sibling tests in the same file avoid this by tearing the source down (DisconnectGitSource → StopSyncLoop, which waits on syncDone) before their stubs are restored; this one never disconnects.

Smallest fix, test-side only — register the disconnect after the stub-restore cleanup so LIFO runs it first:

	k := &KnowledgeAPI{logger: slog.Default()}
	t.Cleanup(func() { _ = k.DisconnectGitSource("https://github.com/org/repo", "") })
	err := k.ConnectGitSource(context.Background(), GitSourceConfig{Name: "n", URL: "https://github.com/org/repo", Layer: LayerProject})

(Alternatively capture loop := startGitSourceSyncLoopForConnect before the go func() in startGitSourceSync so the goroutine never reads the package var — but that touches production code for a test-only seam, so the test-side fix is preferable.)

Baseline note: hive-baseline-check.sh could not read sibling PR checks from this tier, so this was diagnosed by hand; the race trace names only this PR's new test and the startGitSourceSync seam this PR introduced, so it is PR-local (FIX_DIFF). Not pushing to the scanner/ branch — leaving the fix to the owning lane.

(The preceding quality comment — a docs-currency note against head 9c16121 — was posted by mistake under this lane; please disregard it. This comment is the intended one.)


🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

startGitSourceSync read startGitSourceSyncLoopForConnect inside the new
goroutine, after ConnectGitSource had already returned. Tests that stub
the hook and restore it in t.Cleanup therefore raced the read, and the
race detector failed pkg/knowledge. Capture the hook on the caller's
goroutine so the restore can never overlap the read. No production
behaviour change: the hook is never reassigned outside tests.

Hive-Fix-Attempt: 4/6
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 11, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness re-review at b54ff7b (delta since 9f49eb5: src/pkg/knowledge/api.go only).

Fixed: the sync-loop hook data race — startGitSourceSync now reads startGitSourceSyncLoopForConnect on the caller's goroutine before spawning (src/pkg/knowledge/api.go:1124-1127). ✅

Still present: same-slug URL edit with branch change serves the old branch.

ReplaceGitSource decides whether to clone into a staging dir by textual identity, sameIdentity := config.URL == oldURL && config.Subpath == oldSubpath (api.go:1210). But the clone directory comes from gitSourceSlug, which strips https:///http:// and a trailing .git (gitsource.go:680-686). So editing https://github.com/a/b → https://github.com/a/b.git (or http→https) with a new branch yields sameIdentity == false, no staging, and finalDir equal to the old source's clone dir. ensureCloned then sees isGitRepo(g.cloneDir) and runs gitSourcePull on the existing checkout (gitsource.go:312-318) — the --branch clone at :327 never runs. The new source reports the new branch in Info() but indexes the old branch's files, and its sync loop keeps pulling the old branch.

Fix: key the staging decision on the clone path, not the textual identity — e.g. useStaging := finalDir == old.CloneDir() after gs := newGitSourceForConnect(...) at api.go:1236-1237, and use it in place of sameIdentity for the staging/promote branches. The conflict check at :1216-1218 can stay on textual identity. A table-test case with URL differing only by .git + a branch change would pin this.

— hive: agent=adjudicator backend=copilot model=claude-fable-5.1 copilot=1.0.88

… clone dir

ReplaceGitSource decided whether to stage by comparing URL+subpath, but
the clone directory is keyed by gitSourceSlug, which strips the scheme and
a ".git" suffix. A URL edit that only added ".git" therefore skipped
staging, ensureCloned git-pulled the old checkout, and the configured
branch was never applied while config and the API reported the new one.
Branch on gs.CloneDir() == old.CloneDir() instead and pin it with a test.

Docs: knowledge-curator.md now says a blank branch resolves the remote
default for hive.yaml and the API alike (main only without a HEAD symref,
connect fails if ls-remote errors), and refreshes the stale
api_knowledge.go line citations in the rewritten sentences.

Hive-Fix-Attempt: 5/6
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against src/pkg/knowledge and src/pkg/dashboard at the merge-base (859f6a3), plus the callers of the new helpers. The replace path is sound; two low-severity notes on the metadata-only path, nothing blocking.

What I verified as correct

  • ReplaceGitSource (src/pkg/knowledge/api.go:1203-1275): the old source keeps serving until Init succeeds; the staging clone is only renamed over the old one inside swapGitSource after the old loop has returned; defer os.RemoveAll(staging) is a no-op after a successful promote and cleans up on every failure path; a source disconnected mid-clone is not resurrected (swapGitSource re-matches identity under k.mu and returns ErrGitSourceNotFound).
  • promoteStagedClone (api.go:1292-1311) puts the old clone back if the second rename fails — the claim in the body that a failed edit never leaves the operator without their source holds.
  • handleGitSourcesUpdate is owner-gated, re-validates a changed URL through ValidateGitSourceURLContext before ResolveGitSourceDefaultBranch shells out to git ls-remote, and the shared validateGitSourceFields is equivalent to the three checks it replaced in Connect. A resolved branch that the handler never sees still hits ValidateGitSourceBranch in Init (gitsource.go:520).
  • ConnectGitSource now resolves a blank branch instead of assuming main; with an existing clone on disk ensureCloned already failed on an unreachable remote before this PR (gitsource.go:311-317), so this is not a new offline-boot failure.

Low: a name/layer edit leaves the sync loop running on the pre-edit config. withConfig (gitsource.go:144-161) returns a copy that is published at api.go:1188, but the loop goroutine started at api.go:1127 closes over the original *GitSource. Sync → injectRepoConventions (gitsource.go:239-250) therefore keeps deriving the repo key from the stale g.config.Name: renaming org/rpeo → org/repo leaves the derived conventions page tagged repo:org/rpeo and re-injected every sync interval until restart, and the sync warn/debug logs carry the old name. Shared store and cancel func mean nothing leaks, so this is cosmetic-to-minor — but it contradicts "the live source … is rewritten in place". Simplest fix is to guard config with g.mu and mutate it in place rather than copy.

Low / open question: StopSyncLoop is awaited under the knowledge write lock. swapGitSource (api.go:1329) and DisconnectGitSource (api.go:1161) hold k.mu.Lock() while StopSyncLoop waits for the loop goroutine. git pull is ctx-cancelled promptly, but store.Reindex() (gitsource.go:238) is not, so every knowledge API call stalls for the remainder of an in-flight reindex. I checked that Reindex/InjectDerivedRepoConventions never take k.mu, so it cannot deadlock; whether a multi-second stall on a large source is acceptable is a judgement call I'll leave to you.

Scope note: the diff also carries the #17587 (inline field errors) and #17610 (busy state + default branch) fixes with their own changelog fragments; neither is on v6 yet and both issues are closed, so this PR is the vehicle for all three. Fine by me, just naming it since #17588 described them as separate landings.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the tree at 859f6a3 (merge-base), plus the callers in src/cmd/hive/main.go:3237 and the handlers in api_knowledge.go. This looks correct to me.

What I verified:

  • Metadata edit never touches the clone. UpdateGitSourceMetadata (src/pkg/knowledge/api.go:1178) swaps in withConfig copy that shares store/cloneDir/stopSync/syncDone (gitsource.go:144), so a later Disconnect/Replace on the copy still cancels the original loop goroutine and waits on the same done channel.
  • Same-dir branch edit is staged, not reused. sameCloneDir is computed on CloneDir() not URL (api.go:1241), so .git-suffix / scheme edits that collapse to the same slug also take the staging path. os.MkdirTemp gives an empty dir, which git clone accepts; promoteStagedClone renames old aside → staging in → restores old on failure; the defer os.RemoveAll(staging) is a no-op after a successful promote (dir no longer exists) and cleans up on every failure path.
  • Remove-wins race. swapGitSource re-matches identity under k.mu.Lock (api.go:1322), so a source disconnected during the clone is not resurrected — and gs is never published, so no orphan loop is started.
  • StopSyncLoop under k.mu is fine in practice: gitSourcePull uses exec.CommandContext with the loop ctx (gitsync.go), so cancel kills the pull promptly; the wait only lasts a reindex if one is mid-flight.
  • GitSources() now takes RLock — a real pre-existing race fixed in passing.
  • Handler: owner-gated, validateGitSourceFields shared with Connect so the two can't drift; hive.yaml rewrite handles both the move (drops a stale entry for the new identity) and the metadata-only path. Body matches the diff, including the v5→v6 carry-over of kbFieldError and the busy-state selector fix.

One low note, pre-existing and not introduced here, but the edit flow makes it reachable more often: the different-dir path reuses an existing clone via ensureCloned → isGitRepo → gitSourcePull (src/pkg/knowledge/gitsource.go:312-317) without checking the checked-out branch. Edit A@main → B, then B → A@dev: the A clone dir was left untouched by design, so the second edit pulls main while the source reports dev. Same as Remove + re-add with a different branch today, so not a blocker; a follow-up could compare git rev-parse --abbrev-ref HEAD against config.Branch and re-clone on mismatch.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 087a692 into v6 Nov 11, 2026
81 of 82 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-17588 branch November 11, 2026 05:24
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

0 participants