diff --git a/adr/4173-issue-tracker-seam.md b/adr/4173-issue-tracker-seam.md index 6c594c6bf0..2bb6500cca 100644 --- a/adr/4173-issue-tracker-seam.md +++ b/adr/4173-issue-tracker-seam.md @@ -1,15 +1,15 @@ # ADR-4173: The issue tracker seam -**Status**: Accepted (PRD #4173, M4 landed; M5–M6 pending) -**Date**: 2026-11-11 (updated 2026-11-17) +**Status**: Accepted (PRD #4173, M5 landed; M6 pending) +**Date**: 2026-11-11 (updated 2026-11-18) **Deciders**: the maintainer + agent team -**PRD**: [prds/4173-external-issue-trackers.md](../prds/4173-external-issue-trackers.md) — the PRD carries the milestones and the Decision Log; this ADR carries the durable shape of what M3 and M4 built and the invariants a later change must keep. +**PRD**: [prds/4173-external-issue-trackers.md](../prds/4173-external-issue-trackers.md) — the PRD carries the milestones and the Decision Log; this ADR carries the durable shape of what M3 to M5 built and the invariants a later change must keep. ## Decision (summary) Issue trackers other than the repo's forge sit behind a `Driver` in their own package, `api/internal/issuetracker`. A driver is `forge.IssueTracker` plus four connection operations. Connections live apart from forge connections (`issuetrackersvc`, table `issue_tracker_connections`) with their own base URL allowlist. The YouTrack driver adapts YouTrack's string ids inside itself, is built for exactly one project, and treats tags as user-owned objects chosen by id. From M4 a repo created after the M4 migration can link to one before its first enable (D13); a repo with no link behaves as before. -Shipped in M4: the repo link and `IssueTrackerForRepo` returning the linked issue tracker, with intake and runs from YouTrack issues. Not shipped yet: issue writes and board moves on YouTrack (M5), the web and CLI settings surfaces, docs and the live pass against a real YouTrack (M6). On a linked repo every issue write is refused with `ErrIssueWriteNotOffered`. +Shipped in M4: the repo link and `IssueTrackerForRepo` returning the linked issue tracker, with intake and runs from YouTrack issues. Shipped in M5: the board and issue writes on YouTrack. A card move goes through `MoveToColumn` and writes the issue's YouTrack State, through a per-repo column-to-State mapping set with `GET` and `PUT /api/repos/{id}/issue-tracker-link/states` (session-only, no UI); an unmapped column is never guessed (`ErrColumnNotMapped`, a 409 on a board drag). Close and reopen write the stored closed, open or mapped column States, and never land on a won't-fix State: closability is an allowlist of done-style names, checked when the mapping is saved and again against the live bundle before every State write. A column move refuses an issue YouTrack already shows as resolved unless it is the explicit reopen of a closed card (`ColumnMove.Reopen`), so an automatic move or a board drag never reopens an issue a person resolved there. A linked card's board column is read from its State and cached in `issues.issue_tracker_column`. Issues uzi creates on a linked repo are created in the linked project and carry the eligibility tag where the forge path would carry the uzi label. Comments and the PRD link patch reach the issue. The one issue write still refused is removing the eligibility tag (`ErrIssueWriteNotOffered`); `EnsureRepoLabels` is refused with a 409 before any request. Not shipped yet: the web and CLI settings surfaces (the mapping has no UI), docs, the live pass against a real YouTrack, and the issue detail page's column (all M6). There is no close-on-merge: a linked MR body is non-closing, so the issue stays in its column after the merge until a person closes it. ## Context @@ -59,12 +59,23 @@ Port note (2026-11-17): M4 was rebased onto the final M3. Of the M3 review chang - **S3. Credentials follow the forge rules.** Sealed token with AAD and no reveal path, https-only base URL on the separate empty-by-default allowlist re-checked in `DriverFor`, redirects refused, errors redacted. The redirect, oversized-body and token-in-error cases each have a test in `api/internal/issuetracker`. - **S4. Issue tracker text is untrusted input.** The driver bounds what it reads (response bytes, listing pages and items, nested items, label name length). YouTrack titles, descriptions and comments go through the same `issueinput` fencing as forge text; an oversized description is refused and an oversized comment by the connection's own account is cut to the same bound as on a forge, while an unmapped author's comment is withheld; M4 tests each. - **S5. The worker never holds an issue tracker token.** The allowlist does not feed worker egress, and the token is opened only inside `DriverFor` in the API. The worker read routes go through the API's resolver, so the worker never holds the token. +- **S6. Every YouTrack State write re-checks the live bundle.** The open state and every column state must be unresolved and not archived, the closed state must pass `issuetracker.ClosableState` (an allowlist on the State's `name` alone, never `localizedName`), and all mapped states must be distinct. All of this runs when the mapping is saved (a 400); before every write the target State is checked again in its role, failing with `ErrStateNotAllowed`; an unmapped column fails with `ErrColumnNotMapped` before any request. Do not turn the allowlist into a denylist. +- **S7. The eligibility tag is never stripped.** The write guard refuses any label write on a linked repo that removes the link's eligibility name, and remove-label-on-dispatch compares the selector with the eligibility label. Every other issue write is lifted one operation at a time and passes through `markNotFound`, so a moved-away issue stays skippable. +- **S8. `issues.issue_tracker_column` has three writers.** The linked `UpsertIssueIfChanged`, `SetIssueTrackerColumn` (after a successful move) and `ClearIssueTrackerColumnsNotIn` (a linked `ConfigureColumns`, in the transaction that prunes the mapping). Every reader goes through `board.CardColumnLabels`, keyed on the repo being linked, which leaves an unlinked repo's labels unchanged. +- **S9. `forge.MoveByLabels` keeps the forge label calls identical.** `EnsureLabels` only when `Ensure` is non-empty, then one `UpdateIssueLabels` with the slices passed through. `forgetest.Recorder` logs `MoveToColumn` in a separate `Moves()` log so the `Calls()` goldens do not change. The YouTrack driver ignores the label fields, because columns are not tags. +- **S10. A linked run's MR-state moves happen in the board-free MR-watch lane.** It moves cards only on the close and reopen edges and only for the issue's latest issue-lane run; the board lane excludes linked repos so the two stay disjoint. ## Open questions for M6's live pass - Whether YouTrack accepts `readSharingSettings` and `tagSharingSettings` in the tag-create body. The fake accepts them; no real server has been asked. - Whether an authenticated token receives `usersCount` on a tag's sharing groups (the anonymous recording has none). - Whether the id-scoped listing `GET /api/tags/{id}/issues` behaves on a real instance as the fake and YouTrack's documented shape say. The earlier question, whether `tag: {name}` resolves every tag of that name, is settled: intake sends no tag name in a search query (PRD Decision Log 2026-11-12). If the listing proves unusable, the documented fallback walks the whole project and filters by stored id, bounded by the same listing ceilings. +- Whether the real YouTrack State write (the custom field update on an issue) behaves as the fake and the documented shape say. +- Whether the bounded `State` custom field read with each issue behaves on a real instance, including the cap that fails a listing when the field list is over it. +- Whether the done-name allowlist matches real bundles. A done state named only in another language is refused, so a real project may need its State renamed or the list extended. +- Whether a non-admin token can read `admin/projects/{id}/customFields`, which the live bundle check uses before every State write. +- The window between the bundle check and the State write (as in D10): a bundle edited inside it can still be landed on. +- The duplicate-on-retry risk of a create: YouTrack's create has no rollback, so if the POST lands but attaching the tag or the read-back fails, the untagged issue stays in YouTrack and a retry can file a duplicate. ## Consequences diff --git a/api/internal/board/board.go b/api/internal/board/board.go index c6f1a0cda3..bddb6ae4d1 100644 --- a/api/internal/board/board.go +++ b/api/internal/board/board.go @@ -41,6 +41,22 @@ func ResolveColumn(labels []string, state string, position map[string]int) (colu return best, false, matches > 1 } +// CardColumnLabels returns the labels a column-inferring ResolveColumn call +// should read. A card of a repo linked to an issue tracker sits in the column +// cached from its State (issues.issue_tracker_column), never in a label, because +// columns are not tags there (D9, D14): it returns just that column, or nil for +// the Open lane (column ""). On any other repo it returns labels unchanged. +// linked is the repo's linkage, never a property of one issue's key. +func CardColumnLabels(linked bool, labels []string, column string) []string { + if !linked { + return labels + } + if column == "" { + return nil + } + return []string{column} +} + // PlanLabelMove computes the atomic add/remove sets for a single-column move and // the resulting label set for the cache. add is the target column (if not // already present); remove is every OTHER column label the issue currently diff --git a/api/internal/board/board_test.go b/api/internal/board/board_test.go index e8539d06ff..213c131ec5 100644 --- a/api/internal/board/board_test.go +++ b/api/internal/board/board_test.go @@ -128,3 +128,43 @@ func TestPlanLabelMove(t *testing.T) { }) } } + +func TestCardColumnLabels(t *testing.T) { + labels := []string{"uzi", "Planned"} + tests := []struct { + name string + linked bool + labels []string + column string + want []string + }{ + {"unlinked keeps the labels", false, labels, "", labels}, + {"unlinked ignores a stray cached column", false, labels, "In Progress", labels}, + {"unlinked nil stays nil", false, nil, "", nil}, + {"linked reads the cached column only", true, labels, "In Progress", []string{"In Progress"}}, + {"linked with no cached column is the Open lane", true, labels, "", nil}, + {"linked never reads a column label", true, []string{"Planned"}, "", nil}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + got := CardColumnLabels(tc.linked, tc.labels, tc.column) + if len(got) != len(tc.want) || (tc.want == nil) != (got == nil) { + t.Fatalf("CardColumnLabels = %#v, want %#v", got, tc.want) + } + for i := range got { + if got[i] != tc.want[i] { + t.Fatalf("CardColumnLabels = %#v, want %#v", got, tc.want) + } + } + }) + } + // The unlinked result is the input slice itself, so ResolveColumn sees identical input. + if got := CardColumnLabels(false, labels, "x"); &got[0] != &labels[0] { + t.Fatal("an unlinked repo must get its labels back unchanged") + } + pos := map[string]int{"In Progress": 1, "Planned": 0} + col, closed, conflict := ResolveColumn(CardColumnLabels(true, labels, "In Progress"), "opened", pos) + if col != "In Progress" || closed || conflict { + t.Fatalf("ResolveColumn = %q %v %v", col, closed, conflict) + } +} diff --git a/api/internal/forge/forgejo_issues.go b/api/internal/forge/forgejo_issues.go index 24441f4b77..7665792bbb 100644 --- a/api/internal/forge/forgejo_issues.go +++ b/api/internal/forge/forgejo_issues.go @@ -132,6 +132,11 @@ func (f *forgejo) UpdateIssueDescription(ctx context.Context, projectID, issueII }{Body: description}) } +// MoveToColumn moves the issue to a board column through its labels. +func (f *forgejo) MoveToColumn(ctx context.Context, projectID, issueIID int64, move ColumnMove) error { + return MoveByLabels(ctx, f, projectID, issueIID, move) +} + // SetIssueState closes or reopens an issue. // // This sends a STATE-ONLY PATCH and does NOT read the issue first, for the same diff --git a/api/internal/forge/forgetest/basefake.go b/api/internal/forge/forgetest/basefake.go index 2ca82d7684..6030843428 100644 --- a/api/internal/forge/forgetest/basefake.go +++ b/api/internal/forge/forgetest/basefake.go @@ -103,6 +103,11 @@ func (*BaseFake) UpdateIssueLabels(context.Context, int64, int64, []string, []st return notStubbed("UpdateIssueLabels") } +// MoveToColumn implements forge.Forge. +func (*BaseFake) MoveToColumn(context.Context, int64, int64, forge.ColumnMove) error { + return notStubbed("MoveToColumn") +} + // UpdateIssueDescription implements forge.Forge. func (*BaseFake) UpdateIssueDescription(context.Context, int64, int64, string) error { return notStubbed("UpdateIssueDescription") diff --git a/api/internal/forge/forgetest/basefake_test.go b/api/internal/forge/forgetest/basefake_test.go index 5c96230349..6a7cecaf63 100644 --- a/api/internal/forge/forgetest/basefake_test.go +++ b/api/internal/forge/forgetest/basefake_test.go @@ -58,6 +58,7 @@ func actionMethods() []struct { return err }}, {"UpdateIssueLabels", func(b *BaseFake) error { return b.UpdateIssueLabels(ctx, 1, 2, nil, nil) }}, + {"MoveToColumn", func(b *BaseFake) error { return b.MoveToColumn(ctx, 1, 2, forge.ColumnMove{}) }}, {"UpdateIssueDescription", func(b *BaseFake) error { return b.UpdateIssueDescription(ctx, 1, 2, "d") }}, {"SetIssueState", func(b *BaseFake) error { return b.SetIssueState(ctx, 1, 2, forge.StateClosed) }}, {"UserExists", func(b *BaseFake) error { _, err := b.UserExists(ctx, "u"); return err }}, @@ -166,8 +167,8 @@ func actionMethods() []struct { func TestBaseFakeActionMethodsNotStubbed(t *testing.T) { methods := actionMethods() - if len(methods) != 37 { - t.Fatalf("expected 37 action methods, got %d", len(methods)) + if len(methods) != 38 { + t.Fatalf("expected 38 action methods, got %d", len(methods)) } b := &BaseFake{} for _, m := range methods { diff --git a/api/internal/forge/forgetest/recorder.go b/api/internal/forge/forgetest/recorder.go index b16a6e8cfc..7e864a670b 100644 --- a/api/internal/forge/forgetest/recorder.go +++ b/api/internal/forge/forgetest/recorder.go @@ -16,9 +16,15 @@ import ( // the method name plus its arguments, formatted by formatArg. It exists so a // test can pin the exact set of forge calls a code path makes (a golden // record, see AssertGolden) and notice when a later refactor changes it. -// Only calls are recorded, never results; the wrapped Forge supplies those. +// Only calls are recorded, never results; the wrapped Forge supplies the +// results of every method except MoveToColumn. // -// Safe for concurrent use. Calls() keeps call order. +// MoveToColumn is the exception: its line goes to Moves(), not Calls(), and it +// runs forge.MoveByLabels over the Recorder, so the wrapped Forge's own +// MoveToColumn is never called; only the EnsureLabels and UpdateIssueLabels +// calls reach it. +// +// Safe for concurrent use. Calls() and Moves() keep call order. // // Recorder implements only forge.Forge. It hides any optional interface the // wrapped value also implements (for example ProjectBoardSyncer), so code that @@ -27,8 +33,9 @@ import ( type Recorder struct { inner forge.Forge - mu sync.Mutex // guards calls + mu sync.Mutex // guards calls and moves calls []string + moves []string } var _ forge.Forge = (*Recorder)(nil) @@ -44,17 +51,43 @@ func (r *Recorder) Calls() []string { return append([]string(nil), r.calls...) } -func (r *Recorder) record(method string, args ...any) { +// Moves returns a copy of the recorded MoveToColumn lines in call order. They are +// kept apart from Calls so the recorded-call goldens stay byte-identical. +func (r *Recorder) Moves() []string { + r.mu.Lock() + defer r.mu.Unlock() + return append([]string(nil), r.moves...) +} + +func formatCall(method string, args ...any) string { parts := make([]string, len(args)) for i, a := range args { parts[i] = formatArg(reflect.ValueOf(a)) } - line := method + "(" + strings.Join(parts, ", ") + ")" + return method + "(" + strings.Join(parts, ", ") + ")" +} + +func (r *Recorder) record(method string, args ...any) { + line := formatCall(method, args...) r.mu.Lock() r.calls = append(r.calls, line) r.mu.Unlock() } +// MoveToColumn records the move in Moves, not Calls, then performs the +// label-backed move over the Recorder itself, so the EnsureLabels and +// UpdateIssueLabels calls land in Calls and are delegated to the wrapped Forge +// exactly as before. It never calls the wrapped Forge's MoveToColumn: the +// recorded-call goldens stay byte-identical, and the Recorder records the +// label-backed move. +func (r *Recorder) MoveToColumn(ctx context.Context, a0 int64, a1 int64, move forge.ColumnMove) error { + line := formatCall("MoveToColumn", a0, a1, move) + r.mu.Lock() + r.moves = append(r.moves, line) + r.mu.Unlock() + return forge.MoveByLabels(ctx, r, a0, a1, move) +} + var timeType = reflect.TypeOf(time.Time{}) // formatArg renders v deterministically: times as RFC3339 UTC (zero time as diff --git a/api/internal/forge/forgetest/recorder_test.go b/api/internal/forge/forgetest/recorder_test.go index 6ecb6f03bd..d0d799f32d 100644 --- a/api/internal/forge/forgetest/recorder_test.go +++ b/api/internal/forge/forgetest/recorder_test.go @@ -61,3 +61,55 @@ func TestAssertGoldenWritesThenMatches(t *testing.T) { } func reflectValue(v any) reflect.Value { return reflect.ValueOf(v) } + +// moveInner stubs the label writes and fails the test if the Recorder ever +// forwards MoveToColumn to it. +type moveInner struct { + BaseFake + t *testing.T + underlying []string +} + +func (m *moveInner) EnsureLabels(context.Context, int64, []forge.Label) error { + m.underlying = append(m.underlying, "EnsureLabels") + return nil +} + +func (m *moveInner) UpdateIssueLabels(context.Context, int64, int64, []string, []string) error { + m.underlying = append(m.underlying, "UpdateIssueLabels") + return nil +} + +func (m *moveInner) MoveToColumn(context.Context, int64, int64, forge.ColumnMove) error { + m.t.Error("Recorder forwarded MoveToColumn to the wrapped Forge") + return nil +} + +func TestRecorderMoveToColumnRecordsTheLabelBackedMove(t *testing.T) { + inner := &moveInner{t: t} + rec := NewRecorder(inner) + move := forge.ColumnMove{ + Column: "In Progress", + Ensure: []forge.Label{{Name: "In Progress", Color: "#123456"}}, + Add: []string{"In Progress"}, + Remove: []string{"Planned"}, + } + if err := rec.MoveToColumn(context.Background(), 42, 7, move); err != nil { + t.Fatal(err) + } + + wantMoves := []string{`MoveToColumn(42, 7, {Column:"In Progress" Ensure:[{Name:"In Progress" Color:"#123456"}] Add:["In Progress"] Remove:["Planned"] Reopen:false})`} + if got := rec.Moves(); !slices.Equal(got, wantMoves) { + t.Fatalf("moves = %q, want %q", got, wantMoves) + } + wantCalls := []string{ + `EnsureLabels(42, [{Name:"In Progress" Color:"#123456"}])`, + `UpdateIssueLabels(42, 7, ["In Progress"], ["Planned"])`, + } + if got := rec.Calls(); !slices.Equal(got, wantCalls) { + t.Fatalf("calls = %q, want %q", got, wantCalls) + } + if want := []string{"EnsureLabels", "UpdateIssueLabels"}; !slices.Equal(inner.underlying, want) { + t.Fatalf("wrapped calls = %v, want %v", inner.underlying, want) + } +} diff --git a/api/internal/forge/github_issues.go b/api/internal/forge/github_issues.go index ec289976b8..16150bdc54 100644 --- a/api/internal/forge/github_issues.go +++ b/api/internal/forge/github_issues.go @@ -134,6 +134,11 @@ func (g *github) UpdateIssueDescription(ctx context.Context, projectID, issueIID return nil } +// MoveToColumn moves the issue to a board column through its labels. +func (g *github) MoveToColumn(ctx context.Context, projectID, issueIID int64, move ColumnMove) error { + return MoveByLabels(ctx, g, projectID, issueIID, move) +} + // SetIssueState closes or reopens an issue. go-github's UpdateIssueRequest // leaves Title/Body nil (unchanged) and Labels/Assignees omitzero (unchanged) // when only State and StateReason are set, so this sends the state ALONE and diff --git a/api/internal/forge/gitlab.go b/api/internal/forge/gitlab.go index 39dbd98048..1a94631959 100644 --- a/api/internal/forge/gitlab.go +++ b/api/internal/forge/gitlab.go @@ -431,6 +431,11 @@ func (g *gitLab) UpdateIssueLabels(ctx context.Context, projectID, issueIID int6 return nil } +// MoveToColumn moves the issue to a board column through its labels. +func (g *gitLab) MoveToColumn(ctx context.Context, projectID, issueIID int64, move ColumnMove) error { + return MoveByLabels(ctx, g, projectID, issueIID, move) +} + // SetIssueState closes or reopens an issue via GitLab's state_event verb. // gitlab.UpdateIssueOptions.StateEvent is a *string with `omitempty` and takes // the MUTATE vocabulary "close"/"reopen" (not the query-filter opened/closed the diff --git a/api/internal/forge/issuetracker.go b/api/internal/forge/issuetracker.go index 9a74fc4c5b..63fa9c04b7 100644 --- a/api/internal/forge/issuetracker.go +++ b/api/internal/forge/issuetracker.go @@ -113,6 +113,39 @@ type ListIssuesOptions struct { Limit int } +// ColumnMove is one board move of an issue, carrying both the target column and +// the label delta a label-backed issue tracker applies to reach it. +type ColumnMove struct { + // Column is the target board column label. The empty string is the implicit + // Open column. + Column string + // Ensure is the labels to ensure exist before the move; empty means no + // EnsureLabels call is made. + Ensure []Label + // Add and Remove are the label delta, exactly board.PlanLabelMove's output. + Add []string + Remove []string + // Reopen is set only by an explicit reopen of a closed card. Any other + // move (an automatic one, or a drag of a card the cache shows open) leaves it + // false, so an issue tracker that keeps its own resolved state refuses to + // write over an issue a person resolved there. A + // label-backed forge ignores it: a forge label move never reopens. + Reopen bool +} + +// MoveByLabels is the label-backed implementation of IssueTracker.MoveToColumn: +// EnsureLabels (only when move.Ensure is non-empty), then exactly one +// UpdateIssueLabels with move.Add and move.Remove passed through untouched, so a +// nil slice stays nil. +func MoveByLabels(ctx context.Context, tr IssueTracker, projectID, issueIID int64, move ColumnMove) error { + if len(move.Ensure) > 0 { + if err := tr.EnsureLabels(ctx, projectID, move.Ensure); err != nil { + return err + } + } + return tr.UpdateIssueLabels(ctx, projectID, issueIID, move.Add, move.Remove) +} + // IssueTracker is the issue-side half of the driver contract: issues, labels, // comments and issue state. It is split out of Forge (PRD #4173) so a later phase // can let a non-forge issue tracker implement it without the code-host methods. @@ -140,6 +173,11 @@ type IssueTracker interface { // GitLab this is atomic (a single add_labels/remove_labels update); // single-column enforcement relies on that atomicity. UpdateIssueLabels(ctx context.Context, projectID, issueIID int64, add, remove []string) error + // MoveToColumn moves an issue to the board column move.Column. A label-backed + // issue tracker (GitLab, GitHub, Forgejo) applies the label delta in move via + // MoveByLabels; an issue tracker with its own workflow field (YouTrack) writes + // the value mapped to move.Column and ignores the label delta. + MoveToColumn(ctx context.Context, projectID, issueIID int64, move ColumnMove) error // UpdateIssueDescription replaces an issue's description wholesale (PRD #72 M5). // It is a read-modify-write from the CALLER's point of view — the caller reads // via GetIssue, rewrites, and writes back — because no forge here offers a patch diff --git a/api/internal/forge/move_to_column_test.go b/api/internal/forge/move_to_column_test.go new file mode 100644 index 0000000000..5d30b3fb8e --- /dev/null +++ b/api/internal/forge/move_to_column_test.go @@ -0,0 +1,215 @@ +package forge + +import ( + "context" + "errors" + "io" + "net/http" + "net/http/httptest" + "reflect" + "strings" + "sync" + "testing" + "time" +) + +// moveWire is an httptest server that records every request as +// "METHOD request-uri body" and answers from a stateless per-dialect responder, +// so two fresh drivers fed the same calls see the same wire. +type moveWire struct { + mu sync.Mutex + seen []string + srv *httptest.Server +} + +func newMoveWire(t *testing.T, respond func(w http.ResponseWriter, r *http.Request)) *moveWire { + t.Helper() + m := &moveWire{} + m.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + m.mu.Lock() + m.seen = append(m.seen, r.Method+" "+r.URL.RequestURI()+" "+string(body)) + m.mu.Unlock() + w.Header().Set("Content-Type", "application/json") + respond(w, r) + })) + t.Cleanup(m.srv.Close) + return m +} + +func (m *moveWire) requests() []string { + m.mu.Lock() + defer m.mu.Unlock() + return append([]string(nil), m.seen...) +} + +func respondGitLab(w http.ResponseWriter, r *http.Request) { + switch { + case r.Method == http.MethodGet && strings.HasSuffix(r.URL.Path, "/labels"): + _, _ = io.WriteString(w, `[]`) + case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/labels"): + _, _ = io.WriteString(w, `{"id":1,"name":"x"}`) + default: + _, _ = io.WriteString(w, `{"id":1,"iid":5,"labels":[]}`) + } +} + +func respondGitHub(w http.ResponseWriter, r *http.Request) { + switch { + case r.URL.Path == "/api/v3/repositories/7": + _, _ = io.WriteString(w, `{"id":7,"name":"widgets","owner":{"id":1,"login":"acme"}}`) + case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/labels") && !strings.Contains(r.URL.Path, "/issues/"): + _, _ = io.WriteString(w, `{"id":1,"name":"x"}`) + default: + _, _ = io.WriteString(w, `[]`) + } +} + +func respondForgejo(w http.ResponseWriter, r *http.Request) { + switch { + case strings.HasSuffix(r.URL.Path, "/version"): + _, _ = io.WriteString(w, `{"version":"11.0.0"}`) + case r.URL.Path == "/api/v1/repositories/7": + _, _ = io.WriteString(w, `{"id":7,"name":"widgets","owner":{"id":1,"login":"acme"}}`) + case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/labels") && !strings.Contains(r.URL.Path, "/issues/"): + _, _ = io.WriteString(w, `{"id":9,"name":"x"}`) + case r.Method == http.MethodGet && strings.Contains(r.URL.Path, "/issues/") && strings.HasSuffix(r.URL.Path, "/labels"): + _, _ = io.WriteString(w, `[{"id":3,"name":"Planned"}]`) + default: + _, _ = io.WriteString(w, `[]`) + } +} + +func TestMoveToColumnWireEqualsEnsureThenUpdate(t *testing.T) { + dialects := []struct { + name string + typ Type + respond func(http.ResponseWriter, *http.Request) + }{ + {"gitlab", TypeGitLab, respondGitLab}, + {"github", TypeGitHub, respondGitHub}, + {"forgejo", TypeForgejo, respondForgejo}, + } + moves := []struct { + name string + move ColumnMove + }{ + {"full move", ColumnMove{ + Column: "In Progress", + Ensure: []Label{{Name: "In Progress", Color: "#123456"}}, + Add: []string{"In Progress"}, + Remove: []string{"Planned"}, + }}, + {"move to Open: no ensure, nil add", ColumnMove{Column: "", Remove: []string{"Planned"}}}, + {"nothing to do", ColumnMove{Column: "Planned"}}, + } + for _, d := range dialects { + for _, mv := range moves { + t.Run(d.name+"/"+mv.name, func(t *testing.T) { + ctx := context.Background() + const p, i = int64(7), int64(5) + fresh := func() (Forge, *moveWire) { + w := newMoveWire(t, d.respond) + drv, err := New(d.typ, w.srv.URL, "tok", 5*time.Second) + if err != nil { + t.Fatalf("New: %v", err) + } + return drv, w + } + + viaMove, wireMove := fresh() + if err := viaMove.MoveToColumn(ctx, p, i, mv.move); err != nil { + t.Fatalf("MoveToColumn: %v", err) + } + + viaParts, wireParts := fresh() + // The drivers' EnsureLabels lists the project's labels even for an + // empty set, so an empty Ensure means no EnsureLabels call at all + // (the pre-MoveToColumn AutoMove skipped it the same way). + if len(mv.move.Ensure) > 0 { + if err := viaParts.EnsureLabels(ctx, p, mv.move.Ensure); err != nil { + t.Fatalf("EnsureLabels: %v", err) + } + } + if err := viaParts.UpdateIssueLabels(ctx, p, i, mv.move.Add, mv.move.Remove); err != nil { + t.Fatalf("UpdateIssueLabels: %v", err) + } + + got, want := wireMove.requests(), wireParts.requests() + if !reflect.DeepEqual(got, want) { + t.Fatalf("MoveToColumn wire differs from EnsureLabels+UpdateIssueLabels\n got: %q\nwant: %q", got, want) + } + }) + } + } +} + +// spyTracker records the IssueTracker calls MoveByLabels makes. +type spyTracker struct { + IssueTracker // nil: any other method would panic + calls []string + ensure []Label + add, remove []string + ensureErr error +} + +func (s *spyTracker) EnsureLabels(_ context.Context, _ int64, l []Label) error { + s.calls = append(s.calls, "EnsureLabels") + s.ensure = l + return s.ensureErr +} + +func (s *spyTracker) UpdateIssueLabels(_ context.Context, _, _ int64, add, remove []string) error { + s.calls = append(s.calls, "UpdateIssueLabels") + s.add, s.remove = add, remove + return nil +} + +func TestMoveByLabels(t *testing.T) { + ctx := context.Background() + t.Run("ensure then one update", func(t *testing.T) { + s := &spyTracker{} + m := ColumnMove{Ensure: []Label{{Name: "a"}}, Add: []string{"a"}, Remove: []string{"b"}} + if err := MoveByLabels(ctx, s, 1, 2, m); err != nil { + t.Fatal(err) + } + if want := []string{"EnsureLabels", "UpdateIssueLabels"}; !reflect.DeepEqual(s.calls, want) { + t.Fatalf("calls = %v, want %v", s.calls, want) + } + if !reflect.DeepEqual(s.add, []string{"a"}) || !reflect.DeepEqual(s.remove, []string{"b"}) { + t.Fatalf("add/remove = %v/%v", s.add, s.remove) + } + }) + t.Run("no ensure call when Ensure is empty, nil stays nil", func(t *testing.T) { + s := &spyTracker{} + if err := MoveByLabels(ctx, s, 1, 2, ColumnMove{Remove: []string{"b"}}); err != nil { + t.Fatal(err) + } + if want := []string{"UpdateIssueLabels"}; !reflect.DeepEqual(s.calls, want) { + t.Fatalf("calls = %v, want %v", s.calls, want) + } + if s.add != nil { + t.Fatalf("add = %#v, want nil", s.add) + } + }) + t.Run("empty but non-nil slices stay non-nil", func(t *testing.T) { + s := &spyTracker{} + if err := MoveByLabels(ctx, s, 1, 2, ColumnMove{Add: []string{}, Remove: []string{}}); err != nil { + t.Fatal(err) + } + if s.add == nil || s.remove == nil { + t.Fatalf("add/remove = %#v/%#v, want non-nil", s.add, s.remove) + } + }) + t.Run("ensure failure stops before the update", func(t *testing.T) { + boom := errors.New("boom") + s := &spyTracker{ensureErr: boom} + err := MoveByLabels(ctx, s, 1, 2, ColumnMove{Ensure: []Label{{Name: "a"}}, Add: []string{"a"}}) + if !errors.Is(err, boom) { + t.Fatalf("err = %v, want boom", err) + } + if want := []string{"EnsureLabels"}; !reflect.DeepEqual(s.calls, want) { + t.Fatalf("calls = %v, want %v", s.calls, want) + } + }) +} diff --git a/api/internal/forgesvc/board_columns.go b/api/internal/forgesvc/board_columns.go index 4102f73f4f..9505905ccd 100644 --- a/api/internal/forgesvc/board_columns.go +++ b/api/internal/forgesvc/board_columns.go @@ -31,8 +31,9 @@ func (e *LabelSeedError) Unwrap() error { return e.Err } // EnsureBoardColumnRows is the store-only half of EnsureBoardColumns, for a repo linked // to an external issue tracker (PRD #4173 M4): it records the DefaultColumns rows when -// the repo has none and never calls an issue tracker, because uzi creates no labels or -// tags there (writes arrive in M5). Same one-statement insert, same idempotence. +// the repo has none and never calls an issue tracker, because a column is not a tag +// there (D9; a card move writes the State) and no column tags are wanted. Same +// one-statement insert, same idempotence. func EnsureBoardColumnRows(ctx context.Context, q BoardColumnStore, repoID uuid.UUID) error { count, err := q.CountBoardColumns(ctx, repoID) if err != nil { diff --git a/api/internal/forgesvc/board_free_mr_watch.go b/api/internal/forgesvc/board_free_mr_watch.go index 005e2ee9d9..5a8a6145b2 100644 --- a/api/internal/forgesvc/board_free_mr_watch.go +++ b/api/internal/forgesvc/board_free_mr_watch.go @@ -17,7 +17,10 @@ import ( // card: these runs are issue-less (no card) or share a non-board-promoted tracking issue. It // only records runs.mr_state (via the existing recordMRState, the sole SQL writer — invariant // preserved) and, on the opened->closed / ->merged edge, cancels an in-flight rework via the -// existing cancelReworkOnClosedMR (issue #853 contract, reused verbatim). Reads the run set +// existing cancelReworkOnClosedMR (issue #853 contract, reused verbatim). It is +// SyncBoardFreeMRStatesWithIssueSide with no issue tracker, which is the whole of its +// behaviour on an unlinked repo; it is the tr=nil seam kept for the board-free tests and +// equals the pre-M5 behaviour. Reads the run set // from ListBoardFreeMRStateWatchCandidates, which self-evicts a run once its mr_state is // terminal, except a linked repo's closed issue-lane run (below). // @@ -33,14 +36,59 @@ import ( // Like SyncMRStates it never returns a per-candidate error (log-and-skip); only a failure // to enumerate candidates surfaces to the poller. func (s *Service) SyncBoardFreeMRStates(ctx context.Context, repoID uuid.UUID, forgeProjectID int64, f forge.CodeHost) error { + return s.SyncBoardFreeMRStatesWithIssueSide(ctx, repoID, forgeProjectID, f, nil) +} + +// SyncBoardFreeMRStatesWithIssueSide is SyncBoardFreeMRStates for a repo whose issue side +// may be a linked issue tracker (PRD #4173 M5). When tr is a real linked issue tracker, a candidate +// that is an issue-lane run (issue_iid set, kind not prompt/self_improve) is reconciled +// through syncOneMRState, the board lane's edge switch, so a linked card moves Human Review +// -> In Progress when its MR closes and back when it reopens, exactly as an unlinked card +// does. The card moves only for the issue's latest such run (the board lane's DISTINCT ON +// semantics): a superseded run records its MR state with no move. A transient error from the +// real linked driver defers the edge and retries it next tick, like the board lane. +// +// Every other candidate goes through recordBoardFreeMRState unchanged: that is every +// candidate when tr is nil or is not a linked issue tracker (an unlinked repo's forge driver, whose +// issue-lane runs the candidate query keeps out anyway), and every candidate when tr is the +// UnavailableIssueTracker the poller hands a linked repo whose resolver failed. Recording +// consumes the close edge (mr_state advances, so no later tick sees it and the card is +// never moved for it). That is the trade-off: the cause can be permanent (a connection +// removed from the allowlist), and deferring would leave mr_state opened and the rework gate +// open on a closed MR forever, so the edge is recorded without a move. The one exception is a +// failed repo-source database read (errRepoSourceRead), which is transient: the linked +// issue-lane candidates are skipped this tick, neither recorded nor moved, and retried next +// tick. Candidates that are not linked issue-lane runs are recorded as usual. +// +// A missing cached issue row skips the move (the MR state is still recorded) rather than +// deferring it: this lane's candidates do not JOIN the issue cache, so deferring would poll +// an evicted issue's MR forever. A move the issue tracker refuses (ColumnMoveRefused) is +// likewise consumed without a move; see guardedMRMove. +func (s *Service) SyncBoardFreeMRStatesWithIssueSide(ctx context.Context, repoID uuid.UUID, forgeProjectID int64, f forge.CodeHost, tr forge.IssueTracker) error { candidates, err := s.q.ListBoardFreeMRStateWatchCandidates(ctx, repoID) if err != nil { return err } + movesCard := tr != nil && isLinkedTracker(tr) && !isUnavailableTracker(tr) + deferLinked := tr != nil && isTransientUnavailableTracker(tr) + deferLogged := false for _, c := range candidates { if ctx.Err() != nil { return ctx.Err() } + if movesCard && c.IssueIid.Valid && c.Kind != "prompt" && c.Kind != "self_improve" { + s.syncOneMRState(ctx, repoID, forgeProjectID, f, tr, mrWatchCandidate{ + RunID: c.ID, IssueIID: c.IssueIid.Int64, MrIID: c.MrIid.Int64, MrState: c.MrState, + }, mrLane{missingRow: moveSkipped, noMove: !c.IsLatest}) + continue + } + if deferLinked && c.IssueIid.Valid && c.Kind != "prompt" && c.Kind != "self_improve" { + if !deferLogged { + slog.Warn("forgesvc: repo issue source unreadable, deferring linked MR edges to the next tick", "repo", repoID) + deferLogged = true + } + continue + } s.recordBoardFreeMRState(ctx, repoID, forgeProjectID, f, c) } return nil @@ -48,7 +96,8 @@ func (s *Service) SyncBoardFreeMRStates(ctx context.Context, repoID uuid.UUID, f // recordBoardFreeMRState reconciles one board-free run's MR state against the stored value, // board-free. It is a second implementation of syncOneMRState's observe->persist->cancel -// contract WITHOUT the board move; the two must agree on: unknown-state -> no write; the +// contract WITHOUT the board move (a linked repo's issue-lane runs use syncOneMRState itself, +// see SyncBoardFreeMRStatesWithIssueSide); the two must agree on: unknown-state -> no write; the // NULL bootstrap records without acting; opened->closed and ->merged cancel exactly once; // locked is transient (no cancel); a forge read failure or a cancel failure leaves mr_state // unadvanced so the next tick retries. diff --git a/api/internal/forgesvc/close_reopen.go b/api/internal/forgesvc/close_reopen.go index 4c391b2b75..0c696bab4e 100644 --- a/api/internal/forgesvc/close_reopen.go +++ b/api/internal/forgesvc/close_reopen.go @@ -55,7 +55,12 @@ func (s *Service) CloseIssue(ctx context.Context, f forge.IssueTracker, forgePro // Partial failure: if the move half fails, the reopened-but-unmoved row (state opened, // board_position nulled) is returned with the error so the handler can surface a 502 // for the move while the card still renders open — never lost. +// +// A linked repo reopens through reopenLinkedIssue instead, with a single write. func (s *Service) ReopenIssue(ctx context.Context, f forge.IssueTracker, forgeProjectID int64, issue store.Issue, columns []store.BoardColumn, target string) (store.Issue, error) { + if isLinkedTracker(f) { + return s.reopenLinkedIssue(ctx, f, forgeProjectID, issue, target) + } if issue.State != string(forge.StateOpened) { // Forge-first: reopen remotely before touching the cache. On failure the cache // is untouched (no ReopenIssueState, no UpsertIssueLabels) and the card snaps back. @@ -81,3 +86,39 @@ func (s *Service) ReopenIssue(ctx context.Context, f forge.IssueTracker, forgePr } return moved, nil } + +// reopenLinkedIssue reopens an issue of a repo linked to an issue tracker (PRD +// #4173 M5). It calls MoveToColumn directly and makes no separate SetIssueState: the +// driver checks the column mapping and the State bundle before its single State write, +// and every column State is unresolved, so that one write reopens the issue. A refused +// or failed move therefore changes neither the issue tracker nor the cache. Only on +// success does the cache change: ReopenIssueState (state opened, board_position +// nulled), then SetIssueTrackerColumn. A failure between the two statements leaves +// the row opened in the Open lane, the same partial-failure contract as the unlinked +// reopen. The target "" is the Open column and uses the stored open State. The move +// sets Reopen, so the driver writes over an issue resolved in the issue tracker; an +// automatic move leaves it false and is refused for such an issue. +func (s *Service) reopenLinkedIssue(ctx context.Context, f forge.IssueTracker, forgeProjectID int64, issue store.Issue, target string) (store.Issue, error) { + if err := f.MoveToColumn(ctx, forgeProjectID, issue.ForgeIssueIid, forge.ColumnMove{Column: target, Reopen: true}); err != nil { + return store.Issue{}, err + } + if issue.State != string(forge.StateOpened) { + reopened, err := s.q.ReopenIssueState(ctx, store.ReopenIssueStateParams{ + RepoID: issue.RepoID, + ForgeIssueIid: issue.ForgeIssueIid, + }) + if err != nil { + return store.Issue{}, err + } + issue = reopened + } + moved, err := s.q.SetIssueTrackerColumn(ctx, store.SetIssueTrackerColumnParams{ + RepoID: issue.RepoID, ForgeIssueIid: issue.ForgeIssueIid, IssueTrackerColumn: target, + }) + if err != nil { + // Reopened-but-unplaced: return the opened row with the error, as the unlinked + // reopen does for its move half. + return issue, err + } + return moved, nil +} diff --git a/api/internal/forgesvc/issue_key_livedb_test.go b/api/internal/forgesvc/issue_key_livedb_test.go index 4b06b8bc89..94767d9292 100644 --- a/api/internal/forgesvc/issue_key_livedb_test.go +++ b/api/internal/forgesvc/issue_key_livedb_test.go @@ -134,21 +134,21 @@ func TestLinkedSyncWritesOnlyChangedRowsLiveDB(t *testing.T) { } is := issueAt(5, recordedMarkBase) is.Title = "first" - if err := svc.upsertIssues(ctx, repoID, []forge.Issue{is}, "ABC"); err != nil { + if err := svc.upsertIssues(ctx, repoID, []forge.Issue{is}, "ABC", nil); err != nil { t.Fatal(err) } title, key, first := read() if title != "first" || key != "ABC-5" { t.Fatalf("inserted row = %q, %q; want first, ABC-5", title, key) } - if err := svc.upsertIssues(ctx, repoID, []forge.Issue{is}, "ABC"); err != nil { + if err := svc.upsertIssues(ctx, repoID, []forge.Issue{is}, "ABC", nil); err != nil { t.Fatal(err) } if _, _, again := read(); !again.Equal(first) { t.Fatalf("synced_at moved from %v to %v on an unchanged linked sync", first, again) } is.Title = "second" - if err := svc.upsertIssues(ctx, repoID, []forge.Issue{is}, "ABC"); err != nil { + if err := svc.upsertIssues(ctx, repoID, []forge.Issue{is}, "ABC", nil); err != nil { t.Fatal(err) } if title, _, changed := read(); title != "second" || !changed.After(first) { diff --git a/api/internal/forgesvc/issue_write_guard.go b/api/internal/forgesvc/issue_write_guard.go index ead89042b3..54986a362a 100644 --- a/api/internal/forgesvc/issue_write_guard.go +++ b/api/internal/forgesvc/issue_write_guard.go @@ -4,22 +4,26 @@ import ( "context" "errors" "fmt" + "slices" "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/issueinput" "github.com/vtmocanu/uzi/api/internal/issuesource" + "github.com/vtmocanu/uzi/api/internal/issuetracker" ) -// ErrIssueWriteNotOffered is returned by the write guard for every write to a -// linked external issue tracker: YouTrack writes arrive in PRD #4173 M5, so until -// then no request is sent (S1: uzi stays read-only against the tracker). +// ErrIssueWriteNotOffered is returned by the write guard for the one write to a linked +// external issue tracker that is not offered: an UpdateIssueLabels that removes the +// eligibility name (the eligibility tag is never stripped). No request is sent. +// MoveToColumn, SetIssueState, CreateIssue, CreateIssueNote, UpdateIssueDescription, +// EnsureLabels and the other label writes pass through (PRD #4173 M5). // // It is the issuesource sentinel itself, so a package that cannot import forgesvc // (workersvc) can wrap and match the same value. var ErrIssueWriteNotOffered = issuesource.ErrIssueWriteNotOffered // linkedIssueError marks an *forge.IssueNotFoundError returned by a linked -// repo's issue reads. A forge not-found is never marked, so IssueWriteSkippable +// repo's issue reads and writes. A forge not-found is never marked, so IssueWriteSkippable // can tell "the linked issue moved or is gone" (D10: skip, never retry forever) // from an ordinary forge not-found that existing callers handle on their own. type linkedIssueError struct{ err error } @@ -35,21 +39,56 @@ func IssueWriteSkippable(err error) bool { if errors.Is(err, ErrIssueWriteNotOffered) { return true } + return LinkedIssueGone(err) +} + +// LinkedIssueGone reports whether err says a linked repo's issue is no longer in +// the linked project (D10), as opposed to the write not being offered. +func LinkedIssueGone(err error) bool { var marked *linkedIssueError return errors.As(err, &marked) } +// ColumnMoveRefused reports whether err is the issue tracker driver refusing a +// card move or State write before any request: the column has no State mapped, +// the mapping is unset, or the target State is not allowed (PRD #4173 M5). It is +// deliberately not part of IssueWriteSkippable: a refused move may succeed once +// the owner saves the mapping, so a caller keeps its retry marker for it. +func ColumnMoveRefused(err error) bool { + return errors.Is(err, issuetracker.ErrColumnNotMapped) || + errors.Is(err, issuetracker.ErrStateMappingUnset) || + errors.Is(err, issuetracker.ErrStateNotAllowed) +} + +// LinkedIssueResolved reports whether err is the issue tracker driver refusing a +// card move because the issue is resolved in the issue tracker +// (issuetracker.ErrIssueResolved). Any column move (MoveToColumn) other than an +// explicit reopen of a closed card is refused this way: an automatic move, or a board +// drag to a column of a card the cache shows open. A drag to Closed is a close, not a +// column move, and is not refused. It is deliberately not part of IssueWriteSkippable +// or ColumnMoveRefused: retrying cannot help, and the person's resolution wins until +// the next sync brings it into the cache. +func LinkedIssueResolved(err error) bool { + return errors.Is(err, issuetracker.ErrIssueResolved) +} + // issueWriteGuard wraps a linked repo's issue tracker driver. Reads pass -// through; every write is refused before any request. +// through, as do all writes (PRD #4173 M5), each marked when the issue moved away; the +// only write refused before any request is a label write that removes the eligibility +// name. type issueWriteGuard struct { inner forge.IssueTracker + // eligibility is the link's stored eligibility tag name; "" when none is known + // (UnavailableIssueTracker), where the inner driver returns its cause instead. + eligibility string } var ( - _ forge.IssueTracker = (*issueWriteGuard)(nil) - _ issueinput.AuthorMapper = (*issueWriteGuard)(nil) - _ issueinput.IssueTextMapper = (*issueWriteGuard)(nil) - _ issueinput.LinkedIssueSide = (*issueWriteGuard)(nil) + _ forge.IssueTracker = (*issueWriteGuard)(nil) + _ issueinput.AuthorMapper = (*issueWriteGuard)(nil) + _ issueinput.IssueTextMapper = (*issueWriteGuard)(nil) + _ issueinput.LinkedIssueSide = (*issueWriteGuard)(nil) + _ issuesource.IssueStateLister = (*issueWriteGuard)(nil) ) func markNotFound(err error) error { @@ -68,6 +107,21 @@ func (g *issueWriteGuard) ListIssues(ctx context.Context, projectID int64, opts return g.inner.ListIssues(ctx, projectID, opts) } +// ListIssuesWithState forwards the listing with State values when the inner +// driver offers it. A driver that cannot (unavailableIssueTracker) is an error, +// never an empty result, because the linked full sync evicts every cached issue +// the listing does not return. +func (g *issueWriteGuard) ListIssuesWithState(ctx context.Context, projectID int64, opts forge.ListIssuesOptions) ([]issuesource.StatedIssue, error) { + l, ok := g.inner.(issuesource.IssueStateLister) + if !ok { + return nil, fmt.Errorf("list issues with state: the issue tracker driver cannot list State values: %w", errNoStateListing) + } + return l.ListIssuesWithState(ctx, projectID, opts) +} + +// errNoStateListing marks an inner driver that has no ListIssuesWithState. +var errNoStateListing = errors.New("listing with issue tracker States is unavailable") + func (g *issueWriteGuard) GetIssue(ctx context.Context, projectID, issueIID int64) (forge.Issue, error) { iss, err := g.inner.GetIssue(ctx, projectID, issueIID) return iss, markNotFound(err) @@ -106,36 +160,60 @@ func (g *issueWriteGuard) refuse(op string) error { return fmt.Errorf("%s: %w", op, ErrIssueWriteNotOffered) } -func (g *issueWriteGuard) EnsureLabels(context.Context, int64, []forge.Label) error { - return g.refuse("ensure labels") +// EnsureLabels passes through (PRD #4173 M5). The driver never creates the eligibility +// tag; the handler's explicit label-ensure endpoint refuses a linked repo before this. +func (g *issueWriteGuard) EnsureLabels(ctx context.Context, projectID int64, labels []forge.Label) error { + return markNotFound(g.inner.EnsureLabels(ctx, projectID, labels)) +} + +// CreateIssue passes through (PRD #4173 M5); the caller has already rewritten the label +// set with issuesource.LinkedLabels. +func (g *issueWriteGuard) CreateIssue(ctx context.Context, projectID int64, title, description string, labels []string) (forge.Issue, error) { + iss, err := g.inner.CreateIssue(ctx, projectID, title, description, labels) + return iss, markNotFound(err) } -func (g *issueWriteGuard) CreateIssue(context.Context, int64, string, string, []string) (forge.Issue, error) { - return forge.Issue{}, g.refuse("create issue") +// UpdateIssueLabels passes through unless the removal set contains the eligibility name: +// that is refused before any request, so the eligibility tag is never stripped. +func (g *issueWriteGuard) UpdateIssueLabels(ctx context.Context, projectID, issueIID int64, add, remove []string) error { + if g.eligibility != "" && slices.Contains(remove, g.eligibility) { + return g.refuse("update issue labels") + } + return markNotFound(g.inner.UpdateIssueLabels(ctx, projectID, issueIID, add, remove)) } -func (g *issueWriteGuard) UpdateIssueLabels(context.Context, int64, int64, []string, []string) error { - return g.refuse("update issue labels") +// MoveToColumn passes through to the issue tracker driver (PRD #4173 M5). The +// driver refuses an unmapped column or a State it may not write before any +// request; a moved-away issue is marked so IssueWriteSkippable stays true (D10). +func (g *issueWriteGuard) MoveToColumn(ctx context.Context, projectID, issueIID int64, move forge.ColumnMove) error { + return markNotFound(g.inner.MoveToColumn(ctx, projectID, issueIID, move)) } -func (g *issueWriteGuard) UpdateIssueDescription(context.Context, int64, int64, string) error { - return g.refuse("update issue description") +// UpdateIssueDescription passes through (PRD #4173 M5); a moved-away issue is marked. +func (g *issueWriteGuard) UpdateIssueDescription(ctx context.Context, projectID, issueIID int64, description string) error { + return markNotFound(g.inner.UpdateIssueDescription(ctx, projectID, issueIID, description)) } -func (g *issueWriteGuard) SetIssueState(context.Context, int64, int64, forge.IssueState) error { - return g.refuse("set issue state") +// SetIssueState passes through to the issue tracker driver, which writes the +// stored closed or open State (PRD #4173 M5); a moved-away issue is marked. +func (g *issueWriteGuard) SetIssueState(ctx context.Context, projectID, issueIID int64, state forge.IssueState) error { + return markNotFound(g.inner.SetIssueState(ctx, projectID, issueIID, state)) } -func (g *issueWriteGuard) CreateIssueNote(context.Context, int64, int64, string) (forge.IssueNote, error) { - return forge.IssueNote{}, g.refuse("create issue note") +// CreateIssueNote passes through (PRD #4173 M5): the note lands as a comment on the +// linked issue; a moved-away issue is marked. +func (g *issueWriteGuard) CreateIssueNote(ctx context.Context, projectID, issueIID int64, body string) (forge.IssueNote, error) { + n, err := g.inner.CreateIssueNote(ctx, projectID, issueIID, body) + return n, markNotFound(err) } // UnavailableIssueTracker is the issue side handed to a linked repo whose resolver // failed (for example a connection removed from the allowlist): every read returns -// cause, which is NOT skippable so no marker is settled over an outage, and every -// write is refused by the write guard. It lets the poller's issue-side steps fail -// or skip per candidate while the code-host steps carry on, instead of dereferencing -// a nil tracker. +// cause, which is NOT skippable so no marker is settled over an outage, as does a +// move, State, create, note, description or label write (it has no eligibility name, +// so the guard refuses nothing). It lets the poller's issue-side steps fail or skip +// per candidate while the code-host steps carry on, instead of dereferencing a nil +// tracker. func UnavailableIssueTracker(cause error) forge.IssueTracker { return &issueWriteGuard{inner: unavailableIssueTracker{cause: cause}} } @@ -151,6 +229,9 @@ func (u unavailableIssueTracker) EnsureLabels(context.Context, int64, []forge.La func (u unavailableIssueTracker) ListIssues(context.Context, int64, forge.ListIssuesOptions) ([]forge.Issue, error) { return nil, u.cause } +func (u unavailableIssueTracker) ListIssuesWithState(context.Context, int64, forge.ListIssuesOptions) ([]issuesource.StatedIssue, error) { + return nil, u.cause +} func (u unavailableIssueTracker) GetIssue(context.Context, int64, int64) (forge.Issue, error) { return forge.Issue{}, u.cause } @@ -160,6 +241,9 @@ func (u unavailableIssueTracker) CreateIssue(context.Context, int64, string, str func (u unavailableIssueTracker) UpdateIssueLabels(context.Context, int64, int64, []string, []string) error { return u.cause } +func (u unavailableIssueTracker) MoveToColumn(context.Context, int64, int64, forge.ColumnMove) error { + return u.cause +} func (u unavailableIssueTracker) UpdateIssueDescription(context.Context, int64, int64, string) error { return u.cause } @@ -188,3 +272,27 @@ func isLinkedTracker(tr forge.IssueTracker) bool { _, ok := tr.(interface{ LinkedIssueSource() }) return ok } + +// isUnavailableTracker reports whether tr is the UnavailableIssueTracker placeholder, a +// linked issue side whose every read, move and State write fails with its cause. +func isUnavailableTracker(tr forge.IssueTracker) bool { + g, ok := tr.(*issueWriteGuard) + if !ok { + return false + } + _, unavailable := g.inner.(unavailableIssueTracker) + return unavailable +} + +// isTransientUnavailableTracker reports whether tr is the UnavailableIssueTracker built +// from a failed repo-source database read (errRepoSourceRead), the only unavailable cause +// classified as transient. Other transient failures, such as the connection-row read inside +// DriverFor, are not classified: they are recorded without a move. +func isTransientUnavailableTracker(tr forge.IssueTracker) bool { + g, ok := tr.(*issueWriteGuard) + if !ok { + return false + } + u, unavailable := g.inner.(unavailableIssueTracker) + return unavailable && errors.Is(u.cause, errRepoSourceRead) +} diff --git a/api/internal/forgesvc/issue_write_guard_test.go b/api/internal/forgesvc/issue_write_guard_test.go index cf3b5b76d4..4107f3929c 100644 --- a/api/internal/forgesvc/issue_write_guard_test.go +++ b/api/internal/forgesvc/issue_write_guard_test.go @@ -7,9 +7,11 @@ import ( "testing" "github.com/google/uuid" + "github.com/jackc/pgx/v5/pgtype" "github.com/vtmocanu/uzi/api/internal/board" "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuetracker" "github.com/vtmocanu/uzi/api/internal/store" ) @@ -93,6 +95,28 @@ func TestGuardedMRMoveOnALinkedRepoRecordsTheMRState(t *testing.T) { assertRecorded(t, st, runID, "closed") } +// A card a person resolved in the issue tracker (cache still open) is not moved: the +// edge is consumed, the MR state is recorded, and nothing is retried. +func TestGuardedMRMoveOnAnIssueResolvedInTheIssueTrackerRecordsAndSkips(t *testing.T) { + runID, repoID := uuid.New(), uuid.New() + st := &fakeStore{ + candidates: []store.ListMRWatchCandidatesRow{candidate(runID, 9, 13, mrTxt("opened"))}, + issue: mrIssue(repoID, 9, "opened", "PRD", board.ColumnHumanReview), + columns: mrCols(), + } + st.issue.IssueTrackerColumn = pgtype.Text{String: board.ColumnHumanReview, Valid: true} + f := &fakeForge{mr: forgeMR(13, "closed")} + d := &movingDriver{err: fmt.Errorf("move: %w", issuetracker.ErrIssueResolved)} + + if err := newTestService(st).SyncMRStates(context.Background(), repoID, testProjectID, f, &issueWriteGuard{inner: d}); err != nil { + t.Fatal(err) + } + if len(d.moves) != 1 { + t.Fatalf("move attempts = %v, want exactly one (no retry)", d.moves) + } + assertRecorded(t, st, runID, "closed") +} + // A plain (unlinked) forge failure on the same edge still leaves the edge. func TestGuardedMRMoveForgeFailureStillDefers(t *testing.T) { runID, repoID := uuid.New(), uuid.New() @@ -108,23 +132,52 @@ func TestGuardedMRMoveForgeFailureStillDefers(t *testing.T) { assertNoRecord(t, st) } -// The PRD link patch settles its marker on a linked repo: the description write is -// not offered, so retrying it would never end. -func TestPRDLinkPatchSettlesWhenWritesAreNotOffered(t *testing.T) { +// Converted from TestPRDLinkPatchSettlesWhenWritesAreNotOffered (PRD #4173 M5 U6): the +// description write is offered on a linked repo, so it reaches the issue tracker and the marker +// settles because the patch landed. +func TestPRDLinkPatchWritesTheDescriptionOnALinkedRepo(t *testing.T) { s, st, f, id := newPRDFixture(t, nil) guarded := &issueWriteGuard{inner: f} if err := s.SyncPRDLinkPatches(context.Background(), uuid.New(), fixProject, f, guarded); err != nil { t.Fatal(err) } enumerated(t, st) - if len(f.descriptionUpdates) != 0 { - t.Fatalf("a request reached the tracker: %+v", f.descriptionUpdates) + if len(f.descriptionUpdates) != 1 { + t.Fatalf("description writes = %+v, want exactly one to reach the issue tracker", f.descriptionUpdates) + } + if len(st.prdSettled) != 1 || st.prdSettled[0] != id { + t.Fatalf("settled = %v, want the run %v", st.prdSettled, id) } +} + +// An issue that moved away between the read and the write is skippable: the marker settles. +func TestPRDLinkPatchSettlesWhenTheIssueMovesAwayOnTheWrite(t *testing.T) { + s, st, f, id := newPRDFixture(t, nil) + f.updateDescErr = &forge.IssueNotFoundError{Err: errors.New("404")} + guarded := &issueWriteGuard{inner: f} + if err := s.SyncPRDLinkPatches(context.Background(), uuid.New(), fixProject, f, guarded); err != nil { + t.Fatal(err) + } + enumerated(t, st) if len(st.prdSettled) != 1 || st.prdSettled[0] != id { t.Fatalf("settled = %v, want the run %v", st.prdSettled, id) } } +// A transient write failure keeps the marker so the next tick retries. +func TestPRDLinkPatchKeepsTheMarkerOnATransientLinkedWriteFailure(t *testing.T) { + s, st, f, _ := newPRDFixture(t, nil) + f.updateDescErr = errors.New("503") + guarded := &issueWriteGuard{inner: f} + if err := s.SyncPRDLinkPatches(context.Background(), uuid.New(), fixProject, f, guarded); err != nil { + t.Fatal(err) + } + enumerated(t, st) + if len(st.prdSettled) != 0 { + t.Fatalf("a transient failure must not settle: %v", st.prdSettled) + } +} + func TestPRDLinkPatchSettlesWhenTheLinkedIssueIsGone(t *testing.T) { s, st, f, id := newPRDFixture(t, nil) gone := &stubDriverNotFound{fakeForge: *f, err: &forge.IssueNotFoundError{Err: errors.New("404")}} @@ -151,9 +204,9 @@ func TestPRDLinkPatchKeepsTheMarkerOnAPlainForgeNotFound(t *testing.T) { } // An unresolvable linked repo's issue side must never be nil and must not settle any -// marker: reads fail with the cause (not skippable), writes are refused. -func TestUnavailableIssueTrackerReadsFailAndWritesAreRefused(t *testing.T) { - cause := errors.New("tracker unavailable") +// marker: reads and writes fail with the cause (not skippable). +func TestUnavailableIssueTrackerReadsAndWritesFailWithTheCause(t *testing.T) { + cause := errors.New("issue tracker unavailable") tr := UnavailableIssueTracker(cause) ctx := context.Background() if _, err := tr.GetIssue(ctx, 1, 2); !errors.Is(err, cause) || IssueWriteSkippable(err) { @@ -162,11 +215,11 @@ func TestUnavailableIssueTrackerReadsFailAndWritesAreRefused(t *testing.T) { if _, err := tr.ListIssues(ctx, 1, forge.ListIssuesOptions{}); !errors.Is(err, cause) { t.Fatalf("ListIssues error = %v", err) } - if err := tr.UpdateIssueDescription(ctx, 1, 2, "d"); !IssueWriteSkippable(err) { - t.Fatalf("a write must be refused as not offered, got %v", err) + if err := tr.UpdateIssueDescription(ctx, 1, 2, "d"); !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Fatalf("a write must fail with the cause and not be skippable, got %v", err) } if !isLinkedTracker(tr) { - t.Fatal("the unavailable tracker must dispatch as linked") + t.Fatal("the unavailable issue tracker must dispatch as linked") } if isLinkedTracker(&fakeForge{}) { t.Fatal("a forge driver must not dispatch as linked") diff --git a/api/internal/forgesvc/linked_board_test.go b/api/internal/forgesvc/linked_board_test.go new file mode 100644 index 0000000000..699b89c52a --- /dev/null +++ b/api/internal/forgesvc/linked_board_test.go @@ -0,0 +1,534 @@ +package forgesvc + +import ( + "context" + "errors" + "fmt" + "testing" + "time" + + "github.com/google/uuid" + + "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuesource" + "github.com/vtmocanu/uzi/api/internal/issuetracker" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// PRD #4173 M5: the linked board. The guard lets the move and the State write through, +// the cache keeps a linked card's column in issue_tracker_column, and a refused move +// changes neither the issue tracker nor the cache. + +// movingDriver is the inner driver of a linked guard: it records the moves and State +// writes that reach it and answers them with err. +type movingDriver struct { + forge.IssueTracker // nil: only what a test drives exists + moves []forge.ColumnMove + states []forge.IssueState + err error + // The create and label writes PRD #4173 M5 lifts through the guard. + creates [][]string + ensured [][]forge.Label + labelOps [][2][]string + // The description and note writes PRD #4173 M5 U6 lifts through the guard. + descriptions []string + notes []string +} + +func (d *movingDriver) UpdateIssueDescription(_ context.Context, _, _ int64, desc string) error { + d.descriptions = append(d.descriptions, desc) + return d.err +} + +func (d *movingDriver) CreateIssueNote(_ context.Context, _, _ int64, body string) (forge.IssueNote, error) { + d.notes = append(d.notes, body) + return forge.IssueNote{}, d.err +} + +func (d *movingDriver) CreateIssue(_ context.Context, _ int64, _, _ string, labels []string) (forge.Issue, error) { + d.creates = append(d.creates, labels) + return forge.Issue{IID: 5}, d.err +} + +func (d *movingDriver) EnsureLabels(_ context.Context, _ int64, labels []forge.Label) error { + d.ensured = append(d.ensured, labels) + return d.err +} + +func (d *movingDriver) UpdateIssueLabels(_ context.Context, _, _ int64, add, remove []string) error { + d.labelOps = append(d.labelOps, [2][]string{add, remove}) + return d.err +} + +func (d *movingDriver) MoveToColumn(_ context.Context, _, _ int64, m forge.ColumnMove) error { + d.moves = append(d.moves, m) + return d.err +} + +func (d *movingDriver) SetIssueState(_ context.Context, _, _ int64, s forge.IssueState) error { + d.states = append(d.states, s) + return d.err +} + +func TestLinkedIssueResolvedNamesOnlyTheResolvedRefusal(t *testing.T) { + if !LinkedIssueResolved(fmt.Errorf("move: %w", issuetracker.ErrIssueResolved)) { + t.Error("a wrapped ErrIssueResolved must be reported") + } + if !LinkedIssueResolved((&issueWriteGuard{inner: &movingDriver{err: issuetracker.ErrIssueResolved}}).MoveToColumn(context.Background(), 1, 2, forge.ColumnMove{})) { + t.Error("the write guard must keep ErrIssueResolved in the chain") + } + for _, e := range []error{nil, issuetracker.ErrColumnNotMapped, ErrIssueWriteNotOffered, errors.New("503")} { + if LinkedIssueResolved(e) { + t.Errorf("LinkedIssueResolved(%v) = true, want false", e) + } + } + // Neither a skip nor a refusal: each has its own caller handling. + if IssueWriteSkippable(issuetracker.ErrIssueResolved) || ColumnMoveRefused(issuetracker.ErrIssueResolved) { + t.Error("ErrIssueResolved must be neither IssueWriteSkippable nor ColumnMoveRefused") + } +} + +func TestColumnMoveRefusedNamesOnlyTheDriversRefusals(t *testing.T) { + tests := []struct { + name string + err error + want bool + }{ + {"nil", nil, false}, + {"column not mapped", issuetracker.ErrColumnNotMapped, true}, + {"mapping unset", issuetracker.ErrStateMappingUnset, true}, + {"state not allowed", issuetracker.ErrStateNotAllowed, true}, + {"wrapped", fmt.Errorf("move: %w", issuetracker.ErrColumnNotMapped), true}, + {"write not offered is not a refusal", ErrIssueWriteNotOffered, false}, + {"moved away is not a refusal", &linkedIssueError{err: &forge.IssueNotFoundError{Err: errors.New("404")}}, false}, + {"other error", errors.New("youtrack unreachable"), false}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + if got := ColumnMoveRefused(tc.err); got != tc.want { + t.Fatalf("ColumnMoveRefused(%v) = %v, want %v", tc.err, got, tc.want) + } + }) + } + // IssueWriteSkippable is NOT widened: a refusal keeps its retry marker. + for _, e := range []error{issuetracker.ErrColumnNotMapped, issuetracker.ErrStateMappingUnset, issuetracker.ErrStateNotAllowed} { + if IssueWriteSkippable(e) { + t.Errorf("IssueWriteSkippable(%v) = true, want false", e) + } + } +} + +func TestGuardPassesMovesAndStateWritesThroughAndMarksNotFound(t *testing.T) { + ctx := context.Background() + moved := forge.ColumnMove{Column: "In Progress"} + + t.Run("reach the driver", func(t *testing.T) { + d := &movingDriver{} + g := &issueWriteGuard{inner: d} + if err := g.MoveToColumn(ctx, 1, 2, moved); err != nil { + t.Fatal(err) + } + if err := g.SetIssueState(ctx, 1, 2, forge.StateClosed); err != nil { + t.Fatal(err) + } + if len(d.moves) != 1 || d.moves[0].Column != "In Progress" || len(d.states) != 1 || d.states[0] != forge.StateClosed { + t.Fatalf("driver saw moves=%v states=%v", d.moves, d.states) + } + }) + + t.Run("a moved-away issue is a skippable linked error", func(t *testing.T) { + d := &movingDriver{err: &forge.IssueNotFoundError{Err: errors.New("404")}} + g := &issueWriteGuard{inner: d} + for name, err := range map[string]error{ + "move": g.MoveToColumn(ctx, 1, 2, moved), + "state": g.SetIssueState(ctx, 1, 2, forge.StateClosed), + } { + if !LinkedIssueGone(err) || !IssueWriteSkippable(err) { + t.Errorf("%s: %v must be a marked, skippable not-found", name, err) + } + var missing *forge.IssueNotFoundError + if !errors.As(err, &missing) { + t.Errorf("%s: the marked error must still unwrap to the not-found", name) + } + } + }) + + t.Run("refusals and other errors pass unmarked", func(t *testing.T) { + for _, cause := range []error{issuetracker.ErrColumnNotMapped, issuetracker.ErrStateNotAllowed, errors.New("503")} { + g := &issueWriteGuard{inner: &movingDriver{err: cause}} + for _, err := range []error{g.MoveToColumn(ctx, 1, 2, moved), g.SetIssueState(ctx, 1, 2, forge.StateOpened)} { + if !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Errorf("error %v for cause %v must keep the cause and not be skippable", err, cause) + } + } + } + }) + + // Converted from "every other write is still refused without reaching the driver": create, + // ensure and label writes are lifted by PRD #4173 M5; the description and note writes are + // lifted by U6. + t.Run("create and label writes reach the driver", func(t *testing.T) { + d := &movingDriver{} + g := &issueWriteGuard{inner: d, eligibility: "ready"} + if err := g.EnsureLabels(ctx, 1, []forge.Label{{Name: "x"}}); err != nil { + t.Fatalf("ensure: %v", err) + } + if iss, err := g.CreateIssue(ctx, 1, "t", "d", []string{"ready"}); err != nil || iss.IID != 5 { + t.Fatalf("create: %v %v", iss, err) + } + if err := g.UpdateIssueLabels(ctx, 1, 2, []string{"ready"}, []string{"other"}); err != nil { + t.Fatalf("labels: %v", err) + } + if len(d.ensured) != 1 || len(d.creates) != 1 || len(d.labelOps) != 1 { + t.Fatalf("driver saw ensure=%d create=%d labels=%d, want 1 each", len(d.ensured), len(d.creates), len(d.labelOps)) + } + }) + + t.Run("a moved-away issue on a label write is a skippable linked error", func(t *testing.T) { + g := &issueWriteGuard{inner: &movingDriver{err: &forge.IssueNotFoundError{Err: errors.New("404")}}, eligibility: "ready"} + err := g.UpdateIssueLabels(ctx, 1, 2, []string{"a"}, nil) + if !LinkedIssueGone(err) || !IssueWriteSkippable(err) { + t.Fatalf("%v must be a marked, skippable not-found", err) + } + }) + + t.Run("removing the eligibility name is refused without reaching the driver", func(t *testing.T) { + d := &movingDriver{} + g := &issueWriteGuard{inner: d, eligibility: "ready"} + err := g.UpdateIssueLabels(ctx, 1, 2, nil, []string{"other", "ready"}) + if !errors.Is(err, ErrIssueWriteNotOffered) || !IssueWriteSkippable(err) { + t.Fatalf("err = %v, want the skippable ErrIssueWriteNotOffered", err) + } + if len(d.labelOps) != 0 { + t.Fatalf("the driver saw %v", d.labelOps) + } + // Adding it, or removing another label, is not a removal of the eligibility name. + if err := g.UpdateIssueLabels(ctx, 1, 2, []string{"ready"}, []string{"other"}); err != nil { + t.Fatalf("add eligibility: %v", err) + } + // The comparison is exact: a different case or a longer name is not the eligibility name. + d.labelOps = nil + if err := g.UpdateIssueLabels(ctx, 1, 2, nil, []string{"Ready", "ready-x", "other"}); err != nil { + t.Fatalf("near-miss removal: %v", err) + } + if len(d.labelOps) != 1 || len(d.labelOps[0][1]) != 3 { + t.Fatalf("the driver saw %v, want the near-miss removal passed through", d.labelOps) + } + }) + + t.Run("a moved-away issue on create and ensure is a marked not-found", func(t *testing.T) { + g := &issueWriteGuard{inner: &movingDriver{err: &forge.IssueNotFoundError{Err: errors.New("404")}}, eligibility: "ready"} + _, createErr := g.CreateIssue(ctx, 1, "t", "d", []string{"ready"}) + for name, err := range map[string]error{ + "create": createErr, + "ensure": g.EnsureLabels(ctx, 1, []forge.Label{{Name: "x"}}), + } { + if !LinkedIssueGone(err) || !IssueWriteSkippable(err) { + t.Errorf("%s: %v must be a marked, skippable not-found", name, err) + } + } + }) + + // Converted from "description and note writes are still refused without reaching the + // driver": PRD #4173 M5 U6 lifts both, so they reach the driver. + t.Run("description and note writes reach the driver", func(t *testing.T) { + d := &movingDriver{} + g := &issueWriteGuard{inner: d} + if err := g.UpdateIssueDescription(ctx, 1, 2, "d"); err != nil { + t.Errorf("desc: %v", err) + } + if _, err := g.CreateIssueNote(ctx, 1, 2, "n"); err != nil { + t.Errorf("note: %v", err) + } + if len(d.descriptions) != 1 || d.descriptions[0] != "d" || len(d.notes) != 1 || d.notes[0] != "n" { + t.Fatalf("driver saw descriptions=%v notes=%v", d.descriptions, d.notes) + } + }) + + t.Run("a moved-away issue on a note and a description is a skippable linked error", func(t *testing.T) { + g := &issueWriteGuard{inner: &movingDriver{err: &forge.IssueNotFoundError{Err: errors.New("404")}}} + _, noteErr := g.CreateIssueNote(ctx, 1, 2, "n") + for name, err := range map[string]error{ + "note": noteErr, + "desc": g.UpdateIssueDescription(ctx, 1, 2, "d"), + } { + if !LinkedIssueGone(err) || !IssueWriteSkippable(err) || errors.Is(err, ErrIssueWriteNotOffered) { + t.Errorf("%s: %v must be a marked, skippable not-found", name, err) + } + } + }) + + t.Run("an unavailable driver's note and description writes return its cause", func(t *testing.T) { + cause := errors.New("connection removed") + g := UnavailableIssueTracker(cause) + if _, err := g.CreateIssueNote(ctx, 1, 2, "n"); !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Errorf("note: %v", err) + } + if err := g.UpdateIssueDescription(ctx, 1, 2, "d"); !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Errorf("desc: %v", err) + } + }) + + t.Run("an unavailable driver's create and label writes return its cause", func(t *testing.T) { + cause := errors.New("connection removed") + g := UnavailableIssueTracker(cause) + if _, err := g.CreateIssue(ctx, 1, "t", "d", nil); !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Errorf("create: %v", err) + } + if err := g.EnsureLabels(ctx, 1, nil); !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Errorf("ensure: %v", err) + } + if err := g.UpdateIssueLabels(ctx, 1, 2, []string{"a"}, nil); !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Errorf("labels: %v", err) + } + }) +} + +type statedInner struct { + movingDriver + got forge.ListIssuesOptions + list []issuesource.StatedIssue + err error +} + +func (s *statedInner) ListIssuesWithState(_ context.Context, _ int64, opts forge.ListIssuesOptions) ([]issuesource.StatedIssue, error) { + s.got = opts + return s.list, s.err +} + +func TestGuardListIssuesWithState(t *testing.T) { + ctx := context.Background() + opts := forge.ListIssuesOptions{Labels: []string{"uzi"}} + + t.Run("forwards to a driver that offers it", func(t *testing.T) { + in := &statedInner{list: []issuesource.StatedIssue{{Issue: forge.Issue{IID: 5}, State: "In Progress"}}} + got, err := (&issueWriteGuard{inner: in}).ListIssuesWithState(ctx, 1, opts) + if err != nil || len(got) != 1 || got[0].State != "In Progress" || len(in.got.Labels) != 1 { + t.Fatalf("got (%v, %v) with options %+v", got, err, in.got) + } + }) + + t.Run("an inner failure is returned", func(t *testing.T) { + boom := errors.New("boom") + got, err := (&issueWriteGuard{inner: &statedInner{err: boom}}).ListIssuesWithState(ctx, 1, opts) + if !errors.Is(err, boom) || got != nil { + t.Fatalf("got (%v, %v), want the cause", got, err) + } + }) + + t.Run("a driver without it is an error, never an empty listing", func(t *testing.T) { + got, err := (&issueWriteGuard{inner: &movingDriver{}}).ListIssuesWithState(ctx, 1, opts) + if err == nil || !errors.Is(err, errNoStateListing) || got != nil { + t.Fatalf("got (%v, %v), want an error and no listing", got, err) + } + }) + + t.Run("the unavailable issue tracker returns its cause", func(t *testing.T) { + cause := errors.New("issue tracker unavailable") + tr := UnavailableIssueTracker(cause) + lister, ok := tr.(issuesource.IssueStateLister) + if !ok { + t.Fatal("the linked guard must offer ListIssuesWithState") + } + got, err := lister.ListIssuesWithState(ctx, 1, opts) + if !errors.Is(err, cause) || got != nil || IssueWriteSkippable(err) { + t.Fatalf("got (%v, %v), want the cause, no listing, not skippable", got, err) + } + // Moves and State writes of an unavailable side fail with the cause, so a + // resolver outage is retried, never skipped. + if err := tr.MoveToColumn(ctx, 1, 2, forge.ColumnMove{Column: "x"}); !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Fatalf("move error = %v, want the cause, not skippable", err) + } + if err := tr.SetIssueState(ctx, 1, 2, forge.StateClosed); !errors.Is(err, cause) || IssueWriteSkippable(err) { + t.Fatalf("state error = %v, want the cause, not skippable", err) + } + }) +} + +// ── AutoMove, close and reopen on a linked repo ─────────────────────────────── + +func linkedIssue(state, column string) store.Issue { + iss := stateIssue2(state) + if column != "" { + iss.IssueTrackerColumn.String, iss.IssueTrackerColumn.Valid = column, true + } + return iss +} + +func stateIssue2(state string) store.Issue { + return store.Issue{RepoID: uuid.New(), ForgeIssueIid: 42, Title: "t", State: state, Labels: []byte(`["uzi","Planned"]`)} +} + +func TestAutoMoveOnALinkedRepoCachesOnlyTheColumn(t *testing.T) { + st := &fakeStore{} + d := &movingDriver{} + cols := []store.BoardColumn{{LabelName: "In Progress", Position: 0}} + + got, err := newTestService(st).AutoMove(context.Background(), &issueWriteGuard{inner: d}, 7, linkedIssue("opened", "Planned"), cols, "In Progress") + if err != nil { + t.Fatal(err) + } + if len(d.moves) != 1 || d.moves[0].Column != "In Progress" { + t.Fatalf("driver moves = %v", d.moves) + } + if len(st.labelUpserts) != 0 { + t.Fatalf("a linked move wrote labels: %+v", st.labelUpserts) + } + if len(st.columnWrites) != 1 || st.columnWrites[0].IssueTrackerColumn != "In Progress" || st.columnWrites[0].ForgeIssueIid != 42 { + t.Fatalf("column writes = %+v, want In Progress for issue 42", st.columnWrites) + } + if got.IssueTrackerColumn.String != "In Progress" { + t.Fatalf("returned column = %+v", got.IssueTrackerColumn) + } +} + +func TestAutoMoveOnALinkedRepoLeavesTheCacheOnAFailure(t *testing.T) { + for name, cause := range map[string]error{ + "refused": issuetracker.ErrColumnNotMapped, + "unreachable": errors.New("503"), + "moved away": &forge.IssueNotFoundError{Err: errors.New("404")}, + } { + t.Run(name, func(t *testing.T) { + st := &fakeStore{} + _, err := newTestService(st).AutoMove(context.Background(), &issueWriteGuard{inner: &movingDriver{err: cause}}, 7, linkedIssue("opened", ""), nil, "In Progress") + if !errors.Is(err, cause) { + t.Fatalf("error = %v, want the cause", err) + } + if len(st.columnWrites) != 0 || len(st.labelUpserts) != 0 { + t.Fatalf("a failed move changed the cache: columns=%v labels=%v", st.columnWrites, st.labelUpserts) + } + }) + } +} + +// A move to the Open lane stores no column (the Open lane is NULL). +func TestAutoMoveOnALinkedRepoToOpenWritesAnEmptyColumn(t *testing.T) { + st := &fakeStore{} + got, err := newTestService(st).AutoMove(context.Background(), &issueWriteGuard{inner: &movingDriver{}}, 7, linkedIssue("opened", "Planned"), nil, "") + if err != nil { + t.Fatal(err) + } + if len(st.columnWrites) != 1 || st.columnWrites[0].IssueTrackerColumn != "" || got.IssueTrackerColumn.Valid { + t.Fatalf("column writes = %+v, returned %+v", st.columnWrites, got.IssueTrackerColumn) + } +} + +func TestCloseIssueOnALinkedRepoWritesTheClosedState(t *testing.T) { + st := &fakeStore{} + d := &movingDriver{} + if _, err := newTestService(st).CloseIssue(context.Background(), &issueWriteGuard{inner: d}, 7, linkedIssue("opened", "Planned")); err != nil { + t.Fatal(err) + } + if len(d.states) != 1 || d.states[0] != forge.StateClosed || len(st.stateUpdates) != 1 { + t.Fatalf("states=%v stateUpdates=%v", d.states, st.stateUpdates) + } + if len(st.columnWrites) != 0 { + t.Fatalf("close must not touch the column: %v", st.columnWrites) + } + + // A refused close (no closed State saved) changes nothing. + st = &fakeStore{} + _, err := newTestService(st).CloseIssue(context.Background(), &issueWriteGuard{inner: &movingDriver{err: issuetracker.ErrStateMappingUnset}}, 7, linkedIssue("opened", "")) + if !errors.Is(err, issuetracker.ErrStateMappingUnset) || len(st.stateUpdates) != 0 { + t.Fatalf("err=%v stateUpdates=%v", err, st.stateUpdates) + } +} + +func TestReopenIssueOnALinkedRepoIsOneMoveThenTheCache(t *testing.T) { + t.Run("one write, then the cache", func(t *testing.T) { + st := &fakeStore{} + d := &movingDriver{} + got, err := newTestService(st).ReopenIssue(context.Background(), &issueWriteGuard{inner: d}, 7, linkedIssue("closed", ""), nil, "In Progress") + if err != nil { + t.Fatal(err) + } + if len(d.states) != 0 { + t.Fatalf("a linked reopen must not write the State separately: %v", d.states) + } + if len(d.moves) != 1 || d.moves[0].Column != "In Progress" { + t.Fatalf("moves = %v, want one move to In Progress", d.moves) + } + if !d.moves[0].Reopen { + t.Fatalf("move = %+v, want Reopen set so the driver writes over a resolved issue", d.moves[0]) + } + if len(st.reopens) != 1 || len(st.columnWrites) != 1 || st.columnWrites[0].IssueTrackerColumn != "In Progress" { + t.Fatalf("reopens=%v columnWrites=%v", st.reopens, st.columnWrites) + } + if got.IssueTrackerColumn.String != "In Progress" { + t.Fatalf("returned column=%+v", got.IssueTrackerColumn) + } + if len(st.labelUpserts) != 0 { + t.Fatalf("linked reopen wrote labels: %v", st.labelUpserts) + } + }) + + t.Run("a refused or failed move changes neither side", func(t *testing.T) { + for name, cause := range map[string]error{ + "unmapped column": issuetracker.ErrColumnNotMapped, + "not allowed": issuetracker.ErrStateNotAllowed, + "unreachable": errors.New("503"), + } { + st := &fakeStore{} + d := &movingDriver{err: cause} + _, err := newTestService(st).ReopenIssue(context.Background(), &issueWriteGuard{inner: d}, 7, linkedIssue("closed", ""), nil, "Nowhere") + if !errors.Is(err, cause) { + t.Fatalf("%s: error = %v", name, err) + } + if len(d.states) != 0 || len(st.reopens) != 0 || len(st.columnWrites) != 0 { + t.Fatalf("%s: states=%v reopens=%v columns=%v", name, d.states, st.reopens, st.columnWrites) + } + } + }) +} + +// ── the linked sync ─────────────────────────────────────────────────────────── + +func TestLinkedSyncCachesTheColumnOfEachStatesMapping(t *testing.T) { + user, repo := uuid.New(), uuid.New() + closed := issueAt(9, recordedMarkBase) + closed.State = "closed" + f := &fakeForge{ + issues: []forge.Issue{issueAt(7, recordedMarkBase), issueAt(8, recordedMarkBase), closed, issueAt(10, recordedMarkBase), issueAt(11, recordedMarkBase), issueAt(12, recordedMarkBase)}, + states: map[int64]string{7: "In Progress", 8: "Some Custom State", 9: "In Progress", 10: "Open", 11: "", 12: "In Review"}, + } + st := newLinkedSyncStore() + st.row.ColumnStates = []byte(`{"In Progress":"In Progress","Human Review":"In Review"}`) + + if _, err := newTestService(st).SyncIssues(context.Background(), user, repo, testProjectID, linkedTracker(f)); err != nil { + t.Fatal(err) + } + want := map[int64]string{7: "In Progress", 8: "", 9: "", 10: "", 11: "", 12: "Human Review"} + if len(st.upserts) != len(want) { + t.Fatalf("upserts = %d", len(st.upserts)) + } + for i, up := range st.upserts { + col := st.changedColumns[i] + if col.String != want[up.ForgeIssueIid] || col.Valid != (want[up.ForgeIssueIid] != "") { + t.Errorf("issue %d cached column %+v, want %q (NULL for the Open lane)", up.ForgeIssueIid, col, want[up.ForgeIssueIid]) + } + } +} + +// The forge sync never writes the column. +func TestForgeSyncDoesNotWriteTheColumn(t *testing.T) { + st := &fakeStore{} + f := &fakeForge{issues: []forge.Issue{issueAt(1, time.Unix(100, 0))}, states: map[int64]string{1: "In Progress"}} + if _, err := newTestService(st).FullSync(context.Background(), uuid.New(), testProjectID, f); err != nil { + t.Fatal(err) + } + if st.changedUpserts != 0 || len(st.columnWrites) != 0 { + t.Fatalf("the forge sync touched the column path: changed=%d columns=%v", st.changedUpserts, st.columnWrites) + } +} + +// A linked side that cannot list States fails the sync with no write and no eviction. +func TestLinkedSyncFailsWhenTheDriverCannotListStates(t *testing.T) { + st := newLinkedSyncStore() + _, err := newTestService(st).SyncIssues(context.Background(), uuid.New(), uuid.New(), testProjectID, &issueWriteGuard{inner: &movingDriver{}}) + if !errors.Is(err, errNoStateListing) { + t.Fatalf("error = %v, want errNoStateListing", err) + } + if len(st.upserts) != 0 || len(st.deleteCalls) != 0 { + t.Fatalf("a failed listing wrote or evicted: upserts=%d deletes=%v", len(st.upserts), st.deleteCalls) + } +} diff --git a/api/internal/forgesvc/linked_issue_sync.go b/api/internal/forgesvc/linked_issue_sync.go index 783aa54f24..ab85b16ddf 100644 --- a/api/internal/forgesvc/linked_issue_sync.go +++ b/api/internal/forgesvc/linked_issue_sync.go @@ -2,11 +2,13 @@ package forgesvc import ( "context" + "encoding/json" "fmt" "github.com/google/uuid" "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuesource" "github.com/vtmocanu/uzi/api/internal/store" ) @@ -35,7 +37,8 @@ func (s *Service) SyncIssuesIncremental(ctx context.Context, userID, repoID uuid // syncLinkedIssues caches the issues of a linked repo's eligibility tag (D10, D14). // It reads the link from the owner-scoped repo row (the row repoSource reads), lists -// the tag's issues, and upserts every one with issue_key = short + "-" + number. No +// the tag's issues, and upserts every one with issue_key = short + "-" + number and the board column its +// State maps to. No // open-issue and no finding fetch: only tagged issues are cached. // // The listing never carries UpdatedAfter, in either form: the id-scoped listing has @@ -56,11 +59,36 @@ func (s *Service) syncLinkedIssues(ctx context.Context, userID, repoID uuid.UUID if !src.IssueTrackerProjectShort.Valid || !src.EligibilityTagName.Valid { return m, ErrLinkedRepoUnresolvable } - issues, err := tr.ListIssues(ctx, forgeProjectID, forge.ListIssuesOptions{Labels: []string{src.EligibilityTagName.String}}) + // The listing carries each issue's State so the board column can be cached (PRD + // #4173 M5). The write guard of a linked repo always offers it; a linked side that + // cannot is a fault, never an empty listing, which the full form would read as + // "evict everything". + lister, ok := tr.(issuesource.IssueStateLister) + if !ok { + return m, fmt.Errorf("linked issue sync: %w", errNoStateListing) + } + stated, err := lister.ListIssuesWithState(ctx, forgeProjectID, forge.ListIssuesOptions{Labels: []string{src.EligibilityTagName.String}}) + if err != nil { + return m, err + } + stateColumns, err := reverseColumnStates(src.ColumnStates) if err != nil { return m, err } - if err := s.upsertIssues(ctx, repoID, issues, src.IssueTrackerProjectShort.String); err != nil { + issues := make([]forge.Issue, 0, len(stated)) + columns := make(map[int64]string, len(stated)) + for _, si := range stated { + issues = append(issues, si.Issue) + // A resolved issue, the open state and a State no column maps to are all the Open + // lane (empty column, stored as NULL); closed is decided by issues.state. + if si.Issue.State == string(forge.StateClosed) { + continue + } + if col, mapped := stateColumns[si.State]; mapped { + columns[si.Issue.IID] = col + } + } + if err := s.upsertIssues(ctx, repoID, issues, src.IssueTrackerProjectShort.String, columns); err != nil { return m, err } if full { @@ -74,3 +102,21 @@ func (s *Service) syncLinkedIssues(ctx context.Context, userID, repoID uuid.UUID } return m.Advance(Marks{PRD: maxUpdatedAt(issues)}), nil } + +// reverseColumnStates inverts the stored column-to-State mapping (a JSON object of +// board column label to State name) into State name to column label. The stored +// States are distinct per repo, so the inverse is a function. +func reverseColumnStates(raw []byte) (map[string]string, error) { + if len(raw) == 0 { + return nil, nil + } + var columns map[string]string + if err := json.Unmarshal(raw, &columns); err != nil { + return nil, fmt.Errorf("decode stored column states: %w", err) + } + out := make(map[string]string, len(columns)) + for col, state := range columns { + out[state] = col + } + return out, nil +} diff --git a/api/internal/forgesvc/linked_mr_watch_livedb_test.go b/api/internal/forgesvc/linked_mr_watch_livedb_test.go new file mode 100644 index 0000000000..8aef5a6dd0 --- /dev/null +++ b/api/internal/forgesvc/linked_mr_watch_livedb_test.go @@ -0,0 +1,529 @@ +package forgesvc + +import ( + "context" + "errors" + "fmt" + "net/http" + "os" + "testing" + "time" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5/pgtype" + "github.com/jackc/pgx/v5/pgxpool" + + "github.com/vtmocanu/uzi/api/internal/board" + "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuetracker" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// PRD #4173 M5: the board-free MR-state recorder moves a linked repo's card on an MR +// close/reopen edge through the issue tracker, for the issue's latest run only. + +type linkedMRWatch struct { + pool *pgxpool.Pool + q *store.Queries + svc *Service + srv *youtracktest.Server + tr forge.IssueTracker + repo uuid.UUID + user uuid.UUID + forge uuid.UUID // the forge connection id +} + +// newLinkedMRWatch starts a linked repo over the YouTrack fake. columnStates maps the +// board columns to State names; the bundle holds the Open and Fixed defaults plus +// "In Progress" and "In Review". +func newLinkedMRWatch(t *testing.T, columnStates map[string]string) *linkedMRWatch { + t.Helper() + ctx := context.Background() + dsn := os.Getenv("UZI_TEST_DATABASE_URL") + if dsn == "" { + t.Skip("UZI_TEST_DATABASE_URL not set; run via e2e/run-store-it.sh for live-DB coverage") + } + if err := store.Migrate(ctx, dsn); err != nil { + t.Fatalf("migrate: %v", err) + } + pool, err := store.OpenPool(ctx, dsn) + if err != nil { + t.Fatalf("open pool: %v", err) + } + t.Cleanup(pool.Close) + + srv := youtracktest.NewServer(t) + srv.SeedState("In Progress", false) + srv.SeedState("In Review", false) + def, ok := http.DefaultTransport.(*http.Transport) + if !ok { + t.Fatal("http.DefaultTransport is not an *http.Transport") + } + saved := def.TLSClientConfig + def.TLSClientConfig = srv.Client().Transport.(*http.Transport).TLSClientConfig.Clone() + t.Cleanup(func() { def.TLSClientConfig = saved }) + d, err := issuetracker.New(issuetracker.KindYouTrack, issuetracker.Options{ + BaseURL: srv.URL, + Token: srv.Token, + Project: issuetracker.ProjectRef{ID: youtracktest.ProjectID, ShortName: youtracktest.ProjectShortName}, + ProjectID: 77, + EligibilityTag: issuetracker.TagRef{ID: youtracktest.EligibilityTagID, Name: youtracktest.EligibilityTagName}, + OpenState: "Open", + ClosedState: "Fixed", + ColumnStates: columnStates, + }) + if err != nil { + t.Fatal(err) + } + + f := &linkedMRWatch{ + pool: pool, q: store.New(pool), svc: New(store.New(pool), nil, time.Second, nil), srv: srv, + tr: &issueWriteGuard{inner: d}, repo: uuid.New(), user: uuid.New(), forge: uuid.New(), + } + itConn := uuid.New() + f.exec(t, `INSERT INTO users (id, email, password_hash) VALUES ($1, $2, 'x')`, f.user, fmt.Sprintf("lmw-%s@e2e", f.user)) + f.exec(t, `INSERT INTO forge_connections (id, user_id, forge_type, base_url, bot_username, bot_forge_user_id, token_ciphertext) + VALUES ($1, $2, 'gitlab', $3, 'bot', 1, $4)`, f.forge, f.user, "https://"+uuid.NewString()+".forge.e2e", []byte{1}) + f.exec(t, `INSERT INTO issue_tracker_connections (id, user_id, kind, base_url, account_id, account_login, token_ciphertext) + VALUES ($1, $2, 'youtrack', $3, '1-1', 'bot', $4)`, itConn, f.user, "https://"+uuid.NewString()+".yt.e2e", []byte{1}) + f.exec(t, `INSERT INTO repos (id, connection_id, forge_project_id, path_with_namespace, web_url, default_branch, enabled, + issue_tracker_connection_id, issue_tracker_project_id, issue_tracker_project_short, eligibility_tag_id, eligibility_tag_name, issue_tracker_linked_at) + VALUES ($1, $2, 77, $3, 'https://forge.e2e/g/l', 'main', true, $4, '0-1', 'UZI', '5-1', 'uzi', now())`, + f.repo, f.forge, "g/lmw-"+f.repo.String(), itConn) + if err := EnsureBoardColumnRows(ctx, f.q, f.repo); err != nil { + t.Fatal(err) + } + return f +} + +func (f *linkedMRWatch) exec(t *testing.T, sql string, args ...any) { + t.Helper() + if _, err := f.pool.Exec(context.Background(), sql, args...); err != nil { + t.Fatalf("exec %q: %v", sql, err) + } +} + +// card seeds a YouTrack issue sitting in column (its State is written to match) and +// caches it with that column. An empty column caches the Open lane. +func (f *linkedMRWatch) card(t *testing.T, column string) int64 { + t.Helper() + n := f.srv.SeedIssue("a card", "b", youtracktest.EligibilityTagID) + if column != "" { + if err := f.tr.MoveToColumn(context.Background(), 77, n, forge.ColumnMove{Column: column}); err != nil { + t.Fatalf("seed the State: %v", err) + } + } + f.cache(t, n, column) + return n +} + +func (f *linkedMRWatch) cache(t *testing.T, n int64, column string) { + t.Helper() + f.exec(t, `INSERT INTO issues (repo_id, forge_issue_iid, title, state, labels, web_url, forge_updated_at, synced_at, issue_tracker_column) + VALUES ($1, $2, 't', 'opened', '[]', 'https://yt.e2e/i', now(), now(), $3)`, f.repo, n, column) +} + +// run inserts a completed issue-lane run for issue n with an MR in mrState. +func (f *linkedMRWatch) run(t *testing.T, n, mr int64, mrState string, age time.Duration) uuid.UUID { + t.Helper() + id := uuid.New() + f.exec(t, `INSERT INTO runs (id, user_id, repo_id, kind, issue_iid, issue_title, issue_description, branch, mr_iid, mr_state, status, created_at) + VALUES ($1, $2, $3, 'issue', $4, 't', 'd', $5, $6, $7, 'completed', now() - $8::interval)`, + id, f.user, f.repo, n, "agent/issue-"+id.String(), mr, mrState, fmt.Sprintf("%d seconds", int(age.Seconds()))) + return id +} + +func (f *linkedMRWatch) mrState(t *testing.T, run uuid.UUID) string { + t.Helper() + var s pgtype.Text + if err := f.pool.QueryRow(context.Background(), `SELECT mr_state FROM runs WHERE id = $1`, run).Scan(&s); err != nil { + t.Fatal(err) + } + return s.String +} + +func (f *linkedMRWatch) column(t *testing.T, n int64) string { + t.Helper() + var c pgtype.Text + if err := f.pool.QueryRow(context.Background(), `SELECT issue_tracker_column FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2`, f.repo, n).Scan(&c); err != nil { + t.Fatal(err) + } + return c.String +} + +func (f *linkedMRWatch) writes() []youtracktest.Request { + var out []youtracktest.Request + for _, r := range f.srv.Requests() { + if r.Method != http.MethodGet { + out = append(out, r) + } + } + return out +} + +func (f *linkedMRWatch) sync(t *testing.T, mrs map[int64]string) { + t.Helper() + cf := &fakeForge{mrByIID: map[int64]forge.MergeRequest{}} + for iid, st := range mrs { + cf.mrByIID[iid] = forgeMR(iid, st) + } + if err := f.svc.SyncBoardFreeMRStatesWithIssueSide(context.Background(), f.repo, 77, cf, f.tr); err != nil { + t.Fatal(err) + } +} + +var humanReviewMapping = map[string]string{board.ColumnHumanReview: "In Review", board.ColumnInProgress: "In Progress"} + +// A closed MR moves the cached card Human Review -> In Progress (State written, column +// cached), a reopened one moves it back, and a merge records with no move. +func TestLinkedMRCloseAndReopenMoveTheCardLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + n := f.card(t, board.ColumnHumanReview) + run := f.run(t, n, 13, "opened", time.Minute) + f.srv.ResetRequests() + + f.sync(t, map[int64]string{13: forge.MRStateClosed}) + if got := f.srv.IssueState(n); got != "In Progress" { + t.Fatalf("State after close = %q, want In Progress", got) + } + if got := f.column(t, n); got != board.ColumnInProgress { + t.Fatalf("cached column after close = %q, want In Progress", got) + } + if got := f.mrState(t, run); got != forge.MRStateClosed { + t.Fatalf("mr_state after close = %q, want closed", got) + } + + f.sync(t, map[int64]string{13: forge.MRStateOpened}) + if got := f.srv.IssueState(n); got != "In Review" { + t.Fatalf("State after reopen = %q, want In Review", got) + } + if got := f.column(t, n); got != board.ColumnHumanReview { + t.Fatalf("cached column after reopen = %q, want Human Review", got) + } + if got := f.mrState(t, run); got != forge.MRStateOpened { + t.Fatalf("mr_state after reopen = %q, want opened", got) + } + + f.sync(t, map[int64]string{13: forge.MRStateMerged}) + if got := f.mrState(t, run); got != forge.MRStateMerged { + t.Fatalf("mr_state after merge = %q, want merged", got) + } + if got := f.srv.IssueState(n); got != "In Review" { + t.Fatalf("a merge changed the State to %q", got) + } + if got := f.column(t, n); got != board.ColumnHumanReview { + t.Fatalf("a merge changed the cached column to %q", got) + } +} + +// A run that a newer run of the same issue superseded records its MR state and never +// moves the card; the latest run's edge does. +func TestLinkedMRSupersededRunRecordsWithoutMovingLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + n := f.card(t, board.ColumnHumanReview) + old := f.run(t, n, 13, "opened", time.Hour) + latest := f.run(t, n, 14, "opened", time.Minute) + f.srv.ResetRequests() + + f.sync(t, map[int64]string{13: forge.MRStateClosed, 14: forge.MRStateOpened}) + if got := f.mrState(t, old); got != forge.MRStateClosed { + t.Fatalf("superseded mr_state = %q, want closed recorded", got) + } + if w := f.writes(); len(w) != 0 { + t.Fatalf("a superseded run wrote to the issue tracker: %+v", w) + } + if got := f.column(t, n); got != board.ColumnHumanReview { + t.Fatalf("cached column = %q, want it untouched", got) + } + + f.sync(t, map[int64]string{13: forge.MRStateClosed, 14: forge.MRStateClosed}) + if got := f.mrState(t, latest); got != forge.MRStateClosed { + t.Fatalf("latest mr_state = %q, want closed", got) + } + if got := f.column(t, n); got != board.ColumnInProgress { + t.Fatalf("cached column = %q, want In Progress after the latest run's close", got) + } +} + +// An issue that left the cache is not deferred: its MR state is recorded, nothing is +// written, and the run stops being a candidate. +func TestLinkedMREvictedIssueRecordsAndIsNotPolledForeverLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + n := f.srv.SeedIssue("evicted", "b", youtracktest.EligibilityTagID) + run := f.run(t, n, 13, "opened", time.Minute) + f.srv.ResetRequests() + + f.sync(t, map[int64]string{13: forge.MRStateClosed}) + if got := f.mrState(t, run); got != forge.MRStateClosed { + t.Fatalf("mr_state = %q, want closed recorded for an uncached issue", got) + } + if w := f.writes(); len(w) != 0 { + t.Fatalf("writes for an uncached issue: %+v", w) + } + rows, err := f.q.ListBoardFreeMRStateWatchCandidates(context.Background(), f.repo) + if err != nil { + t.Fatal(err) + } + if len(rows) != 0 { + t.Fatalf("candidates = %d, want the evicted issue's run dropped", len(rows)) + } +} + +// A target column the issue tracker refuses (no State mapped) records the MR state and +// leaves the card where it is, without a write. +func TestLinkedMRRefusedMoveRecordsAndLeavesTheCardLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, map[string]string{board.ColumnHumanReview: "In Review"}) // In Progress unmapped + n := f.card(t, board.ColumnHumanReview) + run := f.run(t, n, 13, "opened", time.Minute) + f.srv.ResetRequests() + + f.sync(t, map[int64]string{13: forge.MRStateClosed}) + if got := f.mrState(t, run); got != forge.MRStateClosed { + t.Fatalf("mr_state = %q, want closed recorded", got) + } + if got := f.column(t, n); got != board.ColumnHumanReview { + t.Fatalf("cached column = %q, want the card left in Human Review", got) + } + if got := f.srv.IssueState(n); got != "In Review" { + t.Fatalf("State = %q, want it untouched", got) + } + if w := f.writes(); len(w) != 0 { + t.Fatalf("a refused move sent a write: %+v", w) + } +} + +// The same MR sequence over a cached card yields the same edges and the same card +// column through the board lane (unlinked repo, SyncMRStates) as through the linked +// board-free lane. +func TestLinkedMRWatchMatchesTheBoardLaneLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + ctx := context.Background() + + plain := uuid.New() + f.exec(t, `INSERT INTO repos (id, connection_id, forge_project_id, path_with_namespace, web_url, default_branch, enabled) + VALUES ($1, $2, 2, $3, 'https://forge.e2e/g/p', 'main', true)`, plain, f.forge, "g/plain-"+plain.String()) + if err := EnsureBoardColumnRows(ctx, f.q, plain); err != nil { + t.Fatal(err) + } + f.exec(t, `INSERT INTO issues (repo_id, forge_issue_iid, title, state, labels, web_url, forge_updated_at, synced_at) + VALUES ($1, 7, 't', 'opened', '["Human Review"]', 'https://forge.e2e/i/7', now(), now())`, plain) + plainRun := uuid.New() + f.exec(t, `INSERT INTO runs (id, user_id, repo_id, kind, issue_iid, issue_title, issue_description, branch, mr_iid, mr_state, status) + VALUES ($1, $2, $3, 'issue', 7, 't', 'd', $4, 13, 'opened', 'completed')`, plainRun, f.user, plain, "agent/issue-7-"+plain.String()) + + n := f.card(t, board.ColumnHumanReview) + linkedRun := f.run(t, n, 13, "opened", time.Minute) + + plainColumn := func() string { + t.Helper() + var labels []byte + if err := f.pool.QueryRow(ctx, `SELECT labels FROM issues WHERE repo_id = $1 AND forge_issue_iid = 7`, plain).Scan(&labels); err != nil { + t.Fatal(err) + } + switch s := string(labels); s { + case `["Human Review"]`: + return board.ColumnHumanReview + case `["In Progress"]`: + return board.ColumnInProgress + default: + return s + } + } + type step struct{ mrState, column string } + steps := []string{forge.MRStateClosed, forge.MRStateClosed, forge.MRStateOpened, forge.MRStateLocked, forge.MRStateClosed, forge.MRStateMerged} + for i, observed := range steps { + cf := &fakeForge{mrByIID: map[int64]forge.MergeRequest{13: forgeMR(13, observed)}} + if err := f.svc.SyncMRStates(ctx, plain, 2, cf, cf); err != nil { + t.Fatal(err) + } + f.sync(t, map[int64]string{13: observed}) + got := step{f.mrState(t, linkedRun), f.column(t, n)} + want := step{f.mrState(t, plainRun), plainColumn()} + if got != want { + t.Fatalf("step %d (MR %s): linked lane = %+v, board lane = %+v", i, observed, got, want) + } + } +} + +// ListBoardFreeMRStateWatchCandidates reports issue_iid, kind and is_latest without +// changing which runs it returns or their order. +func TestBoardFreeCandidatesReportLatestnessLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + ctx := context.Background() + ins := func(kind string, iid any, mr int64, status string, age time.Duration) uuid.UUID { + t.Helper() + id := uuid.New() + f.exec(t, `INSERT INTO runs (id, user_id, repo_id, kind, issue_iid, issue_title, issue_description, branch, mr_iid, status, created_at) + VALUES ($1, $2, $3, $4, $5, 't', 'd', $6, $7, $8, now() - $9::interval)`, + id, f.user, f.repo, kind, iid, "b/"+id.String(), mr, status, fmt.Sprintf("%d seconds", int(age.Seconds()))) + return id + } + // issue 7: an older run and a newer run, both completed with an MR. + old7 := ins("issue", 7, 1, "completed", 50*time.Minute) + new7 := ins("issue", 7, 2, "completed", 40*time.Minute) + // issue 8: only a newer self_improve run exists beside it: scheduled kinds never supersede. + run8 := ins("issue", 8, 3, "completed", 30*time.Minute) + ins("self_improve", 8, 4, "running", 5*time.Minute) + // issue 9: a newer run that has not completed still supersedes it. + old9 := ins("issue", 9, 5, "completed", 20*time.Minute) + ins("issue", 9, 6, "running", 2*time.Minute) + // issue-less and scheduled runs are never superseded. + prompt := ins("prompt", nil, 7, "completed", 10*time.Minute) + selfImprove := ins("self_improve", 7, 8, "completed", 15*time.Minute) + rework := uuid.New() + f.exec(t, `INSERT INTO runs (id, user_id, repo_id, kind, issue_title, issue_description, pipeline_ref, mr_iid, target_run_id, status, created_at) + VALUES ($1, $2, $3, 'mr_rework', 't', 'd', 'b/rw', 9, $4, 'completed', now() - interval '12 minutes')`, rework, f.user, f.repo, old7) + + // A newer issue 8 run in ANOTHER repo does not supersede this repo's. + other := uuid.New() + f.exec(t, `INSERT INTO repos (id, connection_id, forge_project_id, path_with_namespace, web_url, default_branch, enabled) + VALUES ($1, $2, 3, $3, 'https://forge.e2e/g/o', 'main', true)`, other, f.forge, "g/o-"+other.String()) + f.exec(t, `INSERT INTO runs (id, user_id, repo_id, kind, issue_iid, issue_title, issue_description, branch, status) + VALUES ($1, $2, $3, 'issue', 8, 't', 'd', 'b/o', 'running')`, uuid.New(), f.user, other) + + rows, err := f.q.ListBoardFreeMRStateWatchCandidates(ctx, f.repo) + if err != nil { + t.Fatal(err) + } + type view struct { + iid pgtype.Int8 + kind string + latest bool + } + want := []struct { + id uuid.UUID + v view + }{ // created_at DESC among live rows: prompt, rework, self_improve, old9, run8, new7, old7 + {prompt, view{pgtype.Int8{}, "prompt", true}}, + {rework, view{pgtype.Int8{}, "mr_rework", true}}, + {selfImprove, view{pgtype.Int8{Int64: 7, Valid: true}, "self_improve", true}}, + {old9, view{pgtype.Int8{Int64: 9, Valid: true}, "issue", false}}, + {run8, view{pgtype.Int8{Int64: 8, Valid: true}, "issue", true}}, + {new7, view{pgtype.Int8{Int64: 7, Valid: true}, "issue", true}}, + {old7, view{pgtype.Int8{Int64: 7, Valid: true}, "issue", false}}, + } + if len(rows) != len(want) { + t.Fatalf("candidates = %d, want %d", len(rows), len(want)) + } + for i, w := range want { + r := rows[i] + if r.ID != w.id { + t.Fatalf("row %d is run %s, want %s (membership or order changed)", i, r.ID, w.id) + } + if got := (view{r.IssueIid, r.Kind, r.IsLatest}); got != w.v { + t.Errorf("row %d = %+v, want %+v", i, got, w.v) + } + } +} + +// When the poller could not build the linked issue side it hands over the +// UnavailableIssueTracker. The close edge is still recorded (a permanent cause would +// otherwise leave mr_state opened and the rework gate open on a closed MR), nothing is +// moved or written, and the closed run stays a reopen-watch candidate like any linked +// issue-lane run whose cached issue is still opened. +func TestLinkedMRCloseIsRecordedWhenTheIssueSideIsUnavailableLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + n := f.card(t, board.ColumnHumanReview) + run := f.run(t, n, 13, "opened", time.Minute) + f.tr = UnavailableIssueTracker(errors.New("down")) + f.srv.ResetRequests() + + f.sync(t, map[int64]string{13: forge.MRStateClosed}) + if got := f.mrState(t, run); got != forge.MRStateClosed { + t.Fatalf("mr_state = %q, want closed recorded despite the unavailable issue side", got) + } + if got := f.column(t, n); got != board.ColumnHumanReview { + t.Fatalf("cached column = %q, want it untouched", got) + } + if got := f.srv.IssueState(n); got != "In Review" { + t.Fatalf("State = %q, want it untouched", got) + } + if reqs := f.srv.Requests(); len(reqs) != 0 { + t.Fatalf("requests reached the issue tracker: %+v", reqs) + } + rows, err := f.q.ListBoardFreeMRStateWatchCandidates(context.Background(), f.repo) + if err != nil { + t.Fatal(err) + } + if len(rows) != 1 || rows[0].ID != run || rows[0].MrState.String != forge.MRStateClosed { + t.Fatalf("candidates = %+v, want only the closed run, watched for a reopen", rows) + } +} + +// A failed repo-source read is the one transient unavailable cause: the linked issue-lane +// close edge is deferred (mr_state stays opened, so the next tick sees it again), while a +// permanent cause is recorded as in the test above. +func TestLinkedMRCloseIsDeferredOnATransientRepoSourceReadFailureLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + n := f.card(t, board.ColumnHumanReview) + run := f.run(t, n, 13, "opened", time.Minute) + real := f.tr + f.tr = UnavailableIssueTracker(fmt.Errorf("read repo issue source: %w: %w", errRepoSourceRead, errors.New("conn reset"))) + f.srv.ResetRequests() + + f.sync(t, map[int64]string{13: forge.MRStateClosed}) + if got := f.mrState(t, run); got != forge.MRStateOpened { + t.Fatalf("mr_state = %q, want opened (edge deferred, not consumed)", got) + } + if reqs := f.srv.Requests(); len(reqs) != 0 { + t.Fatalf("requests reached the issue tracker: %+v", reqs) + } + + // The read recovers: the same edge is seen again and moves the card. + f.tr = real + f.sync(t, map[int64]string{13: forge.MRStateClosed}) + if got := f.mrState(t, run); got != forge.MRStateClosed { + t.Fatalf("mr_state = %q after recovery, want closed", got) + } +} + +// During a deferral only linked issue-lane candidates are skipped: a self_improve run +// (issue_iid set) and an issue-less helper run on the same repo are still recorded. +func TestLinkedMRDeferralStillRecordsNonIssueLaneRunsLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + n := f.card(t, board.ColumnHumanReview) + issueRun := f.run(t, n, 13, "opened", time.Minute) + selfRun := f.run(t, n, 14, "opened", 2*time.Minute) + f.exec(t, `UPDATE runs SET kind = 'self_improve' WHERE id = $1`, selfRun) + helperRun := f.run(t, n, 15, "opened", 3*time.Minute) + f.exec(t, `UPDATE runs SET kind = 'task', issue_iid = NULL WHERE id = $1`, helperRun) + f.tr = UnavailableIssueTracker(fmt.Errorf("read repo issue source: %w: %w", errRepoSourceRead, errors.New("conn reset"))) + f.srv.ResetRequests() + + f.sync(t, map[int64]string{13: forge.MRStateClosed, 14: forge.MRStateClosed, 15: forge.MRStateClosed}) + if got := f.mrState(t, issueRun); got != forge.MRStateOpened { + t.Fatalf("issue-lane mr_state = %q, want opened (deferred)", got) + } + if got := f.mrState(t, selfRun); got != forge.MRStateClosed { + t.Fatalf("self_improve mr_state = %q, want closed (recorded during the deferral)", got) + } + if got := f.mrState(t, helperRun); got != forge.MRStateClosed { + t.Fatalf("issue-less mr_state = %q, want closed (recorded during the deferral)", got) + } +} + +// A self_improve run carries an issue_iid but is not an issue-lane run: its close edge is +// recorded and moves no card, even over a Human Review card. (A prompt run cannot carry an +// issue_iid: runs_kind_shape refuses it, so only self_improve can reach the kind check.) +func TestLinkedMRSelfImproveRunNeverMovesTheCardLiveDB(t *testing.T) { + f := newLinkedMRWatch(t, humanReviewMapping) + n := f.card(t, board.ColumnHumanReview) + run := f.run(t, n, 13, "opened", time.Minute) + f.exec(t, `UPDATE runs SET kind = 'self_improve' WHERE id = $1`, run) + f.srv.ResetRequests() + + f.sync(t, map[int64]string{13: forge.MRStateClosed}) + if got := f.mrState(t, run); got != forge.MRStateClosed { + t.Fatalf("mr_state = %q, want closed recorded", got) + } + if got := f.column(t, n); got != board.ColumnHumanReview { + t.Fatalf("cached column = %q, want it untouched", got) + } + if got := f.srv.IssueState(n); got != "In Review" { + t.Fatalf("State = %q, want it untouched", got) + } + if w := f.writes(); len(w) != 0 { + t.Fatalf("a self_improve run wrote to the issue tracker: %+v", w) + } +} diff --git a/api/internal/forgesvc/mr_watch.go b/api/internal/forgesvc/mr_watch.go index 1be0b4d802..3ac9c9f65e 100644 --- a/api/internal/forgesvc/mr_watch.go +++ b/api/internal/forgesvc/mr_watch.go @@ -35,11 +35,45 @@ func (s *Service) SyncMRStates(ctx context.Context, repoID uuid.UUID, forgeProje if ctx.Err() != nil { return ctx.Err() } - s.syncOneMRState(ctx, repoID, forgeProjectID, ch, tr, c) + s.syncOneMRState(ctx, repoID, forgeProjectID, ch, tr, mrWatchCandidate{ + RunID: c.ID, IssueIID: c.IssueIid.Int64, MrIID: c.MrIid.Int64, MrState: c.MrState, + }, mrLane{missingRow: moveDeferred}) } return nil } +// mrWatchCandidate is the part of a watch candidate syncOneMRState reads, so the +// board lane (ListMRWatchCandidates) and the linked issue-lane of the board-free +// recorder (ListBoardFreeMRStateWatchCandidates) share one edge switch. +type mrWatchCandidate struct { + RunID uuid.UUID + IssueIID int64 + MrIID int64 + MrState pgtype.Text +} + +// mrLane is how a lane differs in the card moves syncOneMRState asks for. The edge +// logic itself is identical across lanes. +type mrLane struct { + // missingRow is what a move reports when the issue has no cached row. The board + // lane defers (its candidates JOIN the issue cache, so a vanished row self-heals + // next tick); the linked board-free lane skips, because its candidate set does + // not require the row and an evicted issue's MR must not be polled forever. + missingRow moveOutcome + // noMove records the edge without moving the card: a superseded run of the issue + // (a newer issue-lane run owns the card) never moves it. + noMove bool +} + +// laneMRMove runs one guarded card move for the candidate, or skips it on a lane that does +// not move (the edge is consumed). +func (s *Service) laneMRMove(ctx context.Context, lane mrLane, repoID uuid.UUID, forgeProjectID, issueIID int64, tr forge.IssueTracker, wantSource, target string) moveOutcome { + if lane.noMove { + return moveSkipped + } + return s.guardedMRMove(ctx, repoID, forgeProjectID, issueIID, tr, wantSource, target, lane.missingRow) +} + // syncOneMRState reconciles one candidate's MR state against the stored value. // // As of #527 the candidate set (Lane B of ListMRWatchCandidates) also includes @@ -56,11 +90,11 @@ func (s *Service) SyncMRStates(ctx context.Context, repoID uuid.UUID, forgeProje // read, or the move itself) mr_state is LEFT as-is, so the next poller tick // re-observes the same edge and retries — the poller cadence is the retry loop. // Consuming the edge on failure would re-create the stuck-card bug this PRD fixes. -func (s *Service) syncOneMRState(ctx context.Context, repoID uuid.UUID, forgeProjectID int64, ch forge.CodeHost, tr forge.IssueTracker, c store.ListMRWatchCandidatesRow) { - mr, err := ch.GetMergeRequest(ctx, forgeProjectID, c.MrIid.Int64) +func (s *Service) syncOneMRState(ctx context.Context, repoID uuid.UUID, forgeProjectID int64, ch forge.CodeHost, tr forge.IssueTracker, c mrWatchCandidate, lane mrLane) { + mr, err := ch.GetMergeRequest(ctx, forgeProjectID, c.MrIID) if err != nil { // A read failure is not an edge: log-and-skip, leave mr_state for next tick. - slog.Warn("forgesvc: MR-state read failed", "repo", repoID, "issue", c.IssueIid.Int64, "mr", c.MrIid.Int64, "error", err) + slog.Warn("forgesvc: MR-state read failed", "repo", repoID, "issue", c.IssueIID, "mr", c.MrIID, "error", err) return } observed := mr.State @@ -73,7 +107,7 @@ func (s *Service) syncOneMRState(ctx context.Context, repoID uuid.UUID, forgePro // baseline; ignoring it instead lets a transient glitch self-heal // (stored="opened" → glitch skipped → a later real "closed" still fires). if !forge.IsKnownMRState(observed) { - slog.Warn("forgesvc: ignoring unknown MR state", "repo", repoID, "issue", c.IssueIid.Int64, "mr", c.MrIid.Int64, "state", observed) + slog.Warn("forgesvc: ignoring unknown MR state", "repo", repoID, "issue", c.IssueIID, "mr", c.MrIID, "state", observed) return } @@ -82,7 +116,7 @@ func (s *Service) syncOneMRState(ctx context.Context, repoID uuid.UUID, forgePro // Acting on NULL→closed cannot distinguish "closed and stuck" from "closed // yesterday, already triaged elsewhere", so pre-existing closed MRs at // upgrade time never cause a wave of moves — a manual drag fixes those. - s.recordMRState(ctx, c.ID, observed) + s.recordMRState(ctx, c.RunID, observed) return } stored := c.MrState.String @@ -106,24 +140,24 @@ func (s *Service) syncOneMRState(ctx context.Context, repoID uuid.UUID, forgePro // re-entry re-stamps stop_kind and re-enqueues a cancel verdict, which is // harmless (the worker cancels on the first it consumes) and only occurs in // the narrow window where the move keeps deferring before the worker acks. - cancelErr := s.cancelReworkOnClosedMR(ctx, repoID, c.MrIid.Int64) + cancelErr := s.cancelReworkOnClosedMR(ctx, repoID, c.MrIID) if cancelErr != nil { - slog.Warn("forgesvc: cancel rework on MR close failed, will retry", "repo", repoID, "issue", c.IssueIid.Int64, "mr", c.MrIid.Int64, "error", cancelErr) + slog.Warn("forgesvc: cancel rework on MR close failed, will retry", "repo", repoID, "issue", c.IssueIID, "mr", c.MrIID, "error", cancelErr) } - moved := s.guardedMRMove(ctx, repoID, forgeProjectID, c.IssueIid.Int64, tr, board.ColumnHumanReview, board.ColumnInProgress) + moved := s.laneMRMove(ctx, lane, repoID, forgeProjectID, c.IssueIID, tr, board.ColumnHumanReview, board.ColumnInProgress) if moved == moveDeferred || cancelErr != nil { // forge failure / vanished card, or a cancel that must be retried: leave // mr_state unadvanced so the next tick re-observes the edge and retries. return } - s.recordMRState(ctx, c.ID, observed) + s.recordMRState(ctx, c.RunID, observed) case stored == forge.MRStateClosed && observed == forge.MRStateOpened: // reopen-edge (Decision 6): restore the card Human Review-ward, // symmetrically, guarded from In Progress (where the close-edge left it). - if s.guardedMRMove(ctx, repoID, forgeProjectID, c.IssueIid.Int64, tr, board.ColumnInProgress, board.ColumnHumanReview) == moveDeferred { + if s.laneMRMove(ctx, lane, repoID, forgeProjectID, c.IssueIID, tr, board.ColumnInProgress, board.ColumnHumanReview) == moveDeferred { return } - s.recordMRState(ctx, c.ID, observed) + s.recordMRState(ctx, c.RunID, observed) default: // A KNOWN non-edge transition: record, never move (this arm makes no board // move). Unknown states never reach here — they are ignored before the @@ -142,12 +176,12 @@ func (s *Service) syncOneMRState(ctx context.Context, repoID uuid.UUID, forgePro // `locked` was mid-merge. A `locked`->`opened` settle is a non-terminal // transition and must NOT cancel — the rework gate stays open. if observed == forge.MRStateMerged || observed == forge.MRStateClosed { - if err := s.cancelReworkOnClosedMR(ctx, repoID, c.MrIid.Int64); err != nil { - slog.Warn("forgesvc: cancel rework on MR terminal state failed, will retry", "repo", repoID, "issue", c.IssueIid.Int64, "mr", c.MrIid.Int64, "state", observed, "error", err) + if err := s.cancelReworkOnClosedMR(ctx, repoID, c.MrIID); err != nil { + slog.Warn("forgesvc: cancel rework on MR terminal state failed, will retry", "repo", repoID, "issue", c.IssueIID, "mr", c.MrIID, "state", observed, "error", err) return // leave mr_state unadvanced so the next tick retries } } - s.recordMRState(ctx, c.ID, observed) + s.recordMRState(ctx, c.RunID, observed) } } @@ -185,15 +219,18 @@ const ( // source-column guard makes the retry a no-op (the card already moved) — except // the narrow case where a human dragged the card back to wantSource in the crash // window, which then gets re-moved once; accepted (crash-window-sized exposure). -func (s *Service) guardedMRMove(ctx context.Context, repoID uuid.UUID, forgeProjectID, issueIID int64, tr forge.IssueTracker, wantSource, target string) moveOutcome { +func (s *Service) guardedMRMove(ctx context.Context, repoID uuid.UUID, forgeProjectID, issueIID int64, tr forge.IssueTracker, wantSource, target string, missingRow moveOutcome) moveOutcome { issue, err := s.q.GetIssueByIID(ctx, store.GetIssueByIIDParams{RepoID: repoID, ForgeIssueIid: issueIID}) if err != nil { - // No cached card (evicted/de-labeled since the candidate scan): leave the - // edge. If the issue is genuinely gone it stops being a candidate next tick - // (the query JOINs issues), so this does not retry forever. - if !errors.Is(err, pgx.ErrNoRows) { - slog.Warn("forgesvc: MR-state load issue", "repo", repoID, "issue", issueIID, "error", err) + // No cached card (evicted/de-labeled since the candidate scan): the caller's + // lane decides. The board lane leaves the edge: if the issue is genuinely gone + // it stops being a candidate next tick (the query JOINs issues), so this does + // not retry forever. The linked board-free lane skips (its query does not JOIN + // issues) so the MR state is still recorded and the run stops being polled. + if errors.Is(err, pgx.ErrNoRows) { + return missingRow } + slog.Warn("forgesvc: MR-state load issue", "repo", repoID, "issue", issueIID, "error", err) return moveDeferred } // Never auto-move a closed issue: Closed is a terminal placement, not a @@ -216,7 +253,7 @@ func (s *Service) guardedMRMove(ctx context.Context, repoID uuid.UUID, forgeProj if err := json.Unmarshal(issue.Labels, &labels); err != nil { labels = []string{} } - current, _, _ := board.ResolveColumn(labels, issue.State, position) + current, _, _ := board.ResolveColumn(board.CardColumnLabels(isLinkedTracker(tr), labels, issue.IssueTrackerColumn.String), issue.State, position) if current != wantSource { // A human placed the card elsewhere since the run parked it — manual drags // win (same baseline discipline as runlifecycle.apply). Consume the edge; @@ -226,12 +263,28 @@ func (s *Service) guardedMRMove(ctx context.Context, repoID uuid.UUID, forgeProj if _, err := s.AutoMove(ctx, tr, forgeProjectID, issue, cols, target); err != nil { if IssueWriteSkippable(err) { - // A linked repo's card is never moved by uzi (writes arrive in M5) or its - // issue is gone: consume the edge so recordMRState still records the MR - // state, and never retry forever (PRD #4173 D10). + // The linked issue is gone or the write is not offered: + // consume the edge so recordMRState still records the MR state, and never + // retry forever (PRD #4173 D10). slog.Info("forgesvc: MR-state move skipped for a linked repo", "repo", repoID, "issue", issueIID, "target", target, "reason", err) return moveSkipped } + if LinkedIssueResolved(err) { + // A person resolved the issue in the issue tracker since the last sync, so the + // cache still shows it open. Their resolution wins: consume the edge without a + // move, as for a closed issue in the cache. + slog.Info("forgesvc: MR-state move skipped, the issue is resolved in the issue tracker", "repo", repoID, "issue", issueIID, "target", target, "reason", err) + return moveSkipped + } + if ColumnMoveRefused(err) { + // The linked issue tracker refused the move before any request: the target + // column has no State mapped, the mapping is unset, or the State is not + // allowed. Unlike the run lifecycle, the MR watch does not replay it: the + // MR state is recorded and the card stays where it is (a manual drag + // fixes it). Accepted exception, PRD #4173 M5. + slog.Info("forgesvc: MR-state move refused by the issue tracker, edge consumed without a move", "repo", repoID, "issue", issueIID, "target", target, "reason", err) + return moveSkipped + } // Forge error: leave mr_state unadvanced so the next tick re-observes this // edge and retries once the forge is reachable again. slog.Warn("forgesvc: MR-state move failed, will retry", "repo", repoID, "issue", issueIID, "target", target, "error", err) @@ -248,7 +301,8 @@ func (s *Service) guardedMRMove(ctx context.Context, repoID uuid.UUID, forgeProj // while ListBoardFreeMRStateWatchCandidates owns exactly the complement // (issue_iid IS NULL OR kind IN ('prompt','self_improve'), plus EVERY run of a repo linked // to an external issue tracker, which the board lane excludes — PRD #4173 M4) — a run -// cannot satisfy both predicates at once. Best-effort: a write failure only means the same edge is +// cannot satisfy both predicates at once. A linked repo's issue-lane runs are still +// reconciled by syncOneMRState, through SyncBoardFreeMRStatesWithIssueSide (PRD #4173 M5). Best-effort: a write failure only means the same edge is // re-observed next tick, and the guards make the retry a no-op if the card has // already moved. func (s *Service) recordMRState(ctx context.Context, runID uuid.UUID, state string) { diff --git a/api/internal/forgesvc/prd_link_patch.go b/api/internal/forgesvc/prd_link_patch.go index 734c233e0f..e5649a8bca 100644 --- a/api/internal/forgesvc/prd_link_patch.go +++ b/api/internal/forgesvc/prd_link_patch.go @@ -183,7 +183,7 @@ func (s *Service) patchOnePRDLink(ctx context.Context, forgeProjectID int64, ch if err := tr.UpdateIssueDescription(ctx, forgeProjectID, c.IssueIid.Int64, desc); err != nil { if IssueWriteSkippable(err) { - s.settlePRDLink(ctx, lg, c.ID, "PRD link patch: issue writes are not offered for this repo; nothing to patch") + s.settlePRDLink(ctx, lg, c.ID, "PRD link patch: the linked issue is gone; nothing to patch") return } // Leave the marker: the write is what we are retrying. diff --git a/api/internal/forgesvc/repo_sides.go b/api/internal/forgesvc/repo_sides.go index 65b825e219..7c3e17a9ee 100644 --- a/api/internal/forgesvc/repo_sides.go +++ b/api/internal/forgesvc/repo_sides.go @@ -2,10 +2,12 @@ package forgesvc import ( "context" + "encoding/json" "errors" "fmt" "github.com/google/uuid" + "github.com/jackc/pgx/v5" "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/issuetracker" @@ -16,7 +18,7 @@ import ( // one repo binding. *issuetrackersvc.Service satisfies it. Optional on a Service: // without it a linked repo fails closed (PRD #4173 DL-5). type IssueTrackerDrivers interface { - DriverFor(ctx context.Context, userID, connID uuid.UUID, binding issuetracker.Binding, stateField, open, closed string) (issuetracker.Driver, error) + DriverFor(ctx context.Context, userID, connID uuid.UUID, binding issuetracker.Binding, states issuetracker.StateMapping) (issuetracker.Driver, error) } // SetIssueTrackerDrivers wires the linked-issue-tracker driver source. @@ -26,6 +28,11 @@ func (s *Service) SetIssueTrackerDrivers(d IssueTrackerDrivers) { s.itDrivers = // resolver must fail, not dereference a nil store. var errNoRepoStore = errors.New("forgesvc: no repo store wired") +// errRepoSourceRead marks a failed database read of the repo's issue source (not a missing +// row, which is permanent). It is the one UnavailableIssueTracker cause the board-free MR +// watch treats as transient, see SyncBoardFreeMRStatesWithIssueSide. +var errRepoSourceRead = errors.New("forgesvc: repo issue source read failed") + // repoSource reads the one owner-scoped row both resolvers use. func (s *Service) repoSource(ctx context.Context, userID, repoID uuid.UUID) (store.GetRepoIssueSourceForUserRow, error) { if s.q == nil { @@ -47,7 +54,10 @@ var ErrLinkedRepoUnresolvable = errors.New("repo is linked to an issue tracker t func (s *Service) IssueTrackerForRepo(ctx context.Context, userID, repoID uuid.UUID) (forge.IssueTracker, error) { src, err := s.repoSource(ctx, userID, repoID) if err != nil { - return nil, fmt.Errorf("read repo issue source: %w", err) + if errors.Is(err, pgx.ErrNoRows) || errors.Is(err, errNoRepoStore) { + return nil, fmt.Errorf("read repo issue source: %w", err) + } + return nil, fmt.Errorf("read repo issue source: %w: %w", errRepoSourceRead, err) } if !src.IssueTrackerConnectionID.Valid { f, err := s.ForgeForConnection(src.ForgeType, src.BaseUrl, src.TokenCiphertext) @@ -61,15 +71,21 @@ func (s *Service) IssueTrackerForRepo(ctx context.Context, userID, repoID uuid.U !src.EligibilityTagID.Valid || !src.EligibilityTagName.Valid { return nil, ErrLinkedRepoUnresolvable } + var columns map[string]string + if len(src.ColumnStates) > 0 { + if err := json.Unmarshal(src.ColumnStates, &columns); err != nil { + return nil, fmt.Errorf("decode stored column states: %w", err) + } + } d, err := s.itDrivers.DriverFor(ctx, userID, uuid.UUID(src.LinkedConnectionID.Bytes), issuetracker.Binding{ Project: issuetracker.ProjectRef{ID: src.IssueTrackerProjectID.String, ShortName: src.IssueTrackerProjectShort.String}, ProjectID: src.ForgeProjectID, EligibilityTag: issuetracker.TagRef{ID: src.EligibilityTagID.String, Name: src.EligibilityTagName.String}, - }, "", "", "") + }, issuetracker.StateMapping{Open: src.OpenState.String, Closed: src.ClosedState.String, Columns: columns}) if err != nil { return nil, fmt.Errorf("build linked issue tracker: %w", err) } - return &issueWriteGuard{inner: d}, nil + return &issueWriteGuard{inner: d, eligibility: src.EligibilityTagName.String}, nil } // CodeHostForRepo returns the code-host side of a repo: always the repo's own diff --git a/api/internal/forgesvc/repo_sides_test.go b/api/internal/forgesvc/repo_sides_test.go index 9316e07cf6..7ad7eb469b 100644 --- a/api/internal/forgesvc/repo_sides_test.go +++ b/api/internal/forgesvc/repo_sides_test.go @@ -3,6 +3,7 @@ package forgesvc import ( "context" "errors" + "slices" "testing" "time" @@ -12,6 +13,7 @@ import ( "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/forge/forgetest" + "github.com/vtmocanu/uzi/api/internal/issuesource" "github.com/vtmocanu/uzi/api/internal/issuetracker" "github.com/vtmocanu/uzi/api/internal/secretbox" "github.com/vtmocanu/uzi/api/internal/store" @@ -70,6 +72,14 @@ func (d *stubDriver) UpdateIssueDescription(context.Context, int64, int64, strin d.writes = append(d.writes, "UpdateIssueDescription") return nil } +func (d *stubDriver) MoveToColumn(context.Context, int64, int64, forge.ColumnMove) error { + d.writes = append(d.writes, "MoveToColumn") + return nil +} +func (d *stubDriver) ListIssuesWithState(context.Context, int64, forge.ListIssuesOptions) ([]issuesource.StatedIssue, error) { + d.reads = append(d.reads, "ListIssuesWithState") + return []issuesource.StatedIssue{{Issue: forge.Issue{IID: 3}, State: "In Progress"}}, nil +} func (d *stubDriver) SetIssueState(context.Context, int64, int64, forge.IssueState) error { d.writes = append(d.writes, "SetIssueState") return nil @@ -84,18 +94,18 @@ func (d *stubDriver) CreateIssueNote(context.Context, int64, int64, string) (for } type fakeDrivers struct { - driver *stubDriver - err error - calls int - gotUser, gotConn uuid.UUID - gotBinding issuetracker.Binding - gotState, gotOpen, gotC string + driver *stubDriver + err error + calls int + gotUser, gotConn uuid.UUID + gotBinding issuetracker.Binding + gotStates issuetracker.StateMapping } -func (f *fakeDrivers) DriverFor(_ context.Context, userID, connID uuid.UUID, b issuetracker.Binding, stateField, open, closed string) (issuetracker.Driver, error) { +func (f *fakeDrivers) DriverFor(_ context.Context, userID, connID uuid.UUID, b issuetracker.Binding, states issuetracker.StateMapping) (issuetracker.Driver, error) { f.calls++ f.gotUser, f.gotConn, f.gotBinding = userID, connID, b - f.gotState, f.gotOpen, f.gotC = stateField, open, closed + f.gotStates = states if f.err != nil { return nil, f.err } @@ -171,6 +181,7 @@ func writeAll(ctx context.Context, tr forge.IssueTracker, projectID int64) (acce func() error { _, err := tr.CreateIssueNote(ctx, projectID, 9, "note"); return err }, func() error { _, err := tr.CreateIssue(ctx, projectID, "t", "d", nil); return err }, func() error { return tr.SetIssueState(ctx, projectID, 9, forge.StateClosed) }, + func() error { return tr.MoveToColumn(ctx, projectID, 9, forge.ColumnMove{Column: "In Progress"}) }, func() error { return tr.UpdateIssueDescription(ctx, projectID, 9, "d") }, func() error { return tr.EnsureLabels(ctx, projectID, []forge.Label{{Name: "x"}}) }, } @@ -202,8 +213,13 @@ func TestIssueTrackerForRepoUnlinkedWritesReachTheForge(t *testing.T) { t.Fatalf("an unlinked repo's write was refused by the guard: %v", e) } } - if got := len(h.rec.Calls()); got != 6 { - t.Fatalf("recorder saw %d calls, want 6 writes: %v", got, h.rec.Calls()) + // The six other writes are one call each; the move is logged in Moves() and reaches + // the recorder as its one label call (forge.MoveByLabels), seven calls in all. + if got := len(h.rec.Calls()); got != 7 { + t.Fatalf("recorder saw %d calls, want 7: %v", got, h.rec.Calls()) + } + if got := len(h.rec.Moves()); got != 1 { + t.Fatalf("recorder saw %d moves, want 1: %v", got, h.rec.Moves()) } if h.drivers.calls != 0 { t.Fatalf("the issue tracker driver source was used for an unlinked repo") @@ -232,24 +248,33 @@ func TestIssueTrackerForRepoLinkedBuildsTheBoundDriverBehindTheGuard(t *testing. if d.calls != 1 || d.gotUser != h.user || d.gotConn != conn || d.gotBinding != want { t.Fatalf("DriverFor got (calls %d, user %v, conn %v, binding %+v), want the repo row's binding for its owner", d.calls, d.gotUser, d.gotConn, d.gotBinding) } - if d.gotState != "" || d.gotOpen != "" || d.gotC != "" { - t.Errorf("state mapping arguments must be empty until M5, got %q %q %q", d.gotState, d.gotOpen, d.gotC) + if d.gotStates.Field != "" || d.gotStates.Open != "" || d.gotStates.Closed != "" || len(d.gotStates.Columns) != 0 { + t.Errorf("a repo with no saved state mapping must pass the zero mapping, got %+v", d.gotStates) } if h.built != 0 { t.Errorf("a linked repo's issue side must not build the forge driver") } + // Every write is offered (PRD #4173 M5, U6 lifts the note and description); only the + // eligibility-tag removal is refused, below. accepted, errs := writeAll(context.Background(), tr, 4242) - if accepted != 0 || len(errs) != 6 { - t.Fatalf("accepted %d writes with %d errors, want none accepted", accepted, len(errs)) + if accepted != 7 || len(errs) != 0 { + t.Fatalf("accepted %d writes with errors %v, want all 7 accepted", accepted, errs) } - for _, e := range errs { - if !errors.Is(e, ErrIssueWriteNotOffered) || !IssueWriteSkippable(e) { - t.Errorf("write error = %v, want ErrIssueWriteNotOffered", e) - } + wantWrites := []string{"UpdateIssueLabels", "CreateIssueNote", "CreateIssue", "SetIssueState", "MoveToColumn", "UpdateIssueDescription", "EnsureLabels"} + if !slices.Equal(d.driver.writes, wantWrites) { + t.Fatalf("writes that reached the driver = %v, want %v", d.driver.writes, wantWrites) + } + // The guard was built with the row's stored eligibility name ("uzi"): removing it is + // refused before the driver, adding it is not. + if err := tr.UpdateIssueLabels(context.Background(), 4242, 9, nil, []string{"uzi"}); !errors.Is(err, ErrIssueWriteNotOffered) { + t.Fatalf("removing the eligibility name: %v, want ErrIssueWriteNotOffered", err) + } + if err := tr.UpdateIssueLabels(context.Background(), 4242, 9, []string{"uzi"}, nil); err != nil { + t.Fatalf("adding the eligibility name: %v", err) } - if len(d.driver.writes) != 0 { - t.Fatalf("writes reached the driver: %v", d.driver.writes) + if len(d.driver.writes) != len(wantWrites)+1 { + t.Fatalf("writes that reached the driver = %v, want only the add after the refused removal", d.driver.writes) } if _, err := tr.ListIssues(context.Background(), 4242, forge.ListIssuesOptions{}); err != nil { t.Fatal(err) @@ -259,6 +284,40 @@ func TestIssueTrackerForRepoLinkedBuildsTheBoundDriverBehindTheGuard(t *testing. } } +// The stored open, closed and column States of the repo row reach DriverFor (PRD #4173 M5); +// the State field stays empty so the driver's default applies. +func TestIssueTrackerForRepoPassesTheStoredStateMapping(t *testing.T) { + h, src := newSidesHarness(t) + h.svc.SetIssueTrackerDrivers(h.drivers) + repo := h.addRepo(t, src, h.user, func(r *store.GetRepoIssueSourceForUserRow) { + linkTo(uuid.New(), true)(r) + r.OpenState = pgtype.Text{String: "To Do", Valid: true} + r.ClosedState = pgtype.Text{String: "Done", Valid: true} + r.ColumnStates = []byte(`{"In Progress":"Doing","Review":"In Review"}`) + }) + if _, err := h.svc.IssueTrackerForRepo(context.Background(), h.user, repo); err != nil { + t.Fatal(err) + } + got := h.drivers.gotStates + if got.Field != "" || got.Open != "To Do" || got.Closed != "Done" || + len(got.Columns) != 2 || got.Columns["In Progress"] != "Doing" || got.Columns["Review"] != "In Review" { + t.Fatalf("DriverFor got %+v, want the stored mapping", got) + } +} + +// A malformed stored column map fails closed rather than building a driver without it. +func TestIssueTrackerForRepoRefusesAMalformedColumnMap(t *testing.T) { + h, src := newSidesHarness(t) + h.svc.SetIssueTrackerDrivers(h.drivers) + repo := h.addRepo(t, src, h.user, func(r *store.GetRepoIssueSourceForUserRow) { + linkTo(uuid.New(), true)(r) + r.ColumnStates = []byte(`[1]`) + }) + if _, err := h.svc.IssueTrackerForRepo(context.Background(), h.user, repo); err == nil || h.drivers.calls != 0 { + t.Fatalf("err = %v, DriverFor calls = %d, want an error before any driver is built", err, h.drivers.calls) + } +} + func TestIssueTrackerForRepoLinkedFailsClosed(t *testing.T) { t.Run("connection missing for the owner", func(t *testing.T) { h, src := newSidesHarness(t) @@ -344,3 +403,46 @@ func TestRepoSidesPassBuilderErrorThrough(t *testing.T) { t.Errorf("CodeHostForRepo = (%v, %v), want (nil, builder error)", ch, err) } } + +// errStore fails the repo-source read with a fixed error. +type errStore struct { + IssueStore + err error +} + +func (s errStore) GetRepoIssueSourceForUser(context.Context, store.GetRepoIssueSourceForUserParams) (store.GetRepoIssueSourceForUserRow, error) { + return store.GetRepoIssueSourceForUserRow{}, s.err +} + +func TestIssueTrackerForRepoClassifiesARepoSourceReadFailure(t *testing.T) { + box, err := secretbox.New([]byte("0123456789abcdefghijklmnopqrstuv")) + if err != nil { + t.Fatal(err) + } + build := func(q IssueStore) *Service { + return NewWithForgeBuilder(q, box, time.Second, nil, func(forge.Type, string, string, time.Duration) (forge.Forge, error) { + return nil, errors.New("unused") + }) + } + ctx := context.Background() + + t.Run("a database error is transient and keeps its cause", func(t *testing.T) { + inner := errors.New("conn reset") + _, err := build(errStore{err: inner}).IssueTrackerForRepo(ctx, uuid.New(), uuid.New()) + if !errors.Is(err, errRepoSourceRead) || !errors.Is(err, inner) { + t.Fatalf("err = %v, want errRepoSourceRead wrapping the inner error", err) + } + }) + t.Run("a missing row is permanent", func(t *testing.T) { + _, err := build(errStore{err: pgx.ErrNoRows}).IssueTrackerForRepo(ctx, uuid.New(), uuid.New()) + if err == nil || errors.Is(err, errRepoSourceRead) { + t.Fatalf("err = %v, want a non-transient error", err) + } + }) + t.Run("no store is not transient", func(t *testing.T) { + _, err := build(nil).IssueTrackerForRepo(ctx, uuid.New(), uuid.New()) + if !errors.Is(err, errNoRepoStore) || errors.Is(err, errRepoSourceRead) { + t.Fatalf("err = %v, want errNoRepoStore only", err) + } + }) +} diff --git a/api/internal/forgesvc/service.go b/api/internal/forgesvc/service.go index 27f00a6305..292cdc146f 100644 --- a/api/internal/forgesvc/service.go +++ b/api/internal/forgesvc/service.go @@ -127,6 +127,8 @@ type IssueStore interface { // board_position so a reopened card lands at the bottom of its lane. UpdateIssueState(ctx context.Context, arg store.UpdateIssueStateParams) (store.Issue, error) ReopenIssueState(ctx context.Context, arg store.ReopenIssueStateParams) (store.Issue, error) + // SetIssueTrackerColumn writes only a linked card's cached board column (PRD #4173 M5). + SetIssueTrackerColumn(ctx context.Context, arg store.SetIssueTrackerColumnParams) (store.Issue, error) DeleteIssuesNotIn(ctx context.Context, arg store.DeleteIssuesNotInParams) (int64, error) // Used by the MR-close watcher (mr_watch.go). ListMRWatchCandidates(ctx context.Context, repoID uuid.UUID) ([]store.ListMRWatchCandidatesRow, error) @@ -518,7 +520,8 @@ func (s *Service) ForgeForConnection(forgeType, baseURL string, tokenCiphertext // AutoMove plans a column label delta, writes it forge-first, then applies it to // the current cached row. Forge errors leave the cache untouched; a driver can // partially change remote labels before returning an error. No compensation is -// attempted. The returned issue is the re-cached row. Caller guards own closed +// attempted. The returned issue is the re-cached row. On a linked repo the cache +// write is SetIssueTrackerColumn instead of the label write. Caller guards own closed // issue and manual-drag policy. func (s *Service) AutoMove(ctx context.Context, f forge.IssueTracker, forgeProjectID int64, issue store.Issue, columns []store.BoardColumn, target string) (store.Issue, error) { var current []string @@ -537,22 +540,29 @@ func (s *Service) AutoMove(ctx context.Context, f forge.IssueTracker, forgeProje // be missing or drifted (e.g. the Upcoming→Planned rename, #176). add is the // target column and has at most one element; an empty add (already in the target // column, or a move to Open) ensures nothing and keeps the cheap no-forge-call path. + move := forge.ColumnMove{Column: target, Add: add, Remove: remove} if len(add) > 0 { - labels := make([]forge.Label, 0, len(add)) + move.Ensure = make([]forge.Label, 0, len(add)) for _, name := range add { - labels = append(labels, forge.Label{Name: name, Color: ColumnColor(name)}) - } - if err := f.EnsureLabels(ctx, forgeProjectID, labels); err != nil { - return store.Issue{}, err + move.Ensure = append(move.Ensure, forge.Label{Name: name, Color: ColumnColor(name)}) } } - // Forge-first: apply the label change remotely before touching the cache. On - // failure the cache is untouched (UpdateIssueLabels no-ops on empty sets, so a - // move to a column the card already sits in costs no forge call). - if err := f.UpdateIssueLabels(ctx, forgeProjectID, issue.ForgeIssueIid, add, remove); err != nil { + // Forge-first: apply the move remotely before touching the cache. On failure + // the cache is untouched (a label-backed issue tracker's UpdateIssueLabels + // no-ops on empty sets, so a move to a column the card already sits in costs + // no forge call). + if err := f.MoveToColumn(ctx, forgeProjectID, issue.ForgeIssueIid, move); err != nil { return store.Issue{}, err } + if isLinkedTracker(f) { + // A linked card's column is not a label: cache it in issue_tracker_column and + // touch nothing else (PRD #4173 M5), so a stale snapshot of state or labels is + // never written back. + return s.q.SetIssueTrackerColumn(ctx, store.SetIssueTrackerColumnParams{ + RepoID: issue.RepoID, ForgeIssueIid: issue.ForgeIssueIid, IssueTrackerColumn: target, + }) + } labelsJSON, err := json.Marshal(append([]string{}, newLabels...)) if err != nil { @@ -846,13 +856,13 @@ func (s *Service) FullSync(ctx context.Context, repoID uuid.UUID, forgeProjectID } advanceGroups() - if err := s.upsertIssues(ctx, repoID, issues, ""); err != nil { + if err := s.upsertIssues(ctx, repoID, issues, "", nil); err != nil { return Marks{}, err } - if err := s.upsertIssues(ctx, repoID, extra, ""); err != nil { + if err := s.upsertIssues(ctx, repoID, extra, "", nil); err != nil { return Marks{}, err } - if err := s.upsertIssues(ctx, repoID, findingExtra, ""); err != nil { + if err := s.upsertIssues(ctx, repoID, findingExtra, "", nil); err != nil { return Marks{}, err } // A clean SET of fetches that legitimately returns zero issues DOES evict @@ -946,13 +956,13 @@ func (s *Service) IncrementalSync(ctx context.Context, repoID uuid.UUID, forgePr // deliberately ignored), so incremental passes cannot shift which page the // next FullSync examines. Recorded-op settlement on later pages therefore // happens as FullSync rotates to them. - if err := s.upsertIssues(ctx, repoID, issues, ""); err != nil { + if err := s.upsertIssues(ctx, repoID, issues, "", nil); err != nil { return m, err } - if err := s.upsertIssues(ctx, repoID, withoutLabel(openIssues, uziLabel), ""); err != nil { + if err := s.upsertIssues(ctx, repoID, withoutLabel(openIssues, uziLabel), "", nil); err != nil { return m, err } - if err := s.upsertIssues(ctx, repoID, withoutLabel(findingIssues, uziLabel), ""); err != nil { + if err := s.upsertIssues(ctx, repoID, withoutLabel(findingIssues, uziLabel), "", nil); err != nil { return m, err } return m.Advance(Marks{PRD: maxUpdatedAt(issues), Open: maxUpdatedAt(openIssues), Finding: maxUpdatedAt(findingIssues)}), nil @@ -1010,10 +1020,12 @@ func maxUpdatedAt(issues []forge.Issue) time.Time { // upsertIssues writes each forge issue into the cache, computing has_prd_link at // write time. issueKeyShort is the linked repo's issue tracker project short name: // non-empty, every row is stored with issue_key = short + "-" + iid (D10); empty (the -// forge sync), issue_key is written NULL. It reports only an error: no high-water mark is read off this path, +// forge sync), issue_key is written NULL. columns maps an issue iid to its cached board +// column and is read only on the linked path (nil from the forge sync, which never +// writes the column). It reports only an error: no high-water mark is read off this path, // deliberately, because the time.Now() substitution below would put one past the // instant its fetch observed (see maxUpdatedAt). -func (s *Service) upsertIssues(ctx context.Context, repoID uuid.UUID, issues []forge.Issue, issueKeyShort string) error { +func (s *Service) upsertIssues(ctx context.Context, repoID uuid.UUID, issues []forge.Issue, issueKeyShort string, columns map[int64]string) error { for _, is := range issues { labelsJSON, err := json.Marshal(is.Labels) if err != nil { @@ -1053,8 +1065,27 @@ func (s *Service) upsertIssues(ctx context.Context, repoID uuid.UUID, issues []f } if issueKeyShort != "" { // The linked sync re-reads every tagged issue each tick, so it writes only rows - // that changed; the forge sync keeps the unconditional write. - err = s.q.UpsertIssueIfChanged(ctx, store.UpsertIssueIfChangedParams(params)) + // that changed; the forge sync keeps the unconditional write. The cached board + // column has three writers: this linked upsert, SetIssueTrackerColumn (a move) and + // ClearIssueTrackerColumnsNotIn (a linked ConfigureColumns); an empty column (the + // Open lane) is stored as NULL. + changed := store.UpsertIssueIfChangedParams{ + RepoID: params.RepoID, + ForgeIssueIid: params.ForgeIssueIid, + Title: params.Title, + State: params.State, + Labels: params.Labels, + AssigneeIds: params.AssigneeIds, + WebUrl: params.WebUrl, + Author: params.Author, + HasPrdLink: params.HasPrdLink, + ForgeUpdatedAt: params.ForgeUpdatedAt, + IssueKey: params.IssueKey, + } + if col := columns[is.IID]; col != "" { + changed.IssueTrackerColumn = pgtype.Text{String: col, Valid: true} + } + err = s.q.UpsertIssueIfChanged(ctx, changed) } else { _, err = s.q.UpsertIssue(ctx, params) } diff --git a/api/internal/forgesvc/sync_test.go b/api/internal/forgesvc/sync_test.go index 08aab06345..139be4d8b3 100644 --- a/api/internal/forgesvc/sync_test.go +++ b/api/internal/forgesvc/sync_test.go @@ -14,6 +14,7 @@ import ( "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/forge/forgetest" + "github.com/vtmocanu/uzi/api/internal/issuesource" "github.com/vtmocanu/uzi/api/internal/settings" "github.com/vtmocanu/uzi/api/internal/store" ) @@ -27,6 +28,9 @@ type fakeForge struct { issues []forge.Issue listErr error listCalls []forge.ListIssuesOptions + // states scripts the issue tracker State name ListIssuesWithState reports per issue + // iid (PRD #4173 M5); a missing iid reports "" (no readable State). + states map[int64]string // PRD #102 M6 Decision 9 made both sync paths issue TWO ListIssues calls: the // PRD-labelled one (state=all) and an additive open, no-label one. openIssues / @@ -271,6 +275,23 @@ func (f *fakeForge) UpdateIssueLabels(_ context.Context, _, _ int64, add, remove } return f.updateErr } + +// MoveToColumn performs the label-backed move over this fake's own label +// methods; the embedded BaseFake cannot dispatch to them. +func (f *fakeForge) ListIssuesWithState(ctx context.Context, p int64, opts forge.ListIssuesOptions) ([]issuesource.StatedIssue, error) { + issues, err := f.ListIssues(ctx, p, opts) + if err != nil { + return nil, err + } + out := make([]issuesource.StatedIssue, 0, len(issues)) + for _, is := range issues { + out = append(out, issuesource.StatedIssue{Issue: is, State: f.states[is.IID]}) + } + return out, nil +} +func (f *fakeForge) MoveToColumn(ctx context.Context, p, i int64, m forge.ColumnMove) error { + return forge.MoveByLabels(ctx, f, p, i, m) +} func (f *fakeForge) GetMergeRequest(_ context.Context, _, mrIID int64) (forge.MergeRequest, error) { f.mrCalls = append(f.mrCalls, mrIID) if err, ok := f.mrErrByIID[mrIID]; ok { @@ -316,8 +337,12 @@ type fakeStore struct { upserts []store.UpsertIssueParams // changedUpserts counts the UpsertIssueIfChanged calls (the linked sync's write); they also land in upserts. changedUpserts int - labelUpserts []store.UpsertIssueLabelsParams // AutoMove / SetIssueLabel writes (PRD #767) - deleteCalls []store.DeleteIssuesNotInParams + // changedColumns is the issue_tracker_column each UpsertIssueIfChanged call carried (PRD #4173 M5). + changedColumns []pgtype.Text + // columnWrites records SetIssueTrackerColumn calls (a linked move's cache write). + columnWrites []store.SetIssueTrackerColumnParams + labelUpserts []store.UpsertIssueLabelsParams // AutoMove / SetIssueLabel writes (PRD #767) + deleteCalls []store.DeleteIssuesNotInParams // Close/reopen state flips (PRD #1034 M2). stateUpdates records UpdateIssueState // (bare close) calls, reopens records ReopenIssueState calls — each by identity so a @@ -392,7 +417,12 @@ func (s *fakeStore) UpsertIssue(_ context.Context, arg store.UpsertIssueParams) return store.Issue{RepoID: arg.RepoID, ForgeIssueIid: arg.ForgeIssueIid, State: arg.State, Labels: arg.Labels}, nil } func (s *fakeStore) UpsertIssueIfChanged(_ context.Context, arg store.UpsertIssueIfChangedParams) error { - s.upserts = append(s.upserts, store.UpsertIssueParams(arg)) + s.upserts = append(s.upserts, store.UpsertIssueParams{ + RepoID: arg.RepoID, ForgeIssueIid: arg.ForgeIssueIid, Title: arg.Title, State: arg.State, + Labels: arg.Labels, AssigneeIds: arg.AssigneeIds, WebUrl: arg.WebUrl, Author: arg.Author, + HasPrdLink: arg.HasPrdLink, ForgeUpdatedAt: arg.ForgeUpdatedAt, IssueKey: arg.IssueKey, + }) + s.changedColumns = append(s.changedColumns, arg.IssueTrackerColumn) s.changedUpserts++ return nil } @@ -448,6 +478,13 @@ func (s *fakeStore) ReopenIssueState(_ context.Context, arg store.ReopenIssueSta // (the real query leaves them untouched); default empty mimics a card with no column. return store.Issue{RepoID: arg.RepoID, ForgeIssueIid: arg.ForgeIssueIid, State: string(forge.StateOpened), Labels: s.reopenLabels}, nil } +func (s *fakeStore) SetIssueTrackerColumn(_ context.Context, arg store.SetIssueTrackerColumnParams) (store.Issue, error) { + s.columnWrites = append(s.columnWrites, arg) + row := s.issue + row.RepoID, row.ForgeIssueIid = arg.RepoID, arg.ForgeIssueIid + row.IssueTrackerColumn = pgtype.Text{String: arg.IssueTrackerColumn, Valid: arg.IssueTrackerColumn != ""} + return row, nil +} func (s *fakeStore) DeleteIssuesNotIn(_ context.Context, arg store.DeleteIssuesNotInParams) (int64, error) { s.deleteCalls = append(s.deleteCalls, arg) return 0, nil @@ -889,6 +926,45 @@ func TestAutoMoveEnsuresTargetColumnLabel(t *testing.T) { }) } +// moveRecordingForge records the ColumnMove each MoveToColumn receives, then +// delegates to fakeForge's label-backed move. +type moveRecordingForge struct { + fakeForge + moves []forge.ColumnMove +} + +func (f *moveRecordingForge) MoveToColumn(ctx context.Context, p, i int64, m forge.ColumnMove) error { + f.moves = append(f.moves, m) + return f.fakeForge.MoveToColumn(ctx, p, i, m) +} + +// TestAutoMovePassesTargetAsMoveColumn pins the Column AutoMove hands to the +// issue tracker: the target column's name, and "" for the Open target. An issue +// tracker with its own workflow field reads only this field. +func TestAutoMovePassesTargetAsMoveColumn(t *testing.T) { + for _, tc := range []struct { + name, target string + labels []string + }{ + {"named column", "Planned", nil}, + {"open target", "", []string{"Planned"}}, + } { + t.Run(tc.name, func(t *testing.T) { + svc := newTestService(&fakeStore{}) + f := &moveRecordingForge{} + if _, err := svc.AutoMove(context.Background(), f, 7, autoMoveIssue(t, tc.labels...), boardColumns(), tc.target); err != nil { + t.Fatalf("AutoMove: %v", err) + } + if len(f.moves) != 1 { + t.Fatalf("MoveToColumn calls = %d, want 1: %+v", len(f.moves), f.moves) + } + if f.moves[0].Column != tc.target { + t.Fatalf("ColumnMove.Column = %q, want %q", f.moves[0].Column, tc.target) + } + }) + } +} + // assertBound compares one fetch's UpdatedAfter against a want, where nil means // "this fetch must carry no lower bound at all". Split out so the nil case is // asserted as deliberately as the set case: `got != nil` on its own reads as a diff --git a/api/internal/handler/account_cooldown_livedb_test.go b/api/internal/handler/account_cooldown_livedb_test.go index 21d4c6a66e..75709b6d6d 100644 --- a/api/internal/handler/account_cooldown_livedb_test.go +++ b/api/internal/handler/account_cooldown_livedb_test.go @@ -89,7 +89,7 @@ func TestAccountCooldownReadersLiveDB(t *testing.T) { if single.AccountCooldownClass.Valid || single.AccountCooldownEligibleAt.Valid { t.Fatal("single board projection survived invalid authority") } - cards := assembleCards([]store.Issue{{ForgeIssueIid: r.IssueIid.Int64}}, rows, nil, nil, nil, fx.userID, "github", nil, 0) + cards := assembleCards([]store.Issue{{ForgeIssueIid: r.IssueIid.Int64}}, rows, nil, nil, nil, fx.userID, "github", false, nil, 0) board := cards[0].LatestRun if board.Health != "ok" || board.HealthReason != nil || board.AccountCooldownClass != nil { t.Fatalf("stale board: %+v", board) diff --git a/api/internal/handler/account_cooldown_test.go b/api/internal/handler/account_cooldown_test.go index 32de2fe5e9..40118eb8be 100644 --- a/api/internal/handler/account_cooldown_test.go +++ b/api/internal/handler/account_cooldown_test.go @@ -25,7 +25,7 @@ func TestAccountCooldownBoardReaders(t *testing.T) { if !mine { viewer = uuid.New() } - cards := assembleCards([]store.Issue{{ForgeIssueIid: 1}}, []store.ListLatestRunsForRepoRow{row}, nil, nil, nil, viewer, "github", nil, 0) + cards := assembleCards([]store.Issue{{ForgeIssueIid: 1}}, []store.ListLatestRunsForRepoRow{row}, nil, nil, nil, viewer, "github", false, nil, 0) dto := cards[0].LatestRun if mine { if dto.Health != "waiting_account_cooldown" || dto.AccountCooldownClass == nil || string(*dto.AccountCooldownClass) != class || !dto.AccountCooldownEligibleAt.Equal(at) { @@ -81,7 +81,7 @@ func TestAccountCooldownBoardPrecedenceAndExpiry(t *testing.T) { at = time.Now().Add(-time.Second) } row := store.ListLatestRunsForRepoRow{ID: uuid.New(), UserID: owner, Kind: "issue", Status: "running", IssueIid: pgtype.Int8{Int64: 1, Valid: true}, Health: health, AccountCooldownClass: pgconv.TextOrNull("overload"), AccountCooldownEligibleAt: pgconv.Time(at)} - cards := assembleCards([]store.Issue{{ForgeIssueIid: 1}}, []store.ListLatestRunsForRepoRow{row}, nil, nil, nil, owner, "github", nil, 0) + cards := assembleCards([]store.Issue{{ForgeIssueIid: 1}}, []store.ListLatestRunsForRepoRow{row}, nil, nil, nil, owner, "github", false, nil, 0) dto := cards[0].LatestRun want := health if health == "ok" && future { diff --git a/api/internal/handler/board.go b/api/internal/handler/board.go index fe483738d4..7b39b538a3 100644 --- a/api/internal/handler/board.go +++ b/api/internal/handler/board.go @@ -21,6 +21,7 @@ import ( "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/forgesvc" "github.com/vtmocanu/uzi/api/internal/httpx" + "github.com/vtmocanu/uzi/api/internal/issuetracker" mw "github.com/vtmocanu/uzi/api/internal/middleware" "github.com/vtmocanu/uzi/api/internal/schedsvc" "github.com/vtmocanu/uzi/api/internal/store" @@ -474,8 +475,8 @@ func (h *Handler) seedBoard(w http.ResponseWriter, r *http.Request, repo store.G linked := repo.IssueTrackerConnectionID.Valid var boardErr error if linked { - // No labels are created on a linked issue tracker (writes arrive in M5): the - // column rows are seeded from the store alone. + // No column labels are created on a linked issue tracker (a column is not a tag + // there, D9): the column rows are seeded from the store alone. boardErr = forgesvc.EnsureBoardColumnRows(r.Context(), h.q, repo.ID) } else { boardErr = forgesvc.EnsureBoardColumns(r.Context(), h.q, tr, repo.ID, repo.ForgeProjectID) @@ -548,7 +549,7 @@ func (h *Handler) buildBoard(w http.ResponseWriter, r *http.Request, repo store. // repo.UserID is the board viewer (the connection owner); IsMine gates the // owner-only run-view link. repo.ForgeType stamps every card's forge for the // per-card MR/PR noun (all cards on one board share the repo's connection). - cards := assembleCards(issues, runRows, cardPipelines, autofixHalts, position, repo.UserID, repo.ForgeType, revising, h.cfg.RunTimeout) + cards := assembleCards(issues, runRows, cardPipelines, autofixHalts, position, repo.UserID, repo.ForgeType, repo.IssueTrackerConnectionID.Valid, revising, h.cfg.RunTimeout) return boardDTO{ RepoID: repo.ID.String(), @@ -696,7 +697,7 @@ func nonNilAssigneeIDs(ids []int64) []int64 { // ids whose latest plan-ish message is a plan_revising (issue #750); a nil map leaves // every IsRevising false. autofixHalts is cardAutofixHalts' iid -> attempts map (PRD // #1650 D3a); a nil map leaves every card un-halted. -func assembleCards(issues []store.Issue, runRows []store.ListLatestRunsForRepoRow, cardPipelines map[int64]*apitypes.PipelineDTO, autofixHalts map[int64]int32, position map[string]int, viewerID uuid.UUID, forgeType string, revising map[uuid.UUID]bool, globalTimeout time.Duration) []cardDTO { +func assembleCards(issues []store.Issue, runRows []store.ListLatestRunsForRepoRow, cardPipelines map[int64]*apitypes.PipelineDTO, autofixHalts map[int64]int32, position map[string]int, viewerID uuid.UUID, forgeType string, linked bool, revising map[uuid.UUID]bool, globalTimeout time.Duration) []cardDTO { latestByIID := make(map[int64]*latestRunDTO, len(runRows)) for _, rr := range runRows { dto := mapLatestRun(rr.ID, rr.UserID, rr.Status, rr.Kind, rr.IterationCount, rr.HasPlanMd.Bool, rr.MrIid, rr.MrWebUrl, @@ -713,7 +714,7 @@ func assembleCards(issues []store.Issue, runRows []store.ListLatestRunsForRepoRo cards := make([]cardDTO, 0, len(issues)) for _, is := range issues { labels := decodeLabels(is.Labels) - col, closed, conflict := board.ResolveColumn(labels, is.State, position) + col, closed, conflict := board.ResolveColumn(board.CardColumnLabels(linked, labels, is.IssueTrackerColumn.String), is.State, position) card := cardDTO{ IID: is.ForgeIssueIid, Title: is.Title, @@ -751,7 +752,9 @@ type configureColumnsRequest struct { // ConfigureColumns replaces a repo's column set. Positions are taken from array // order. Each column name is ensured to exist as a label on the forge (so a // newly typed column becomes a real, addressable label), then the board_columns -// rows are replaced wholesale. +// rows are replaced wholesale. A repo linked to an issue tracker has no column labels to +// ensure: only the rows are replaced, and the state mapping of a removed column is pruned +// (replaceLinkedBoardColumns). func (h *Handler) ConfigureColumns(w http.ResponseWriter, r *http.Request) { repo, ok := h.repoForRequest(w, r) if !ok { @@ -798,6 +801,29 @@ func (h *Handler) ConfigureColumns(w http.ResponseWriter, r *http.Request) { return } + // A linked repo's columns are uzi's own: the issue tracker has no column labels to + // ensure, so no issue tracker is resolved and nothing is sent to it (PRD #4173 M5). + if repo.IssueTrackerConnectionID.Valid { + // The labels are stored and shown back, so they get the rule the driver applies to a + // tag name and a State name: no braces, no invisible, blank-filler or + // variation-selector character, at most 200 bytes. + for _, n := range names { + if issuetracker.LabelNameProblem(n) { + httpx.Error(w, http.StatusBadRequest, "a column name is too long (max 200 bytes) or has a brace or an invisible, blank-filler or variation-selector character") + return + } + } + if !h.replaceLinkedBoardColumns(w, r, repo, names) { + return + } + b, ok := h.buildBoard(w, r, repo) + if !ok { + return + } + httpx.JSON(w, http.StatusOK, map[string]any{"board": b}) + return + } + tr, err := h.issueTrackerForRepo(r.Context(), repo.UserID, repo.ID, repo.IssueTrackerConnectionID, repo.ForgeType, repo.BaseUrl, repo.TokenCiphertext) if err != nil { slog.Error("build forge for connection", "error", err) @@ -842,6 +868,55 @@ func (h *Handler) ConfigureColumns(w http.ResponseWriter, r *http.Request) { httpx.JSON(w, http.StatusOK, map[string]any{"board": b}) } +// replaceLinkedBoardColumns replaces the board_columns rows of a linked repo and prunes the +// state mapping rows and cached card columns of every column that is gone, in one transaction that first takes the repo +// row lock the state mapping route takes, so a mapping save and a reconfigure serialise. It +// writes the response itself on failure and reports whether it succeeded. +func (h *Handler) replaceLinkedBoardColumns(w http.ResponseWriter, r *http.Request, repo store.GetRepoForUserRow, names []string) bool { + ctx := r.Context() + fail := func(msg string, err error) bool { + slog.Error(msg, "error", err) + httpx.Error(w, http.StatusInternalServerError, "internal error") + return false + } + tx, err := h.pool.Begin(ctx) + if err != nil { + return fail("begin linked board columns tx", err) + } + defer tx.Rollback(ctx) //nolint:errcheck // no-op after a successful Commit + qtx := h.q.WithTx(tx) + if _, err := qtx.LockRepoForStateMappingForUser(ctx, store.LockRepoForStateMappingForUserParams{ID: repo.ID, UserID: repo.UserID}); err != nil { + if errors.Is(err, pgx.ErrNoRows) { + httpx.Error(w, http.StatusNotFound, "repo not found") + return false + } + return fail("lock linked repo for board columns", err) + } + if err := qtx.DeleteBoardColumnsByRepo(ctx, repo.ID); err != nil { + return fail("delete board columns", err) + } + for i, n := range names { + if err := qtx.InsertBoardColumn(ctx, store.InsertBoardColumnParams{RepoID: repo.ID, LabelName: n, Position: int32(i)}); err != nil { + return fail("insert board column", err) + } + } + // No column left prunes every mapping row, which is right. The keep list is never nil: a + // nil slice is sent as NULL, and "<> ALL(NULL)" matches nothing, so nothing would be pruned. + keep := append([]string{}, names...) + if err := qtx.PruneRepoIssueTrackerColumnStates(ctx, store.PruneRepoIssueTrackerColumnStatesParams{RepoID: repo.ID, Keep: keep}); err != nil { + return fail("prune issue tracker column states", err) + } + // A cached card whose column is gone returns to the Open lane in the same transaction, + // or its cached column would name a column no board shows (PRD #4173 M5). + if err := qtx.ClearIssueTrackerColumnsNotIn(ctx, store.ClearIssueTrackerColumnsNotInParams{RepoID: repo.ID, Keep: keep}); err != nil { + return fail("clear cached columns of removed board columns", err) + } + if err := tx.Commit(ctx); err != nil { + return fail("commit linked board columns tx", err) + } + return true +} + // ── Manual order ──────────────────────────────────────────────────────────── // maxBoardOrderIids caps how many iids one freeze may carry. Far above the PRD's @@ -1094,6 +1169,11 @@ func (h *Handler) MoveIssue(w http.ResponseWriter, r *http.Request) { if writeIssueWriteNotOffered(w, err) { return } + // On a linked repo a refused move (409) or a moved-away issue (404) leaves the + // cache unchanged, so the card snaps back; any other failure is a 502. + if repo.IssueTrackerConnectionID.Valid && writeLinkedMoveError(w, err) { + return + } httpx.Error(w, http.StatusBadGateway, "could not update the issue on the forge: "+err.Error()) return } @@ -1122,7 +1202,7 @@ func (h *Handler) MoveIssue(w http.ResponseWriter, r *http.Request) { for _, c := range cols { position[c.LabelName] = int(c.Position) } - card := issueToCard(updated, position, repo.ForgeType) + card := issueToCard(updated, position, repo.ForgeType, repo.IssueTrackerConnectionID.Valid) // Carry the issue's latest run on the single-card response too, so a drag never // blanks the run badge the board is showing (the client replaces the card). if lr, err := h.q.GetLatestRunForIssue(r.Context(), store.GetLatestRunForIssueParams{RepoID: repo.ID, IssueIid: pgtype.Int8{Int64: iid, Valid: true}}); err == nil { @@ -1194,6 +1274,11 @@ func (h *Handler) PromoteIssue(w http.ResponseWriter, r *http.Request) { // UziLabel already falls back to the compiled-in default when unset, so label is // never empty here. label, _ := h.settings.UziLabel(r.Context()) + // On a repo linked to an issue tracker the label is the link's stored eligibility + // name (PRD #4173 D14), which the driver resolves to the stored tag id. + if eligibility, linked := linkedEligibility(repo); linked { + label = eligibility + } issue, err := h.q.GetIssueByIID(r.Context(), store.GetIssueByIIDParams{RepoID: repo.ID, ForgeIssueIid: iid}) if err != nil { @@ -1243,7 +1328,7 @@ func (h *Handler) PromoteIssue(w http.ResponseWriter, r *http.Request) { for _, c := range cols { position[c.LabelName] = int(c.Position) } - card := issueToCard(updated, position, repo.ForgeType) + card := issueToCard(updated, position, repo.ForgeType, repo.IssueTrackerConnectionID.Valid) if lr, err := h.q.GetLatestRunForIssue(r.Context(), store.GetLatestRunForIssueParams{RepoID: repo.ID, IssueIid: pgtype.Int8{Int64: iid, Valid: true}}); err == nil { card.LatestRun = mapLatestRun(lr.ID, lr.UserID, lr.Status, lr.Kind, lr.IterationCount, lr.HasPlanMd.Bool, lr.MrIid, lr.MrWebUrl, lr.MrState, lr.FailureReason, lr.StopKind, lr.StopReason, lr.Health, lr.HealthReason, lr.HealthSince, lr.HoldReason, @@ -1264,9 +1349,9 @@ func (h *Handler) PromoteIssue(w http.ResponseWriter, r *http.Request) { // issueToCard resolves a cached issue row into a card DTO. forgeType stamps the // card's forge for the per-card MR/PR noun (PRD #65 D2), from the repo's connection. // Three production callers: MoveIssue, PromoteIssue and CreateIssue (issues.go). -func issueToCard(is store.Issue, position map[string]int, forgeType string) cardDTO { +func issueToCard(is store.Issue, position map[string]int, forgeType string, linked bool) cardDTO { labels := decodeLabels(is.Labels) - col, closed, conflict := board.ResolveColumn(labels, is.State, position) + col, closed, conflict := board.ResolveColumn(board.CardColumnLabels(linked, labels, is.IssueTrackerColumn.String), is.State, position) card := cardDTO{ IID: is.ForgeIssueIid, Title: is.Title, diff --git a/api/internal/handler/board_latestrun_test.go b/api/internal/handler/board_latestrun_test.go index 07c8d44d8a..2355cc3ef3 100644 --- a/api/internal/handler/board_latestrun_test.go +++ b/api/internal/handler/board_latestrun_test.go @@ -272,7 +272,7 @@ func TestAssembleCards(t *testing.T) { } position := map[string]int{"In Progress": 0} - cards := assembleCards(issues, runRows, nil, nil, position, viewer, "forgejo", nil, 2*time.Hour) + cards := assembleCards(issues, runRows, nil, nil, position, viewer, "forgejo", false, nil, 2*time.Hour) byIID := make(map[int64]cardDTO, len(cards)) for _, c := range cards { byIID[c.IID] = c @@ -373,7 +373,7 @@ func TestAssembleCardsCarriesAutofixHalt(t *testing.T) { {ForgeIssueIid: 10, Title: "ten", State: "opened", Labels: []byte(`[]`)}, {ForgeIssueIid: 20, Title: "twenty", State: "opened", Labels: []byte(`[]`)}, } - cards := assembleCards(issues, nil, nil, map[int64]int32{20: 2}, nil, uuid.Nil, "gitlab", nil, 0) + cards := assembleCards(issues, nil, nil, map[int64]int32{20: 2}, nil, uuid.Nil, "gitlab", false, nil, 0) byIID := map[int64]cardDTO{} for _, c := range cards { byIID[c.IID] = c @@ -384,7 +384,7 @@ func TestAssembleCardsCarriesAutofixHalt(t *testing.T) { if c := byIID[10]; c.CIAutofixHalted || c.CIAutofixAttempts != 0 { t.Fatalf("card 10 = (halted %v, attempts %d), want (false, 0)", c.CIAutofixHalted, c.CIAutofixAttempts) } - for _, c := range assembleCards(issues, nil, nil, nil, nil, uuid.Nil, "gitlab", nil, 0) { + for _, c := range assembleCards(issues, nil, nil, nil, nil, uuid.Nil, "gitlab", false, nil, 0) { if c.CIAutofixHalted || c.CIAutofixAttempts != 0 { t.Fatalf("nil halt map: card %d = (halted %v, attempts %d), want (false, 0)", c.IID, c.CIAutofixHalted, c.CIAutofixAttempts) } diff --git a/api/internal/handler/card_labels_test.go b/api/internal/handler/card_labels_test.go index 3c15d60506..5e34d4fa18 100644 --- a/api/internal/handler/card_labels_test.go +++ b/api/internal/handler/card_labels_test.go @@ -62,12 +62,12 @@ func TestCardLabelsNeverMarshalAsNull(t *testing.T) { is := store.Issue{ForgeIssueIid: 1, Title: "t", State: "opened", Labels: []byte(tc.stored)} t.Run("issueToCard", func(t *testing.T) { - if got := labelsFieldOf(t, issueToCard(is, nil, "gitlab")); got != tc.want { + if got := labelsFieldOf(t, issueToCard(is, nil, "gitlab", false)); got != tc.want { t.Fatalf("labels = %s, want %s", got, tc.want) } }) t.Run("assembleCards", func(t *testing.T) { - cards := assembleCards([]store.Issue{is}, nil, nil, nil, nil, uuid.Nil, "gitlab", nil, 0) + cards := assembleCards([]store.Issue{is}, nil, nil, nil, nil, uuid.Nil, "gitlab", false, nil, 0) if len(cards) != 1 { t.Fatalf("expected one card, got %d", len(cards)) } diff --git a/api/internal/handler/findings_file.go b/api/internal/handler/findings_file.go index 9fe87b8804..ccc43cbde5 100644 --- a/api/internal/handler/findings_file.go +++ b/api/internal/handler/findings_file.go @@ -14,6 +14,7 @@ import ( "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/httpx" "github.com/vtmocanu/uzi/api/internal/issuedraft" + "github.com/vtmocanu/uzi/api/internal/issuesource" mw "github.com/vtmocanu/uzi/api/internal/middleware" "github.com/vtmocanu/uzi/api/internal/settings" "github.com/vtmocanu/uzi/api/internal/store" @@ -133,7 +134,7 @@ func (h *Handler) FileFinding(w http.ResponseWriter, r *http.Request) { // in the body. A user-chosen run-eligible label is therefore the owner's own deliberate // action, not an agent-injected one. marker, _ := h.settings.FindingLabel(ctx) - filedLabels, requiredLabels := h.findingFilingLabels(ctx, marker, req.Labels, req.SendToUzi) + filedLabels, requiredLabels := h.findingFilingLabels(ctx, repo, marker, req.Labels, req.SendToUzi) // (5) Claim-first (D4): a guarded open→filing UPDATE. 0 rows affected means the coordinate // is already filed or being filed (the single row is not `open`), so exactly one of two @@ -308,7 +309,29 @@ func assembleFindingLabels(marker string, userLabels []string) []string { // findingFilingLabels preserves assembleFindingLabels' explicit-label budget, then appends // the requested run labels without displacing accepted user labels. Required forge labels // are built independently so an explicit-label collision never skips ensuring a run label. -func (h *Handler) findingFilingLabels(ctx context.Context, marker string, userLabels []string, sendToUzi bool) ([]string, []forge.Label) { +// +// On a repo linked to an issue tracker the filed set maps the uzi label to the stored +// eligibility name and drops the autopilot label, and neither run label is ensured: the +// eligibility tag already exists by id and is never created, and autopilot is not +// offered (PRD #4173 D12, D14). +func (h *Handler) findingFilingLabels(ctx context.Context, repo store.GetRepoForUserRow, marker string, userLabels []string, sendToUzi bool) ([]string, []forge.Label) { + filed, required := h.forgeFindingFilingLabels(ctx, marker, userLabels, sendToUzi) + eligibility, linked := linkedEligibility(repo) + if !linked { + return filed, required + } + uzi, autopilot := h.runLabelNames(ctx) + linkedRequired := make([]forge.Label, 0, len(required)) + for _, l := range required { + if l.Name != uzi && l.Name != autopilot && l.Name != eligibility { + linkedRequired = append(linkedRequired, l) + } + } + return issuesource.LinkedLabels(filed, uzi, autopilot, eligibility), linkedRequired +} + +// forgeFindingFilingLabels is the unlinked assembly findingFilingLabels starts from. +func (h *Handler) forgeFindingFilingLabels(ctx context.Context, marker string, userLabels []string, sendToUzi bool) ([]string, []forge.Label) { filed := assembleFindingLabels(marker, userLabels) required := []forge.Label{{Name: marker}} if !sendToUzi { diff --git a/api/internal/handler/findings_group_file.go b/api/internal/handler/findings_group_file.go index 4e3445926f..c13cb665c8 100644 --- a/api/internal/handler/findings_group_file.go +++ b/api/internal/handler/findings_group_file.go @@ -209,7 +209,7 @@ func (h *Handler) FileFindingGroup(w http.ResponseWriter, r *http.Request) { writeStoppedFindingGroup(w, op.ID, dispositionIDs, "pre_call") return } - filedLabels, requiredLabels := h.findingFilingLabels(callCtx, marker, req.Labels, req.SendToUzi) + filedLabels, requiredLabels := h.findingFilingLabels(callCtx, repo, marker, req.Labels, req.SendToUzi) if err := tr.EnsureLabels(callCtx, repo.ForgeProjectID, requiredLabels); err != nil { if callCtx.Err() != nil { writeStoppedFindingGroup(w, op.ID, dispositionIDs, "pre_call") @@ -239,8 +239,10 @@ func (h *Handler) FileFindingGroup(w http.ResponseWriter, r *http.Request) { } created, err := tr.CreateIssue(callCtx, repo.ForgeProjectID, title, description, filedLabels) if err != nil { - // The write guard refuses before any request, so nothing was created: release the - // group like a definitive rejection instead of marking it uncertain. + // A definitive rejection created nothing: release the group instead of marking it + // uncertain. The ErrIssueWriteNotOffered branch is defence only, since the guard + // passes a create through. On a linked repo every other create failure is treated + // as uncertain, because the YouTrack driver returns no definitive rejection. if errors.Is(err, forgesvc.ErrIssueWriteNotOffered) || forge.IsCreateIssueDefinitiveRejection(err) { if !h.releaseDefinitiveFindingGroup(user.ID, op.ID) { writeStoppedFindingGroup(w, op.ID, dispositionIDs, "in_flight") diff --git a/api/internal/handler/issue_tracker_states.go b/api/internal/handler/issue_tracker_states.go new file mode 100644 index 0000000000..bad5de86f8 --- /dev/null +++ b/api/internal/handler/issue_tracker_states.go @@ -0,0 +1,107 @@ +package handler + +import ( + "net/http" + + "github.com/vtmocanu/uzi/api/internal/httpx" + "github.com/vtmocanu/uzi/api/internal/issuetrackersvc" +) + +type issueTrackerColumnStateDTO struct { + LabelName string `json:"label_name"` + State string `json:"state"` +} + +// issueTrackerStateMappingDTO is the wire form of a repo's state mapping, for both the +// PUT body and the responses. +type issueTrackerStateMappingDTO struct { + OpenState string `json:"open_state"` + ClosedState string `json:"closed_state"` + Columns []issueTrackerColumnStateDTO `json:"columns"` +} + +// issueTrackerStateDTO is one State of the linked project with what uzi may do with it. +type issueTrackerStateDTO struct { + Name string `json:"name"` + Resolved bool `json:"resolved"` + Archived bool `json:"archived"` + Closable bool `json:"closable"` + Openable bool `json:"openable"` +} + +func toIssueTrackerStateMappingDTO(m issuetrackersvc.StateMapping) issueTrackerStateMappingDTO { + out := issueTrackerStateMappingDTO{ + OpenState: m.OpenState, ClosedState: m.ClosedState, + Columns: make([]issueTrackerColumnStateDTO, 0, len(m.Columns)), + } + for _, c := range m.Columns { + out.Columns = append(out.Columns, issueTrackerColumnStateDTO{LabelName: cleanTrackerText(c.LabelName), State: c.State}) + } + return out +} + +// GetIssueTrackerStateMapping returns the saved state mapping of a linked repo (null +// until one is saved) and the States of its project, each with whether uzi may write it as +// the closed state (closable) or as the open or a column state (openable) +// (GET /api/repos/{id}/issue-tracker-link/states). 404 for a repo that is not the user's, +// 409 for one that is not linked. +func (h *Handler) GetIssueTrackerStateMapping(w http.ResponseWriter, r *http.Request) { + userID, svc, ok := h.issueTrackerRequest(w, r) + if !ok { + return + } + repoID, ok := httpx.PathUUID(w, r, "id", "repo") + if !ok { + return + } + view, err := svc.GetStateMapping(r.Context(), userID, repoID) + if err != nil { + writeIssueTrackerError(w, err) + return + } + states := make([]issueTrackerStateDTO, 0, len(view.States)) + for _, s := range view.States { + states = append(states, issueTrackerStateDTO{ + Name: cleanTrackerText(s.Name), Resolved: s.Resolved, Archived: s.Archived, + Closable: s.Closable, Openable: s.Openable, + }) + } + var mapping *issueTrackerStateMappingDTO + if view.Mapping != nil { + m := toIssueTrackerStateMappingDTO(*view.Mapping) + mapping = &m + } + httpx.JSON(w, http.StatusOK, map[string]any{"mapping": mapping, "states": states}) +} + +// PutIssueTrackerStateMapping replaces the whole state mapping of a linked repo +// (PUT /api/repos/{id}/issue-tracker-link/states) and returns what it saved. Refusals: 404 +// for a repo that is not the user's, 409 for one that is not linked, 400 for a name that is +// out of bounds, not a State of the linked project, or not allowed in its role (a closed +// state must mean done, never "Won't fix"), for a column label that is not a board column +// and for a State that is mapped twice. +func (h *Handler) PutIssueTrackerStateMapping(w http.ResponseWriter, r *http.Request) { + userID, svc, ok := h.issueTrackerRequest(w, r) + if !ok { + return + } + repoID, ok := httpx.PathUUID(w, r, "id", "repo") + if !ok { + return + } + var req issueTrackerStateMappingDTO + if err := httpx.DecodeJSON(r, &req); err != nil { + httpx.Error(w, http.StatusBadRequest, "invalid request body") + return + } + in := issuetrackersvc.StateMapping{OpenState: req.OpenState, ClosedState: req.ClosedState} + for _, c := range req.Columns { + in.Columns = append(in.Columns, issuetrackersvc.ColumnState{LabelName: c.LabelName, State: c.State}) + } + saved, err := svc.SetStateMapping(r.Context(), userID, repoID, in) + if err != nil { + writeIssueTrackerError(w, err) + return + } + httpx.JSON(w, http.StatusOK, map[string]any{"mapping": toIssueTrackerStateMappingDTO(saved)}) +} diff --git a/api/internal/handler/issue_tracker_states_livedb_test.go b/api/internal/handler/issue_tracker_states_livedb_test.go new file mode 100644 index 0000000000..9836de8543 --- /dev/null +++ b/api/internal/handler/issue_tracker_states_livedb_test.go @@ -0,0 +1,281 @@ +package handler + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/go-chi/chi/v5" + "github.com/google/uuid" + "github.com/jackc/pgx/v5" + + "github.com/vtmocanu/uzi/api/internal/issuetrackersvc" + mw "github.com/vtmocanu/uzi/api/internal/middleware" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// stateMappingFixture is the linked repo of newLinkedHandlerFixture (columns "In Progress" and +// "Human Review") with the issue tracker service wired into the Handler and a few States +// seeded in the YouTrack fake beside its default Open, Fixed and Won't fix. +func newStateMappingFixture(t *testing.T) *linkedHandlerFixture { + t.Helper() + f := newLinkedHandlerFixture(t, true) + f.h.SetIssueTrackerService(f.itSvc.UseTransactions(f.pool, func(tx pgx.Tx) issuetrackersvc.Querier { return f.h.q.WithTx(tx) })) + f.srv.SeedState("Doing", false) + f.srv.SeedState("In Review", false) + f.srv.SeedState("Shipped", true) + return f +} + +// callAs drives a handler as user over the repo of the fixture. +func (f *linkedHandlerFixture) callAs(user store.User, handler http.HandlerFunc, method string, body any) *httptest.ResponseRecorder { + var rdr *strings.Reader + if body != nil { + b, _ := json.Marshal(body) + rdr = strings.NewReader(string(b)) + } else { + rdr = strings.NewReader("") + } + r := httptest.NewRequest(method, "/x", rdr) + rctx := chi.NewRouteContext() + rctx.URLParams.Add("id", f.repo.String()) + r = r.WithContext(context.WithValue(mw.ContextWithUser(r.Context(), user), chi.RouteCtxKey, rctx)) + rr := httptest.NewRecorder() + handler(rr, r) + return rr +} + +func (f *linkedHandlerFixture) putStates(body any) *httptest.ResponseRecorder { + return f.callAs(f.user, f.h.PutIssueTrackerStateMapping, http.MethodPut, body) +} + +func (f *linkedHandlerFixture) getStates(t *testing.T) (mapping map[string]any, states []map[string]any) { + t.Helper() + rr := f.callAs(f.user, f.h.GetIssueTrackerStateMapping, http.MethodGet, nil) + if rr.Code != http.StatusOK { + t.Fatalf("GET states = %d\n%s", rr.Code, rr.Body.String()) + } + var out struct { + Mapping map[string]any `json:"mapping"` + States []map[string]any `json:"states"` + } + if err := json.Unmarshal(rr.Body.Bytes(), &out); err != nil { + t.Fatal(err) + } + return out.Mapping, out.States +} + +func mappingBody(open, closed string, cols ...[2]string) map[string]any { + cs := make([]map[string]string, 0, len(cols)) + for _, c := range cols { + cs = append(cs, map[string]string{"label_name": c[0], "state": c[1]}) + } + return map[string]any{"open_state": open, "closed_state": closed, "columns": cs} +} + +func TestIssueTrackerStateMappingRouteLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + + if m, _ := f.getStates(t); m != nil { + t.Fatalf("mapping before a save = %v, want null", m) + } + rr := f.putStates(mappingBody("Open", "Fixed", [2]string{"In Progress", "Doing"}, [2]string{"Human Review", "In Review"})) + if rr.Code != http.StatusOK { + t.Fatalf("PUT = %d\n%s", rr.Code, rr.Body.String()) + } + var saved struct { + Mapping struct { + OpenState string `json:"open_state"` + ClosedState string `json:"closed_state"` + Columns []struct{ LabelName, State string } + } `json:"mapping"` + } + if err := json.Unmarshal(rr.Body.Bytes(), &saved); err != nil { + t.Fatal(err) + } + if saved.Mapping.OpenState != "Open" || saved.Mapping.ClosedState != "Fixed" || len(saved.Mapping.Columns) != 2 { + t.Fatalf("PUT body = %s", rr.Body.String()) + } + + // Stored: the rows hold what was saved, and a GET reads it back. + var open, closed string + if err := f.pool.QueryRow(context.Background(), `SELECT open_state, closed_state FROM repo_issue_tracker_states WHERE repo_id = $1`, f.repo).Scan(&open, &closed); err != nil || open != "Open" || closed != "Fixed" { + t.Fatalf("stored pair = %q %q (%v)", open, closed, err) + } + var state string + if err := f.pool.QueryRow(context.Background(), `SELECT state_name FROM repo_issue_tracker_column_states WHERE repo_id = $1 AND label_name = 'In Progress'`, f.repo).Scan(&state); err != nil || state != "Doing" { + t.Fatalf("stored column state = %q (%v)", state, err) + } + m, states := f.getStates(t) + if m["open_state"] != "Open" || m["closed_state"] != "Fixed" || len(m["columns"].([]any)) != 2 { + t.Fatalf("GET mapping = %v", m) + } + + // The States carry what uzi may do with each. + by := map[string]map[string]any{} + for _, s := range states { + by[s["name"].(string)] = s + } + for name, want := range map[string][2]bool{ // closable, openable + "Open": {false, true}, "Doing": {false, true}, "Fixed": {true, false}, "Shipped": {true, false}, "Won't fix": {false, false}, + } { + s := by[name] + if s == nil || s["closable"] != want[0] || s["openable"] != want[1] { + t.Errorf("state %q = %v, want closable %v openable %v", name, s, want[0], want[1]) + } + } + if by["Won't fix"]["resolved"] != true { + t.Errorf("Won't fix must read as resolved: %v", by["Won't fix"]) + } + + // A second PUT replaces the mapping, columns included. + if rr := f.putStates(mappingBody("Doing", "Shipped", [2]string{"Human Review", "In Review"})); rr.Code != http.StatusOK { + t.Fatalf("second PUT = %d\n%s", rr.Code, rr.Body.String()) + } + var n int + if err := f.pool.QueryRow(context.Background(), `SELECT count(*) FROM repo_issue_tracker_column_states WHERE repo_id = $1`, f.repo).Scan(&n); err != nil || n != 1 { + t.Fatalf("column rows after the second PUT = %d (%v), want 1", n, err) + } +} + +func TestIssueTrackerStateMappingRouteRefusalsLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + cases := []struct { + name string + body any + code int + msg string + }{ + {"unknown state", mappingBody("Nope", "Fixed"), http.StatusBadRequest, "Nope"}, + {"resolved column state", mappingBody("Open", "Fixed", [2]string{"In Progress", "Shipped"}), http.StatusBadRequest, "column state"}, + {"won't fix closed", mappingBody("Open", "Won't fix"), http.StatusBadRequest, "Won't fix"}, + {"unresolved closed", mappingBody("Open", "Doing"), http.StatusBadRequest, "closed state"}, + {"label that is not a column", mappingBody("Open", "Fixed", [2]string{"Backlog", "Doing"}), http.StatusBadRequest, "not a board column"}, + {"duplicate states", mappingBody("Open", "Fixed", [2]string{"In Progress", "Doing"}, [2]string{"Human Review", "Doing"}), http.StatusBadRequest, "more than once"}, + {"column state equals open", mappingBody("Open", "Fixed", [2]string{"In Progress", "Open"}), http.StatusBadRequest, "more than once"}, + {"duplicate label", mappingBody("Open", "Fixed", [2]string{"In Progress", "Doing"}, [2]string{"In Progress", "In Review"}), http.StatusBadRequest, "more than once"}, + {"open equals closed", mappingBody("Fixed", "Fixed"), http.StatusBadRequest, "different"}, + {"control character", mappingBody("Op\ten", "Fixed"), http.StatusBadRequest, "control"}, + {"overlong name", mappingBody(strings.Repeat("x", 201), "Fixed"), http.StatusBadRequest, "200"}, + {"not json", "x", http.StatusBadRequest, "invalid request body"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + rr := f.putStates(tc.body) + if rr.Code != tc.code || !strings.Contains(rr.Body.String(), tc.msg) { + t.Fatalf("PUT = %d %s, want %d containing %q", rr.Code, rr.Body.String(), tc.code, tc.msg) + } + }) + } + var n int + if err := f.pool.QueryRow(context.Background(), `SELECT count(*) FROM repo_issue_tracker_states WHERE repo_id = $1`, f.repo).Scan(&n); err != nil || n != 0 { + t.Fatalf("states rows after every refusal = %d (%v), want 0", n, err) + } + + // A foreign user gets 404 on both verbs; a repo that is not linked gets 409. + stranger := store.User{ID: uuid.New()} + if rr := f.callAs(stranger, f.h.PutIssueTrackerStateMapping, http.MethodPut, mappingBody("Open", "Fixed")); rr.Code != http.StatusNotFound { + t.Errorf("PUT as a stranger = %d, want 404\n%s", rr.Code, rr.Body.String()) + } + if rr := f.callAs(stranger, f.h.GetIssueTrackerStateMapping, http.MethodGet, nil); rr.Code != http.StatusNotFound { + t.Errorf("GET as a stranger = %d, want 404\n%s", rr.Code, rr.Body.String()) + } + plain := *f + plain.repo = uuid.New() + if _, err := f.pool.Exec(context.Background(), `INSERT INTO repos (id, connection_id, forge_project_id, path_with_namespace, web_url, default_branch, enabled) + SELECT $1::uuid, connection_id, 78, 'g/plain-' || $1::text, 'https://forge.test/g/p', 'main', true FROM repos WHERE id = $2`, plain.repo, f.repo); err != nil { + t.Fatal(err) + } + if rr := plain.putStates(mappingBody("Open", "Fixed")); rr.Code != http.StatusConflict { + t.Errorf("PUT on an unlinked repo = %d, want 409\n%s", rr.Code, rr.Body.String()) + } + if rr := plain.callAs(plain.user, plain.h.GetIssueTrackerStateMapping, http.MethodGet, nil); rr.Code != http.StatusConflict { + t.Errorf("GET on an unlinked repo = %d, want 409\n%s", rr.Code, rr.Body.String()) + } +} + +// ConfigureColumns on a linked repo writes the board_columns rows, prunes the mapping rows of +// the columns that are gone, and sends nothing to the issue tracker or the forge. +func TestConfigureColumnsOnALinkedRepoPrunesTheMappingLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + if rr := f.putStates(mappingBody("Open", "Fixed", [2]string{"In Progress", "Doing"}, [2]string{"Human Review", "In Review"})); rr.Code != http.StatusOK { + t.Fatalf("PUT = %d\n%s", rr.Code, rr.Body.String()) + } + f.srv.ResetRequests() + + rr := f.call(f.h.ConfigureColumns, http.MethodPut, map[string]any{"columns": []map[string]string{{"label_name": "In Progress"}, {"label_name": "Backlog"}}}) + if rr.Code != http.StatusOK { + t.Fatalf("ConfigureColumns = %d\n%s", rr.Code, rr.Body.String()) + } + f.assertNothingSent(t) + + rows, err := f.pool.Query(context.Background(), `SELECT label_name FROM board_columns WHERE repo_id = $1 ORDER BY position`, f.repo) + if err != nil { + t.Fatal(err) + } + var cols []string + for rows.Next() { + var l string + if err := rows.Scan(&l); err != nil { + t.Fatal(err) + } + cols = append(cols, l) + } + rows.Close() + if len(cols) < 2 || cols[0] != "In Progress" || cols[1] != "Backlog" { + t.Fatalf("board columns = %v, want In Progress then Backlog first", cols) + } + m, _ := f.getStates(t) + mapped := m["columns"].([]any) + if len(mapped) != 1 || mapped[0].(map[string]any)["label_name"] != "In Progress" { + t.Fatalf("mapping after the reconfigure = %v, want only In Progress", m) + } + if m["open_state"] != "Open" { + t.Errorf("the open and closed pair must survive a reconfigure: %v", m) + } + + // No column at all prunes every mapping row and keeps the pair. + if rr := f.call(f.h.ConfigureColumns, http.MethodPut, map[string]any{"columns": []map[string]string{}}); rr.Code != http.StatusOK { + t.Fatalf("empty ConfigureColumns = %d\n%s", rr.Code, rr.Body.String()) + } + var n int + if err := f.pool.QueryRow(context.Background(), `SELECT count(*) FROM repo_issue_tracker_column_states WHERE repo_id = $1`, f.repo).Scan(&n); err != nil || n != 0 { + t.Fatalf("column mapping rows after clearing the columns = %d (%v), want 0", n, err) + } +} + +// A linked repo's column labels are stored and shown back, so ConfigureColumns refuses one the +// YouTrack driver would refuse as a tag name: bidi or zero-width characters, braces, an +// overlong label. Nothing is stored and nothing is sent. +func TestConfigureColumnsOnALinkedRepoRefusesBadLabelsLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + var before int + if err := f.pool.QueryRow(context.Background(), `SELECT count(*) FROM board_columns WHERE repo_id = $1`, f.repo).Scan(&before); err != nil { + t.Fatal(err) + } + for name, label := range map[string]string{ + "bidi override": "Do\u202eing", + "zero width": "Do\u200bing", + "line sep": "Do\u2028ing", + "brace": "Do{ing", + "hangul filler": "Review\u3164", + "blank filler": "\u2800", + "variation sel": "Review\uFE0F", + "overlong": strings.Repeat("x", 201), + } { + t.Run(name, func(t *testing.T) { + rr := f.call(f.h.ConfigureColumns, http.MethodPut, map[string]any{"columns": []map[string]string{{"label_name": label}}}) + if rr.Code != http.StatusBadRequest { + t.Fatalf("ConfigureColumns = %d, want 400\n%s", rr.Code, rr.Body.String()) + } + }) + } + var after int + if err := f.pool.QueryRow(context.Background(), `SELECT count(*) FROM board_columns WHERE repo_id = $1`, f.repo).Scan(&after); err != nil || after != before { + t.Fatalf("board columns after the refusals = %d (%v), want the %d it had", after, err, before) + } + f.assertNothingSent(t) +} diff --git a/api/internal/handler/issue_trackers_test.go b/api/internal/handler/issue_trackers_test.go index a5fa8d9555..dc6d586f0d 100644 --- a/api/internal/handler/issue_trackers_test.go +++ b/api/internal/handler/issue_trackers_test.go @@ -23,6 +23,8 @@ var issueTrackerRoutes = []struct{ method, path, body string }{ {http.MethodGet, "/api/issue-trackers/connections/11111111-1111-1111-1111-111111111111/projects", ""}, {http.MethodGet, "/api/issue-trackers/connections/11111111-1111-1111-1111-111111111111/tags", ""}, {http.MethodPut, "/api/repos/22222222-2222-2222-2222-222222222222/issue-tracker-link", `{"connection_id":"11111111-1111-1111-1111-111111111111","project_id":"0-1","tag_id":"5-1"}`}, + {http.MethodGet, "/api/repos/22222222-2222-2222-2222-222222222222/issue-tracker-link/states", ""}, + {http.MethodPut, "/api/repos/22222222-2222-2222-2222-222222222222/issue-tracker-link/states", `{"open_state":"Open","closed_state":"Fixed","columns":[]}`}, } func issueTrackerTestRouter() http.Handler { diff --git a/api/internal/handler/issue_write_refusal.go b/api/internal/handler/issue_write_refusal.go index 5a88f3fcea..fa62903803 100644 --- a/api/internal/handler/issue_write_refusal.go +++ b/api/internal/handler/issue_write_refusal.go @@ -1,21 +1,89 @@ package handler import ( + "context" "errors" "net/http" "github.com/vtmocanu/uzi/api/internal/forgesvc" "github.com/vtmocanu/uzi/api/internal/httpx" + "github.com/vtmocanu/uzi/api/internal/issuesource" + "github.com/vtmocanu/uzi/api/internal/issuetracker" + "github.com/vtmocanu/uzi/api/internal/settings" + "github.com/vtmocanu/uzi/api/internal/store" ) +// issueWriteNotOfferedMessage is the 409 body for a write a linked repo does not offer. +const issueWriteNotOfferedMessage = "this issue write is not offered on a repo linked to an issue tracker" + +// linkedEligibility returns the stored eligibility tag name of a repo linked to an issue +// tracker (PRD #4173 D14), and whether the repo is linked at all. +func linkedEligibility(repo store.GetRepoForUserRow) (string, bool) { + if !repo.IssueTrackerConnectionID.Valid { + return "", false + } + return repo.EligibilityTagName.String, true +} + +// createLabelsFor returns the labels an issue uzi creates on repo carries: labels +// untouched for an unlinked repo, and for a linked one the set with the configured uzi +// label replaced by the stored eligibility name and the autopilot label dropped (PRD +// #4173 D12, D14). +func (h *Handler) createLabelsFor(ctx context.Context, repo store.GetRepoForUserRow, labels []string) []string { + eligibility, linked := linkedEligibility(repo) + if !linked { + return labels + } + uzi, autopilot := h.runLabelNames(ctx) + return issuesource.LinkedLabels(labels, uzi, autopilot, eligibility) +} + +// runLabelNames returns the configured uzi and autopilot labels, each falling back to +// its compiled-in default when unset. +func (h *Handler) runLabelNames(ctx context.Context) (uzi, autopilot string) { + uzi, _ = h.settings.UziLabel(ctx) + if uzi == "" { + uzi = settings.DefaultUziLabel + } + autopilot, _ = h.settings.AutopilotLabel(ctx) + if autopilot == "" { + autopilot = settings.DefaultAutopilotLabel + } + return uzi, autopilot +} + // writeIssueWriteNotOffered answers the typed refusal the linked-repo write guard -// returns (PRD #4173 M4: YouTrack writes arrive in M5) as a 409 instead of a 502/500, +// returns (PRD #4173 M4; after M5 only the writes no unit has lifted) as a 409 instead of a 502/500, // and reports whether it did. Callers that claimed something before the write have // already released it by the time they call this. func writeIssueWriteNotOffered(w http.ResponseWriter, err error) bool { if !errors.Is(err, forgesvc.ErrIssueWriteNotOffered) { return false } - httpx.Error(w, http.StatusConflict, "issue writes on a repo linked to an issue tracker are not offered yet") + httpx.Error(w, http.StatusConflict, issueWriteNotOfferedMessage) + return true +} + +// writeLinkedMoveError answers a failed card move or State write on a repo linked +// to an issue tracker (PRD #4173 M5) and reports whether it did. A column or State +// the driver refused before any request is a 409, and a moved-away issue is a 404 +// (D10); an issue the issue tracker shows as resolved is a 409 too, refused after the +// reads and before any write. In each the cache is unchanged, so the card snaps back. Any other error +// is left to the caller's 502. +func writeLinkedMoveError(w http.ResponseWriter, err error) bool { + switch { + case errors.Is(err, issuetracker.ErrStateNotAllowed): + httpx.Error(w, http.StatusConflict, "the target issue tracker State is not allowed for this move") + case errors.Is(err, issuetracker.ErrStateMappingUnset): + httpx.Error(w, http.StatusConflict, "the open and closed issue tracker States are not set for this repo") + case forgesvc.LinkedIssueResolved(err): + httpx.Error(w, http.StatusConflict, "the issue was resolved in the issue tracker; reopen it from the Closed lane after the next sync") + case forgesvc.ColumnMoveRefused(err): + httpx.Error(w, http.StatusConflict, "map this column to an issue tracker State first") + case forgesvc.LinkedIssueGone(err): + httpx.Error(w, http.StatusNotFound, "the issue is no longer in the linked project") + default: + return false + } return true } diff --git a/api/internal/handler/issues.go b/api/internal/handler/issues.go index 062c4b4696..0ba1d53ac2 100644 --- a/api/internal/handler/issues.go +++ b/api/internal/handler/issues.go @@ -67,7 +67,8 @@ func (h *Handler) CreateIssue(w http.ResponseWriter, r *http.Request) { // best-effort — a cold settings read yields the compiled-in default ("uzi"), // never an empty label. uziLabel, _ := h.settings.UziLabel(r.Context()) - created, err := tr.CreateIssue(r.Context(), repo.ForgeProjectID, title, req.Description, []string{uziLabel}) + // On a linked repo the label is the link's stored eligibility name (PRD #4173 D14). + created, err := tr.CreateIssue(r.Context(), repo.ForgeProjectID, title, req.Description, h.createLabelsFor(r.Context(), repo, []string{uziLabel})) if err != nil { // err is already PAT-redacted by the driver. if writeIssueWriteNotOffered(w, err) { @@ -132,7 +133,7 @@ func (h *Handler) CreateIssue(w http.ResponseWriter, r *http.Request) { for _, c := range cols { position[c.LabelName] = int(c.Position) } - httpx.JSON(w, http.StatusCreated, map[string]any{"card": issueToCard(upserted, position, repo.ForgeType)}) + httpx.JSON(w, http.StatusCreated, map[string]any{"card": issueToCard(upserted, position, repo.ForgeType, repo.IssueTrackerConnectionID.Valid)}) } // issueDetailDTO is the in-app issue view payload (PRD #12 §3): the board card diff --git a/api/internal/handler/judge_admin_issue.go b/api/internal/handler/judge_admin_issue.go index 6a44fc9cca..6a764233a1 100644 --- a/api/internal/handler/judge_admin_issue.go +++ b/api/internal/handler/judge_admin_issue.go @@ -270,7 +270,7 @@ func (h *Handler) AdminFileJudgeIssue(w http.ResponseWriter, r *http.Request) { httpx.Error(w, http.StatusInternalServerError, "internal error") return } - created, err := tr.CreateIssue(ctx, repo.ForgeProjectID, title, description, labels) + created, err := tr.CreateIssue(ctx, repo.ForgeProjectID, title, description, h.createLabelsFor(ctx, repo, labels)) if err != nil { // Forge rejected the write: the claim reverts, nothing persists, the coordinate is // fileable again. err is already PAT-redacted by the driver. diff --git a/api/internal/handler/labels.go b/api/internal/handler/labels.go index 693135d8df..4233ecd5fd 100644 --- a/api/internal/handler/labels.go +++ b/api/internal/handler/labels.go @@ -58,6 +58,13 @@ func (h *Handler) EnsureRepoLabels(w http.ResponseWriter, r *http.Request) { if refuseNeverEnabled(w, repo.AwaitingFirstEnable, repo.IssueSourceLocked) { return } + // A linked repo's eligibility tag is chosen at link time and column and autopilot tags + // are not wanted, so the endpoint is refused before any forge or issue tracker request + // (PRD #4173 D14). + if repo.IssueTrackerConnectionID.Valid { + httpx.Error(w, http.StatusConflict, issueWriteNotOfferedMessage) + return + } var req apitypes.LabelEnsureRequest if err := httpx.DecodeJSONLimited(w, r, &req); err != nil { httpx.RespondDecodeError(w, err, "invalid request body") diff --git a/api/internal/handler/linked_board_livedb_test.go b/api/internal/handler/linked_board_livedb_test.go new file mode 100644 index 0000000000..cfd398f475 --- /dev/null +++ b/api/internal/handler/linked_board_livedb_test.go @@ -0,0 +1,409 @@ +package handler + +import ( + "context" + "encoding/json" + "net/http" + "strings" + "testing" + + "github.com/jackc/pgx/v5/pgtype" + + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// PRD #4173 M5: the linked board. A drag writes the mapped State to the YouTrack fake and +// caches the column; a refused or failed move leaves the cache unchanged so the card snaps +// back; close and reopen use the stored States; a sync keeps what a move wrote. + +// boardMapping saves the mapping the board tests use: Open, Fixed, and the two board columns. +func (f *linkedHandlerFixture) boardMapping(t *testing.T) { + t.Helper() + if rr := f.putStates(mappingBody("Open", "Fixed", [2]string{"In Progress", "Doing"}, [2]string{"Human Review", "In Review"})); rr.Code != http.StatusOK { + t.Fatalf("PUT mapping = %d\n%s", rr.Code, rr.Body.String()) + } + f.srv.ResetRequests() +} + +func (f *linkedHandlerFixture) move(to string) (int, string) { + rr := f.call(f.h.MoveIssue, http.MethodPost, map[string]string{"to_column": to}) + return rr.Code, rr.Body.String() +} + +// cached reads the card's cached state and column. +func (f *linkedHandlerFixture) cached(t *testing.T) (state string, column pgtype.Text) { + t.Helper() + if err := f.pool.QueryRow(context.Background(), `SELECT state, issue_tracker_column FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2`, f.repo, f.issueN).Scan(&state, &column); err != nil { + t.Fatal(err) + } + return state, column +} + +func (f *linkedHandlerFixture) assertCache(t *testing.T, wantState, wantColumn string) { + t.Helper() + state, col := f.cached(t) + if state != wantState || col.String != wantColumn || col.Valid != (wantColumn != "") { + t.Fatalf("cache = state %q column %+v, want state %q column %q (NULL when empty)", state, col, wantState, wantColumn) + } +} + +// labelsAndPosition reads the card's labels and board_position. +func (f *linkedHandlerFixture) labelsAndPosition(t *testing.T) (labels string, position pgtype.Int8) { + t.Helper() + if err := f.pool.QueryRow(context.Background(), `SELECT labels::text, board_position FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2`, f.repo, f.issueN).Scan(&labels, &position); err != nil { + t.Fatal(err) + } + return labels, position +} + +// assertOnlyColumnChanged fails when labels or board_position differ from the values read +// before the move. +func (f *linkedHandlerFixture) assertOnlyColumnChanged(t *testing.T, wantLabels string, wantPosition pgtype.Int8) { + t.Helper() + labels, position := f.labelsAndPosition(t) + if labels != wantLabels || position != wantPosition || !position.Valid { + t.Fatalf("labels %s position %+v after the move, want %s %+v (a move writes only the column)", labels, position, wantLabels, wantPosition) + } +} + +// boardCard reads the card of the fixture's issue off the board the handler serves. +func (f *linkedHandlerFixture) boardCard(t *testing.T) (column string, closed bool) { + t.Helper() + rr := f.call(f.h.GetBoard, http.MethodGet, nil) + if rr.Code != http.StatusOK { + t.Fatalf("board = %d\n%s", rr.Code, rr.Body.String()) + } + var b struct { + Board struct { + Cards []struct { + IID int64 `json:"iid"` + Column string `json:"column"` + Closed bool `json:"closed"` + } `json:"cards"` + } `json:"board"` + } + if err := json.Unmarshal(rr.Body.Bytes(), &b); err != nil { + t.Fatal(err) + } + for _, c := range b.Board.Cards { + if c.IID == f.issueN { + return c.Column, c.Closed + } + } + t.Fatalf("the board has no card %d: %s", f.issueN, rr.Body.String()) + return "", false +} + +func (f *linkedHandlerFixture) assertNoYouTrackWrites(t *testing.T) { + t.Helper() + for _, r := range f.srv.Requests() { + if r.Method != http.MethodGet { + t.Errorf("the issue tracker saw a write %s %s", r.Method, r.Path) + } + } +} + +func TestLinkedBoardMoveWritesTheStateAndCachesTheColumnLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + f.boardMapping(t) + // A move writes only the column: give the row labels and a position to prove they survive. + if _, err := f.pool.Exec(context.Background(), `UPDATE issues SET labels = '["uzi-ready","keep-me"]'::jsonb, board_position = 7 WHERE repo_id = $1 AND forge_issue_iid = $2`, f.repo, f.issueN); err != nil { + t.Fatal(err) + } + labels, position := f.labelsAndPosition(t) + + code, body := f.move("In Progress") + if code != http.StatusOK { + t.Fatalf("move = %d %s", code, body) + } + if got := f.srv.IssueState(f.issueN); got != "Doing" { + t.Fatalf("YouTrack State = %q, want Doing", got) + } + f.assertCache(t, "opened", "In Progress") + f.assertOnlyColumnChanged(t, labels, position) + if col, closed := f.boardCard(t); col != "In Progress" || closed { + t.Fatalf("board card in %q closed=%v, want In Progress", col, closed) + } + + // Moving to Open writes the stored open State and caches NULL (the Open lane). + if code, body := f.move(""); code != http.StatusOK { + t.Fatalf("move to Open = %d %s", code, body) + } + if got := f.srv.IssueState(f.issueN); got != "Open" { + t.Fatalf("YouTrack State = %q, want Open", got) + } + f.assertCache(t, "opened", "") + if col, _ := f.boardCard(t); col != "" { + t.Fatalf("board card in %q, want the Open lane", col) + } +} + +func TestLinkedBoardMoveToAnUnmappedColumnIsRefusedLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + // Only In Progress is mapped; Human Review is a board column with no State. + if rr := f.putStates(mappingBody("Open", "Fixed", [2]string{"In Progress", "Doing"})); rr.Code != http.StatusOK { + t.Fatalf("PUT mapping = %d\n%s", rr.Code, rr.Body.String()) + } + if code, _ := f.move("In Progress"); code != http.StatusOK { + t.Fatalf("move = %d", code) + } + f.srv.ResetRequests() + + code, body := f.move("Human Review") + if code != http.StatusConflict || !strings.Contains(body, "map this column to an issue tracker State first") { + t.Fatalf("move = %d %s, want 409 with the mapping hint", code, body) + } + f.assertNoYouTrackWrites(t) + if got := f.srv.IssueState(f.issueN); got != "Doing" { + t.Fatalf("YouTrack State = %q, want it unchanged", got) + } + f.assertCache(t, "opened", "In Progress") + if col, _ := f.boardCard(t); col != "In Progress" { + t.Fatalf("the card snapped to %q, want it to stay in In Progress", col) + } +} + +// A card a person resolved in YouTrack while the cache still shows it open is not moved by a +// drag: 409, no State write, the cache unchanged so the card snaps back. +func TestLinkedBoardDragOfAnIssueResolvedInYouTrackIs409LiveDB(t *testing.T) { + f := newStateMappingFixture(t) + f.boardMapping(t) + // Resolve the issue in YouTrack, then put the cache back as it was before the next sync. + if code, body := f.move("closed"); code != http.StatusOK { + t.Fatalf("close = %d %s", code, body) + } + if _, err := f.pool.Exec(context.Background(), `UPDATE issues SET state = 'opened' WHERE repo_id = $1 AND forge_issue_iid = $2`, f.repo, f.issueN); err != nil { + t.Fatal(err) + } + f.srv.ResetRequests() + + code, body := f.move("In Progress") + if code != http.StatusConflict || !strings.Contains(body, "resolved in the issue tracker") { + t.Fatalf("move = %d %s, want 409 naming the resolved issue", code, body) + } + f.assertNoYouTrackWrites(t) + if got := f.srv.IssueState(f.issueN); got != "Fixed" { + t.Fatalf("YouTrack State = %q, want Fixed untouched", got) + } + f.assertCache(t, "opened", "") +} + +// A failing issue tracker answers 502 and leaves the cache as it was. +func TestLinkedBoardMoveWhenYouTrackFailsIs502LiveDB(t *testing.T) { + f := newStateMappingFixture(t) + f.boardMapping(t) + f.srv.Close() // every request now fails to connect + + code, body := f.move("In Progress") + if code != http.StatusBadGateway { + t.Fatalf("move = %d %s, want 502", code, body) + } + f.assertCache(t, "opened", "") +} + +func TestLinkedBoardCloseAndReopenLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + f.boardMapping(t) + + if code, body := f.move("closed"); code != http.StatusOK { + t.Fatalf("close = %d %s", code, body) + } + if got := f.srv.IssueState(f.issueN); got != "Fixed" { + t.Fatalf("YouTrack State after close = %q, want Fixed", got) + } + f.assertCache(t, "closed", "") + if _, closed := f.boardCard(t); !closed { + t.Fatal("the board must show the card closed") + } + + // Reopen onto a mapped column: one State write, the cache opened and in the column. + f.srv.ResetRequests() + if code, body := f.move("In Progress"); code != http.StatusOK { + t.Fatalf("reopen = %d %s", code, body) + } + if got := f.srv.IssueState(f.issueN); got != "Doing" { + t.Fatalf("YouTrack State after reopen = %q, want Doing", got) + } + posts := 0 + for _, r := range f.srv.Requests() { + if r.Method == http.MethodPost { + posts++ + } + } + if posts != 1 { + t.Fatalf("a linked reopen made %d write requests, want exactly 1", posts) + } + f.assertCache(t, "opened", "In Progress") + if col, closed := f.boardCard(t); col != "In Progress" || closed { + t.Fatalf("board card in %q closed=%v after the reopen", col, closed) + } +} + +func TestLinkedBoardCloseWithNoMappingIs409LiveDB(t *testing.T) { + f := newStateMappingFixture(t) // no mapping saved + code, body := f.move("closed") + if code != http.StatusConflict { + t.Fatalf("close = %d %s, want 409", code, body) + } + if !strings.Contains(body, "open and closed issue tracker States are not set") { + t.Fatalf("close body = %s, want the unset open/closed States message", body) + } + f.assertNoYouTrackWrites(t) + f.assertCache(t, "opened", "") +} + +// A closed State renamed to Won't fix after the mapping was saved is refused with no write. +func TestLinkedBoardCloseOntoARenamedWontFixIsRefusedLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + f.boardMapping(t) + f.srv.RenameState("Fixed", "Won't fix") + f.srv.ResetRequests() + + code, body := f.move("closed") + if code != http.StatusConflict { + t.Fatalf("close = %d %s, want 409", code, body) + } + f.assertNoYouTrackWrites(t) + if got := f.srv.IssueState(f.issueN); got == "Won't fix" { + t.Fatalf("the issue was closed as %q", got) + } + f.assertCache(t, "opened", "") +} + +// A reopen onto a column with no State changes neither YouTrack nor the cache. +func TestLinkedBoardReopenOntoAnUnmappedColumnChangesNothingLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + if rr := f.putStates(mappingBody("Open", "Fixed", [2]string{"In Progress", "Doing"})); rr.Code != http.StatusOK { + t.Fatalf("PUT mapping = %d\n%s", rr.Code, rr.Body.String()) + } + if code, body := f.move("closed"); code != http.StatusOK { + t.Fatalf("close = %d %s", code, body) + } + f.srv.ResetRequests() + + code, body := f.move("Human Review") + if code != http.StatusConflict { + t.Fatalf("reopen = %d %s, want 409", code, body) + } + f.assertNoYouTrackWrites(t) + if got := f.srv.IssueState(f.issueN); got != "Fixed" { + t.Fatalf("YouTrack State = %q, want it still Fixed", got) + } + f.assertCache(t, "closed", "") +} + +// A card dragged after its issue left the linked project is a 404, and the cache is unchanged. +func TestLinkedBoardMoveOfAMovedAwayIssueIs404LiveDB(t *testing.T) { + f := newStateMappingFixture(t) + f.boardMapping(t) + f.srv.MoveIssue(f.issueN, youtracktest.OtherProjectID) + f.srv.ResetRequests() + + code, body := f.move("In Progress") + if code != http.StatusNotFound || !strings.Contains(body, "no longer in the linked project") { + t.Fatalf("move = %d %s, want 404", code, body) + } + f.assertNoYouTrackWrites(t) + f.assertCache(t, "opened", "") +} + +// The linked sync reads the State of each issue into the column: a mapped State shows the card in +// its column, a move then a sync keeps the column, and an unmapped State is the Open lane. +func TestLinkedSyncCachesTheColumnOfTheStateLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + f.boardMapping(t) + if code, body := f.move("In Progress"); code != http.StatusOK { + t.Fatalf("move = %d %s", code, body) + } + + // Forget the cached column, as a sync of an older build would have left it. + if _, err := f.pool.Exec(context.Background(), `UPDATE issues SET issue_tracker_column = NULL WHERE repo_id = $1`, f.repo); err != nil { + t.Fatal(err) + } + if rr := f.call(f.h.SyncRepo, http.MethodPost, nil); rr.Code != http.StatusOK { + t.Fatalf("sync = %d %s", rr.Code, rr.Body.String()) + } + f.assertCache(t, "opened", "In Progress") + if col, _ := f.boardCard(t); col != "In Progress" { + t.Fatalf("board card in %q after the sync, want In Progress", col) + } + + // A sync right after a move keeps the column: the move was written to YouTrack first. + if code, body := f.move("Human Review"); code != http.StatusOK { + t.Fatalf("second move = %d %s", code, body) + } + if rr := f.call(f.h.SyncRepo, http.MethodPost, nil); rr.Code != http.StatusOK { + t.Fatalf("sync = %d %s", rr.Code, rr.Body.String()) + } + f.assertCache(t, "opened", "Human Review") + + // The mapping loses "Human Review": its State is now unmapped, so the card is the Open lane. + if rr := f.putStates(mappingBody("Open", "Fixed", [2]string{"In Progress", "Doing"})); rr.Code != http.StatusOK { + t.Fatalf("PUT mapping = %d\n%s", rr.Code, rr.Body.String()) + } + if rr := f.call(f.h.SyncRepo, http.MethodPost, nil); rr.Code != http.StatusOK { + t.Fatalf("sync = %d %s", rr.Code, rr.Body.String()) + } + f.assertCache(t, "opened", "") + if col, _ := f.boardCard(t); col != "" { + t.Fatalf("board card in %q for an unmapped State, want the Open lane", col) + } +} + +// Removing a board column clears the cached column of its cards in the same transaction, so the +// card is back in the Open lane and no cached column names a column the board lacks. +func TestConfigureColumnsClearsTheCachedColumnOfARemovedColumnLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + f.boardMapping(t) + if code, body := f.move("Human Review"); code != http.StatusOK { + t.Fatalf("move = %d %s", code, body) + } + + rr := f.call(f.h.ConfigureColumns, http.MethodPut, map[string]any{"columns": []map[string]string{{"label_name": "In Progress"}}}) + if rr.Code != http.StatusOK { + t.Fatalf("ConfigureColumns = %d\n%s", rr.Code, rr.Body.String()) + } + f.assertCache(t, "opened", "") + if col, _ := f.boardCard(t); col != "" { + t.Fatalf("board card in %q after its column was removed, want the Open lane", col) + } + + // A card in a column that stays keeps it. + f.boardMapping(t) + if code, body := f.move("In Progress"); code != http.StatusOK { + t.Fatalf("move = %d %s", code, body) + } + if rr := f.call(f.h.ConfigureColumns, http.MethodPut, map[string]any{"columns": []map[string]string{{"label_name": "In Progress"}, {"label_name": "Backlog"}}}); rr.Code != http.StatusOK { + t.Fatalf("ConfigureColumns = %d\n%s", rr.Code, rr.Body.String()) + } + f.assertCache(t, "opened", "In Progress") +} + +// The unlinked upserts never touch issue_tracker_column: a forge repo's sync leaves it as it is +// (NULL on every forge row). +func TestUnlinkedUpsertsLeaveTheCachedColumnAloneLiveDB(t *testing.T) { + f := newStateMappingFixture(t) + ctx := context.Background() + if _, err := f.pool.Exec(ctx, `UPDATE issues SET issue_tracker_column = 'In Progress' WHERE repo_id = $1`, f.repo); err != nil { + t.Fatal(err) + } + row, err := f.h.q.GetIssueByIID(ctx, store.GetIssueByIIDParams{RepoID: f.repo, ForgeIssueIid: f.issueN}) + if err != nil { + t.Fatal(err) + } + if _, err := f.h.q.UpsertIssue(ctx, store.UpsertIssueParams{ + RepoID: row.RepoID, ForgeIssueIid: row.ForgeIssueIid, Title: "renamed", State: row.State, Labels: row.Labels, + AssigneeIds: row.AssigneeIds, WebUrl: row.WebUrl, ForgeUpdatedAt: row.ForgeUpdatedAt, + }); err != nil { + t.Fatal(err) + } + f.assertCache(t, "opened", "In Progress") + if _, err := f.h.q.UpsertIssueLabels(ctx, store.UpsertIssueLabelsParams{ + RepoID: row.RepoID, ForgeIssueIid: row.ForgeIssueIid, Title: row.Title, State: row.State, Labels: row.Labels, + WebUrl: row.WebUrl, ForgeUpdatedAt: row.ForgeUpdatedAt, AddLabels: []byte(`["x"]`), RemoveLabels: []byte(`[]`), + }); err != nil { + t.Fatal(err) + } + f.assertCache(t, "opened", "In Progress") +} diff --git a/api/internal/handler/linked_repo_create_livedb_test.go b/api/internal/handler/linked_repo_create_livedb_test.go new file mode 100644 index 0000000000..cae88eeaa8 --- /dev/null +++ b/api/internal/handler/linked_repo_create_livedb_test.go @@ -0,0 +1,294 @@ +package handler + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "slices" + "testing" + + "github.com/vtmocanu/uzi/api/internal/apitypes" + "github.com/vtmocanu/uzi/api/internal/issuetracker" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" + "github.com/vtmocanu/uzi/api/internal/issuetrackersvc" +) + +// PRD #4173 M5 (D12, D14): an issue uzi creates on a repo linked to a YouTrack project is +// created in that project. The configured uzi label becomes the stored eligibility tag, +// addressed by id (never the same-named tag the connection's account owns), the autopilot +// label is dropped, and an issue filed without send-to-uzi carries no eligibility tag. + +// trustYouTrack makes the process-wide default transport trust srv's certificate for the +// test, as the linked fixture does, so the issue tracker driver built by issuetrackersvc +// reaches the fake over TLS. +func trustYouTrack(t *testing.T, srv *youtracktest.Server) { + t.Helper() + def, ok := http.DefaultTransport.(*http.Transport) + if !ok { + t.Fatal("http.DefaultTransport is not an *http.Transport") + } + saved := def.TLSClientConfig + def.TLSClientConfig = srv.Client().Transport.(*http.Transport).TLSClientConfig.Clone() + t.Cleanup(func() { def.TLSClientConfig = saved }) +} + +// linkNeverEnabledFixture links the fixture's repo to a fresh YouTrack fake and wires the +// real issue tracker driver source into its forgesvc. +func linkNeverEnabledFixture(t *testing.T, f *neverEnabledFixture) *youtracktest.Server { + t.Helper() + ctx := context.Background() + srv := youtracktest.NewServer(t) + trustYouTrack(t, srv) + itSvc := issuetrackersvc.New(f.h.q, f.h.box, func(u string) bool { return u == srv.URL }, issuetracker.New) + f.h.svc.SetIssueTrackerDrivers(itSvc) + yt, err := itSvc.Connect(ctx, f.admin.ID, "youtrack", srv.URL, srv.Token) + if err != nil { + t.Fatal(err) + } + mustExecT(ctx, t, f.pool, `UPDATE repos SET enabled = true, awaiting_first_enable = false, + issue_tracker_connection_id = $2, issue_tracker_project_id = $3, issue_tracker_project_short = $4, + eligibility_tag_id = $5, eligibility_tag_name = $6, issue_tracker_linked_at = now() WHERE id = $1`, + f.repoID, yt.ID, youtracktest.ProjectID, youtracktest.ProjectShortName, youtracktest.EligibilityTagID, youtracktest.EligibilityTagName) + srv.ResetRequests() + return srv +} + +// createdIID returns the first "iid" in a JSON response body, however deeply nested. +func createdIID(t *testing.T, body []byte) int64 { + t.Helper() + var v any + if err := json.Unmarshal(body, &v); err != nil { + t.Fatalf("decode %s: %v", body, err) + } + var find func(any) (int64, bool) + find = func(v any) (int64, bool) { + switch x := v.(type) { + case map[string]any: + if n, ok := x["iid"].(float64); ok { + return int64(n), true + } + for _, c := range x { + if n, ok := find(c); ok { + return n, true + } + } + case []any: + for _, c := range x { + if n, ok := find(c); ok { + return n, true + } + } + } + return 0, false + } + n, ok := find(v) + if !ok { + t.Fatalf("no iid in %s", body) + } + return n +} + +// issueTags returns the tag ids and the tag names on YouTrack issue n. +func issueTags(srv *youtracktest.Server, n int64) (ids, names []string) { + ids = srv.IssueTagIDs(n) + for _, id := range ids { + names = append(names, srv.TagByID(id).Name) + } + slices.Sort(names) + return ids, names +} + +func wantTags(t *testing.T, srv *youtracktest.Server, n int64, wantEligibility bool, wantNames ...string) { + t.Helper() + ids, names := issueTags(srv, n) + if got := slices.Contains(ids, youtracktest.EligibilityTagID); got != wantEligibility { + t.Errorf("eligibility tag on the issue = %v, want %v (tags %v)", got, wantEligibility, ids) + } + if slices.Contains(ids, youtracktest.SameNameTagID) { + t.Errorf("the same-named tag the connection's account owns was applied: %v", ids) + } + slices.Sort(wantNames) + if !slices.Equal(names, wantNames) { + t.Errorf("tag names = %v, want %v", names, wantNames) + } + for _, stray := range []string{"uzi", "autopilot"} { + if len(srv.TagsNamed(stray)) != 0 { + t.Errorf("a %q tag was created on the issue tracker", stray) + } + } +} + +func TestLinkedRepoFindingFilingCreatesInTheLinkedProjectLiveDB(t *testing.T) { + cases := []struct { + name string + sendToUzi bool + wantEligibility bool + wantNames []string + }{ + {"send to uzi carries the eligibility tag only", true, true, []string{"agent-found", youtracktest.EligibilityTagName}}, + {"without send to uzi carries no eligibility tag", false, false, []string{"agent-found"}}, + } + for _, tc := range cases { + t.Run("FileFinding/"+tc.name, func(t *testing.T) { + f := seedNeverEnabledFixture(t) + srv := linkNeverEnabledFixture(t, f) + rr := httptest.NewRecorder() + f.h.FileFinding(rr, f.request(http.MethodPost, apitypes.FileFindingRequest{SendToUzi: tc.sendToUzi}, map[string]string{"id": f.findID.String()})) + if rr.Code != http.StatusCreated { + t.Fatalf("status = %d body = %s", rr.Code, rr.Body.String()) + } + wantTags(t, srv, createdIID(t, rr.Body.Bytes()), tc.wantEligibility, tc.wantNames...) + if n := f.hits.Load(); n != 0 { + t.Errorf("the forge saw %d request(s), want 0", n) + } + }) + t.Run("FileFindingGroup/"+tc.name, func(t *testing.T) { + f := seedNeverEnabledFixture(t) + srv := linkNeverEnabledFixture(t, f) + rr := httptest.NewRecorder() + f.h.FileFindingGroup(rr, f.request(http.MethodPost, apitypes.FindingGroupFileRequest{IDs: []string{f.dispID.String()}, SendToUzi: tc.sendToUzi}, nil)) + if rr.Code != http.StatusCreated { + t.Fatalf("status = %d body = %s", rr.Code, rr.Body.String()) + } + wantTags(t, srv, createdIID(t, rr.Body.Bytes()), tc.wantEligibility, tc.wantNames...) + if n := f.hits.Load(); n != 0 { + t.Errorf("the forge saw %d request(s), want 0", n) + } + }) + } +} + +func TestLinkedRepoReviewAndJudgeIssuesCarryTheEligibilityTagLiveDB(t *testing.T) { + t.Run("FileIssue", func(t *testing.T) { + f := seedNeverEnabledFixture(t) + srv := linkNeverEnabledFixture(t, f) + rr := httptest.NewRecorder() + f.h.FileIssue(rr, f.request(http.MethodPost, map[string]string{"repo_id": f.repoID.String(), "title": "t", "description": "d"}, + map[string]string{"id": f.runID.String(), "recID": f.recID.String()})) + if rr.Code != http.StatusCreated { + t.Fatalf("status = %d body = %s", rr.Code, rr.Body.String()) + } + wantTags(t, srv, createdIID(t, rr.Body.Bytes()), true, youtracktest.EligibilityTagName) + }) + t.Run("AdminFileJudgeIssue", func(t *testing.T) { + f := seedNeverEnabledFixture(t) + srv := linkNeverEnabledFixture(t, f) + rr := httptest.NewRecorder() + f.h.AdminFileJudgeIssue(rr, f.request(http.MethodPost, adminFileIssueRequest{ + Category: "improve_agent", Target: "ne-" + f.repoID.String(), RepoID: f.repoID.String(), Title: "t", Description: "d", + }, nil)) + if rr.Code != http.StatusCreated { + t.Fatalf("status = %d body = %s", rr.Code, rr.Body.String()) + } + wantTags(t, srv, createdIID(t, rr.Body.Bytes()), true, youtracktest.EligibilityTagName) + }) +} + +// The board's new issue is created in the linked project with the stored eligibility tag +// and is cached under the eligibility name. +func TestLinkedRepoBoardCreateIssueCarriesTheEligibilityTagLiveDB(t *testing.T) { + f := newLinkedHandlerFixture(t, true) + rr := f.call(f.h.CreateIssue, http.MethodPost, map[string]string{"title": "t", "description": "d"}) + if rr.Code != http.StatusCreated { + t.Fatalf("status = %d body = %s", rr.Code, rr.Body.String()) + } + n := createdIID(t, rr.Body.Bytes()) + wantTags(t, f.srv, n, true, youtracktest.EligibilityTagName) + var labels []string + var raw []byte + if err := f.pool.QueryRow(context.Background(), `SELECT labels FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2`, f.repo, n).Scan(&raw); err != nil { + t.Fatal(err) + } + if err := json.Unmarshal(raw, &labels); err != nil || !slices.Equal(labels, []string{youtracktest.EligibilityTagName}) { + t.Errorf("cached labels = %v (%v), want only the eligibility name", labels, err) + } + if c := f.rec.Calls(); len(c) != 0 { + t.Errorf("the forge saw calls %v, want none", c) + } +} + +// Promote adds the stored eligibility tag by id, not the configured uzi label, and the +// same-named tag the connection's account owns is untouched. +func TestLinkedRepoPromoteAddsTheStoredEligibilityTagLiveDB(t *testing.T) { + f := newLinkedHandlerFixture(t, true) + untagged := f.srv.SeedIssue("untagged card", "body") + if _, err := f.pool.Exec(context.Background(), `INSERT INTO issues (repo_id, forge_issue_iid, title, state, labels, web_url, forge_updated_at, synced_at) + VALUES ($1, $2, 'untagged', 'opened', '[]', 'https://yt.test/u', now(), now())`, f.repo, untagged); err != nil { + t.Fatal(err) + } + f.issueN = untagged + f.srv.ResetRequests() + rr := f.call(f.h.PromoteIssue, http.MethodPost, nil) + if rr.Code != http.StatusOK { + t.Fatalf("status = %d body = %s", rr.Code, rr.Body.String()) + } + wantTags(t, f.srv, untagged, true, youtracktest.EligibilityTagName) + var resp struct { + Card struct { + Labels []string `json:"labels"` + } `json:"card"` + } + if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil || !slices.Equal(resp.Card.Labels, []string{youtracktest.EligibilityTagName}) { + t.Errorf("promoted card labels = %v (%v), want the eligibility name", resp.Card.Labels, err) + } + if c := f.rec.Calls(); len(c) != 0 { + t.Errorf("the forge saw calls %v, want none", c) + } +} + +// PRD #4173 M5: an issue uzi creates without the eligibility tag (a finding filed without +// send-to-uzi) lives in the linked project but is not on the board after the linked full +// sync, while the board's own new issue (which carries the tag) is. +func TestLinkedRepoUntaggedCreateIsNotOnTheBoardAfterTheFullSyncLiveDB(t *testing.T) { + f := seedNeverEnabledFixture(t) + srv := linkNeverEnabledFixture(t, f) + params := map[string]string{"id": f.repoID.String()} + + rr := httptest.NewRecorder() + f.h.FileFinding(rr, f.request(http.MethodPost, apitypes.FileFindingRequest{SendToUzi: false}, map[string]string{"id": f.findID.String()})) + if rr.Code != http.StatusCreated { + t.Fatalf("file finding: status = %d body = %s", rr.Code, rr.Body.String()) + } + untagged := createdIID(t, rr.Body.Bytes()) + wantTags(t, srv, untagged, false, "agent-found") + + rr = httptest.NewRecorder() + f.h.CreateIssue(rr, f.request(http.MethodPost, map[string]string{"title": "board card", "description": "d"}, params)) + if rr.Code != http.StatusCreated { + t.Fatalf("create issue: status = %d body = %s", rr.Code, rr.Body.String()) + } + tagged := createdIID(t, rr.Body.Bytes()) + wantTags(t, srv, tagged, true, youtracktest.EligibilityTagName) + + rr = httptest.NewRecorder() + f.h.SyncRepo(rr, f.request(http.MethodPost, nil, params)) + if rr.Code != http.StatusOK { + t.Fatalf("sync: status = %d body = %s", rr.Code, rr.Body.String()) + } + rr = httptest.NewRecorder() + f.h.GetBoard(rr, f.request(http.MethodGet, nil, params)) + if rr.Code != http.StatusOK { + t.Fatalf("board: status = %d body = %s", rr.Code, rr.Body.String()) + } + var body struct { + Board struct { + Cards []struct { + IID int64 `json:"iid"` + } `json:"cards"` + } `json:"board"` + } + if err := json.Unmarshal(rr.Body.Bytes(), &body); err != nil { + t.Fatalf("decode board: %v", err) + } + var onBoard []int64 + for _, c := range body.Board.Cards { + onBoard = append(onBoard, c.IID) + } + if slices.Contains(onBoard, untagged) { + t.Errorf("the issue created without the eligibility tag (%d) is on the board: %v", untagged, onBoard) + } + if !slices.Contains(onBoard, tagged) { + t.Errorf("the issue created with the eligibility tag (%d) is not on the board: %v", tagged, onBoard) + } +} diff --git a/api/internal/handler/linked_repo_filing_livedb_test.go b/api/internal/handler/linked_repo_filing_livedb_test.go index 5b88a912d1..440a560fe5 100644 --- a/api/internal/handler/linked_repo_filing_livedb_test.go +++ b/api/internal/handler/linked_repo_filing_livedb_test.go @@ -4,7 +4,7 @@ import ( "context" "net/http" "net/http/httptest" - "strings" + "sync/atomic" "testing" "github.com/google/uuid" @@ -12,20 +12,24 @@ import ( "github.com/vtmocanu/uzi/api/internal/issuetracker" ) -// linkedDrivers hands out the BaseFake-backed driver; every write is refused by the -// guard before it could reach it. -type linkedDrivers struct{} +// linkedDrivers hands out the BaseFake-backed driver; a test that uses it never drives an +// issue write through it. +type linkedDrivers struct{ built *atomic.Int32 } -func (linkedDrivers) DriverFor(_ context.Context, _, _ uuid.UUID, _ issuetracker.Binding, _, _, _ string) (issuetracker.Driver, error) { +func (d linkedDrivers) DriverFor(_ context.Context, _, _ uuid.UUID, _ issuetracker.Binding, _ issuetracker.StateMapping) (issuetracker.Driver, error) { + d.built.Add(1) return &itDriver{}, nil } -// Every filing entry point on a repo linked to an issue tracker answers 409 (never 502 or -// an "uncertain" 202), releases what it claimed, and sends nothing to the forge. -func TestLinkedRepoFilingRoutesAreRefusedAndReleaseTheirClaimLiveDB(t *testing.T) { +// Reconfiguring the columns on a linked repo writes only board_columns rows. Converted from +// TestLinkedRepoFilingRoutesAreRefusedAndReleaseTheirClaimLiveDB: the four filing routes it +// refused with a 409 are accepted since PRD #4173 M5 (see +// linked_repo_create_livedb_test.go), and this was its remaining subtest. +func TestLinkedRepoConfigureColumnsTouchesNoIssueTrackerLiveDB(t *testing.T) { f := seedNeverEnabledFixture(t) ctx := context.Background() - f.h.svc.SetIssueTrackerDrivers(linkedDrivers{}) + var built atomic.Int32 + f.h.svc.SetIssueTrackerDrivers(linkedDrivers{built: &built}) itConn := uuid.New() mustExecT(ctx, t, f.pool, `INSERT INTO issue_tracker_connections (id, user_id, kind, base_url, account_id, account_login, token_ciphertext) VALUES ($1, $2, 'youtrack', $3, '1-1', 'bot', $4)`, itConn, f.admin.ID, "https://"+uuid.NewString()+".yt.test", []byte{1}) @@ -33,50 +37,22 @@ func TestLinkedRepoFilingRoutesAreRefusedAndReleaseTheirClaimLiveDB(t *testing.T mustExecT(ctx, t, f.pool, `UPDATE repos SET enabled = true, awaiting_first_enable = false, issue_tracker_connection_id = $2, issue_tracker_project_id = '0-1', issue_tracker_project_short = 'UZI', eligibility_tag_id = '5-1', eligibility_tag_name = 'uzi', issue_tracker_linked_at = now() WHERE id = $1`, f.repoID, itConn) - repoParam := map[string]string{"id": f.repoID.String()} - cases := []struct { - name string - call func(w http.ResponseWriter) - }{ - {"FileFinding", func(w http.ResponseWriter) { - f.h.FileFinding(w, f.request(http.MethodPost, map[string]any{}, map[string]string{"id": f.findID.String()})) - }}, - {"FileFindingGroup", func(w http.ResponseWriter) { - f.h.FileFindingGroup(w, f.request(http.MethodPost, map[string]any{"ids": []string{f.dispID.String()}}, nil)) - }}, - {"FileIssue", func(w http.ResponseWriter) { - f.h.FileIssue(w, f.request(http.MethodPost, map[string]string{"repo_id": f.repoID.String(), "title": "t", "description": "d"}, - map[string]string{"id": f.runID.String(), "recID": f.recID.String()})) - }}, - {"AdminFileJudgeIssue", func(w http.ResponseWriter) { - f.h.AdminFileJudgeIssue(w, f.request(http.MethodPost, adminFileIssueRequest{ - Category: "improve_agent", Target: "ne-" + f.repoID.String(), RepoID: f.repoID.String(), Title: "t", Description: "d", - }, nil)) - }}, - {"ConfigureColumns", func(w http.ResponseWriter) { - f.h.ConfigureColumns(w, f.request(http.MethodPut, map[string]any{"columns": []map[string]string{{"label_name": "a"}}}, repoParam)) - }}, + before := built.Load() + rr := httptest.NewRecorder() + f.h.ConfigureColumns(rr, f.request(http.MethodPut, map[string]any{"columns": []map[string]string{{"label_name": "a"}}}, + map[string]string{"id": f.repoID.String()})) + if rr.Code != http.StatusOK { + t.Fatalf("status = %d body = %s, want 200", rr.Code, rr.Body.String()) } - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - rr := httptest.NewRecorder() - tc.call(rr) - if rr.Code != http.StatusConflict || !strings.Contains(rr.Body.String(), "not offered yet") { - t.Fatalf("status = %d body = %s, want the 409 refusal", rr.Code, rr.Body.String()) - } - if n := f.hits.Load(); n != 0 { - t.Errorf("the forge saw %d request(s), want 0", n) - } - }) + if n := f.hits.Load(); n != 0 { + t.Errorf("the forge saw %d request(s), want 0", n) } - - var status string - if err := f.pool.QueryRow(ctx, `SELECT status FROM finding_dispositions WHERE repo_id = $1`, f.repoID).Scan(&status); err != nil || status != "open" { - t.Errorf("finding disposition = %q (%v), want open: the group claim must be released, not left uncertain", status, err) + if n := built.Load() - before; n != 0 { + t.Errorf("ConfigureColumns resolved %d issue tracker driver(s), want 0", n) } - var filed int - if err := f.pool.QueryRow(ctx, `SELECT count(*) FROM recommendation_filed_issues WHERE filed_repo_id = $1`, f.repoID).Scan(&filed); err != nil || filed != 0 { - t.Errorf("recommendation filed rows = %d (%v), want 0", filed, err) + var cols int + if err := f.pool.QueryRow(ctx, `SELECT count(*) FROM board_columns WHERE repo_id = $1 AND label_name = 'a'`, f.repoID).Scan(&cols); err != nil || cols != 1 { + t.Errorf("board column rows = %d (%v), want 1", cols, err) } } diff --git a/api/internal/handler/linked_repo_runnow_livedb_test.go b/api/internal/handler/linked_repo_runnow_livedb_test.go index fb520533e4..21b9d0bfe5 100644 --- a/api/internal/handler/linked_repo_runnow_livedb_test.go +++ b/api/internal/handler/linked_repo_runnow_livedb_test.go @@ -10,6 +10,7 @@ import ( "github.com/go-chi/chi/v5" "github.com/google/uuid" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" mw "github.com/vtmocanu/uzi/api/internal/middleware" ) @@ -26,15 +27,17 @@ func TestLinkedRepoRunNowResolvesTheIssueTrackerLiveDB(t *testing.T) { ctx := context.Background() mustExecT(ctx, t, f.pool, `INSERT INTO run_schedules (id, user_id, repo_id, target, labels, timing, cron_expr, timezone, next_fire_at, auto_approve, status, enabled, max_issues) - VALUES ($1, $2, $3, 'sweep', '["uzi"]'::jsonb, 'recurring', '0 3 * * *', 'UTC', now() + interval '1 day', false, 'active', true, 1)`, - sweep, f.user.ID, f.repo) + VALUES ($1, $2, $3, 'sweep', jsonb_build_array($4::text), 'recurring', '0 3 * * *', 'UTC', now() + interval '1 day', false, 'active', true, 1)`, + sweep, f.user.ID, f.repo, youtracktest.EligibilityTagName) mustExecT(ctx, t, f.pool, `INSERT INTO run_schedules (id, user_id, repo_id, target, issue_iid, timing, cron_expr, timezone, next_fire_at, auto_approve, status, enabled) VALUES ($1, $2, $3, 'issue', $4, 'recurring', '0 3 * * *', 'UTC', now() + interval '1 day', false, 'active', true)`, issue, f.user.ID, f.repo, f.issueN) - // The sweep reads its candidates from the cached cards' labels; make the seeded card one. - mustExecT(ctx, t, f.pool, `UPDATE issues SET labels = '["uzi"]' WHERE repo_id = $1`, f.repo) + // The sweep reads its candidates from the cached cards' labels, and on a linked repo the + // eligibility label is the link's stored eligibility tag name (PRD #4173 D14), not "uzi": + // tag the seeded card with it so the sweep has a real candidate. + mustExecT(ctx, t, f.pool, `UPDATE issues SET labels = jsonb_build_array($2::text) WHERE repo_id = $1`, f.repo, youtracktest.EligibilityTagName) for name, id := range map[string]uuid.UUID{"sweep": sweep, "issue": issue} { t.Run(name, func(t *testing.T) { diff --git a/api/internal/handler/linked_repo_writes_livedb_test.go b/api/internal/handler/linked_repo_writes_livedb_test.go index 8cfd6fc48f..f4398998bc 100644 --- a/api/internal/handler/linked_repo_writes_livedb_test.go +++ b/api/internal/handler/linked_repo_writes_livedb_test.go @@ -30,6 +30,7 @@ type linkedHandlerFixture struct { h *Handler pool *pgxpool.Pool srv *youtracktest.Server + itSvc *issuetrackersvc.Service rec *forgetest.Recorder user store.User repo uuid.UUID @@ -61,7 +62,7 @@ func newLinkedHandlerFixture(t *testing.T, withColumns bool) *linkedHandlerFixtu if err != nil { t.Fatal(err) } - f := &linkedHandlerFixture{h: h, pool: pool, srv: srv, rec: rec, repo: uuid.New(), + f := &linkedHandlerFixture{h: h, pool: pool, srv: srv, itSvc: itSvc, rec: rec, repo: uuid.New(), user: store.User{ID: uuid.New(), Email: fmt.Sprintf("lw-%s@e2e", uuid.NewString()[:8])}} exec := func(sql string, args ...any) { t.Helper() @@ -119,35 +120,20 @@ func (f *linkedHandlerFixture) assertNothingSent(t *testing.T) { } } -// Every issue write on a linked repo is a clear 409 and sends nothing anywhere. -func TestLinkedRepoIssueWritesAreRefusedWith409LiveDB(t *testing.T) { +// The one issue write on a linked repo that no unit offers, the label ensure endpoint, is a +// clear 409 and sends nothing anywhere: the eligibility tag is chosen at link time and column +// and autopilot tags are not wanted. The board move is offered since PRD #4173 M5 (see +// linked_board_livedb_test.go), and so are the board's new issue and Promote (see +// linked_repo_create_livedb_test.go). Converted from +// TestLinkedRepoIssueWritesAreRefusedWith409LiveDB, whose "create issue" and "promote issue" +// subtests became those acceptance tests. +func TestLinkedRepoEnsureLabelsIsRefusedWith409LiveDB(t *testing.T) { f := newLinkedHandlerFixture(t, true) - cases := []struct { - name string - do func() *httptest.ResponseRecorder - }{ - {"create issue", func() *httptest.ResponseRecorder { - return f.call(f.h.CreateIssue, http.MethodPost, map[string]string{"title": "t", "description": "d"}) - }}, - {"move issue", func() *httptest.ResponseRecorder { - return f.call(f.h.MoveIssue, http.MethodPost, map[string]string{"to_column": "In Progress"}) - }}, - {"promote issue", func() *httptest.ResponseRecorder { - return f.call(f.h.PromoteIssue, http.MethodPost, nil) - }}, - {"ensure labels", func() *httptest.ResponseRecorder { - return f.call(f.h.EnsureRepoLabels, http.MethodPost, map[string][]string{"labels": {"uzi"}}) - }}, - } - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - rr := tc.do() - if rr.Code != http.StatusConflict || !strings.Contains(rr.Body.String(), "issue writes on a repo linked to an issue tracker are not offered yet") { - t.Fatalf("status = %d body = %s, want the 409 refusal", rr.Code, rr.Body.String()) - } - f.assertNothingSent(t) - }) + rr := f.call(f.h.EnsureRepoLabels, http.MethodPost, map[string][]string{"labels": {"uzi"}}) + if rr.Code != http.StatusConflict || !strings.Contains(rr.Body.String(), "this issue write is not offered on a repo linked to an issue tracker") { + t.Fatalf("status = %d body = %s, want the 409 refusal", rr.Code, rr.Body.String()) } + f.assertNothingSent(t) } // assertNoWrites fails when the issue tracker saw anything but a read (GET) or the diff --git a/api/internal/handler/review_issue_file.go b/api/internal/handler/review_issue_file.go index c1adbb5ede..8924ac3262 100644 --- a/api/internal/handler/review_issue_file.go +++ b/api/internal/handler/review_issue_file.go @@ -160,7 +160,7 @@ func (h *Handler) FileIssue(w http.ResponseWriter, r *http.Request) { httpx.Error(w, http.StatusInternalServerError, "internal error") return } - created, err := tr.CreateIssue(ctx, repo.ForgeProjectID, title, description, labels) + created, err := tr.CreateIssue(ctx, repo.ForgeProjectID, title, description, h.createLabelsFor(ctx, repo, labels)) if err != nil { // Forge rejected the write (mock state E): the claim reverts, nothing is // persisted, the draft stays open. err is already PAT-redacted by the driver. diff --git a/api/internal/handler/route_limiter_mounts_test.go b/api/internal/handler/route_limiter_mounts_test.go index 669385af3b..88ff06240e 100644 --- a/api/internal/handler/route_limiter_mounts_test.go +++ b/api/internal/handler/route_limiter_mounts_test.go @@ -1107,6 +1107,9 @@ var wantRouteMounts = []routeMount{ {"PUT", "/api/repos/{id}/github-project-sync/visibility", noLimiter}, // PRD #4173 M4: the repo link reads the issue tracker (token, projects, tags). {"PUT", "/api/repos/{id}/issue-tracker-link", limForge}, + // PRD #4173 M5: the State mapping reads the issue tracker's States on both verbs. + {"GET", "/api/repos/{id}/issue-tracker-link/states", limForge}, + {"PUT", "/api/repos/{id}/issue-tracker-link/states", limForge}, {"PUT", "/api/repos/{id}/tool-profile", noLimiter}, // PRD #35 Decision 7, the per-run toggle. noLimiter for the same reason as // /me/wait-on-limit: one owner-scoped boolean UPDATE, no spend, no forge write, diff --git a/api/internal/handler/routes_repos.go b/api/internal/handler/routes_repos.go index f814347fc3..efd5883be5 100644 --- a/api/internal/handler/routes_repos.go +++ b/api/internal/handler/routes_repos.go @@ -107,6 +107,11 @@ func (h *Handler) mountRepoRoutes(r chi.Router, forgeLimiter, boardOrderLimiter // and tag lists), so it carries the per-user forge budget; owner-scoped in the // service (404 for a foreign repo or connection). No unlink or relink route exists. r.With(forgeLimiter.PerUserMiddleware).Put("/{id}/issue-tracker-link", h.LinkRepoToIssueTracker) + // The linked repo's State mapping (PRD #4173 M5): the GET lists the project's States + // from the issue tracker and the PUT validates against them, so both carry the forge + // budget. Session-only and owner-scoped in the service, like the link itself. + r.With(forgeLimiter.PerUserMiddleware).Get("/{id}/issue-tracker-link/states", h.GetIssueTrackerStateMapping) + r.With(forgeLimiter.PerUserMiddleware).Put("/{id}/issue-tracker-link/states", h.PutIssueTrackerStateMapping) // GitHub Projects v2 sync, owner-or-admin (issue #534, PRD #364 follow-up): // relocated out of /admin (D4). Owner path is guarded by a GetRepoForUser // preflight inside each handler; admin skips it. Instance flag still gates. diff --git a/api/internal/issuesource/issuesource.go b/api/internal/issuesource/issuesource.go index e44539b7fa..5f9da417db 100644 --- a/api/internal/issuesource/issuesource.go +++ b/api/internal/issuesource/issuesource.go @@ -24,11 +24,12 @@ type Sources interface { // is deliberately no fallback to the forge, whose issue numbers mean nothing there. var ErrNotWired = errors.New("repo is linked to an issue tracker but no issue tracker source is wired") -// ErrIssueWriteNotOffered is returned for every write to a linked external issue -// tracker (YouTrack writes arrive in PRD #4173 M5). forgesvc.ErrIssueWriteNotOffered -// is this same value; it lives here so a package that cannot import forgesvc can -// wrap and match it. -var ErrIssueWriteNotOffered = errors.New("issue writes on a repo linked to an issue tracker are not offered yet") +// ErrIssueWriteNotOffered is returned for the one write to a linked external issue +// tracker that is not offered: a label write that removes the eligibility name +// (PRD #4173 M5). +// forgesvc.ErrIssueWriteNotOffered is this same value; it lives here so a package that +// cannot import forgesvc can wrap and match it. +var ErrIssueWriteNotOffered = errors.New("this issue write is not offered on a repo linked to an issue tracker") // For returns the issue side a caller should use for one repo row. linked is // whether the row's issue_tracker_connection_id is set. An unlinked row gets @@ -43,3 +44,31 @@ func For(ctx context.Context, src Sources, linked bool, userID, repoID uuid.UUID } return src.IssueTrackerForRepo(ctx, userID, repoID) } + +// LinkedLabels is the label set an issue uzi creates on a repo linked to an issue tracker +// carries (PRD #4173 D12, D14): the configured uzi label becomes the link's stored +// eligibility name, the autopilot label is dropped because autopilot is not offered on a +// linked repo, and each name appears once, in the order first seen. A blank name is +// dropped. Only a linked repo's create site calls it; an unlinked repo keeps its labels +// untouched. +func LinkedLabels(labels []string, uziLabel, autopilotLabel, eligibilityName string) []string { + out := make([]string, 0, len(labels)) + seen := make(map[string]struct{}, len(labels)) + for _, l := range labels { + if l == autopilotLabel { + continue + } + if l == uziLabel { + l = eligibilityName + } + if l == "" { + continue + } + if _, dup := seen[l]; dup { + continue + } + seen[l] = struct{}{} + out = append(out, l) + } + return out +} diff --git a/api/internal/issuesource/issuesource_test.go b/api/internal/issuesource/issuesource_test.go index d225d01101..e8ef6f6b3d 100644 --- a/api/internal/issuesource/issuesource_test.go +++ b/api/internal/issuesource/issuesource_test.go @@ -45,3 +45,40 @@ func TestForLinkedUsesSourcesAndFailsClosedWithout(t *testing.T) { t.Fatalf("resolver error lost: %v", err) } } + +func TestLinkedLabels(t *testing.T) { + cases := []struct { + name string + labels []string + want []string + }{ + {"uzi becomes the eligibility name", []string{"uzi"}, []string{"ready"}}, + {"autopilot is dropped", []string{"uzi", "autopilot"}, []string{"ready"}}, + {"order is kept", []string{"finding", "autopilot", "uzi", "extra"}, []string{"finding", "ready", "extra"}}, + {"dedupe after the mapping", []string{"uzi", "ready", "uzi"}, []string{"ready"}}, + {"no run labels", []string{"finding"}, []string{"finding"}}, + {"empty", nil, []string{}}, + {"only autopilot", []string{"autopilot"}, []string{}}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := LinkedLabels(tc.labels, "uzi", "autopilot", "ready") + if len(got) != len(tc.want) { + t.Fatalf("got %v, want %v", got, tc.want) + } + for i := range got { + if got[i] != tc.want[i] { + t.Fatalf("got %v, want %v", got, tc.want) + } + } + }) + } +} + +func TestLinkedLabelsDoesNotMutateItsInput(t *testing.T) { + in := []string{"uzi", "autopilot"} + _ = LinkedLabels(in, "uzi", "autopilot", "ready") + if in[0] != "uzi" || in[1] != "autopilot" { + t.Fatalf("input mutated: %v", in) + } +} diff --git a/api/internal/issuesource/stated.go b/api/internal/issuesource/stated.go new file mode 100644 index 0000000000..67a055faee --- /dev/null +++ b/api/internal/issuesource/stated.go @@ -0,0 +1,21 @@ +package issuesource + +import ( + "context" + + "github.com/vtmocanu/uzi/api/internal/forge" +) + +// StatedIssue is one listed issue plus the name of its issue tracker State +// value, which forge.Issue has no room for. State is empty when the issue has +// no readable State value. +type StatedIssue struct { + Issue forge.Issue + State string +} + +// IssueStateLister is implemented by an issue side that can list issues with +// their State values from the same walk as forge.IssueTracker.ListIssues. +type IssueStateLister interface { + ListIssuesWithState(ctx context.Context, projectID int64, opts forge.ListIssuesOptions) ([]StatedIssue, error) +} diff --git a/api/internal/issuesource/stated_test.go b/api/internal/issuesource/stated_test.go new file mode 100644 index 0000000000..4b7030a99a --- /dev/null +++ b/api/internal/issuesource/stated_test.go @@ -0,0 +1,32 @@ +package issuesource + +import ( + "context" + "testing" + + "github.com/vtmocanu/uzi/api/internal/forge" +) + +type fakeLister struct{} + +func (fakeLister) ListIssuesWithState(context.Context, int64, forge.ListIssuesOptions) ([]StatedIssue, error) { + return []StatedIssue{{Issue: forge.Issue{IID: 7}, State: "In Progress"}}, nil +} + +// TestIssueStateListerIsOptional: a consumer finds the capability by type +// assertion on a forge.IssueTracker, and a forge driver without it is not one. +func TestIssueStateListerIsOptional(t *testing.T) { + var tr any = fakeLister{} + l, ok := tr.(IssueStateLister) + if !ok { + t.Fatal("fakeLister is not an IssueStateLister") + } + got, err := l.ListIssuesWithState(context.Background(), 1, forge.ListIssuesOptions{}) + if err != nil || len(got) != 1 || got[0].Issue.IID != 7 || got[0].State != "In Progress" { + t.Errorf("ListIssuesWithState = %+v, %v", got, err) + } + var plain any = struct{}{} + if _, ok := plain.(IssueStateLister); ok { + t.Error("an arbitrary value satisfies IssueStateLister") + } +} diff --git a/api/internal/issuetracker/driver.go b/api/internal/issuetracker/driver.go index dee371ff44..2d82875ee7 100644 --- a/api/internal/issuetracker/driver.go +++ b/api/internal/issuetracker/driver.go @@ -27,6 +27,11 @@ type State struct { Name string Resolved bool // true when the state counts as closed Ordinal int + // Archived is true for a value the bundle keeps but no longer offers. + Archived bool + // LocalizedName is the value's name in the server's language, empty when it + // has none or the driver refused it (see stateNameProblem). + LocalizedName string } // Driver is an external issue tracker connection. It embeds forge.IssueTracker diff --git a/api/internal/issuetracker/errors.go b/api/internal/issuetracker/errors.go index 33e5a75da5..0a471ecd43 100644 --- a/api/internal/issuetracker/errors.go +++ b/api/internal/issuetracker/errors.go @@ -14,13 +14,24 @@ var ( // ErrStateMappingUnset is returned when open/closed state mapping is needed // but the connection has not configured it. ErrStateMappingUnset = errors.New("issuetracker: state mapping not configured") + // ErrColumnNotMapped is returned by MoveToColumn for a board column that has + // no issue tracker State mapped to it. + ErrColumnNotMapped = errors.New("issuetracker: board column has no state mapped") + // ErrStateNotAllowed is returned before a State write whose target the + // project's State bundle does not allow: the value is missing, archived, or + // has the wrong resolved flag or meaning for the role it is written in. + ErrStateNotAllowed = errors.New("issuetracker: state value is not allowed for this write") // ErrInvalidLabelName is returned for a label name the issue tracker cannot carry. ErrInvalidLabelName = errors.New("issuetracker: invalid label name") // ErrRedirectRefused is returned when the server answers with a redirect. // The client never follows one, so the token cannot be forwarded. ErrRedirectRefused = errors.New("issuetracker: redirect refused") + // ErrIssueResolved is returned by MoveToColumn for an issue that is resolved + // in the issue tracker when the move is not an explicit reopen. Nothing is + // written. + ErrIssueResolved = errors.New("issuetracker: issue is resolved in the issue tracker") ) // packageSentinels lists every sentinel above. redactor.sentinel keeps only // these in an error chain, so a new sentinel must be added here too. -var packageSentinels = []error{ErrUnsupportedKind, ErrProjectNotBound, ErrProjectMismatch, ErrStateMappingUnset, ErrInvalidLabelName, ErrRedirectRefused} +var packageSentinels = []error{ErrUnsupportedKind, ErrProjectNotBound, ErrProjectMismatch, ErrStateMappingUnset, ErrColumnNotMapped, ErrStateNotAllowed, ErrInvalidLabelName, ErrRedirectRefused, ErrIssueResolved} diff --git a/api/internal/issuetracker/forgefake_contract_test.go b/api/internal/issuetracker/forgefake_contract_test.go index dc2e0e987c..73c5d0e244 100644 --- a/api/internal/issuetracker/forgefake_contract_test.go +++ b/api/internal/issuetracker/forgefake_contract_test.go @@ -10,6 +10,7 @@ import ( "testing" "time" + "github.com/vtmocanu/uzi/api/internal/board" "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/issuetracker/issuetrackertest" ) @@ -48,8 +49,35 @@ func TestIssueTrackerForgeFakeContracts(t *testing.T) { if err != nil { t.Fatalf("forge.New(%s): %v", dialect, err) } + columns := []string{"contract-column-a", "contract-column-b"} + columnSet := map[string]struct{}{columns[0]: {}, columns[1]: {}} return issuetrackertest.Subject{ - IssueTracker: driver, + IssueTracker: driver, + Columns: columns, + PlanMove: func(t *testing.T, iid int64, column string) forge.ColumnMove { + issue, err := driver.GetIssue(t.Context(), forgeFakeProjectID, iid) + if err != nil { + t.Fatalf("GetIssue(%d): %v", iid, err) + } + add, remove, _ := board.PlanLabelMove(issue.Labels, columnSet, column) + move := forge.ColumnMove{Column: column, Add: add, Remove: remove} + for _, name := range add { + move.Ensure = append(move.Ensure, forge.Label{Name: name, Color: "#428bca"}) + } + return move + }, + ColumnOf: func(ctx context.Context, iid int64) (string, error) { + issue, err := driver.GetIssue(ctx, forgeFakeProjectID, iid) + if err != nil { + return "", err + } + for _, l := range issue.Labels { + if _, ok := columnSet[l]; ok { + return l, nil + } + } + return "", nil + }, ProjectID: forgeFakeProjectID, OtherProjectID: forgeFakeOtherProjectID, SeedHumanComment: func(t *testing.T, iid int64, username, body string) { diff --git a/api/internal/issuetracker/issuetrackertest/basefake.go b/api/internal/issuetracker/issuetrackertest/basefake.go index 90edee40f8..21a33316b9 100644 --- a/api/internal/issuetracker/issuetrackertest/basefake.go +++ b/api/internal/issuetracker/issuetrackertest/basefake.go @@ -55,6 +55,11 @@ func (*BaseFake) UpdateIssueLabels(context.Context, int64, int64, []string, []st return notStubbed("UpdateIssueLabels") } +// MoveToColumn implements issuetracker.Driver. +func (*BaseFake) MoveToColumn(context.Context, int64, int64, forge.ColumnMove) error { + return notStubbed("MoveToColumn") +} + // UpdateIssueDescription implements issuetracker.Driver. func (*BaseFake) UpdateIssueDescription(context.Context, int64, int64, string) error { return notStubbed("UpdateIssueDescription") diff --git a/api/internal/issuetracker/issuetrackertest/basefake_test.go b/api/internal/issuetracker/issuetrackertest/basefake_test.go index 80867571f4..4d1d1bacfc 100644 --- a/api/internal/issuetracker/issuetrackertest/basefake_test.go +++ b/api/internal/issuetracker/issuetrackertest/basefake_test.go @@ -26,6 +26,7 @@ func TestBaseFakeMethodsNotStubbed(t *testing.T) { {"GetIssue", func(b *BaseFake) error { _, err := b.GetIssue(ctx, 1, 2); return err }}, {"CreateIssue", func(b *BaseFake) error { _, err := b.CreateIssue(ctx, 1, "t", "d", nil); return err }}, {"UpdateIssueLabels", func(b *BaseFake) error { return b.UpdateIssueLabels(ctx, 1, 2, nil, nil) }}, + {"MoveToColumn", func(b *BaseFake) error { return b.MoveToColumn(ctx, 1, 2, forge.ColumnMove{}) }}, {"UpdateIssueDescription", func(b *BaseFake) error { return b.UpdateIssueDescription(ctx, 1, 2, "d") }}, {"SetIssueState", func(b *BaseFake) error { return b.SetIssueState(ctx, 1, 2, forge.StateClosed) }}, {"ListIssueLabelEvents", func(b *BaseFake) error { _, err := b.ListIssueLabelEvents(ctx, 1, 2); return err }}, @@ -36,8 +37,8 @@ func TestBaseFakeMethodsNotStubbed(t *testing.T) { {"ListTags", func(b *BaseFake) error { _, err := b.ListTags(ctx); return err }}, {"ListStates", func(b *BaseFake) error { _, err := b.ListStates(ctx, "p"); return err }}, } - if len(methods) != 15 { - t.Fatalf("expected 15 methods, got %d", len(methods)) + if len(methods) != 16 { + t.Fatalf("expected 16 methods, got %d", len(methods)) } b := &BaseFake{} for _, m := range methods { diff --git a/api/internal/issuetracker/issuetrackertest/contract.go b/api/internal/issuetracker/issuetrackertest/contract.go index 51e838f7c2..b512d27178 100644 --- a/api/internal/issuetracker/issuetrackertest/contract.go +++ b/api/internal/issuetracker/issuetrackertest/contract.go @@ -38,6 +38,20 @@ type Subject struct { // NoteIDUnset says CreateIssueNote returns ID 0, because the issue tracker's // comment ids have no numeric form. When false the case requires a non-zero ID. NoteIDUnset bool + // Columns are board column labels the subject can move an issue between; + // MoveToColumn cases need at least two. Nil means the subject has no board + // columns, so MoveToColumnMovesTheIssue is skipped and the guarded-write + // cases use a made-up column. + Columns []string + // PlanMove builds the ColumnMove that moves issue iid to column ("" is the + // implicit Open column) from the issue's current state, the way a caller of + // MoveToColumn would. For a label-backed subject it carries the Ensure, Add + // and Remove of a label delta against the issue's current labels; for an + // issue tracker with its own workflow field the column alone is enough. + PlanMove func(t *testing.T, iid int64, column string) forge.ColumnMove + // ColumnOf reports which of Columns issue iid is in now, "" when it is in + // none (the Open column). It reads server side, not through MoveToColumn. + ColumnOf func(ctx context.Context, iid int64) (string, error) // SameName carries the seams of the SameNameTagUsesEligibilityTag case. Nil // means labels on this implementation have no owners, so the case is skipped. SameName *SameNameSeams @@ -102,6 +116,7 @@ func RunContract(t *testing.T, newSubject func(t *testing.T) Subject) { {"CreateIssueRoundTrips", caseCreateIssue}, {"ListIssuesFilters", caseListIssues}, {"UpdateLabelsAndDescription", caseUpdate}, + {"MoveToColumnMovesTheIssue", caseMoveToColumn}, {"SetIssueStateClosesAndReopens", caseSetState}, {"CommentsOldestFirst", caseComments}, {"LabelEventsOldestFirst", caseLabelEvents}, @@ -280,6 +295,51 @@ func caseUpdate(t *testing.T, c *contract) { } } +func caseMoveToColumn(t *testing.T, c *contract) { + if c.ColumnOf == nil || c.PlanMove == nil || len(c.Columns) < 2 { + t.Skip("this implementation has no board columns to move between") + } + issue := c.create(t, "will be moved between columns") + a, b := c.Columns[0], c.Columns[1] + for _, column := range []string{a, b, ""} { + move := c.PlanMove(t, issue.IID, column) + if err := c.IssueTracker.MoveToColumn(c.ctx, c.ProjectID, issue.IID, move); err != nil { + t.Fatalf("MoveToColumn(%q): %v", column, err) + } + got, err := c.ColumnOf(c.ctx, issue.IID) + if err != nil { + t.Fatalf("ColumnOf after moving to %q: %v", column, err) + } + if got != column { + t.Fatalf("column after MoveToColumn(%q) = %q", column, got) + } + } +} + +// guardedMove is a ColumnMove for the cases that expect the write to be +// refused: a column the subject knows when it has any, so the refusal comes +// from the guard under test and not from an unmapped column. +func (c *contract) guardedMove() forge.ColumnMove { + column := "contract-guarded-column" + if len(c.Columns) > 0 { + column = c.Columns[0] + } + return forge.ColumnMove{Column: column, Ensure: []forge.Label{{Name: column}}, Add: []string{column}} +} + +// columnOf is ColumnOf for issue iid, or "" when the subject has no ColumnOf. +func (c *contract) columnOf(t *testing.T, iid int64) string { + t.Helper() + if c.ColumnOf == nil { + return "" + } + column, err := c.ColumnOf(c.ctx, iid) + if err != nil { + t.Fatalf("ColumnOf: %v", err) + } + return column +} + func caseSetState(t *testing.T, c *contract) { label := unique("contract-label") c.ensure(t, label) @@ -413,6 +473,7 @@ func caseMoved(t *testing.T, c *contract) { }, "UpdateIssueDescription": func() error { return it.UpdateIssueDescription(c.ctx, pid, iid, "x") }, "SetIssueState": func() error { return it.SetIssueState(c.ctx, pid, iid, forge.StateClosed) }, + "MoveToColumn": func() error { return it.MoveToColumn(c.ctx, pid, iid, c.guardedMove()) }, "ListIssueLabelEvents": func() error { _, err := it.ListIssueLabelEvents(c.ctx, pid, iid); return err }, "ListIssueComments": func() error { _, err := it.ListIssueComments(c.ctx, pid, iid); return err }, "CreateIssueNote": func() error { _, err := it.CreateIssueNote(c.ctx, pid, iid, "x"); return err }, @@ -435,6 +496,7 @@ func caseProjectGuard(t *testing.T, c *contract) { c.ensure(t, label) issue := c.create(t, "guarded", label) it, pid, iid := c.IssueTracker, c.OtherProjectID, issue.IID + columnBefore := c.columnOf(t, iid) calls := map[string]func() error{ "ListLabels": func() error { _, err := it.ListLabels(c.ctx, pid); return err }, "EnsureLabels": func() error { return it.EnsureLabels(c.ctx, pid, []forge.Label{{Name: unique("contract-label")}}) }, @@ -447,12 +509,13 @@ func caseProjectGuard(t *testing.T, c *contract) { "UpdateIssueLabels": func() error { return it.UpdateIssueLabels(c.ctx, pid, iid, nil, []string{label}) }, "UpdateIssueDescription": func() error { return it.UpdateIssueDescription(c.ctx, pid, iid, "changed") }, "SetIssueState": func() error { return it.SetIssueState(c.ctx, pid, iid, forge.StateClosed) }, + "MoveToColumn": func() error { return it.MoveToColumn(c.ctx, pid, iid, c.guardedMove()) }, "ListIssueLabelEvents": func() error { _, err := it.ListIssueLabelEvents(c.ctx, pid, iid); return err }, "ListIssueComments": func() error { _, err := it.ListIssueComments(c.ctx, pid, iid); return err }, "CreateIssueNote": func() error { _, err := it.CreateIssueNote(c.ctx, pid, iid, "x"); return err }, } - if len(calls) != 11 { - t.Fatalf("table covers %d methods, want 11", len(calls)) + if len(calls) != 12 { + t.Fatalf("table covers %d methods, want 12", len(calls)) } for name, call := range calls { err := call() @@ -466,6 +529,9 @@ func caseProjectGuard(t *testing.T, c *contract) { if got.Description != "guarded" || got.State != string(forge.StateOpened) || !sameSet(got.Labels, []string{label}) { t.Errorf("a guarded call changed the issue: %+v", got) } + if columnAfter := c.columnOf(t, iid); columnAfter != columnBefore { + t.Errorf("a refused MoveToColumn changed the column from %q to %q", columnBefore, columnAfter) + } comments, err := it.ListIssueComments(c.ctx, c.ProjectID, iid) if err != nil { t.Fatalf("ListIssueComments: %v", err) diff --git a/api/internal/issuetracker/options.go b/api/internal/issuetracker/options.go index 75013c9010..a9d4cd9bf4 100644 --- a/api/internal/issuetracker/options.go +++ b/api/internal/issuetracker/options.go @@ -24,6 +24,16 @@ type Binding struct { EligibilityTag TagRef } +// StateMapping is the connection's state configuration as a whole: the State +// field, the open and closed values, and the State each board column maps to. +// Options carries the same four as separate fields. +type StateMapping struct { + Field string + Open string + Closed string + Columns map[string]string +} + // Options configures a driver. There is deliberately no TLS field: a driver // trusts the system roots and nothing else (PRD #4173 S3). type Options struct { @@ -44,6 +54,9 @@ type Options struct { StateField string OpenState string ClosedState string + // ColumnStates maps a board column label to the State name MoveToColumn + // writes for it. A column absent from it is refused with ErrColumnNotMapped. + ColumnStates map[string]string // Timeout bounds one HTTP request; zero selects defaultTimeout. Timeout time.Duration } diff --git a/api/internal/issuetracker/state_rules.go b/api/internal/issuetracker/state_rules.go new file mode 100644 index 0000000000..1e638d1067 --- /dev/null +++ b/api/internal/issuetracker/state_rules.go @@ -0,0 +1,88 @@ +package issuetracker + +import ( + "fmt" + "strings" + "unicode" +) + +// closableNames are the normalised state names that mean the work is done. It +// is an allowlist on purpose: isResolved is true for "Won't fix" and +// "Duplicate" too, and a name this list does not know is refused rather than +// guessed at, so a write never closes an issue as something it was not. +var closableNames = map[string]bool{ + "fixed": true, "done": true, "resolved": true, "verified": true, + "completed": true, "complete": true, "closed": true, "implemented": true, + "shipped": true, "released": true, "delivered": true, "merged": true, + "finished": true, +} + +// ClosableState reports whether s may be written as the closed state: it is +// resolved, not archived, its name passes stateNameProblem, and its name is one +// of closableNames once normalised. Only Name decides. LocalizedName is display +// text that the same bundle administrator sets freely, so consulting it would +// let a won't-fix state relabelled "Fixed" through; a done state named only in +// another language is therefore refused. +func ClosableState(s State) bool { + if !s.Resolved || s.Archived || stateNameProblem(s.Name) { + return false + } + return closableNames[normaliseStateName(s.Name)] +} + +// OpenableState reports whether s may be written as the open state or as a +// board column's state: it is unresolved, not archived, and its name passes +// stateNameProblem. +func OpenableState(s State) bool { + return !s.Resolved && !s.Archived && !stateNameProblem(s.Name) +} + +// normaliseStateName lower-cases name, trims it and collapses each run of +// whitespace, '-' and '_' to one space. +func normaliseStateName(name string) string { + fields := strings.FieldsFunc(strings.ToLower(name), func(r rune) bool { + return unicode.IsSpace(r) || r == '-' || r == '_' + }) + return strings.Join(fields, " ") +} + +// StateNameProblem is stateNameProblem for callers outside the package: a caller that +// stores a State name the driver listed checks it with the same rule. +func StateNameProblem(name string) bool { return stateNameProblem(name) } + +// stateNameProblem reports whether a State name read from the issue tracker is +// one the driver will not pass on: blank, over maxLabelNameLen bytes, or +// carrying an invisible or control character, an ignorable code point, a +// variation selector or a blank filler (Hangul and Braille fillers). Any user +// who administers a bundle can name a state, and the name ends up in errors and +// in the board. +func stateNameProblem(name string) bool { + if strings.TrimSpace(name) == "" || len(name) > maxLabelNameLen { + return true + } + for _, c := range name { + if InvisibleRune(c) || stateNameFiller(c) { + return true + } + } + return false +} + +// stateNameFiller reports whether c is an Other_Default_Ignorable_Code_Point, a +// variation selector, or one of the fillers that render blank: U+115F, U+1160, +// U+2800, U+3164 and U+FFA0. +// InvisibleRune is shared with other callers and is left as it is. +func stateNameFiller(c rune) bool { + switch c { + case 0x115F, 0x1160, 0x2800, 0x3164, 0xFFA0: + return true + } + return unicode.Is(unicode.Other_Default_Ignorable_Code_Point, c) || unicode.Is(unicode.Variation_Selector, c) +} + +// maxQuotedNameLen bounds the name quoteName puts in an error. +const maxQuotedNameLen = 80 + +// quoteName renders a configured state name for an error: cut to +// maxQuotedNameLen bytes and quoted with %q, so no control character survives. +func quoteName(name string) string { return fmt.Sprintf("%q", truncate(name, maxQuotedNameLen)) } diff --git a/api/internal/issuetracker/state_rules_test.go b/api/internal/issuetracker/state_rules_test.go new file mode 100644 index 0000000000..94d3fa17b3 --- /dev/null +++ b/api/internal/issuetracker/state_rules_test.go @@ -0,0 +1,63 @@ +package issuetracker + +import "testing" + +func TestClosableState(t *testing.T) { + for _, tc := range []struct { + name string + s State + want bool + }{ + {"Fixed", State{Name: "Fixed", Resolved: true}, true}, + {"Done", State{Name: "Done", Resolved: true}, true}, + // A tab is a control character stateNameProblem refuses, so use spaces. + {"normalised", State{Name: " FIXED ", Resolved: true}, true}, + {"separators collapse", State{Name: "Won_t--fix", Resolved: true}, false}, + {"Won't fix", State{Name: "Won't fix", Resolved: true}, false}, + {"Duplicate", State{Name: "Duplicate", Resolved: true}, false}, + {"Obsolete", State{Name: "Obsolete", Resolved: true}, false}, + {"Can't Reproduce", State{Name: "Can't Reproduce", Resolved: true}, false}, + {"archived Fixed", State{Name: "Fixed", Resolved: true, Archived: true}, false}, + {"unresolved Done", State{Name: "Done", Resolved: false}, false}, + // Closability is decided on Name alone: the localized name is free text + // set by the same bundle admin, so a non-ASCII name labelled "Fixed" is + // refused (this row expected true before), as is a done state named + // only in another language. + {"non-ASCII name, localized Fixed", State{Name: "Готово", LocalizedName: "Fixed", Resolved: true}, false}, + {"Won't fix localized as Fixed", State{Name: "Won't fix", LocalizedName: "Fixed", Resolved: true}, false}, + {"curly apostrophe, localized Fixed", State{Name: "Won\u2019t fix", LocalizedName: "Fixed", Resolved: true}, false}, + {"NBSP, localized Fixed", State{Name: "Won't\u00a0fix", LocalizedName: "Fixed", Resolved: true}, false}, + {"Russian won't fix, localized Fixed", State{Name: "Не будет исправлено", LocalizedName: "Fixed", Resolved: true}, false}, + {"Cyrillic o, localized Fixed", State{Name: "W\u043en't fix", LocalizedName: "Fixed", Resolved: true}, false}, + {"ASCII name, localized Fixed", State{Name: "Erledigt", LocalizedName: "Fixed", Resolved: true}, false}, + {"Erledigt", State{Name: "Erledigt", Resolved: true}, false}, + {"upper case FIXED", State{Name: "FIXED", Resolved: true}, true}, + {"Fixed with a trailing NBSP", State{Name: "fixed\u00a0", Resolved: true}, true}, + {"Fixed with a filler", State{Name: "Fixed\u3164", Resolved: true}, false}, + {"empty", State{Resolved: true}, false}, + } { + t.Run(tc.name, func(t *testing.T) { + if got := ClosableState(tc.s); got != tc.want { + t.Errorf("ClosableState(%+v) = %v, want %v", tc.s, got, tc.want) + } + }) + } +} + +func TestOpenableState(t *testing.T) { + for name, tc := range map[string]struct { + s State + want bool + }{ + "unresolved": {State{Name: "In Progress"}, true}, + "resolved": {State{Name: "Fixed", Resolved: true}, false}, + "archived unresolved": {State{Name: "Old", Archived: true}, false}, + "filler in the name": {State{Name: "Open\u3164"}, false}, + "variation selector": {State{Name: "Open\uFE0F"}, false}, + "ideographic variation selector": {State{Name: "Open\U000E0100"}, false}, + } { + if got := OpenableState(tc.s); got != tc.want { + t.Errorf("%s: OpenableState = %v, want %v", name, got, tc.want) + } + } +} diff --git a/api/internal/issuetracker/youtrack.go b/api/internal/issuetracker/youtrack.go index 610dde2720..de2be934a4 100644 --- a/api/internal/issuetracker/youtrack.go +++ b/api/internal/issuetracker/youtrack.go @@ -2,6 +2,7 @@ package issuetracker import ( "fmt" + "maps" "net/http" "net/url" "regexp" @@ -35,8 +36,10 @@ type youTrack struct { stateField string openState string closedState string - client *http.Client - red redactor + // columnStates maps a board column label to the State name MoveToColumn writes. + columnStates map[string]string + client *http.Client + red redactor // accountMu guards account, the token's own account. It is read from // users/me the first time tag ownership matters, then kept (see connAccount). @@ -69,16 +72,17 @@ func buildYouTrack(opts Options, base http.RoundTripper) (*youTrack, error) { stateField = defaultStateField } return &youTrack{ - baseURL: baseURL, - token: opts.Token, - project: opts.Project, - projectID: opts.ProjectID, - eligibility: opts.EligibilityTag, - stateField: stateField, - openState: opts.OpenState, - closedState: opts.ClosedState, - client: newHTTPClient(opts.Timeout, base), - red: newRedactor(opts.Token), + baseURL: baseURL, + token: opts.Token, + project: opts.Project, + projectID: opts.ProjectID, + eligibility: opts.EligibilityTag, + stateField: stateField, + openState: opts.OpenState, + closedState: opts.ClosedState, + columnStates: maps.Clone(opts.ColumnStates), + client: newHTTPClient(opts.Timeout, base), + red: newRedactor(opts.Token), }, nil } diff --git a/api/internal/issuetracker/youtrack_contract_test.go b/api/internal/issuetracker/youtrack_contract_test.go index 17b3fddd9c..93a4ebe738 100644 --- a/api/internal/issuetracker/youtrack_contract_test.go +++ b/api/internal/issuetracker/youtrack_contract_test.go @@ -1,10 +1,12 @@ package issuetracker_test import ( + "context" "reflect" "slices" "testing" + "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/issuetracker" "github.com/vtmocanu/uzi/api/internal/issuetracker/issuetrackertest" "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" @@ -19,13 +21,21 @@ const contractProjectID = 4173 func TestYouTrackContract(t *testing.T) { issuetrackertest.RunContract(t, func(t *testing.T) issuetrackertest.Subject { srv := youtracktest.NewServer(t) + const openState = "Open" + // Two columns mapped to unresolved State values the fake's bundle holds. + columns := []string{"contract-column-a", "contract-column-b"} + stateOf := map[string]string{columns[0]: "Contract State A", columns[1]: "Contract State B"} + for _, state := range stateOf { + srv.SeedState(state, false) + } y, err := issuetracker.NewYouTrackOver(issuetracker.Options{ - BaseURL: srv.URL, - Token: srv.Token, - Project: issuetracker.ProjectRef{ID: youtracktest.ProjectID, ShortName: youtracktest.ProjectShortName}, - ProjectID: contractProjectID, - OpenState: "Open", - ClosedState: "Fixed", + BaseURL: srv.URL, + Token: srv.Token, + Project: issuetracker.ProjectRef{ID: youtracktest.ProjectID, ShortName: youtracktest.ProjectShortName}, + ProjectID: contractProjectID, + OpenState: openState, + ClosedState: "Fixed", + ColumnStates: stateOf, EligibilityTag: issuetracker.TagRef{ ID: youtracktest.EligibilityTagID, Name: youtracktest.EligibilityTagName, }, @@ -36,6 +46,23 @@ func TestYouTrackContract(t *testing.T) { return issuetrackertest.Subject{ IssueTracker: y, ProjectID: contractProjectID, + Columns: columns, + PlanMove: func(_ *testing.T, _ int64, column string) forge.ColumnMove { + return forge.ColumnMove{Column: column} + }, + ColumnOf: func(_ context.Context, iid int64) (string, error) { + state := srv.IssueState(iid) + for column, name := range stateOf { + if name == state { + return column, nil + } + } + if state == openState { + return "", nil + } + // Any other unmapped State is a failure, not the Open column. + return "unmapped State " + state, nil + }, SeedHumanComment: func(t *testing.T, iid int64, username, body string) { srv.SeedComment(iid, username, body) }, diff --git a/api/internal/issuetracker/youtrack_decode.go b/api/internal/issuetracker/youtrack_decode.go index 0bd99c5ff9..9d1ffcae51 100644 --- a/api/internal/issuetracker/youtrack_decode.go +++ b/api/internal/issuetracker/youtrack_decode.go @@ -9,6 +9,10 @@ import ( // maxNestedItems caps the tags of one issue or one activity. const maxNestedItems = 1000 +// maxCustomFields caps the custom fields of one issue. A listing whose issue +// carries more fails; it is never read in part. +const maxCustomFields = 64 + // errNestedCeiling is returned when one object carries more than maxNestedItems // nested entries. var errNestedCeiling = errors.New("issuetracker: an item carries more nested entries than the ceiling allows") @@ -25,6 +29,11 @@ var errNestedCeiling = errors.New("issuetracker: an item carries more nested ent // allowed 16*maxNestedItems+256. A single object (an issue holds one tags // array, plus a few members of its own) is allowed 8*maxNestedItems+256, which // holds 8*maxNestedItems for the tags. +// An issue also carries up to maxCustomFields custom fields, each a "$type", +// a name and a value object with its own "$type" and name: 6 values (two +// braces, three commas inside, one between fields), allowed 10 for a value that +// is an array. So a single object is allowed 8*maxNestedItems + +// 10*maxCustomFields + 256; an element of a page already has the room. // The real-shaped tests pin that headroom. Three totals are enforced: // pageValueBudget for a whole list page, itemValueBudget for each element of // it and objectValueBudget for one single-object response. The budgets bound @@ -35,7 +44,7 @@ var errNestedCeiling = errors.New("issuetracker: an item carries more nested ent // one element, at most itemValueBudget values, to amplify. const ( itemValueBudget = 16*maxNestedItems + 256 - objectValueBudget = 8*maxNestedItems + 256 + objectValueBudget = 8*maxNestedItems + 10*maxCustomFields + 256 pageValueBudget = pageSize * itemValueBudget ) @@ -49,10 +58,18 @@ var errNotArray = errors.New("issuetracker: response is not a JSON array") // errTrailingData is returned when a list response has data after its array. var errTrailingData = errors.New("issuetracker: response has data after the JSON array") +// errCustomFieldCeiling is returned when one issue carries more than +// maxCustomFields custom fields. +var errCustomFieldCeiling = errors.New("issuetracker: an issue carries more custom fields than the ceiling allows") + // errTagNotObject is returned when a tags array holds an element that is not an // object, refused before decoding so a bad element costs no error value. var errTagNotObject = errors.New("issuetracker: a tag is not a JSON object") +// errCustomFieldNotObject is returned when a customFields array holds an +// element that is not an object. +var errCustomFieldNotObject = errors.New("issuetracker: a custom field is not a JSON object") + // ytTags is a nested tag array. Its UnmarshalJSON counts the elements with a // byte scan and refuses above maxNestedItems before anything is decoded, so one // object cannot make the decoder allocate a slice as large as the response. It @@ -351,3 +368,55 @@ func forEachElement(b []byte, f func(elem []byte) bool) { } } } + +// ytCustomField is one custom field of an issue, as asked for by issueFields. +type ytCustomField struct { + Name string `json:"name"` + Type string `json:"$type"` + Value ytCustomValue `json:"value"` +} + +// ytCustomValue is the name of a custom field's value. A field's value is an +// object for a state, enum or user field, and a scalar, an array or null for +// others; only an object's name is kept, and anything else leaves Name empty so +// a field the driver does not read cannot fail the listing. +type ytCustomValue struct{ Name string } + +// UnmarshalJSON implements json.Unmarshaler. b is already-validated JSON. +func (v *ytCustomValue) UnmarshalJSON(b []byte) error { + *v = ytCustomValue{} + if t := trimJSONSpace(b); len(t) == 0 || t[0] != '{' { + return nil + } + var o struct { + Name string `json:"name"` + } + if json.Unmarshal(b, &o) == nil { + v.Name = o.Name + } + return nil +} + +// ytCustomFields is a nested custom field array. Like ytTags, its UnmarshalJSON +// counts the elements with a byte scan and refuses above maxCustomFields before +// anything is decoded, and builds no json.Decoder, because encoding/json calls +// it once per repeated "customFields" key. +type ytCustomFields []ytCustomField + +// UnmarshalJSON implements json.Unmarshaler. b is already-validated JSON. +func (c *ytCustomFields) UnmarshalJSON(b []byte) error { + // See ytTags.UnmarshalJSON: reused elements are cleared so a repeated key + // leaves nothing of an earlier one. + clear((*c)[:cap(*c)]) + if trimmed := trimJSONSpace(b); string(trimmed) == "null" { + *c = (*c)[:0] + return nil + } + if n, _ := topLevelCount(b); n > maxCustomFields { + return errCustomFieldCeiling + } + if !onlyObjects(b) { + return errCustomFieldNotObject + } + return json.Unmarshal(b, (*[]ytCustomField)(c)) +} diff --git a/api/internal/issuetracker/youtrack_decode_test.go b/api/internal/issuetracker/youtrack_decode_test.go index 96895f62d0..370414402d 100644 --- a/api/internal/issuetracker/youtrack_decode_test.go +++ b/api/internal/issuetracker/youtrack_decode_test.go @@ -492,3 +492,103 @@ func TestRealShapedActivityIsAccepted(t *testing.T) { t.Fatalf("decoded %d activities, want 3 with %d tags each way", len(got), maxNestedItems) } } + +// TestCustomFieldBombAllocatesLittle: an issue whose "customFields" array holds +// as many empty objects as the object cap allows is refused by the byte scan +// before any element is decoded. +func TestCustomFieldBombAllocatesLittle(t *testing.T) { + prefix := strings.TrimSuffix(rawIssue("2-1", "UZI-1", 1, "0-1", ""), `}`) + `,"customFields":` + n := int(objectBytes-int64(len(prefix))-8) / 3 + body := []byte(prefix + emptyObjects(n) + "}") + if int64(len(body)) > objectBytes { + t.Fatalf("fixture is %d bytes, over the object cap", len(body)) + } + y, _ := loggedDriver(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write(body) }) + var err error + got := totalAlloc(func() { _, err = y.GetIssue(context.Background(), 1, 1) }) + if err == nil { + t.Fatal("GetIssue accepted a custom field bomb") + } + if got > allocBound { + t.Errorf("allocated %d MiB for a %d MiB issue, want under %d MiB", got>>20, len(body)>>20, allocBound>>20) + } +} + +// TestDuplicateCustomFieldsKeysAllocateLittle is TestDuplicateTagsKeysAllocateLittle +// for ytCustomFields: the fixtures sit just under the budgets, so every +// repeated "customFields" key reaches UnmarshalJSON. Measured on 2026-11-17, +// without and with -race: the page allocates about 25 and 78 MiB, the issue +// about 457 and 927 KiB, the same per-call cost as the tags fixtures (about 300 +// bytes a call, from encoding/json itself, with no Decoder built per call). +// Re-measured the same day, the tags fixtures after objectValueBudget grew by +// 10*maxCustomFields: the page about 21 and 84 MiB, the issue about 358 and 778 +// KiB, so their bounds stand. +func TestDuplicateCustomFieldsKeysAllocateLittle(t *testing.T) { + const bound = 112 << 20 + const issueBound = 1200 << 10 + issue := strings.TrimSuffix(rawIssue("2-1", "UZI-1", 1, "0-1", ""), `}`) + `,` + dup := `"customFields":[{}],` + t.Run("page", func(t *testing.T) { + // Each repetition is 3 values and 20 bytes. At the item budget a page of + // pageSize issues would pass the page cap, so the repetitions are the + // fewer of the two limits. + keys := min((itemValueBudget-64)/3, (int(listPageBytes)/pageSize-len(issue)-64)/len(dup)) + elem := issue + strings.Repeat(dup, keys) + `"customFields":[]}` + body := "[" + strings.TrimSuffix(strings.Repeat(elem+",", pageSize), ",") + "]" + if int64(len(body)) > listPageBytes { + t.Fatalf("fixture is %d bytes, over the page cap", len(body)) + } + y, _ := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("$skip") != "0" { + writeRaw(w, "[]") + return + } + _, _ = w.Write([]byte(body)) + }) + var err error + got := totalAlloc(func() { _, err = y.ListIssues(context.Background(), 1, forge.ListIssuesOptions{}) }) + if err != nil { + t.Errorf("ListIssues: %v", err) + } + if got > bound { + t.Errorf("allocated %d KiB for a %d KiB page, want under %d KiB", got>>10, len(body)>>10, bound>>10) + } + }) + t.Run("issue", func(t *testing.T) { + body := []byte(issue + strings.Repeat(dup, (objectValueBudget-64)/3) + `"customFields":[]}`) + if int64(len(body)) > objectBytes { + t.Fatalf("fixture is %d bytes, over the object cap", len(body)) + } + y, _ := loggedDriver(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write(body) }) + got := totalAlloc(func() { _, _ = y.GetIssue(context.Background(), 1, 1) }) + if got > issueBound { + t.Errorf("allocated %d KiB for a %d KiB issue, want under %d KiB", got>>10, len(body)>>10, issueBound>>10) + } + }) +} + +// TestRealShapedIssueWithCustomFieldsIsAccepted: an issue at both nested +// ceilings, 1000 tags and maxCustomFields custom fields, each shaped like +// YouTrack's own output ($type on every entity, a multi-value field among +// them), still sits under the object budget. +func TestRealShapedIssueWithCustomFieldsIsAccepted(t *testing.T) { + tag := `{"$type":"Tag","id":"6-1","name":"x","owner":{"$type":"User","id":"` + rawMeID + `"},"color":null}` + state := `{"$type":"StateIssueCustomField","name":"State","value":{"$type":"StateBundleElement","name":"Open"}}` + multi := `{"$type":"MultiEnumIssueCustomField","name":"m","value":[{"$type":"EnumBundleElement","name":"a"},{"$type":"EnumBundleElement","name":"b"}]}` + fields := []string{state} + for len(fields) < maxCustomFields { + fields = append(fields, multi) + } + issue := `{"$type":"Issue","id":"2-1","idReadable":"UZI-1","numberInProject":1,"summary":"s","created":1,"updated":2000,` + + `"project":{"$type":"Project","id":"0-1","shortName":"UZI"},` + + `"customFields":[` + strings.Join(fields, ",") + `],` + + `"tags":[` + strings.TrimSuffix(strings.Repeat(tag+",", maxNestedItems), ",") + `]}` + if n := valueCount([]byte(issue)); n > objectValueBudget || n <= 8*maxNestedItems+256 { + t.Fatalf("fixture holds %d values, want more than %d without exceeding the %d budget", n, 8*maxNestedItems+256, objectValueBudget) + } + y, _ := loggedDriver(t, withRawMe(func(w http.ResponseWriter, _ *http.Request) { writeRaw(w, issue) })) + got, err := y.GetIssue(context.Background(), 1, 1) + if err != nil || len(got.Labels) != maxNestedItems { + t.Fatalf("%d labels, %v; want %d labels", len(got.Labels), err, maxNestedItems) + } +} diff --git a/api/internal/issuetracker/youtrack_issue_resolve.go b/api/internal/issuetracker/youtrack_issue_resolve.go index 1e15c712f1..45df4ca669 100644 --- a/api/internal/issuetracker/youtrack_issue_resolve.go +++ b/api/internal/issuetracker/youtrack_issue_resolve.go @@ -67,8 +67,11 @@ func (y *youTrack) issueByEntity(ctx context.Context, id string) (ytIssue, error if err != nil { return ytIssue{}, err } - if !y.inBoundProject(w) || w.ID != id { - return ytIssue{}, y.fail("youtrack: created issue is not in the bound project") + if !y.inBoundProject(w) { + return ytIssue{}, &forge.IssueNotFoundError{Err: y.fail("youtrack: created issue is not in the bound project")} + } + if w.ID != id { + return ytIssue{}, y.fail("youtrack: created issue read back under a different id") } return w, nil } diff --git a/api/internal/issuetracker/youtrack_issue_resolve_test.go b/api/internal/issuetracker/youtrack_issue_resolve_test.go index 0a651534d0..25a32f8912 100644 --- a/api/internal/issuetracker/youtrack_issue_resolve_test.go +++ b/api/internal/issuetracker/youtrack_issue_resolve_test.go @@ -70,9 +70,15 @@ func TestCreateIssueReadBackMustBeTheCreatedIssueInTheBoundProject(t *testing.T) } writeRaw(w, readBack) }) - if _, err := y.CreateIssue(context.Background(), 1, "t", "d", nil); err == nil { + _, err := y.CreateIssue(context.Background(), 1, "t", "d", nil) + if err == nil { t.Fatal("CreateIssue returned an issue that failed the read-back check") } + // Only an issue outside the bound project reads as gone; another id + // is a protocol anomaly of an issue that exists. + if got := isNotFound(err); got != (name == "other project") { + t.Errorf("isNotFound = %v for %v", got, err) + } }) } } diff --git a/api/internal/issuetracker/youtrack_issues.go b/api/internal/issuetracker/youtrack_issues.go index 8e48d881b2..14a7ac4b7e 100644 --- a/api/internal/issuetracker/youtrack_issues.go +++ b/api/internal/issuetracker/youtrack_issues.go @@ -8,6 +8,7 @@ import ( "slices" "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuesource" ) // ListIssues implements forge.IssueTracker. Without labels it queries the @@ -22,6 +23,23 @@ import ( // ceilings before the project filter, so a tag with more issues elsewhere than // a ceiling allows fails the listing rather than returning part of it. func (y *youTrack) ListIssues(ctx context.Context, projectID int64, opts forge.ListIssuesOptions) ([]forge.Issue, error) { + stated, err := y.ListIssuesWithState(ctx, projectID, opts) + if err != nil { + return nil, err + } + out := make([]forge.Issue, 0, len(stated)) + for _, s := range stated { + out = append(out, s.Issue) + } + return out, nil +} + +// ListIssuesWithState lists the issues ListIssues does, from the same single +// walk, each with the name of its State field value ("" when it has none, or +// when the name is one stateNameProblem refuses). It implements +// issuesource.IssueStateLister. An issue with more than maxCustomFields custom +// fields fails the listing. +func (y *youTrack) ListIssuesWithState(ctx context.Context, projectID int64, opts forge.ListIssuesOptions) ([]issuesource.StatedIssue, error) { p, err := y.requireProject(projectID) if err != nil { return nil, err @@ -35,13 +53,13 @@ func (y *youTrack) ListIssues(ctx context.Context, projectID int64, opts forge.L } for _, l := range opts.Labels { if _, ok := tagIDs[l]; !ok { - return []forge.Issue{}, nil + return []issuesource.StatedIssue{}, nil } } byTag := len(opts.Labels) > 0 - path, query := []string{"issues"}, url.Values{"query": {issueQuery(p.ShortName, opts)}, "fields": {issueFields}} + path, query := []string{"issues"}, url.Values{"query": {issueQuery(p.ShortName, opts)}, "fields": {listingIssueFields}} if byTag { - path, query = []string{"tags", y.listingTagID(opts.Labels, tagIDs), "issues"}, url.Values{"fields": {issueFields}} + path, query = []string{"tags", y.listingTagID(opts.Labels, tagIDs), "issues"}, url.Values{"fields": {listingIssueFields}} } var matched []ytIssue seen := map[string]bool{} @@ -69,13 +87,13 @@ func (y *youTrack) ListIssues(ctx context.Context, projectID int64, opts forge.L if opts.Limit > 0 && len(matched) > opts.Limit { matched = matched[:opts.Limit] } - out := make([]forge.Issue, 0, len(matched)) + out := make([]issuesource.StatedIssue, 0, len(matched)) for _, w := range matched { issue, err := y.toIssue(ctx, w) if err != nil { return nil, err } - out = append(out, issue) + out = append(out, issuesource.StatedIssue{Issue: issue, State: y.stateName(w)}) } return out, nil } @@ -99,6 +117,8 @@ func (y *youTrack) GetIssue(ctx context.Context, projectID, issueIID int64) (for return y.toIssue(ctx, w) } +var _ issuesource.IssueStateLister = (*youTrack)(nil) + // CreateIssue implements forge.IssueTracker. The labels resolve to tag ids first // (see resolveTagIDs; created when missing), so a failure there leaves no issue // behind. Then the issue is created, each tag is attached, and the result is diff --git a/api/internal/issuetracker/youtrack_move_test.go b/api/internal/issuetracker/youtrack_move_test.go new file mode 100644 index 0000000000..4ab5448986 --- /dev/null +++ b/api/internal/issuetracker/youtrack_move_test.go @@ -0,0 +1,560 @@ +package issuetracker + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "net/http" + "slices" + "strings" + "testing" + + "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" +) + +// columnDriver is newFakeDriver with the board columns mapped to State values +// the fake's bundle has once "In Progress" is seeded. +func columnDriver(t *testing.T, mutate func(*Options)) (*youTrack, *youtracktest.Server) { + t.Helper() + srv := youtracktest.NewServer(t) + srv.SeedState("In Progress", false) + opts := fakeOptions(srv) + opts.ColumnStates = map[string]string{"In Progress": "In Progress", "Done": "Fixed", "Lost": "Won't fix"} + if mutate != nil { + mutate(&opts) + } + y, err := newYouTrackOver(opts, srv.Client().Transport) + if err != nil { + t.Fatal(err) + } + return y, srv +} + +// writesOf returns the fake's logged requests that are not GETs. +func writesOf(srv *youtracktest.Server) []string { + var out []string + for _, r := range srv.Requests() { + if r.Method != http.MethodGet { + out = append(out, r.Method+" "+r.Path) + } + } + return out +} + +func TestSetIssueStateRefusesAClosedStateThatIsNotDone(t *testing.T) { + y, srv := columnDriver(t, func(o *Options) { o.ClosedState = "Won't fix" }) + n := srv.SeedIssue("t", "") + srv.ResetRequests() + err := y.SetIssueState(context.Background(), testProjectID, n, forge.StateClosed) + if !errors.Is(err, ErrStateNotAllowed) { + t.Fatalf("SetIssueState = %v, want ErrStateNotAllowed", err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent: %v", w) + } + if got := srv.IssueState(n); got != "Open" { + t.Errorf("issue state = %q, want it untouched", got) + } +} + +func TestSetIssueStateRefusesABundleEditedAfterMapping(t *testing.T) { + for name, edit := range map[string]func(*youtracktest.Server){ + "Fixed renamed to Won't fix": func(s *youtracktest.Server) { s.RenameState("Fixed", "Won't fix") }, + "Fixed no longer resolved": func(s *youtracktest.Server) { s.SetStateResolved("Fixed", false) }, + "Fixed archived": func(s *youtracktest.Server) { s.SetStateArchived("Fixed", true) }, + } { + t.Run(name, func(t *testing.T) { + y, srv := newFakeDriver(t) + n := srv.SeedIssue("t", "") + ctx := context.Background() + if err := y.SetIssueState(ctx, testProjectID, n, forge.StateClosed); err != nil { + t.Fatalf("before the edit: %v", err) + } + if err := y.SetIssueState(ctx, testProjectID, n, forge.StateOpened); err != nil { + t.Fatalf("reopen: %v", err) + } + edit(srv) + srv.ResetRequests() + err := y.SetIssueState(ctx, testProjectID, n, forge.StateClosed) + if !errors.Is(err, ErrStateNotAllowed) { + t.Fatalf("SetIssueState = %v, want ErrStateNotAllowed", err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent: %v", w) + } + }) + } +} + +func TestSetIssueStateRefusesAResolvedOpenState(t *testing.T) { + y, srv := columnDriver(t, func(o *Options) { o.OpenState = "Fixed" }) + n := srv.SeedIssue("t", "") + srv.ResetRequests() + if err := y.SetIssueState(context.Background(), testProjectID, n, forge.StateOpened); !errors.Is(err, ErrStateNotAllowed) { + t.Fatalf("SetIssueState(opened) = %v, want ErrStateNotAllowed", err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent: %v", w) + } +} + +func TestMoveToColumnUnmappedSendsNothing(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("t", "") + srv.ResetRequests() + err := y.MoveToColumn(context.Background(), testProjectID, n, forge.ColumnMove{Column: "Review"}) + if !errors.Is(err, ErrColumnNotMapped) { + t.Fatalf("MoveToColumn = %v, want ErrColumnNotMapped", err) + } + if got := srv.Requests(); len(got) != 0 { + t.Errorf("requests sent: %v", got) + } +} + +func TestMoveToColumnWritesTheMappedState(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("t", "") + if err := y.MoveToColumn(context.Background(), testProjectID, n, forge.ColumnMove{Column: "In Progress"}); err != nil { + t.Fatalf("MoveToColumn: %v", err) + } + if got := srv.IssueState(n); got != "In Progress" { + t.Errorf("issue state = %q, want In Progress", got) + } +} + +func TestMoveToColumnEmptyUsesTheOpenState(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("t", "") + ctx := context.Background() + if err := y.MoveToColumn(ctx, testProjectID, n, forge.ColumnMove{Column: "In Progress"}); err != nil { + t.Fatal(err) + } + if err := y.MoveToColumn(ctx, testProjectID, n, forge.ColumnMove{Column: ""}); err != nil { + t.Fatalf("MoveToColumn(\"\"): %v", err) + } + if got := srv.IssueState(n); got != "Open" { + t.Errorf("issue state = %q, want the open state", got) + } + y2, srv2 := columnDriver(t, func(o *Options) { o.OpenState = "" }) + n2 := srv2.SeedIssue("t", "") + srv2.ResetRequests() + if err := y2.MoveToColumn(ctx, testProjectID, n2, forge.ColumnMove{Column: ""}); !errors.Is(err, ErrStateMappingUnset) { + t.Errorf("with no open state: %v, want ErrStateMappingUnset", err) + } + if got := srv2.Requests(); len(got) != 0 { + t.Errorf("requests sent: %v", got) + } +} + +func TestMoveToColumnRefusesAResolvedColumnState(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("t", "") + for _, col := range []string{"Done", "Lost"} { + srv.ResetRequests() + if err := y.MoveToColumn(context.Background(), testProjectID, n, forge.ColumnMove{Column: col}); !errors.Is(err, ErrStateNotAllowed) { + t.Errorf("MoveToColumn(%q) = %v, want ErrStateNotAllowed", col, err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("%q: writes sent: %v", col, w) + } + } +} + +func TestMoveToColumnRefusesABundleEditedAfterMapping(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("t", "") + srv.SetStateResolved("In Progress", true) + srv.ResetRequests() + if err := y.MoveToColumn(context.Background(), testProjectID, n, forge.ColumnMove{Column: "In Progress"}); !errors.Is(err, ErrStateNotAllowed) { + t.Fatalf("MoveToColumn = %v, want ErrStateNotAllowed", err) + } + srv.SetStateResolved("In Progress", false) + srv.RenameState("In Progress", "Doing") + if err := y.MoveToColumn(context.Background(), testProjectID, n, forge.ColumnMove{Column: "In Progress"}); !errors.Is(err, ErrStateNotAllowed) { + t.Fatalf("after a rename: %v, want ErrStateNotAllowed", err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent: %v", w) + } +} + +func TestMoveToColumnOnAMovedIssueIsNotFound(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("t", "") + srv.MoveIssue(n, youtracktest.OtherProjectID) + srv.ResetRequests() + err := y.MoveToColumn(context.Background(), testProjectID, n, forge.ColumnMove{Column: "In Progress"}) + var nf *forge.IssueNotFoundError + if !errors.As(err, &nf) { + t.Fatalf("MoveToColumn = %v, want *forge.IssueNotFoundError", err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent: %v", w) + } +} + +func TestMoveToColumnProjectMismatchSendsNothing(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("t", "") + srv.ResetRequests() + if err := y.MoveToColumn(context.Background(), testProjectID+1, n, forge.ColumnMove{Column: "In Progress"}); !errors.Is(err, ErrProjectMismatch) { + t.Fatalf("MoveToColumn = %v, want ErrProjectMismatch", err) + } + if got := srv.Requests(); len(got) != 0 { + t.Errorf("requests sent: %v", got) + } +} + +func TestNewSentinelsRedactTheToken(t *testing.T) { + // Both configured names carry the fake's token, the one secret a bad config + // could plausibly paste into a state name. + y, srv := columnDriver(t, nil) + y.closedState = srv.Token + y.columnStates = map[string]string{"c": srv.Token} + n := srv.SeedIssue("t", "") + ctx := context.Background() + for name, tc := range map[string]struct { + err error + want error + }{ + "closed": {y.SetIssueState(ctx, testProjectID, n, forge.StateClosed), ErrStateNotAllowed}, + "column": {y.MoveToColumn(ctx, testProjectID, n, forge.ColumnMove{Column: "c"}), ErrStateNotAllowed}, + "unmapped": {y.MoveToColumn(ctx, testProjectID, n, forge.ColumnMove{Column: "nope"}), ErrColumnNotMapped}, + } { + if !errors.Is(tc.err, tc.want) { + t.Errorf("%s: %v, want %v", name, tc.err, tc.want) + continue + } + if strings.Contains(tc.err.Error(), srv.Token) { + t.Errorf("%s: the error carries the token: %q", name, tc.err) + } + } +} + +func TestCreateIssueReadBackOutsideTheProjectIsNotFound(t *testing.T) { + y, _ := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { + switch { + case r.Method == http.MethodPost && r.URL.Path == "/api/issues": + writeRaw(w, rawIssue("2-1", "UZI-1", 1, "0-1", "")) + default: + // The read-back answers an issue that sits in another project. + writeRaw(w, rawIssue("2-1", "OTH-1", 1, "0-2", "")) + } + }) + _, err := y.CreateIssue(context.Background(), testProjectID, "t", "d", nil) + var nf *forge.IssueNotFoundError + if !errors.As(err, &nf) { + t.Fatalf("CreateIssue = %v, want *forge.IssueNotFoundError", err) + } +} + +func TestListIssuesWithStateReturnsTheStateNames(t *testing.T) { + y, srv := columnDriver(t, nil) + a := srv.SeedIssue("a", "") + b := srv.SeedIssue("b", "") + ctx := context.Background() + if err := y.MoveToColumn(ctx, testProjectID, b, forge.ColumnMove{Column: "In Progress"}); err != nil { + t.Fatal(err) + } + got, err := y.ListIssuesWithState(ctx, testProjectID, forge.ListIssuesOptions{State: forge.StateAll}) + if err != nil { + t.Fatal(err) + } + states := map[int64]string{} + for _, s := range got { + states[s.Issue.IID] = s.State + } + if len(got) != 2 || states[a] != "Open" || states[b] != "In Progress" { + t.Errorf("listed %v, want a Open and b In Progress", states) + } + // A State field under another name is not read. + srv.SetStateField("Stage") + got, err = y.ListIssuesWithState(ctx, testProjectID, forge.ListIssuesOptions{State: forge.StateAll}) + if err != nil || len(got) != 2 || got[0].State != "" { + t.Errorf("with a renamed field: %+v, %v; want empty State names", got, err) + } +} + +func TestListIssuesWithStateDropsABadStateName(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("a", "") + srv.RenameState("Open", "Op\u202een") + got, err := y.ListIssuesWithState(context.Background(), testProjectID, forge.ListIssuesOptions{State: forge.StateAll}) + if err != nil || len(got) != 1 || got[0].Issue.IID != n || got[0].State != "" { + t.Errorf("listed %+v, %v; want the issue with an empty State", got, err) + } +} + +func TestListIssuesAndListIssuesWithStateSendTheSameRequests(t *testing.T) { + y, srv := columnDriver(t, nil) + srv.SeedIssue("a", "", youtracktest.EligibilityTagID) + srv.SeedIssue("b", "") + ctx := context.Background() + for name, opts := range map[string]forge.ListIssuesOptions{ + "project": {State: forge.StateAll}, + "tag": {State: forge.StateAll, Labels: []string{youtracktest.EligibilityTagName}}, + } { + srv.ResetRequests() + plain, err := y.ListIssues(ctx, testProjectID, opts) + if err != nil { + t.Fatal(err) + } + first := srv.Requests() + srv.ResetRequests() + stated, err := y.ListIssuesWithState(ctx, testProjectID, opts) + if err != nil { + t.Fatal(err) + } + if second := srv.Requests(); !slices.Equal(first, second) { + t.Errorf("%s: ListIssues sent %v, ListIssuesWithState %v", name, first, second) + } + if len(plain) != len(stated) { + t.Fatalf("%s: %d issues against %d", name, len(plain), len(stated)) + } + for i := range plain { + if plain[i].IID != stated[i].Issue.IID { + t.Errorf("%s: issue %d differs", name, i) + } + } + if name == "project" && (len(first) != 1 || first[0].Path != "/api/issues") { + t.Errorf("project listing sent %v, want one GET of /api/issues", first) + } + } +} + +func TestListIssuesWithStateRefusesTooManyCustomFields(t *testing.T) { + field := func(i int) string { + return fmt.Sprintf(`{"$type":"SimpleIssueCustomField","name":"f%d","value":"x"}`, i) + } + issueWith := func(n int) string { + fields := make([]string, 0, n+1) + fields = append(fields, `{"$type":"StateIssueCustomField","name":"State","value":{"$type":"StateBundleElement","name":"Open"}}`) + for i := 1; i < n; i++ { + fields = append(fields, field(i)) + } + return strings.TrimSuffix(rawIssue("2-1", "UZI-1", 1, "0-1", ""), `}`) + `,"customFields":[` + strings.Join(fields, ",") + `]}` + } + for name, tc := range map[string]struct { + n int + wantErr bool + }{"at the cap": {maxCustomFields, false}, "over the cap": {maxCustomFields + 1, true}} { + t.Run(name, func(t *testing.T) { + body := "[" + issueWith(tc.n) + "]" + y, _ := loggedDriver(t, func(w http.ResponseWriter, _ *http.Request) { writeRaw(w, body) }) + got, err := y.ListIssuesWithState(context.Background(), testProjectID, forge.ListIssuesOptions{State: forge.StateAll}) + switch { + case tc.wantErr && (err == nil || !strings.Contains(err.Error(), "custom fields")): + t.Errorf("got %+v, %v; want the custom field ceiling", got, err) + case !tc.wantErr && (err != nil || len(got) != 1 || got[0].State != "Open"): + t.Errorf("got %+v, %v; want one issue in Open", got, err) + } + }) + } +} + +func TestSetIssueStateRefusesAWontFixRelabelledByLocalizedName(t *testing.T) { + y, srv := columnDriver(t, func(o *Options) { o.ClosedState = "Won't fix" }) + srv.SetStateLocalizedName("Won't fix", "Fixed") + n := srv.SeedIssue("t", "") + srv.ResetRequests() + err := y.SetIssueState(context.Background(), testProjectID, n, forge.StateClosed) + if !errors.Is(err, ErrStateNotAllowed) { + t.Fatalf("SetIssueState = %v, want ErrStateNotAllowed", err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent: %v", w) + } +} + +func TestStateChecksRunBeforeTheIssueIsRead(t *testing.T) { + // A moved-away issue and a disallowed target: the state check comes first. + y, srv := columnDriver(t, func(o *Options) { o.ClosedState = "Won't fix" }) + n := srv.SeedIssue("t", "") + srv.MoveIssue(n, youtracktest.OtherProjectID) + srv.ResetRequests() + ctx := context.Background() + for name, err := range map[string]error{ + "SetIssueState": y.SetIssueState(ctx, testProjectID, n, forge.StateClosed), + "MoveToColumn": y.MoveToColumn(ctx, testProjectID, n, forge.ColumnMove{Column: "Lost"}), + } { + if !errors.Is(err, ErrStateNotAllowed) || isNotFound(err) { + t.Errorf("%s = %v, want ErrStateNotAllowed and not an IssueNotFoundError", name, err) + } + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent: %v", w) + } +} + +func TestOnlyTheListingAsksForCustomFields(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedIssue("t", "") + ctx := context.Background() + srv.ResetRequests() + if _, err := y.GetIssue(ctx, testProjectID, n); err != nil { + t.Fatal(err) + } + if err := y.MoveToColumn(ctx, testProjectID, n, forge.ColumnMove{Column: "In Progress"}); err != nil { + t.Fatal(err) + } + if _, err := y.CreateIssue(ctx, testProjectID, "c", "", nil); err != nil { + t.Fatal(err) + } + for _, r := range srv.Requests() { + if r.Method == http.MethodGet && strings.HasPrefix(r.Path, "/api/issues/") || r.Method == http.MethodPost && r.Path == "/api/issues" { + if strings.Contains(r.RawQuery, "customFields") { + t.Errorf("%s %s asked for custom fields: %s", r.Method, r.Path, r.RawQuery) + } + } + } + srv.ResetRequests() + if _, err := y.ListIssuesWithState(ctx, testProjectID, forge.ListIssuesOptions{State: forge.StateAll}); err != nil { + t.Fatal(err) + } + rs := srv.Requests() + if len(rs) == 0 { + t.Fatal("the listing sent no request") + } + for _, r := range rs { + if !strings.Contains(r.RawQuery, "customFields") { + t.Errorf("listing request %s %s did not ask for customFields: %s", r.Method, r.Path, r.RawQuery) + } + } +} + +func TestListIssuesWithStateReadsTheFirstResolvedStateOfAResolvedIssue(t *testing.T) { + y, srv := columnDriver(t, nil) + n := srv.SeedResolvedIssue("old") + srv.SetStateArchived("Fixed", true) + srv.SeedState("Done", true) + got, err := y.ListIssuesWithState(context.Background(), testProjectID, forge.ListIssuesOptions{State: forge.StateAll}) + if err != nil || len(got) != 1 || got[0].Issue.IID != n { + t.Fatalf("listed %+v, %v", got, err) + } + // Open is unresolved and Fixed archived; "Won't fix" is the first resolved, + // non-archived state in the bundle. + if got[0].State != "Won't fix" { + t.Errorf("State = %q, want the first resolved, non-archived state", got[0].State) + } +} + +func TestSetIssueStateRefusesAWontFixRelabelledWithACurlyApostrophe(t *testing.T) { + y, srv := columnDriver(t, nil) + curly := "Won\u2019t fix" + srv.RenameState("Won't fix", curly) + srv.SetStateLocalizedName(curly, "Fixed") + y.closedState = curly + n := srv.SeedIssue("t", "") + srv.ResetRequests() + err := y.SetIssueState(context.Background(), testProjectID, n, forge.StateClosed) + if !errors.Is(err, ErrStateNotAllowed) { + t.Fatalf("SetIssueState = %v, want ErrStateNotAllowed", err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent: %v", w) + } +} + +func TestAFillerInAStateNameIsDroppedAndNotClosable(t *testing.T) { + y, srv := columnDriver(t, nil) + name := "Won't fix\u3164" + srv.RenameState("Won't fix", name) + states, err := y.ListStates(context.Background(), "0-1") + if err != nil { + t.Fatal(err) + } + for _, s := range states { + if s.Name == name { + t.Errorf("ListStates kept %q", name) + } + } + if ClosableState(State{Name: name, Resolved: true}) { + t.Error("ClosableState accepted a name with a filler") + } +} + +func TestListIssuesWithStateReadsAStateMachineField(t *testing.T) { + field := func(typ string) string { + return `{"$type":"` + typ + `","name":"State","value":{"$type":"StateBundleElement","name":"Review"}}` + } + for typ, want := range map[string]string{ + "StateIssueCustomField": "Review", + "StateMachineIssueCustomField": "Review", + "SimpleIssueCustomField": "", + } { + body := "[" + strings.TrimSuffix(rawIssue("2-1", "UZI-1", 1, "0-1", ""), `}`) + `,"customFields":[` + field(typ) + `]}]` + y, _ := loggedDriver(t, func(w http.ResponseWriter, _ *http.Request) { writeRaw(w, body) }) + got, err := y.ListIssuesWithState(context.Background(), testProjectID, forge.ListIssuesOptions{State: forge.StateAll}) + if err != nil || len(got) != 1 || got[0].State != want { + t.Errorf("%s: got %+v, %v; want State %q", typ, got, err, want) + } + } +} + +func TestRepeatedCustomFieldsKeyLeavesNoStaleState(t *testing.T) { + // The second "customFields" key reuses the first one's backing array and + // omits the value; the first key's State name must not survive. + const state = `{"$type":"StateIssueCustomField","name":"State"` + raw := strings.TrimSuffix(rawIssue("2-1", "UZI-1", 1, "0-1", ""), `}`) + + `,"customFields":[` + state + `,"value":{"name":"Open"}}],"customFields":[` + state + `}]}` + var w ytIssue + if err := json.Unmarshal([]byte(raw), &w); err != nil { + t.Fatal(err) + } + y := &youTrack{stateField: "State"} + if got := y.stateName(w); got != "" { + t.Errorf("stateName = %q, want empty after the later key omitted the value", got) + } +} + +func TestCustomFieldNotObjectHasItsOwnError(t *testing.T) { + var c ytCustomFields + if err := c.UnmarshalJSON([]byte(`[1]`)); !errors.Is(err, errCustomFieldNotObject) { + t.Errorf("err = %v, want errCustomFieldNotObject", err) + } +} + +func TestMoveToColumnRefusesAnIssueResolvedInTheIssueTrackerWithoutAReopen(t *testing.T) { + y, srv := columnDriver(t, nil) + ctx := context.Background() + n := srv.SeedIssue("t", "") + if err := y.SetIssueState(ctx, testProjectID, n, forge.StateClosed); err != nil { + t.Fatalf("resolve the issue: %v", err) + } + before := srv.IssueState(n) + srv.ResetRequests() + err := y.MoveToColumn(ctx, testProjectID, n, forge.ColumnMove{Column: "In Progress"}) + if !errors.Is(err, ErrIssueResolved) { + t.Fatalf("MoveToColumn = %v, want ErrIssueResolved", err) + } + if w := writesOf(srv); len(w) != 0 { + t.Errorf("writes sent for a resolved issue: %v", w) + } + if got := srv.IssueState(n); got != before { + t.Errorf("issue state = %q, want it untouched (%q)", got, before) + } +} + +func TestMoveToColumnWithReopenWritesOverAResolvedIssue(t *testing.T) { + y, srv := columnDriver(t, nil) + ctx := context.Background() + n := srv.SeedIssue("t", "") + if err := y.SetIssueState(ctx, testProjectID, n, forge.StateClosed); err != nil { + t.Fatalf("resolve the issue: %v", err) + } + if err := y.MoveToColumn(ctx, testProjectID, n, forge.ColumnMove{Column: "In Progress", Reopen: true}); err != nil { + t.Fatalf("MoveToColumn with Reopen: %v", err) + } + if got := srv.IssueState(n); got != "In Progress" { + t.Errorf("issue state = %q, want In Progress", got) + } + iss, err := y.GetIssue(ctx, testProjectID, n) + if err != nil { + t.Fatal(err) + } + if iss.State != string(forge.StateOpened) { + t.Errorf("issue state after the reopen = %q, want opened", iss.State) + } +} diff --git a/api/internal/issuetracker/youtrack_projects.go b/api/internal/issuetracker/youtrack_projects.go index 733af7d81e..0fda2d5e8e 100644 --- a/api/internal/issuetracker/youtrack_projects.go +++ b/api/internal/issuetracker/youtrack_projects.go @@ -30,14 +30,19 @@ type ytProjectField struct { } type ytStateValue struct { - ID string `json:"id"` - Name string `json:"name"` - IsResolved bool `json:"isResolved"` - Ordinal int `json:"ordinal"` + ID string `json:"id"` + Name string `json:"name"` + LocalizedName string `json:"localizedName"` + IsResolved bool `json:"isResolved"` + Archived bool `json:"archived"` + Ordinal int `json:"ordinal"` } // ListStates implements Driver. It finds the project's State field (named by -// Options.StateField, "State" by default) and lists the values of its bundle. +// Options.StateField, "State" by default) and lists the values of its bundle. A +// value with a malformed id fails the listing. A value whose name +// stateNameProblem refuses is left out, and a localized name it refuses is +// cleared, because any user who administers a bundle can name its values. func (y *youTrack) ListStates(ctx context.Context, projectID string) ([]State, error) { if !entityIDRE.MatchString(projectID) { return nil, y.fail("youtrack: invalid project id") @@ -61,13 +66,23 @@ func (y *youTrack) ListStates(ctx context.Context, projectID string) ([]State, e return nil, y.fail("youtrack: state field with a malformed id in the response") } values, err := ytList[ytStateValue](ctx, y, []string{"admin", "projects", projectID, "customFields", fieldID, "bundle", "values"}, - url.Values{"fields": {"id,name,isResolved,ordinal"}}) + url.Values{"fields": {"id,name,localizedName,isResolved,archived,ordinal"}}) if err != nil { return nil, err } out := make([]State, 0, len(values)) for _, v := range values { - out = append(out, State{ID: v.ID, Name: v.Name, Resolved: v.IsResolved, Ordinal: v.Ordinal}) + if !entityIDRE.MatchString(v.ID) { + return nil, y.fail("youtrack: state with a malformed id in the response") + } + if stateNameProblem(v.Name) { + continue + } + localized := v.LocalizedName + if stateNameProblem(localized) { + localized = "" + } + out = append(out, State{ID: v.ID, Name: v.Name, LocalizedName: localized, Resolved: v.IsResolved, Archived: v.Archived, Ordinal: v.Ordinal}) } return out, nil } diff --git a/api/internal/issuetracker/youtrack_projects_test.go b/api/internal/issuetracker/youtrack_projects_test.go index 4d8f229d8d..98508a57e1 100644 --- a/api/internal/issuetracker/youtrack_projects_test.go +++ b/api/internal/issuetracker/youtrack_projects_test.go @@ -2,6 +2,8 @@ package issuetracker import ( "context" + "net/http" + "strings" "testing" ) @@ -24,3 +26,50 @@ func TestStatesAndProjects(t *testing.T) { t.Fatal("ListStates accepted a malformed project id") } } + +// statesDriver serves the State field of project 0-1 with the given bundle +// values, a raw JSON array. +func statesDriver(t *testing.T, values string) *youTrack { + t.Helper() + y, _ := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { + if strings.HasSuffix(r.URL.Path, "/bundle/values") { + writeRaw(w, values) + return + } + writeRaw(w, `[{"id":"60-1","$type":"StateProjectCustomField","field":{"name":"State"},"bundle":{"id":"40-1"}}]`) + }) + return y +} + +func TestListStatesDropsBadNamesAndRefusesBadIDs(t *testing.T) { + ctx := context.Background() + long := strings.Repeat("a", maxLabelNameLen+1) + y := statesDriver(t, `[ + {"id":"10-1","name":"Open","ordinal":0}, + {"id":"10-2","name":"`+long+`","ordinal":1}, + {"id":"10-3","name":"Zero\u200bwidth","ordinal":2}, + {"id":"10-4","name":" ","ordinal":3}, + {"id":"10-5","name":"Fixed","localizedName":"Er\u202eledigt","isResolved":true,"archived":true,"ordinal":4}, + {"id":"10-6","name":"Done","localizedName":"Erledigt","isResolved":true,"ordinal":5}]`) + got, err := y.ListStates(ctx, "0-1") + if err != nil { + t.Fatal(err) + } + want := []State{ + {ID: "10-1", Name: "Open"}, + {ID: "10-5", Name: "Fixed", Resolved: true, Archived: true, Ordinal: 4}, + {ID: "10-6", Name: "Done", LocalizedName: "Erledigt", Resolved: true, Ordinal: 5}, + } + if len(got) != len(want) { + t.Fatalf("ListStates = %+v, want %+v", got, want) + } + for i := range want { + if got[i] != want[i] { + t.Errorf("state %d = %+v, want %+v", i, got[i], want[i]) + } + } + y = statesDriver(t, `[{"id":"10-1","name":"Open"},{"id":"../x","name":"Fixed"}]`) + if got, err := y.ListStates(ctx, "0-1"); err == nil { + t.Errorf("ListStates accepted a malformed state id: %+v", got) + } +} diff --git a/api/internal/issuetracker/youtrack_query.go b/api/internal/issuetracker/youtrack_query.go index 809b0711d6..2424f5ef1e 100644 --- a/api/internal/issuetracker/youtrack_query.go +++ b/api/internal/issuetracker/youtrack_query.go @@ -31,6 +31,13 @@ func labelNameProblem(name string) bool { return bad } +// LabelNameProblem reports whether name is empty, over 200 bytes, or holds a brace, an +// invisible character, or a character a State name may not hold (a blank filler, a +// variation selector or another ignorable code point, see stateNameProblem). A board +// column of a linked repo sits next to States on the board and in the mapping, so it +// gets both rules. +func LabelNameProblem(name string) bool { return labelNameProblem(name) || stateNameProblem(name) } + func (y *youTrack) checkLabelNames(names ...[]string) error { for _, list := range names { for _, n := range list { diff --git a/api/internal/issuetracker/youtrack_state.go b/api/internal/issuetracker/youtrack_state.go index 68504bce7c..2e6d020fd7 100644 --- a/api/internal/issuetracker/youtrack_state.go +++ b/api/internal/issuetracker/youtrack_state.go @@ -10,38 +10,119 @@ import ( // SetIssueState implements forge.IssueTracker. It writes the explicit state // value Options names for the requested state and never guesses one; with that -// value unset it returns ErrStateMappingUnset. +// value unset it returns ErrStateMappingUnset. The value is checked against the +// project's State bundle first (checkTargetState), before the issue is read. func (y *youTrack) SetIssueState(ctx context.Context, projectID, issueIID int64, state forge.IssueState) error { if _, err := y.requireProject(projectID); err != nil { return err } var value string + var closing bool switch state { case forge.StateOpened: value = y.openState case forge.StateClosed: - value = y.closedState + value, closing = y.closedState, true default: return y.fail("youtrack: state must be opened or closed") } if value == "" { return y.red.sentinel(ErrStateMappingUnset, 0, "no value configured for "+string(state)) } + target, err := y.checkTargetState(ctx, value, closing) + if err != nil { + return err + } + w, err := y.resolveIssue(ctx, projectID, issueIID) + if err != nil { + return err + } + return y.writeState(ctx, w, target) +} + +// MoveToColumn moves the issue to a board column by writing the State value the +// connection maps to it. An empty column is the implicit Open column and writes +// the open state. A column with no mapped State is ErrColumnNotMapped, and +// neither case sends a request before the check. The target is then checked +// against the project's State bundle and the issue against the bound project, as +// SetIssueState does. An issue the issue tracker already shows as resolved is +// ErrIssueResolved with no write unless move.Reopen is set: the move refuses when +// the issue tracker already shows the issue resolved. The resolved read (GET) and +// the State write (POST) are separate requests, so a resolution landing between +// them is still overwritten; that window is accepted, like the check-then-write +// window of checkTargetState. +func (y *youTrack) MoveToColumn(ctx context.Context, projectID, issueIID int64, move forge.ColumnMove) error { + return y.moveToColumn(ctx, projectID, issueIID, move.Column, move.Reopen) +} + +func (y *youTrack) moveToColumn(ctx context.Context, projectID, issueIID int64, column string, reopen bool) error { + if _, err := y.requireProject(projectID); err != nil { + return err + } + var value string + if column == "" { + if y.openState == "" { + return y.red.sentinel(ErrStateMappingUnset, 0, "no value configured for the open column") + } + value = y.openState + } else { + v, ok := y.columnStates[column] + if !ok || v == "" { + return y.red.sentinel(ErrColumnNotMapped, 0, "") + } + value = v + } + target, err := y.checkTargetState(ctx, value, false) + if err != nil { + return err + } w, err := y.resolveIssue(ctx, projectID, issueIID) if err != nil { return err } - _, _, err = y.do(ctx, ytCall{ + if w.Resolved != nil && !reopen { + return y.red.sentinel(ErrIssueResolved, 0, "") + } + return y.writeState(ctx, w, target) +} + +// writeState sets the State field of issue w to s, the value checkTargetState +// approved, addressed by its id so the write cannot land on another value. +func (y *youTrack) writeState(ctx context.Context, w ytIssue, s State) error { + _, _, err := y.do(ctx, ytCall{ method: http.MethodPost, path: []string{"issues", w.ID}, query: url.Values{"fields": {"id"}}, body: map[string]any{"customFields": []any{map[string]any{ "name": y.stateField, "$type": "StateIssueCustomField", - "value": map[string]string{"name": value}, + "value": map[string]string{"id": s.ID}, }}}, limit: objectBytes, prov: forge.ResponseSuccess, }, nil) return err } + +// checkTargetState reads the bound project's State bundle (the ListStates code +// path) and refuses, with ErrStateNotAllowed, a target that the bundle lacks, or +// that is not closable when closing, or not openable otherwise. It runs before +// every State write, so a bundle edited after the mapping was saved is caught. +// It returns the matched State, which is the one to write. +// The error quotes the name through quoteName, which bounds it. +func (y *youTrack) checkTargetState(ctx context.Context, name string, closing bool) (State, error) { + states, err := y.ListStates(ctx, y.project.ID) + if err != nil { + return State{}, err + } + for _, s := range states { + if s.Name != name { + continue + } + if closing && ClosableState(s) || !closing && OpenableState(s) { + return s, nil + } + return State{}, y.red.sentinel(ErrStateNotAllowed, 0, "the state is not allowed in this role: "+quoteName(name)) + } + return State{}, y.red.sentinel(ErrStateNotAllowed, 0, "the project's state bundle has no such state: "+quoteName(name)) +} diff --git a/api/internal/issuetracker/youtrack_wire.go b/api/internal/issuetracker/youtrack_wire.go index 146f3c8365..9a9c8dddab 100644 --- a/api/internal/issuetracker/youtrack_wire.go +++ b/api/internal/issuetracker/youtrack_wire.go @@ -10,13 +10,16 @@ import ( // fields parameters. YouTrack returns only what is asked for. const ( - issueFields = "id,idReadable,numberInProject,summary,description,created,updated,resolved,reporter(id,login),tags(id,name,owner(id)),project(id,shortName)" - tagFields = "id,name" - ownedTagFields = "id,name,owner(id)" - sharingFields = "permittedGroups(id,name,allUsersGroup,usersCount),permittedUsers(id,login)" - sharedTagFields = "id,name,owner(id,login),readSharingSettings(" + sharingFields + "),tagSharingSettings(" + sharingFields + ")" - commentFields = "id,text,created,deleted,author(id,login)" - activityFields = "id,timestamp,author(id,login),added(id,name,owner(id)),removed(id,name,owner(id))" + issueFields = "id,idReadable,numberInProject,summary,description,created,updated,resolved,reporter(id,login),tags(id,name,owner(id)),project(id,shortName)" + // listingIssueFields adds the custom fields the listing reads the State from; + // the single-issue reads do not ask for them. + listingIssueFields = issueFields + ",customFields(name,$type,value(name))" + tagFields = "id,name" + ownedTagFields = "id,name,owner(id)" + sharingFields = "permittedGroups(id,name,allUsersGroup,usersCount),permittedUsers(id,login)" + sharedTagFields = "id,name,owner(id,login),readSharingSettings(" + sharingFields + "),tagSharingSettings(" + sharingFields + ")" + commentFields = "id,text,created,deleted,author(id,login)" + activityFields = "id,timestamp,author(id,login),added(id,name,owner(id)),removed(id,name,owner(id))" ) // idReadableRE matches an id-readable such as "UZI-15"; it ends up in WebURL. @@ -80,10 +83,28 @@ type ytIssue struct { Reporter *ytUser `json:"reporter"` Tags ytTags `json:"tags"` Project *ytProject `json:"project"` + // CustomFields is the issue's custom fields, capped at maxCustomFields + // while decoding (ytCustomFields). + CustomFields ytCustomFields `json:"customFields"` } func (w ytIssue) nestedCount() int { return len(w.Tags) } +// stateName returns the name (a state-machine field is read like a state one; +// writes use StateIssueCustomField only) of the issue's State field value, or "" when the +// issue has none or the name is one stateNameProblem refuses. +func (y *youTrack) stateName(w ytIssue) string { + for _, f := range w.CustomFields { + if f.Name == y.stateField && (f.Type == "StateIssueCustomField" || f.Type == "StateMachineIssueCustomField") { + if stateNameProblem(f.Value.Name) { + return "" + } + return f.Value.Name + } + } + return "" +} + func (a ytActivity) nestedCount() int { return len(a.Added) + len(a.Removed) } type ytComment struct { diff --git a/api/internal/issuetracker/youtracktest/routes.go b/api/internal/issuetracker/youtracktest/routes.go index 03627bd0ad..3a01fd7095 100644 --- a/api/internal/issuetracker/youtracktest/routes.go +++ b/api/internal/issuetracker/youtracktest/routes.go @@ -156,7 +156,11 @@ func (s *Server) listBundleValues(w http.ResponseWriter, r *http.Request) { } var items []map[string]any for _, v := range s.m.states { - items = append(items, map[string]any{"$type": "StateBundleElement", "id": v.id, "name": v.name, "isResolved": v.resolved, "ordinal": v.ordinal}) + var localized any + if v.localizedName != "" { + localized = v.localizedName + } + items = append(items, map[string]any{"$type": "StateBundleElement", "id": v.id, "name": v.name, "isResolved": v.resolved, "archived": v.archived, "localizedName": localized, "ordinal": v.ordinal}) } writePage(w, r, spec, items) } @@ -398,6 +402,7 @@ func (s *Server) updateIssue(w http.ResponseWriter, r *http.Request) { Name string `json:"name"` Type string `json:"$type"` Value struct { + ID string `json:"id"` Name string `json:"name"` } `json:"value"` } `json:"customFields"` @@ -411,13 +416,19 @@ func (s *Server) updateIssue(w http.ResponseWriter, r *http.Request) { i.activities = append(i.activities, &activity{id: s.m.textActivityID(), category: catDescription, timestamp: i.updated, author: BotLogin}) } for _, cf := range body.CustomFields { - v := s.m.stateByName(cf.Value.Name) + // A State value must be addressed by id: a write by name is refused, so + // a driver that sends only a name fails its tests. + var v *stateValue + if cf.Value.ID != "" { + v = s.m.stateByID(cf.Value.ID) + } if cf.Name != s.m.stateField || cf.Type != "StateIssueCustomField" || v == nil { writeError(w, http.StatusBadRequest, "Bad Request", "unsupported custom field update") return } ts := s.m.now() i.updated = ts + i.stateID = v.id if v.resolved { if i.resolved == nil { i.resolved = &ts diff --git a/api/internal/issuetracker/youtracktest/seed.go b/api/internal/issuetracker/youtracktest/seed.go index 8be0fd0337..2877b68de9 100644 --- a/api/internal/issuetracker/youtracktest/seed.go +++ b/api/internal/issuetracker/youtracktest/seed.go @@ -1,6 +1,9 @@ package youtracktest -import "slices" +import ( + "fmt" + "slices" +) // Sharing says who a tag is shared with, by group id and user id. type Sharing struct { @@ -109,6 +112,19 @@ func (s *Server) SeedIssue(summary, description string, tagIDs ...string) int64 return i.number } +// SeedResolvedIssue creates an issue in the main project that is resolved but +// has no State value recorded, as an issue resolved before the State field +// existed, and returns its number. +func (s *Server) SeedResolvedIssue(summary string) int64 { + s.t.Helper() + s.mu.Lock() + defer s.mu.Unlock() + i := s.m.newIssue(s.m.projectByID(ProjectID), summary, "") + ts := s.m.now() + i.resolved = &ts + return i.number +} + // SetReporter makes login the reporter of issue number n. A seeded issue is // reported by BotLogin (the account the token authenticates as) until this says // otherwise, so a test of an author that is not the connection's account sets it. @@ -268,3 +284,68 @@ func (s *Server) SeedCommentEdit(n int64, login, text, newText string) { } s.t.Fatalf("youtracktest: no comment %q on issue %d", text, n) } + +// SeedState adds an unarchived State value to the bundle and returns its id. +func (s *Server) SeedState(name string, resolved bool) string { + s.t.Helper() + s.mu.Lock() + defer s.mu.Unlock() + // Not nextID: its counter would hand out an id the default states hold. + id := fmt.Sprintf("10-%d", 100+len(s.m.states)) + s.m.states = append(s.m.states, stateValue{id: id, name: name, resolved: resolved, ordinal: len(s.m.states)}) + return id +} + +// RenameState renames the State value called from, as a bundle administrator +// would after a mapping was saved. Issues in that state follow the new name. +func (s *Server) RenameState(from, to string) { + s.t.Helper() + s.editState(from, func(v *stateValue) { v.name = to }) +} + +// SetStateResolved sets the resolved flag of the State value called name. +func (s *Server) SetStateResolved(name string, resolved bool) { + s.t.Helper() + s.editState(name, func(v *stateValue) { v.resolved = resolved }) +} + +// SetStateArchived sets the archived flag of the State value called name. +func (s *Server) SetStateArchived(name string, archived bool) { + s.t.Helper() + s.editState(name, func(v *stateValue) { v.archived = archived }) +} + +// SetStateLocalizedName sets the localized name of the State value called name. +func (s *Server) SetStateLocalizedName(name, localized string) { + s.t.Helper() + s.editState(name, func(v *stateValue) { v.localizedName = localized }) +} + +func (s *Server) editState(name string, edit func(*stateValue)) { + s.t.Helper() + s.mu.Lock() + defer s.mu.Unlock() + v := s.m.stateByName(name) + if v == nil { + s.t.Fatalf("youtracktest: no state %q", name) + } + edit(v) +} + +// IssueState returns the name of the State value of issue number n in the main +// project, for assertions on server-side state. +func (s *Server) IssueState(n int64) string { + s.t.Helper() + s.mu.Lock() + defer s.mu.Unlock() + i := s.m.byNumber(n) + if i == nil { + s.t.Fatalf("youtracktest: no issue %d", n) + } + for _, cf := range s.m.customFieldsJSON(i) { + if v, ok := cf["value"].(map[string]any); ok { + return v["name"].(string) + } + } + return "" +} diff --git a/api/internal/issuetracker/youtracktest/state.go b/api/internal/issuetracker/youtracktest/state.go index 9eb544097c..7321fe440d 100644 --- a/api/internal/issuetracker/youtracktest/state.go +++ b/api/internal/issuetracker/youtracktest/state.go @@ -101,9 +101,11 @@ type activity struct { } type stateValue struct { - id, name string - resolved bool - ordinal int + id, name string + localizedName string + resolved bool + archived bool + ordinal int } type issue struct { @@ -116,6 +118,7 @@ type issue struct { updated int64 resolved *int64 reporter string + stateID string // id of the issue's State value; empty reads as the first state tags []string // tag ids, oldest first comments []*comment activities []*activity @@ -303,6 +306,15 @@ func (m *model) removeTag(i *issue, t *tag, author string) { i.activities = append(i.activities, &activity{id: m.nextID(3), category: catTags, timestamp: ts, author: author, removed: []string{t.id}}) } +func (m *model) stateByID(id string) *stateValue { + for k := range m.states { + if m.states[k].id == id { + return &m.states[k] + } + } + return nil +} + func (m *model) stateByName(name string) *stateValue { for k := range m.states { if m.states[k].name == name { @@ -326,6 +338,7 @@ func (m *model) issueJSON(i *issue) map[string]any { resolved = *i.resolved } return map[string]any{ + "customFields": m.customFieldsJSON(i), "$type": "Issue", "id": i.id, "idReadable": i.readable(), @@ -341,6 +354,29 @@ func (m *model) issueJSON(i *issue) map[string]any { } } +// customFieldsJSON is the issue's custom fields: its State, under the current +// name of the State field and of the State value, as YouTrack lists it. +func (m *model) customFieldsJSON(i *issue) []map[string]any { + v := m.stateByID(i.stateID) + if v == nil && i.resolved != nil { + // A resolved issue with no recorded State value reads as the first + // resolved one, so it never reads as open. + for k := range m.states { + if m.states[k].resolved && !m.states[k].archived { + v = &m.states[k] + break + } + } + } else if v == nil && len(m.states) > 0 { + v = &m.states[0] + } + var value any + if v != nil { + value = map[string]any{"$type": "StateBundleElement", "name": v.name} + } + return []map[string]any{{"$type": "StateIssueCustomField", "name": m.stateField, "value": value}} +} + // userJSON is the wire user for a login. A login the fake does not know is // given the bot's id, as comment authors such as "alice" always were. func userJSON(login string) map[string]any { diff --git a/api/internal/issuetrackersvc/driver.go b/api/internal/issuetrackersvc/driver.go index d25aad5213..93ae1e43d1 100644 --- a/api/internal/issuetrackersvc/driver.go +++ b/api/internal/issuetrackersvc/driver.go @@ -15,8 +15,9 @@ import ( // AAD, used only to build the driver, and never returned or logged. binding is // the empty issuetracker.Binding for a connection-level driver, or all of the // project, the repository's forge project id and the eligibility tag; a partial -// or malformed one is refused before anything is read. -func (s *Service) DriverFor(ctx context.Context, userID, connID uuid.UUID, binding issuetracker.Binding, stateField, open, closed string) (issuetracker.Driver, error) { +// or malformed one is refused before anything is read. states is the repo's +// stored state mapping, or the zero StateMapping for a connection-level driver. +func (s *Service) DriverFor(ctx context.Context, userID, connID uuid.UUID, binding issuetracker.Binding, states issuetracker.StateMapping) (issuetracker.Driver, error) { if issuetracker.ValidBinding(binding) != nil { return nil, &Error{Kind: KindBadRequest, Message: "invalid project binding"} } @@ -47,9 +48,10 @@ func (s *Service) DriverFor(ctx context.Context, userID, connID uuid.UUID, bindi Project: binding.Project, ProjectID: binding.ProjectID, EligibilityTag: binding.EligibilityTag, - StateField: stateField, - OpenState: open, - ClosedState: closed, + StateField: states.Field, + OpenState: states.Open, + ClosedState: states.Closed, + ColumnStates: states.Columns, }) if err != nil { s.log.Error("build issue tracker driver", "kind", row.Kind, "error", err) @@ -60,7 +62,7 @@ func (s *Service) DriverFor(ctx context.Context, userID, connID uuid.UUID, bindi // ListProjects lists the projects the connection's token can see. func (s *Service) ListProjects(ctx context.Context, userID, connID uuid.UUID) ([]issuetracker.Project, error) { - d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, "", "", "") + d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, issuetracker.StateMapping{}) if err != nil { return nil, err } @@ -76,7 +78,7 @@ func (s *Service) ListStates(ctx context.Context, userID, connID uuid.UUID, proj if !issuetracker.ValidProjectID(projectID) { return nil, &Error{Kind: KindBadRequest, Message: "invalid project id"} } - d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, "", "", "") + d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, issuetracker.StateMapping{}) if err != nil { return nil, err } @@ -90,7 +92,7 @@ func (s *Service) ListStates(ctx context.Context, userID, connID uuid.UUID, proj // ListTags lists the tags the connection's token can see, with their owners and // sharing settings. func (s *Service) ListTags(ctx context.Context, userID, connID uuid.UUID) ([]issuetracker.Tag, error) { - d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, "", "", "") + d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, issuetracker.StateMapping{}) if err != nil { return nil, err } diff --git a/api/internal/issuetrackersvc/link.go b/api/internal/issuetrackersvc/link.go index 39ffd5d346..a37a3d238e 100644 --- a/api/internal/issuetrackersvc/link.go +++ b/api/internal/issuetrackersvc/link.go @@ -150,7 +150,7 @@ func (s *Service) checkLinkTarget(ctx context.Context, userID, connID uuid.UUID, noProject issuetracker.Project noTag issuetracker.Tag ) - d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, "", "", "") + d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, issuetracker.StateMapping{}) if err != nil { return noAccount, noProject, noTag, err } diff --git a/api/internal/issuetrackersvc/link_fake_test.go b/api/internal/issuetrackersvc/link_fake_test.go index e38f90f216..9a239555eb 100644 --- a/api/internal/issuetrackersvc/link_fake_test.go +++ b/api/internal/issuetrackersvc/link_fake_test.go @@ -20,6 +20,12 @@ type fakeRepo struct { tagID string tagName string storesNumbers bool + // State mapping side (see state_mapping_fake_test.go). + boardColumns []string + openState string + closedState string + hasStates bool + columnStates map[string]string } func (f *fakeQuerier) addRepo(user uuid.UUID, awaiting bool) uuid.UUID { @@ -57,6 +63,11 @@ func (f *fakeQuerier) GetRepoIssueSourceForUser(_ context.Context, a store.GetRe return store.GetRepoIssueSourceForUserRow{ ID: a.ID, AwaitingFirstEnable: r.awaiting, IssueTrackerConnectionID: pgtype.UUID{Bytes: r.linkedConn, Valid: r.linkedConn != uuid.Nil}, + LinkedConnectionID: pgtype.UUID{Bytes: r.linkedConn, Valid: r.linkedConn != uuid.Nil}, + IssueTrackerProjectID: pgtype.Text{String: r.projectID, Valid: r.linkedConn != uuid.Nil}, + OpenState: pgtype.Text{String: r.openState, Valid: r.hasStates}, + ClosedState: pgtype.Text{String: r.closedState, Valid: r.hasStates}, + ColumnStates: columnStatesJSON(r.columnStates), }, nil } diff --git a/api/internal/issuetrackersvc/service.go b/api/internal/issuetrackersvc/service.go index ed7d6a692e..9b8f6288e0 100644 --- a/api/internal/issuetrackersvc/service.go +++ b/api/internal/issuetrackersvc/service.go @@ -26,6 +26,11 @@ type Querier interface { LockRepoForLinkForUser(ctx context.Context, arg store.LockRepoForLinkForUserParams) (store.LockRepoForLinkForUserRow, error) RepoStoresIssueNumber(ctx context.Context, repoID uuid.UUID) (bool, error) SetRepoIssueTrackerLink(ctx context.Context, arg store.SetRepoIssueTrackerLinkParams) (store.Repo, error) + LockRepoForStateMappingForUser(ctx context.Context, arg store.LockRepoForStateMappingForUserParams) (store.LockRepoForStateMappingForUserRow, error) + ListBoardColumns(ctx context.Context, repoID uuid.UUID) ([]store.BoardColumn, error) + UpsertRepoIssueTrackerStates(ctx context.Context, arg store.UpsertRepoIssueTrackerStatesParams) (store.RepoIssueTrackerState, error) + DeleteRepoIssueTrackerColumnStates(ctx context.Context, repoID uuid.UUID) error + InsertRepoIssueTrackerColumnState(ctx context.Context, arg store.InsertRepoIssueTrackerColumnStateParams) error } var _ Querier = (*store.Queries)(nil) diff --git a/api/internal/issuetrackersvc/service_test.go b/api/internal/issuetrackersvc/service_test.go index fe2522fa12..612db7b923 100644 --- a/api/internal/issuetrackersvc/service_test.go +++ b/api/internal/issuetrackersvc/service_test.go @@ -484,17 +484,17 @@ func TestDriverForOpensTokenAndPassesOptions(t *testing.T) { ProjectID: 4173, EligibilityTag: issuetracker.TagRef{ID: "5-3", Name: "uzi"}, } - if _, err := h.svc.DriverFor(ctx, user, c.ID, binding, "State", "Open", "Done"); err != nil { + if _, err := h.svc.DriverFor(ctx, user, c.ID, binding, issuetracker.StateMapping{Field: "State", Open: "Open", Closed: "Done"}); err != nil { t.Fatal(err) } want := issuetracker.Options{ BaseURL: testHost, Token: tok, Project: binding.Project, ProjectID: 4173, EligibilityTag: binding.EligibilityTag, StateField: "State", OpenState: "Open", ClosedState: "Done", } - if len(h.built) != 1 || h.built[0] != want { + if len(h.built) != 1 || !reflect.DeepEqual(h.built[0], want) { t.Errorf("built = %+v, want %+v", h.built, want) } - _, err = h.svc.DriverFor(ctx, uuid.New(), c.ID, binding, "", "", "") + _, err = h.svc.DriverFor(ctx, uuid.New(), c.ID, binding, issuetracker.StateMapping{}) _ = wantKind(t, err, KindNotFound) } @@ -508,7 +508,7 @@ func TestDriverForFailsClosedWhenAllowlistEmptied(t *testing.T) { } h.built = nil h.allow = map[string]bool{} - _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, "", "", "") + _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, issuetracker.StateMapping{}) _ = wantKind(t, err, KindBadRequest) if len(h.built) != 0 { t.Error("driver built after the allowlist was emptied") @@ -530,7 +530,7 @@ func TestDriverForRefusesTamperedRow(t *testing.T) { // A row whose base URL was rewritten no longer opens: the AAD differs. h.allow["https://evil.example.com"] = true h.q.rows[0].BaseUrl = "https://evil.example.com" - _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, "", "", "") + _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, issuetracker.StateMapping{}) e := wantKind(t, err, KindInternal) if e.Message != "internal error" || len(h.built) != 0 { t.Errorf("message = %q, built = %d", e.Message, len(h.built)) @@ -538,7 +538,7 @@ func TestDriverForRefusesTamperedRow(t *testing.T) { // A row with a non-strict URL is refused before opening. h.allow["https://evil.example.com/x"] = true h.q.rows[0].BaseUrl = "https://evil.example.com/x" - _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, "", "", "") + _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, issuetracker.StateMapping{}) _ = wantKind(t, err, KindBadRequest) } @@ -554,7 +554,7 @@ func TestDriverForRefusesNonCanonicalStoredURL(t *testing.T) { // Strict-valid and allowlisted, but not the canonical stored form. h.allow["https://ISSUES.example.com"] = true h.q.rows[0].BaseUrl = "https://ISSUES.example.com" - _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, "", "", "") + _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, issuetracker.StateMapping{}) _ = wantKind(t, err, KindBadRequest) if len(h.built) != 0 { t.Error("driver built for a non-canonical stored URL") @@ -591,7 +591,7 @@ func TestDriverForRefusesPartialOrMalformedBinding(t *testing.T) { "bad tag name": mutate(func(b *issuetracker.Binding) { b.EligibilityTag.Name = "a}b" }), } { t.Run(name, func(t *testing.T) { - _, err := h.svc.DriverFor(ctx, user, c.ID, b, "", "", "") + _, err := h.svc.DriverFor(ctx, user, c.ID, b, issuetracker.StateMapping{}) _ = wantKind(t, err, KindBadRequest) }) } diff --git a/api/internal/issuetrackersvc/state_mapping.go b/api/internal/issuetrackersvc/state_mapping.go new file mode 100644 index 0000000000..2aeb6c0566 --- /dev/null +++ b/api/internal/issuetrackersvc/state_mapping.go @@ -0,0 +1,283 @@ +package issuetrackersvc + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "slices" + "strings" + "unicode/utf8" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5" + + "github.com/vtmocanu/uzi/api/internal/issuetracker" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// Bounds the repo_issue_tracker_states and repo_issue_tracker_column_states CHECKs +// (migration 00468) enforce on every stored name, and the most columns one mapping may +// carry (the board's own column cap). +const ( + maxStateNameLen = 200 + maxMappedColumns = 10 + stateMappingNoLink = "this repo is not linked to an issue tracker project" + stateMappingNoConn = "the repo's issue tracker connection is missing; it cannot be read" + stateMappingBadName = "a state or column name is empty, too long (max 200), or holds a control, invisible, blank-filler or variation-selector character (or, in a column name, a brace)" +) + +// ColumnState maps one board column, by its label name, to the State it stands for. +type ColumnState struct { + LabelName string + State string +} + +// StateMapping is the open and closed State of a linked repo and the State of each +// mapped board column (PRD #4173 M5). A board column that is absent from Columns is +// unmapped. +type StateMapping struct { + OpenState string + ClosedState string + Columns []ColumnState +} + +// StateInfo is one State of the linked project with what uzi may do with it: write it as +// the closed state (Closable), or as the open or a column state (Openable). +type StateInfo struct { + Name string + Resolved bool + Archived bool + Closable bool + Openable bool +} + +// StateMappingView is the saved mapping (nil when none is saved yet) and the States the +// linked project offers. +type StateMappingView struct { + Mapping *StateMapping + States []StateInfo +} + +// linkedSource reads the owner-scoped repo row and refuses a repo that is not linked. +func (s *Service) linkedSource(ctx context.Context, userID, repoID uuid.UUID) (store.GetRepoIssueSourceForUserRow, error) { + src, err := s.q.GetRepoIssueSourceForUser(ctx, store.GetRepoIssueSourceForUserParams{ID: repoID, UserID: userID}) + if errors.Is(err, pgx.ErrNoRows) { + return src, &Error{Kind: KindNotFound, Message: "repo not found"} + } + if err != nil { + s.log.Error("read repo for issue tracker state mapping", "error", err) + return src, internal() + } + if !src.IssueTrackerConnectionID.Valid || !src.IssueTrackerProjectID.Valid { + return src, conflict(stateMappingNoLink) + } + if !src.LinkedConnectionID.Valid { + return src, conflict(stateMappingNoConn) + } + return src, nil +} + +// GetStateMapping returns the saved state mapping of a linked repo and the States of its +// project. A repo that is not the user's is NotFound and one that is not linked is a +// Conflict. +func (s *Service) GetStateMapping(ctx context.Context, userID, repoID uuid.UUID) (StateMappingView, error) { + src, err := s.linkedSource(ctx, userID, repoID) + if err != nil { + return StateMappingView{}, err + } + states, err := s.ListStates(ctx, userID, uuid.UUID(src.LinkedConnectionID.Bytes), src.IssueTrackerProjectID.String) + if err != nil { + return StateMappingView{}, err + } + view := StateMappingView{States: make([]StateInfo, 0, len(states))} + for _, st := range states { + view.States = append(view.States, StateInfo{ + Name: st.Name, Resolved: st.Resolved, Archived: st.Archived, + Closable: issuetracker.ClosableState(st), Openable: issuetracker.OpenableState(st), + }) + } + if src.OpenState.Valid && src.ClosedState.Valid { + m := StateMapping{OpenState: src.OpenState.String, ClosedState: src.ClosedState.String} + var cols map[string]string + if len(src.ColumnStates) > 0 { + if err := json.Unmarshal(src.ColumnStates, &cols); err != nil { + s.log.Error("decode stored column states", "error", err) + return StateMappingView{}, internal() + } + } + for label, state := range cols { + m.Columns = append(m.Columns, ColumnState{LabelName: label, State: state}) + } + slices.SortFunc(m.Columns, func(a, b ColumnState) int { return strings.Compare(a.LabelName, b.LabelName) }) + view.Mapping = &m + } + return view, nil +} + +// SetStateMapping replaces the whole state mapping of a linked repo (PRD #4173 M5). +// +// Every name must be a State of the repo's linked project, by exact name: the open state +// and every column state must be unresolved and not archived, the closed state must be one +// that means done (issuetracker.ClosableState, so never "Won't fix"), and no two mapped +// States may be equal. Every column label must be a current board column of the repo. The +// States are read from the issue tracker before the transaction; the transaction takes the +// repo row lock that a board column reconfigure of a linked repo also takes, then reads the +// board columns and replaces the mapping, so the mapping never lands against columns that a +// concurrent reconfigure is deleting. +func (s *Service) SetStateMapping(ctx context.Context, userID, repoID uuid.UUID, in StateMapping) (StateMapping, error) { + if err := validateMappingShape(in); err != nil { + return StateMapping{}, err + } + src, err := s.linkedSource(ctx, userID, repoID) + if err != nil { + return StateMapping{}, err + } + states, err := s.ListStates(ctx, userID, uuid.UUID(src.LinkedConnectionID.Bytes), src.IssueTrackerProjectID.String) + if err != nil { + return StateMapping{}, err + } + if err := checkMappingStates(in, states); err != nil { + return StateMapping{}, err + } + + err = s.inTx(ctx, func(q Querier) error { + lock, err := q.LockRepoForStateMappingForUser(ctx, store.LockRepoForStateMappingForUserParams{ID: repoID, UserID: userID}) + if errors.Is(err, pgx.ErrNoRows) { + return &Error{Kind: KindNotFound, Message: "repo not found"} + } + if err != nil { + return err + } + if !lock.IssueTrackerConnectionID.Valid { + return conflict(stateMappingNoLink) + } + cols, err := q.ListBoardColumns(ctx, repoID) + if err != nil { + return err + } + have := make(map[string]bool, len(cols)) + for _, c := range cols { + have[c.LabelName] = true + } + for _, c := range in.Columns { + if !have[c.LabelName] { + return &Error{Kind: KindBadRequest, Message: fmt.Sprintf("%q is not a board column of this repo", c.LabelName)} + } + } + _, err = q.UpsertRepoIssueTrackerStates(ctx, store.UpsertRepoIssueTrackerStatesParams{ + OpenState: in.OpenState, ClosedState: in.ClosedState, RepoID: repoID, UserID: userID, + }) + if errors.Is(err, pgx.ErrNoRows) { + return conflict(stateMappingNoLink) + } + if err != nil { + return err + } + if err := q.DeleteRepoIssueTrackerColumnStates(ctx, repoID); err != nil { + return err + } + for _, c := range in.Columns { + if err := q.InsertRepoIssueTrackerColumnState(ctx, store.InsertRepoIssueTrackerColumnStateParams{ + RepoID: repoID, LabelName: c.LabelName, StateName: c.State, + }); err != nil { + return err + } + } + return nil + }) + if err != nil { + return StateMapping{}, s.txError("save issue tracker state mapping", err) + } + return in, nil +} + +// validStateName holds a State name to what migration 00468 checks (1 to 200 characters, no +// control character) and to the rule ListStates applies (issuetracker.StateNameProblem: no +// invisible, blank-filler or variation-selector character). A brace is allowed: a State is +// written by id and never goes into a query. +func validStateName(name string) bool { + n := utf8.RuneCountInString(name) + return n >= 1 && n <= maxStateNameLen && !issuetracker.StateNameProblem(name) +} + +// validColumnLabel holds a board column label to the rule a linked repo's ConfigureColumns +// applies (issuetracker.LabelNameProblem: at most 200 bytes, no brace, no invisible, +// blank-filler or variation-selector character), which is stricter than the 1 to 200 +// character CHECK of migration 00468. +func validColumnLabel(name string) bool { return !issuetracker.LabelNameProblem(name) } + +// validateMappingShape is the part of the validation that needs no issue tracker read: +// name bounds, the column cap, and distinct column labels and distinct States. +func validateMappingShape(in StateMapping) *Error { + bad := func(msg string) *Error { return &Error{Kind: KindBadRequest, Message: msg} } + if len(in.Columns) > maxMappedColumns { + return bad(fmt.Sprintf("too many mapped columns (max %d)", maxMappedColumns)) + } + if !validStateName(in.OpenState) || !validStateName(in.ClosedState) { + return bad(stateMappingBadName) + } + labels := map[string]bool{} + used := map[string]bool{in.OpenState: true} + if used[in.ClosedState] { + return bad("the open and the closed state must be different states") + } + used[in.ClosedState] = true + for _, c := range in.Columns { + if !validColumnLabel(c.LabelName) || !validStateName(c.State) { + return bad(stateMappingBadName) + } + if labels[c.LabelName] { + return bad("a column is mapped more than once") + } + labels[c.LabelName] = true + if used[c.State] { + return bad(fmt.Sprintf("the state %q is mapped more than once; the open state, the closed state and every column need different states", c.State)) + } + used[c.State] = true + } + return nil +} + +// checkMappingStates checks the mapping against the project's States as the issue tracker +// lists them. +func checkMappingStates(in StateMapping, states []issuetracker.State) *Error { + byName := make(map[string]issuetracker.State, len(states)) + for _, st := range states { + byName[st.Name] = st + } + bad := func(msg string, args ...any) *Error { + return &Error{Kind: KindBadRequest, Message: fmt.Sprintf(msg, args...)} + } + find := func(name string) (issuetracker.State, *Error) { + st, ok := byName[name] + if !ok { + return st, bad("the state %q is not a state of the linked project", name) + } + return st, nil + } + open, err := find(in.OpenState) + if err != nil { + return err + } + if !issuetracker.OpenableState(open) { + return bad("the state %q cannot be the open state: it must be an unresolved, unarchived state", in.OpenState) + } + closed, err := find(in.ClosedState) + if err != nil { + return err + } + if !issuetracker.ClosableState(closed) { + return bad("the state %q cannot be the closed state: it must be a resolved state that means done (for example Fixed or Done), never one such as Won't fix", in.ClosedState) + } + for _, c := range in.Columns { + st, err := find(c.State) + if err != nil { + return err + } + if !issuetracker.OpenableState(st) { + return bad("the state %q cannot be a column state: it must be an unresolved, unarchived state", c.State) + } + } + return nil +} diff --git a/api/internal/issuetrackersvc/state_mapping_fake_test.go b/api/internal/issuetrackersvc/state_mapping_fake_test.go new file mode 100644 index 0000000000..7248465932 --- /dev/null +++ b/api/internal/issuetrackersvc/state_mapping_fake_test.go @@ -0,0 +1,82 @@ +package issuetrackersvc + +import ( + "context" + "encoding/json" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5" + "github.com/jackc/pgx/v5/pgtype" + + "github.com/vtmocanu/uzi/api/internal/store" +) + +func columnStatesJSON(m map[string]string) []byte { + if m == nil { + m = map[string]string{} + } + b, _ := json.Marshal(m) + return b +} + +// setBoardColumns replaces the fake repo's board columns. +func (f *fakeQuerier) setBoardColumns(repo uuid.UUID, labels ...string) { + f.mu.Lock() + defer f.mu.Unlock() + f.repos[repo].boardColumns = labels +} + +func (f *fakeQuerier) LockRepoForStateMappingForUser(_ context.Context, a store.LockRepoForStateMappingForUserParams) (store.LockRepoForStateMappingForUserRow, error) { + f.mu.Lock() + defer f.mu.Unlock() + r, err := f.ownedRepo(a.ID, a.UserID) + if err != nil { + return store.LockRepoForStateMappingForUserRow{}, err + } + return store.LockRepoForStateMappingForUserRow{ + ID: a.ID, + IssueTrackerConnectionID: pgtype.UUID{Bytes: r.linkedConn, Valid: r.linkedConn != uuid.Nil}, + IssueTrackerProjectID: pgtype.Text{String: r.projectID, Valid: r.linkedConn != uuid.Nil}, + }, nil +} + +func (f *fakeQuerier) ListBoardColumns(_ context.Context, repoID uuid.UUID) ([]store.BoardColumn, error) { + f.mu.Lock() + defer f.mu.Unlock() + var out []store.BoardColumn + if r := f.repos[repoID]; r != nil { + for i, l := range r.boardColumns { + out = append(out, store.BoardColumn{RepoID: repoID, LabelName: l, Position: int32(i)}) + } + } + return out, nil +} + +func (f *fakeQuerier) UpsertRepoIssueTrackerStates(_ context.Context, a store.UpsertRepoIssueTrackerStatesParams) (store.RepoIssueTrackerState, error) { + f.mu.Lock() + defer f.mu.Unlock() + r, err := f.ownedRepo(a.RepoID, a.UserID) + if err != nil || r.linkedConn == uuid.Nil { + return store.RepoIssueTrackerState{}, pgx.ErrNoRows + } + r.openState, r.closedState, r.hasStates = a.OpenState, a.ClosedState, true + return store.RepoIssueTrackerState{RepoID: a.RepoID, OpenState: a.OpenState, ClosedState: a.ClosedState}, nil +} + +func (f *fakeQuerier) DeleteRepoIssueTrackerColumnStates(_ context.Context, repoID uuid.UUID) error { + f.mu.Lock() + defer f.mu.Unlock() + f.repos[repoID].columnStates = nil + return nil +} + +func (f *fakeQuerier) InsertRepoIssueTrackerColumnState(_ context.Context, a store.InsertRepoIssueTrackerColumnStateParams) error { + f.mu.Lock() + defer f.mu.Unlock() + r := f.repos[a.RepoID] + if r.columnStates == nil { + r.columnStates = map[string]string{} + } + r.columnStates[a.LabelName] = a.StateName + return nil +} diff --git a/api/internal/issuetrackersvc/state_mapping_test.go b/api/internal/issuetrackersvc/state_mapping_test.go new file mode 100644 index 0000000000..d8eb9d42cc --- /dev/null +++ b/api/internal/issuetrackersvc/state_mapping_test.go @@ -0,0 +1,193 @@ +package issuetrackersvc + +import ( + "context" + "errors" + "strings" + "testing" + + "github.com/google/uuid" + + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" +) + +// linkedMappingRepo links a fresh repo to the fake project, gives it the board columns +// and seeds the States the mapping tests use. +func linkedMappingRepo(t *testing.T, h *ytHarness) uuid.UUID { + t.Helper() + repo := h.q.addRepo(h.user, true) + if err := h.link(repo, youtracktest.EligibilityTagID); err != nil { + t.Fatal(err) + } + h.q.setBoardColumns(repo, "In Progress", "Review") + h.srv.SeedState("Doing", false) + h.srv.SeedState("In Review", false) + h.srv.SeedState("Shipped", true) + return repo +} + +func wantMappingErr(t *testing.T, err error, kind Kind, substr string) { + t.Helper() + var e *Error + if !errors.As(err, &e) { + t.Fatalf("err = %v, want a typed *Error", err) + } + if e.Kind != kind || !strings.Contains(e.Message, substr) { + t.Fatalf("err = {%v %q}, want kind %v containing %q", e.Kind, e.Message, kind, substr) + } +} + +func validMapping() StateMapping { + return StateMapping{ + OpenState: "Open", ClosedState: "Fixed", + Columns: []ColumnState{{LabelName: "In Progress", State: "Doing"}, {LabelName: "Review", State: "In Review"}}, + } +} + +func TestSetStateMappingSavesAndReadsBack(t *testing.T) { + h := newYTHarness(t) + repo := linkedMappingRepo(t, h) + ctx := context.Background() + + view, err := h.svc.GetStateMapping(ctx, h.user, repo) + if err != nil { + t.Fatal(err) + } + if view.Mapping != nil { + t.Fatalf("mapping before any save = %+v, want nil", view.Mapping) + } + byName := map[string]StateInfo{} + for _, s := range view.States { + byName[s.Name] = s + } + if s := byName["Fixed"]; !s.Closable || s.Openable { + t.Errorf("Fixed = %+v, want closable and not openable", s) + } + if s := byName["Won't fix"]; s.Closable || s.Openable || !s.Resolved { + t.Errorf("Won't fix = %+v, want resolved, neither closable nor openable", s) + } + if s := byName["Open"]; s.Closable || !s.Openable { + t.Errorf("Open = %+v, want openable and not closable", s) + } + + if _, err := h.svc.SetStateMapping(ctx, h.user, repo, validMapping()); err != nil { + t.Fatal(err) + } + view, err = h.svc.GetStateMapping(ctx, h.user, repo) + if err != nil { + t.Fatal(err) + } + m := view.Mapping + if m == nil || m.OpenState != "Open" || m.ClosedState != "Fixed" || len(m.Columns) != 2 || + m.Columns[0] != (ColumnState{"In Progress", "Doing"}) || m.Columns[1] != (ColumnState{"Review", "In Review"}) { + t.Fatalf("mapping = %+v", m) + } + + // A second save replaces the whole mapping, columns included. + next := StateMapping{OpenState: "Doing", ClosedState: "Shipped", Columns: []ColumnState{{LabelName: "Review", State: "In Review"}}} + if _, err := h.svc.SetStateMapping(ctx, h.user, repo, next); err != nil { + t.Fatal(err) + } + view, _ = h.svc.GetStateMapping(ctx, h.user, repo) + if m := view.Mapping; m == nil || m.OpenState != "Doing" || m.ClosedState != "Shipped" || len(m.Columns) != 1 || m.Columns[0].LabelName != "Review" { + t.Fatalf("replaced mapping = %+v", m) + } +} + +func TestSetStateMappingRefusals(t *testing.T) { + h := newYTHarness(t) + repo := linkedMappingRepo(t, h) + h.srv.SeedState("Archived", false) + h.srv.SetStateArchived("Archived", true) + ctx := context.Background() + + edit := func(f func(*StateMapping)) StateMapping { + m := validMapping() + f(&m) + return m + } + cases := []struct { + name string + in StateMapping + kind Kind + msg string + }{ + {"unknown open state", edit(func(m *StateMapping) { m.OpenState = "Nope" }), KindBadRequest, `"Nope"`}, + {"unknown column state", edit(func(m *StateMapping) { m.Columns[0].State = "Nope" }), KindBadRequest, "not a state"}, + {"resolved column state", edit(func(m *StateMapping) { m.Columns[0].State = "Shipped" }), KindBadRequest, "column state"}, + {"resolved open state", edit(func(m *StateMapping) { m.OpenState = "Shipped" }), KindBadRequest, "open state"}, + {"won't fix closed", edit(func(m *StateMapping) { m.ClosedState = "Won't fix" }), KindBadRequest, "Won't fix"}, + {"unresolved closed", edit(func(m *StateMapping) { m.ClosedState = "Doing" }), KindBadRequest, "closed state"}, + {"archived column state", edit(func(m *StateMapping) { m.Columns[0].State = "Archived" }), KindBadRequest, "column state"}, + {"label not a column", edit(func(m *StateMapping) { m.Columns[0].LabelName = "Backlog" }), KindBadRequest, "not a board column"}, + {"open equals closed", edit(func(m *StateMapping) { m.ClosedState = "Open" }), KindBadRequest, "different"}, + {"column repeats open", edit(func(m *StateMapping) { m.Columns[0].State = "Open" }), KindBadRequest, "more than once"}, + {"two columns, one state", edit(func(m *StateMapping) { m.Columns[1].State = "Doing" }), KindBadRequest, "more than once"}, + {"duplicate label", edit(func(m *StateMapping) { m.Columns[1].LabelName = "In Progress" }), KindBadRequest, "more than once"}, + {"empty name", edit(func(m *StateMapping) { m.OpenState = "" }), KindBadRequest, "empty"}, + {"control character", edit(func(m *StateMapping) { m.Columns[0].LabelName = "a\nb" }), KindBadRequest, "control"}, + {"zero-width label", edit(func(m *StateMapping) { m.Columns[0].LabelName = "In\u200bProgress" }), KindBadRequest, "control"}, + {"bidi state", edit(func(m *StateMapping) { m.OpenState = "Op\u202een" }), KindBadRequest, "control"}, + {"filler label", edit(func(m *StateMapping) { m.Columns[0].LabelName = "Review\u3164" }), KindBadRequest, "blank-filler"}, + {"variation selector state", edit(func(m *StateMapping) { m.OpenState = "Open\uFE0F" }), KindBadRequest, "variation-selector"}, + {"too long", edit(func(m *StateMapping) { m.ClosedState = strings.Repeat("x", 201) }), KindBadRequest, "200"}, + {"too many columns", edit(func(m *StateMapping) { m.Columns = make([]ColumnState, 11) }), KindBadRequest, "too many"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + _, err := h.svc.SetStateMapping(ctx, h.user, repo, tc.in) + wantMappingErr(t, err, tc.kind, tc.msg) + view, _ := h.svc.GetStateMapping(ctx, h.user, repo) + if view.Mapping != nil { + t.Errorf("a refused save stored %+v", view.Mapping) + } + }) + } + // A long name is never echoed whole. + _, err := h.svc.SetStateMapping(ctx, h.user, repo, edit(func(m *StateMapping) { m.OpenState = strings.Repeat("y", 150) + "\u202e" })) + var e *Error + if !errors.As(err, &e) || strings.ContainsRune(e.Message, '\u202e') { + t.Errorf("message %v must not carry the raw bidi character", err) + } +} + +func TestStateMappingScopeAndLink(t *testing.T) { + h := newYTHarness(t) + repo := linkedMappingRepo(t, h) + unlinked := h.q.addRepo(h.user, true) + ctx := context.Background() + + _, err := h.svc.SetStateMapping(ctx, uuid.New(), repo, validMapping()) + wantMappingErr(t, err, KindNotFound, "repo not found") + _, err = h.svc.GetStateMapping(ctx, uuid.New(), repo) + wantMappingErr(t, err, KindNotFound, "repo not found") + _, err = h.svc.SetStateMapping(ctx, h.user, unlinked, validMapping()) + wantMappingErr(t, err, KindConflict, "not linked") + _, err = h.svc.GetStateMapping(ctx, h.user, unlinked) + wantMappingErr(t, err, KindConflict, "not linked") +} + +// A brace is refused in a column label (it is a tag-name rule) but not in a State name: +// a State is written by id and never goes into a query, and ListStates lists it. +func TestSetStateMappingAcceptsABracedState(t *testing.T) { + h := newYTHarness(t) + repo := linkedMappingRepo(t, h) + h.srv.SeedState("In {Review}", false) + m := validMapping() + m.Columns[1].State = "In {Review}" + if _, err := h.svc.SetStateMapping(context.Background(), h.user, repo, m); err != nil { + t.Fatalf("SetStateMapping with a braced State: %v", err) + } +} + +// A brace in a column label is refused even when the label is a board column (one set +// before the repo was linked): the label rule is the driver's tag-name rule. +func TestSetStateMappingRefusesABracedColumnLabel(t *testing.T) { + h := newYTHarness(t) + repo := linkedMappingRepo(t, h) + h.q.setBoardColumns(repo, "In {Progress}", "Review") + m := validMapping() + m.Columns[0].LabelName = "In {Progress}" + _, err := h.svc.SetStateMapping(context.Background(), h.user, repo, m) + wantMappingErr(t, err, KindBadRequest, "brace") +} diff --git a/api/internal/poller/linked_comments_livedb_test.go b/api/internal/poller/linked_comments_livedb_test.go new file mode 100644 index 0000000000..065c467a9f --- /dev/null +++ b/api/internal/poller/linked_comments_livedb_test.go @@ -0,0 +1,260 @@ +package poller + +import ( + "context" + "errors" + "log/slog" + "net/http" + "strconv" + "strings" + "testing" + + "github.com/jackc/pgx/v5/pgtype" + + "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// PRD #4173 M5 U6: the poller's issue comments (CI autofix start and halt, MR rework +// halt) and the PRD link patch reach the YouTrack fake through the real resolver, the +// write guard and the YouTrack driver. The ledgers and the code host are the unit-test +// fakes; the repo row, the resolver and the issue tracker are live. A YouTrack 500 is not +// something the fake can answer, so that case wraps the live tracker in a failing one. + +const linkedCommentsProject = 42 // the forge_project_id newLinkedEnv gives the linked repo + +// failingNoteTracker is a live tracker whose comment write answers a 500. +type failingNoteTracker struct{ forge.IssueTracker } + +func (failingNoteTracker) CreateIssueNote(context.Context, int64, int64, string) (forge.IssueNote, error) { + return forge.IssueNote{}, errors.New("create issue note: youtrack answered 500") +} + +// linkedCommentFixture is a linked repo with one eligible YouTrack issue n. +func linkedCommentFixture(t *testing.T) (env *linkedEnv, srv *youtracktest.Server, tr forge.IssueTracker, n int64) { + t.Helper() + srv = youtracktest.NewServer(t) + trustFake(t, srv) + env = newLinkedEnv(t, srv, nil) + n = srv.SeedIssue("a card", "Implements prds/72-x.md end to end.", youtracktest.EligibilityTagID) + var err error + tr, err = env.svc.IssueTrackerForRepo(env.ctx, env.user, env.repo) + if err != nil { + t.Fatal(err) + } + return env, srv, tr, n +} + +func youtrackComments(t *testing.T, tr forge.IssueTracker, n int64) []string { + t.Helper() + cs, err := tr.ListIssueComments(context.Background(), linkedCommentsProject, n) + if err != nil { + t.Fatal(err) + } + var out []string + for _, c := range cs { + out = append(out, c.Body) + } + return out +} + +func branchFor(n int64) string { return "agent/issue-" + strconv.FormatInt(n, 10) } + +func ciCand(ref string, pipelineID int64) store.ListCIAutofixCandidateRefsRow { + c := cfCand(pipelineID) + c.Ref = pgtype.Text{String: ref, Valid: true} + return c +} + +func TestLinkedCIAutofixStartCommentLandsOnTheYouTrackIssueLiveDB(t *testing.T) { + _, _, tr, n := linkedCommentFixture(t) + ref := branchFor(n) + st := &cfStore{candidates: []store.ListCIAutofixCandidateRefsRow{ciCand(ref, 9001)}} + runs := &cfRuns{} + f := &cfForge{jobs: []forge.Job{cfJob()}, logTail: "panic: boom"} + + newCF(st, runs, &cfNotifier{}).detect(context.Background(), cfRepoRow(), f, tr) + + if len(runs.calls) != 1 { + t.Fatalf("runs started = %d, want 1", len(runs.calls)) + } + got := youtrackComments(t, tr, n) + if len(got) != 1 || got[0] != startCommentBody(ref, cfCand(9001).PipelineWebUrl) { + t.Fatalf("YouTrack comments = %q, want the start comment", got) + } + if len(f.notes) != 0 { + t.Fatalf("the code host received an issue note: %+v", f.notes) + } +} + +func TestLinkedCIAutofixHaltCommentLandsOnTheYouTrackIssueLiveDB(t *testing.T) { + _, _, tr, n := linkedCommentFixture(t) + ref := branchFor(n) + st := &cfStore{ + candidates: []store.ListCIAutofixCandidateRefsRow{ciCand(ref, 9010)}, + attempts: map[string]store.CiAutofixAttempt{ + ref: {Ref: ref, AttemptCount: 2, LastPipelineID: pgtype.Int8{Int64: 9001, Valid: true}}, + }, + } + runs := &cfRuns{} + f := &cfForge{jobs: []forge.Job{cfJob()}, logTail: "boom"} + + newCF(st, runs, &cfNotifier{}).detect(context.Background(), cfRepoRow(), f, tr) + + if len(runs.calls) != 0 || len(st.haltSets) != 1 { + t.Fatalf("runs=%d halt latches=%d, want a halt", len(runs.calls), len(st.haltSets)) + } + got := youtrackComments(t, tr, n) + want := haltCommentBody(haltCap, 2, cfCand(9010).MrIid) + if len(got) != 1 || got[0] != want { + t.Fatalf("YouTrack comments = %q, want the halt comment", got) + } +} + +// A moved-away issue: the comment is not written, the failure is logged, the run was +// already started and the halt latch is set. +func TestLinkedCIAutofixCommentsOnAMovedAwayIssueAreLoggedAndTheRunContinuesLiveDB(t *testing.T) { + _, srv, tr, n := linkedCommentFixture(t) + ref := branchFor(n) + srv.MoveIssue(n, youtracktest.OtherProjectID) + logs := captureLogs(t, slog.LevelDebug) + + st := &cfStore{candidates: []store.ListCIAutofixCandidateRefsRow{ciCand(ref, 9001)}} + runs := &cfRuns{} + f := &cfForge{jobs: []forge.Job{cfJob()}, logTail: "panic: boom"} + newCF(st, runs, &cfNotifier{}).detect(context.Background(), cfRepoRow(), f, tr) + if len(runs.calls) != 1 || len(st.upserts) != 1 { + t.Fatalf("the run must start whatever the comment does: runs=%d upserts=%d", len(runs.calls), len(st.upserts)) + } + if !strings.Contains(logs.String(), "ci-autofix start comment") { + t.Fatalf("the failed start comment was not logged: %s", logs.String()) + } + + halt := &cfStore{ + candidates: []store.ListCIAutofixCandidateRefsRow{ciCand(ref, 9010)}, + attempts: map[string]store.CiAutofixAttempt{ref: {Ref: ref, AttemptCount: 2}}, + } + notifier := &cfNotifier{} + newCF(halt, &cfRuns{}, notifier).detect(context.Background(), cfRepoRow(), f, tr) + if len(halt.haltSets) != 1 || len(notifier.calls) != 1 { + t.Fatalf("the halt must complete: latches=%d notifications=%d", len(halt.haltSets), len(notifier.calls)) + } + if !strings.Contains(logs.String(), "ci-autofix halt comment") { + t.Fatalf("the failed halt comment was not logged: %s", logs.String()) + } +} + +// A failed comment write (a stub standing in for the tracker; youtracktest cannot +// answer a 500) is logged and the run continues. +func TestLinkedCIAutofixCommentFailureIsLoggedAndTheRunContinuesLiveDB(t *testing.T) { + _, _, tr, n := linkedCommentFixture(t) + logs := captureLogs(t, slog.LevelDebug) + st := &cfStore{candidates: []store.ListCIAutofixCandidateRefsRow{ciCand(branchFor(n), 9001)}} + runs := &cfRuns{} + f := &cfForge{jobs: []forge.Job{cfJob()}, logTail: "panic: boom"} + + newCF(st, runs, &cfNotifier{}).detect(context.Background(), cfRepoRow(), f, failingNoteTracker{tr}) + + if len(runs.calls) != 1 || len(st.upserts) != 1 { + t.Fatalf("runs=%d upserts=%d, want the run started", len(runs.calls), len(st.upserts)) + } + if !strings.Contains(logs.String(), "ci-autofix start comment") || !strings.Contains(logs.String(), "500") { + t.Fatalf("the 500 was not logged: %s", logs.String()) + } +} + +func mrReworkCap(ref string) *mrwStore { + cand := mrwCand("success") + cand.Ref = pgtype.Text{String: ref, Valid: true} + return &mrwStore{ + candidates: []store.ListMRReworkCandidatesRow{cand}, + ledgers: map[string]store.GetMRReworkLedgerRow{ref: mrwRow(ref, 5, 120)}, + } +} + +func TestLinkedMRReworkHaltCommentLandsOnTheYouTrackIssueLiveDB(t *testing.T) { + _, _, tr, n := linkedCommentFixture(t) + st := mrReworkCap(branchFor(n)) + runs := &mrwRuns{} + f := landedForge(mrwComment(200, landed(), mrwHeadSHA)) + + newMRW(st, runs, &mrwNotifier{}, mrwSettings{enabled: true, capVal: 5}).detect(context.Background(), mrwRepoRow(), f, tr) + + if len(runs.calls) != 0 || len(st.haltSets) != 1 { + t.Fatalf("runs=%d latches=%d, want a halt", len(runs.calls), len(st.haltSets)) + } + got := youtrackComments(t, tr, n) + if len(got) != 1 || got[0] != mrReworkHaltCommentBody(5, mrwMrIID) { + t.Fatalf("YouTrack comments = %q, want the rework halt comment", got) + } + if len(f.notes) != 0 { + t.Fatalf("the code host received an issue note: %+v", f.notes) + } +} + +func TestLinkedMRReworkHaltCommentOnAMovedAwayIssueIsLoggedAndTheHaltContinuesLiveDB(t *testing.T) { + _, srv, tr, n := linkedCommentFixture(t) + srv.MoveIssue(n, youtracktest.OtherProjectID) + logs := captureLogs(t, slog.LevelDebug) + st := mrReworkCap(branchFor(n)) + notifier := &mrwNotifier{} + f := landedForge(mrwComment(200, landed(), mrwHeadSHA)) + + newMRW(st, &mrwRuns{}, notifier, mrwSettings{enabled: true, capVal: 5}).detect(context.Background(), mrwRepoRow(), f, tr) + + if len(st.haltSets) != 1 || len(notifier.calls) != 1 { + t.Fatalf("the halt must complete: latches=%d notifications=%d", len(st.haltSets), len(notifier.calls)) + } + if !strings.Contains(logs.String(), "mr-rework halt comment") { + t.Fatalf("the failed comment was not logged: %s", logs.String()) + } +} + +// A merged MR's moved PRD path patches the YouTrack description, and the marker settles. +func TestLinkedPRDLinkPatchWritesTheYouTrackDescriptionLiveDB(t *testing.T) { + env, _, tr, n := linkedCommentFixture(t) + if _, err := env.pool.Exec(env.ctx, `UPDATE runs SET issue_iid = $1 WHERE issue_iid = 8 AND repo_id = $2`, n, env.repo); err != nil { + t.Fatal(err) + } + + if err := env.svc.SyncPRDLinkPatches(env.ctx, env.repo, linkedCommentsProject, env.rec, tr); err != nil { + t.Fatal(err) + } + + iss, err := tr.GetIssue(env.ctx, linkedCommentsProject, n) + if err != nil { + t.Fatal(err) + } + if iss.Description != "Implements prds/done/72-x.md end to end." { + t.Fatalf("YouTrack description = %q, want the patched path", iss.Description) + } + var settled pgtype.Timestamptz + if err := env.pool.QueryRow(env.ctx, `SELECT prd_patch_settled_at FROM runs WHERE issue_iid = $1 AND repo_id = $2`, n, env.repo).Scan(&settled); err != nil || !settled.Valid { + t.Fatalf("the marker was not settled: %+v, %v", settled, err) + } +} + +// The issue moved away: the patch reads first, settles on the read, and never writes. +func TestLinkedPRDLinkPatchSettlesOnAMovedAwayIssueLiveDB(t *testing.T) { + env, srv, tr, n := linkedCommentFixture(t) + if _, err := env.pool.Exec(env.ctx, `UPDATE runs SET issue_iid = $1 WHERE issue_iid = 8 AND repo_id = $2`, n, env.repo); err != nil { + t.Fatal(err) + } + srv.MoveIssue(n, youtracktest.OtherProjectID) + srv.ResetRequests() + + if err := env.svc.SyncPRDLinkPatches(env.ctx, env.repo, linkedCommentsProject, env.rec, tr); err != nil { + t.Fatal(err) + } + + for _, r := range srv.Requests() { + if r.Method != http.MethodGet { + t.Fatalf("a write reached YouTrack for a moved issue: %+v", r) + } + } + var settled pgtype.Timestamptz + if err := env.pool.QueryRow(env.ctx, `SELECT prd_patch_settled_at FROM runs WHERE issue_iid = $1 AND repo_id = $2`, n, env.repo).Scan(&settled); err != nil || !settled.Valid { + t.Fatalf("the marker was not settled on the moved-away issue: %+v, %v", settled, err) + } +} diff --git a/api/internal/poller/linked_repo_livedb_test.go b/api/internal/poller/linked_repo_livedb_test.go index 4eb9b9cc95..81d625fb03 100644 --- a/api/internal/poller/linked_repo_livedb_test.go +++ b/api/internal/poller/linked_repo_livedb_test.go @@ -46,7 +46,7 @@ func trustFake(t *testing.T, srv *youtracktest.Server) { type refusingDrivers struct{ calls int } -func (r *refusingDrivers) DriverFor(context.Context, uuid.UUID, uuid.UUID, issuetracker.Binding, string, string, string) (issuetracker.Driver, error) { +func (r *refusingDrivers) DriverFor(context.Context, uuid.UUID, uuid.UUID, issuetracker.Binding, issuetracker.StateMapping) (issuetracker.Driver, error) { r.calls++ return nil, errors.New("base URL is not in the allowed issue tracker list") } @@ -441,3 +441,53 @@ func TestLinkedRepoPollCachesTaggedIssuesLiveDB(t *testing.T) { t.Fatalf("issue_key = %v, want %s", key, want) } } + +// The poller hands a linked repo's issue side to the board-free MR-state recorder: an +// issue run whose MR closed moves its cached Human Review card to In Progress through the +// issue tracker. +func TestLinkedRepoPollMovesTheCardWhenItsMRClosesLiveDB(t *testing.T) { + srv := youtracktest.NewServer(t) + trustFake(t, srv) + srv.SeedState("In Progress", false) + srv.SeedState("In Review", false) + env := newLinkedEnv(t, srv, nil) + exec := func(sql string, args ...any) { + t.Helper() + if _, err := env.pool.Exec(env.ctx, sql, args...); err != nil { + t.Fatal(err) + } + } + exec(`INSERT INTO repo_issue_tracker_states (repo_id, open_state, closed_state) VALUES ($1, 'Open', 'Fixed')`, env.repo) + exec(`INSERT INTO repo_issue_tracker_column_states (repo_id, label_name, state_name) VALUES ($1, 'Human Review', 'In Review'), ($1, 'In Progress', 'In Progress')`, env.repo) + + n := srv.SeedIssue("a card", "body", youtracktest.EligibilityTagID) + tr, err := env.svc.IssueTrackerForRepo(env.ctx, env.user, env.repo) + if err != nil { + t.Fatal(err) + } + if err := tr.MoveToColumn(env.ctx, 42, n, forge.ColumnMove{Column: "Human Review"}); err != nil { + t.Fatal(err) + } + run := uuid.New() + exec("INSERT INTO runs(id,user_id,repo_id,kind,issue_iid,issue_title,issue_description,branch,mr_iid,mr_state,status,created_at) VALUES($1,$2,$3,'issue',$4,'t','d',$5,13,'opened','completed','2020-01-01')", + run, env.user, env.repo, n, "agent/issue-"+run.String()) + + env.poll(t) + if got := srv.IssueState(n); got != "In Progress" { + t.Fatalf("State = %q, want In Progress after the MR closed", got) + } + var column *string + if err := env.pool.QueryRow(env.ctx, `SELECT issue_tracker_column FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2`, env.repo, n).Scan(&column); err != nil { + t.Fatal(err) + } + if column == nil || *column != "In Progress" { + t.Fatalf("cached column = %v, want In Progress", column) + } + var state *string + if err := env.pool.QueryRow(env.ctx, `SELECT mr_state FROM runs WHERE id = $1`, run).Scan(&state); err != nil { + t.Fatal(err) + } + if state == nil || *state != forge.MRStateClosed { + t.Fatalf("mr_state = %v, want closed", state) + } +} diff --git a/api/internal/poller/poller.go b/api/internal/poller/poller.go index 406f758c7e..c00287d7ab 100644 --- a/api/internal/poller/poller.go +++ b/api/internal/poller/poller.go @@ -715,9 +715,11 @@ func (e *Engine) syncRepo(ctx context.Context, r store.ListEnabledReposWithConne // mr_rework detector reading candidates later this same tick. Reads live forge state // (needs the forge reachable: guaranteed by the early returns above on an unlinked repo, // and by the forge client having been built on a linked one); records mr_state - // and cancels an in-flight rework on close/merge, but moves NO board card. Per-candidate + // and cancels an in-flight rework on close/merge. It moves NO board card, except that on + // a linked repo an issue-lane run's MR close/reopen moves the card through the issue + // tracker (PRD #4173 M5), which is why it takes the issue side. Per-candidate // errors are log-and-skipped inside; only a candidate-enumeration failure surfaces here. - if err := e.svc.SyncBoardFreeMRStates(ctx, r.ID, r.ForgeProjectID, f); err != nil { + if err := e.svc.SyncBoardFreeMRStatesWithIssueSide(ctx, r.ID, r.ForgeProjectID, f, tr); err != nil { slog.Error("poller: sync board-free MR states", "repo", r.PathWithNamespace, "error", err) } diff --git a/api/internal/provisionsvc/provisionsvc.go b/api/internal/provisionsvc/provisionsvc.go index 2b54b1194c..22f9cee894 100644 --- a/api/internal/provisionsvc/provisionsvc.go +++ b/api/internal/provisionsvc/provisionsvc.go @@ -382,8 +382,8 @@ func (s *Service) EnableRepo(ctx context.Context, userID, repoID uuid.UUID) (sto return removedLinkedRepoError() } // PRD #4173 D3: board columns are labels, on the issue-tracker half. A repo linked to an - // external issue tracker records the column rows only: uzi creates no labels or tags - // there (writes arrive in M5), and the forge's project is not where its issues live. + // external issue tracker records the column rows only: uzi creates no column labels or + // tags there (a column is not a tag, D9), and the forge's project is not where its issues live. // The choice reads the locked row, not the one read before the guard. if lock.IssueTrackerConnectionID.Valid { if err := forgesvc.EnsureBoardColumnRows(ctx, q, row.ID); err != nil { diff --git a/api/internal/runlifecycle/lifecycle.go b/api/internal/runlifecycle/lifecycle.go index 6a3b804261..8de6993c49 100644 --- a/api/internal/runlifecycle/lifecycle.go +++ b/api/internal/runlifecycle/lifecycle.go @@ -23,6 +23,7 @@ import ( "errors" "fmt" "log/slog" + "slices" "strings" "sync" "time" @@ -162,7 +163,8 @@ func (l *Lifecycle) Interval() time.Duration { return l.interval } type moveContext struct { repoID uuid.UUID userID uuid.UUID - linked bool // the repo's issue_tracker_connection_id is set (PRD #4173) + linked bool // the repo's issue_tracker_connection_id is set (PRD #4173) + eligibilityName string // the link's eligibility tag name; "" on an unlinked repo issueIID int64 forgeProjectID int64 forgeType string @@ -323,6 +325,7 @@ func contextFromRow(mc store.GetRunMoveContextRow) moveContext { repoID: uuid.UUID(mc.RepoID.Bytes), userID: mc.UserID, linked: mc.IssueTrackerConnectionID.Valid, + eligibilityName: mc.EligibilityTagName.String, issueIID: mc.IssueIid.Int64, // valid: notifyOnce and reconcileOne skip card-less runs (NULL issue_iid) forgeProjectID: mc.ForgeProjectID, forgeType: mc.ForgeType, @@ -340,7 +343,12 @@ func contextFromRow(mc store.GetRunMoveContextRow) moveContext { // - forge move succeeds → record board_column and clear the marker; // - manual drag / closed issue / unknown baseline → clear the marker (a // definitive skip that retrying cannot fix); -// - forge error → leave the marker for a later reconcile retry. +// - an issue a person resolved in the linked issue tracker since the last sync +// (the cache still shows it open) → the driver refuses the write and the marker +// is cleared, the same outcome as the cached closed issue; +// - forge error, or a move the linked issue tracker refused (no State mapped for +// the column) → leave the marker for a later reconcile retry; +// - a linked issue missing from the cache → probeUncachedLinkedIssue decides. func (l *Lifecycle) apply(ctx context.Context, runID uuid.UUID, mc moveContext, dec decision) { if !dec.act { if dec.clear { @@ -354,10 +362,10 @@ func (l *Lifecycle) apply(ctx context.Context, runID uuid.UUID, mc moveContext, if !errors.Is(err, pgx.ErrNoRows) { slog.Warn("run lifecycle: load issue for move", "run", runID, "error", err) } else if mc.linked { - // PRD #4173 D10: a linked issue that moved to another project is evicted from - // the cache; it will never be found again, so the move is a permanent skip. - slog.Info("run lifecycle: linked issue is no longer cached (moved or removed), skipping move", "run", runID, "issue", mc.issueIID) - l.clearPending(ctx, runID) + // The cache has no row for this linked issue. It may be gone (moved to another + // project, D10), never eligible (it will never be cached), or only not cached + // yet; the issue tracker itself says which (PRD #4173 M5). + l.probeUncachedLinkedIssue(ctx, runID, mc) return } // No cached card to move (evicted/de-labeled forge-side): leave the marker; @@ -385,7 +393,8 @@ func (l *Lifecycle) apply(ctx context.Context, runID uuid.UUID, mc moveContext, if err := json.Unmarshal(issue.Labels, &labels); err != nil { labels = []string{} } - current, _, _ := board.ResolveColumn(labels, issue.State, position) + // A linked repo's card sits in its cached State column, never in a label (PRD #4173 M5). + current, _, _ := board.ResolveColumn(board.CardColumnLabels(mc.linked, labels, issue.IssueTrackerColumn.String), issue.State, position) baseline, haveBaseline := coalesceColumn(mc.boardColumn, mc.originColumn) if !haveBaseline { @@ -403,18 +412,8 @@ func (l *Lifecycle) apply(ctx context.Context, runID uuid.UUID, mc moveContext, return } - f, err := l.mover.ForgeForConnection(mc.forgeType, mc.baseURL, mc.tokenCiphertext) - if err != nil { - // Likely an undecryptable token; leave the marker (reconcile will retry, - // then give up — visible drift). - slog.Warn("run lifecycle: build forge client for move", "run", runID, "error", err) - return - } - tr, err := issuesource.For(ctx, l.issueSources, mc.linked, mc.userID, mc.repoID, f) - if err != nil { - // A linked repo with no resolver, or one that cannot be built: never the forge. - // Left for the reconcile loop like any other outage. - slog.Warn("run lifecycle: resolve issue tracker for move", "run", runID, "error", err) + tr, ok := l.issueSide(ctx, runID, mc) + if !ok { return } if _, err := l.mover.AutoMove(ctx, tr, mc.forgeProjectID, issue, cols, dec.target); err != nil { @@ -427,6 +426,22 @@ func (l *Lifecycle) apply(ctx context.Context, runID uuid.UUID, mc moveContext, l.clearPending(ctx, runID) return } + if forgesvc.LinkedIssueResolved(err) { + // A person resolved the issue in the issue tracker and the cache has not caught + // up. Their resolution wins: no State was written, and retrying cannot help + // (same outcome as the cached closed-issue guard above). + slog.Info("run lifecycle: issue move skipped, the issue is resolved in the issue tracker", "run", runID, "target", dec.target, "reason", err) + l.clearPending(ctx, runID) + return + } + if forgesvc.ColumnMoveRefused(err) { + // The issue tracker refused the move before any request: the column has no State + // mapped (or the target State is not allowed). Unlike a skip this can heal when + // the owner saves the mapping, so the marker stays and reconciliation retries + // inside its window (PRD #4173 M5; specs/human.md, failed column moves retry). + slog.Warn("run lifecycle: issue tracker refused the column move, will retry", "run", runID, "target", dec.target, "error", err) + return + } // Forge error: the transition already succeeded; leave the marker so the // reconcile loop retries once the forge is reachable again. slog.Warn("run lifecycle: forge move failed, will retry", "run", runID, "target", dec.target, "error", err) @@ -451,6 +466,58 @@ func (l *Lifecycle) apply(ctx context.Context, runID uuid.UUID, mc moveContext, } } +// issueSide builds the forge client and resolves the repo's issue side for a move or a +// probe. On failure it logs and reports false, leaving the marker for the reconcile +// loop like any other outage. +func (l *Lifecycle) issueSide(ctx context.Context, runID uuid.UUID, mc moveContext) (forge.IssueTracker, bool) { + f, err := l.mover.ForgeForConnection(mc.forgeType, mc.baseURL, mc.tokenCiphertext) + if err != nil { + // Likely an undecryptable token; leave the marker (reconcile will retry, + // then give up — visible drift). + slog.Warn("run lifecycle: build forge client for move", "run", runID, "error", err) + return nil, false + } + tr, err := issuesource.For(ctx, l.issueSources, mc.linked, mc.userID, mc.repoID, f) + if err != nil { + // A linked repo with no resolver, or one that cannot be built: never the forge. + // Left for the reconcile loop like any other outage. + slog.Warn("run lifecycle: resolve issue tracker for move", "run", runID, "error", err) + return nil, false + } + return tr, true +} + +// probeUncachedLinkedIssue settles the pending move of a linked run whose issue is not +// in the cache (PRD #4173 M5). It reads the issue from the linked project: +// - not found (moved away, D10): clear the marker, the move can never happen; +// - found but without the link's eligibility tag: it will never be cached (a +// self-improve tracking issue, for example), so clear the marker; +// - found and eligible, or any read error: leave the marker, so reconciliation moves +// the card once the next sync caches it, as on an unlinked repo. +func (l *Lifecycle) probeUncachedLinkedIssue(ctx context.Context, runID uuid.UUID, mc moveContext) { + tr, ok := l.issueSide(ctx, runID, mc) + if !ok { + return + } + issue, err := tr.GetIssue(ctx, mc.forgeProjectID, mc.issueIID) + if err != nil { + if forgesvc.LinkedIssueGone(err) { + slog.Info("run lifecycle: linked issue is no longer in the linked project, skipping move", "run", runID, "issue", mc.issueIID) + l.clearPending(ctx, runID) + return + } + slog.Warn("run lifecycle: probe uncached linked issue", "run", runID, "issue", mc.issueIID, "error", err) + return + } + if mc.eligibilityName == "" { + return + } + if !slices.Contains(issue.Labels, mc.eligibilityName) { + slog.Info("run lifecycle: linked issue does not carry the eligibility tag and will never be cached, skipping move", "run", runID, "issue", mc.issueIID) + l.clearPending(ctx, runID) + } +} + func (l *Lifecycle) clearPending(ctx context.Context, runID uuid.UUID) { if _, err := l.q.ClearRunMovePending(ctx, runID); err != nil { slog.Warn("run lifecycle: clear pending marker", "run", runID, "error", err) @@ -519,8 +586,8 @@ func (l *Lifecycle) maybeTerminalComment(ctx context.Context, runID uuid.UUID, s return } // Only the issue side of the driver is used here. PRD #4173: a linked repo's - // comment goes through the resolver, whose write guard refuses it (writes arrive - // in M5); like any terminal-comment failure it is logged and the lifecycle continues. + // comment goes through the resolver and lands on the linked issue (M5); like any + // terminal-comment failure it is logged and the lifecycle continues. tracker, err := issuesource.For(ctx, l.issueSources, mc.IssueTrackerConnectionID.Valid, mc.UserID, uuid.UUID(mc.RepoID.Bytes), f) if err != nil { slog.Warn("run lifecycle: resolve issue tracker for terminal comment", "run", runID, "error", err) @@ -530,7 +597,7 @@ func (l *Lifecycle) maybeTerminalComment(ctx context.Context, runID uuid.UUID, s if _, err := tracker.CreateIssueNote(ctx, mc.ForgeProjectID, mc.IssueIid.Int64, body); err != nil { // Already PAT-redacted by the driver. Recorded but lost (never retried). if forgesvc.IssueWriteSkippable(err) { - slog.Info("run lifecycle: autopilot terminal comment skipped on linked repo", "run", runID, "reason", err) + slog.Info("run lifecycle: autopilot terminal comment skipped, the linked issue is gone or moved", "run", runID, "reason", err) return } slog.Warn("run lifecycle: post autopilot terminal comment", "run", runID, "error", err) diff --git a/api/internal/runlifecycle/linked_board_livedb_test.go b/api/internal/runlifecycle/linked_board_livedb_test.go new file mode 100644 index 0000000000..6aa631554c --- /dev/null +++ b/api/internal/runlifecycle/linked_board_livedb_test.go @@ -0,0 +1,369 @@ +package runlifecycle + +import ( + "bytes" + "context" + "crypto/sha256" + "fmt" + "log/slog" + "net/http" + "os" + "strings" + "testing" + "time" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5/pgtype" + "github.com/jackc/pgx/v5/pgxpool" + + "github.com/vtmocanu/uzi/api/internal/board" + "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/forge/forgetest" + "github.com/vtmocanu/uzi/api/internal/forgesvc" + "github.com/vtmocanu/uzi/api/internal/issuetracker" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" + "github.com/vtmocanu/uzi/api/internal/issuetrackersvc" + "github.com/vtmocanu/uzi/api/internal/secretbox" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// PRD #4173 M5: the lifecycle against a repo linked to the YouTrack fake. The card move +// writes the State, the cache keeps the column, a refused move keeps the marker, and a +// run whose issue is not cached is probed. + +// saveStateMapping stores the open and closed States of the repo and the State of +// "Human Review", as the state mapping route would. The fake's bundle holds Open and Fixed +// by default; "In Review" is seeded by the caller. +func saveStateMapping(t *testing.T, pool *pgxpool.Pool, repo uuid.UUID) { + t.Helper() + ctx := context.Background() + if _, err := pool.Exec(ctx, `INSERT INTO repo_issue_tracker_states (repo_id, open_state, closed_state) VALUES ($1, 'Open', 'Fixed')`, repo); err != nil { + t.Fatal(err) + } + if _, err := pool.Exec(ctx, `INSERT INTO repo_issue_tracker_column_states (repo_id, label_name, state_name) VALUES ($1, 'Human Review', 'In Review')`, repo); err != nil { + t.Fatal(err) + } +} + +type linkedLifecycleLive struct { + pool *pgxpool.Pool + q *store.Queries + svc *forgesvc.Service + srv *youtracktest.Server + repo uuid.UUID + user uuid.UUID + worker uuid.UUID +} + +func newLinkedLifecycleLive(t *testing.T) *linkedLifecycleLive { + t.Helper() + dsn := os.Getenv("UZI_TEST_DATABASE_URL") + if dsn == "" { + t.Skip("UZI_TEST_DATABASE_URL not set; run via e2e/run-store-it.sh for live-DB coverage") + } + ctx := context.Background() + if err := store.Migrate(ctx, dsn); err != nil { + t.Fatalf("migrate: %v", err) + } + pool, err := store.OpenPool(ctx, dsn) + if err != nil { + t.Fatalf("open pool: %v", err) + } + t.Cleanup(pool.Close) + q := store.New(pool) + key := make([]byte, secretbox.KeySize) + for i := range key { + key[i] = byte(i + 1) + } + box, err := secretbox.New(key) + if err != nil { + t.Fatal(err) + } + srv := youtracktest.NewServer(t) + def, ok := http.DefaultTransport.(*http.Transport) + if !ok { + t.Fatal("http.DefaultTransport is not an *http.Transport") + } + saved := def.TLSClientConfig + def.TLSClientConfig = srv.Client().Transport.(*http.Transport).TLSClientConfig.Clone() + t.Cleanup(func() { def.TLSClientConfig = saved }) + + rec := forgetest.NewRecorder(&mrForge{}) + svc := forgesvc.NewWithForgeBuilder(q, box, 5*time.Second, nil, func(forge.Type, string, string, time.Duration) (forge.Forge, error) { return rec, nil }) + itSvc := issuetrackersvc.New(q, box, func(u string) bool { return u == srv.URL }, issuetracker.New) + svc.SetIssueTrackerDrivers(itSvc) + sealed, err := svc.EncryptToken("forge-token") + if err != nil { + t.Fatal(err) + } + f := &linkedLifecycleLive{pool: pool, q: q, svc: svc, srv: srv, repo: uuid.New(), user: uuid.New(), worker: uuid.New()} + conn := uuid.New() + f.exec(t, `INSERT INTO users (id, email, password_hash) VALUES ($1, $2, 'x')`, f.user, fmt.Sprintf("lb-%s@e2e", f.user)) + f.exec(t, `INSERT INTO forge_connections (id, user_id, forge_type, base_url, bot_username, bot_forge_user_id, token_ciphertext) + VALUES ($1, $2, 'gitlab', $3, 'bot', 1, $4)`, conn, f.user, "https://"+uuid.NewString()+".forge.test", sealed) + yt, err := itSvc.Connect(ctx, f.user, "youtrack", srv.URL, srv.Token) + if err != nil { + t.Fatal(err) + } + f.exec(t, `INSERT INTO repos (id, connection_id, forge_project_id, path_with_namespace, web_url, default_branch, enabled, + issue_tracker_connection_id, issue_tracker_project_id, issue_tracker_project_short, eligibility_tag_id, eligibility_tag_name, issue_tracker_linked_at) + VALUES ($1, $2, 77, $3, 'https://forge.test/g/l', 'main', true, $4, $5, $6, $7, $8, now())`, + f.repo, conn, "g/lb-"+f.repo.String(), yt.ID, youtracktest.ProjectID, youtracktest.ProjectShortName, youtracktest.EligibilityTagID, youtracktest.EligibilityTagName) + if err := forgesvc.EnsureBoardColumnRows(ctx, q, f.repo); err != nil { + t.Fatal(err) + } + sum := sha256.Sum256([]byte("lb-worker-" + f.worker.String())) + f.exec(t, `INSERT INTO workers(id,user_id,name,token_hash,status) VALUES($1,$2,'lb-worker',$3,'online')`, f.worker, f.user, sum[:]) + srv.SeedState("In Review", false) + return f +} + +func (f *linkedLifecycleLive) exec(t *testing.T, sql string, args ...any) { + t.Helper() + if _, err := f.pool.Exec(context.Background(), sql, args...); err != nil { + t.Fatalf("exec %q: %v", sql, err) + } +} + +// cacheIssue caches issue n as an open card with no column. +func (f *linkedLifecycleLive) cacheIssue(t *testing.T, n int64) { + t.Helper() + f.exec(t, `INSERT INTO issues (repo_id, forge_issue_iid, title, state, labels, web_url, forge_updated_at, synced_at) + VALUES ($1, $2, 't', 'opened', '[]', 'https://yt.test/i', now(), now())`, f.repo, n) +} + +// completedRun inserts a run for issue n that completed with an MR, which stamps its +// move marker, and returns its id. Its origin column is the Open lane. +func (f *linkedLifecycleLive) completedRun(t *testing.T, n int64) uuid.UUID { + t.Helper() + runID := uuid.New() + f.exec(t, `INSERT INTO runs (id, user_id, repo_id, kind, issue_iid, issue_title, issue_description, branch, status, auto_approve, + origin_column, worker_id, claim_generation) + VALUES ($1, $2, $3, 'issue', $4, 't', 'd', $5, 'running', false, '', $6, 1)`, runID, f.user, f.repo, n, "agent/issue-"+runID.String(), f.worker) + rows, err := f.q.SetRunCompleted(context.Background(), store.SetRunCompletedParams{ + MrIid: pgtype.Int8{Int64: 13, Valid: true}, ID: runID, WorkerID: pgtype.UUID{Bytes: f.worker, Valid: true}, + }) + if err != nil || rows != 1 { + t.Fatalf("SetRunCompleted rows=%d err=%v", rows, err) + } + return runID +} + +func (f *linkedLifecycleLive) pending(t *testing.T, runID uuid.UUID) (pending bool, boardColumn pgtype.Text) { + t.Helper() + var since pgtype.Timestamptz + if err := f.pool.QueryRow(context.Background(), `SELECT move_pending_since, board_column FROM runs WHERE id = $1`, runID).Scan(&since, &boardColumn); err != nil { + t.Fatal(err) + } + return since.Valid, boardColumn +} + +func (f *linkedLifecycleLive) lifecycle() *Lifecycle { + l := New(f.q, f.svc, "https://uzi.test") + l.SetIssueSources(f.svc) + l.grace = 0 + return l +} + +func (f *linkedLifecycleLive) writes() []youtracktest.Request { + var out []youtracktest.Request + for _, r := range f.srv.Requests() { + if r.Method != http.MethodGet { + out = append(out, r) + } + } + return out +} + +// A run whose issue is not cached is probed in the linked project: a moved-away issue and +// an issue without the eligibility tag clear the marker; an eligible one keeps it (the next +// sync caches it and reconciliation moves the card). +func TestLinkedRunWithAnUncachedIssueIsProbedLiveDB(t *testing.T) { + f := newLinkedLifecycleLive(t) + ctx := context.Background() + + moved := f.srv.SeedIssue("moved away", "b", youtracktest.EligibilityTagID) + f.srv.MoveIssue(moved, youtracktest.OtherProjectID) + untagged := f.srv.SeedIssue("never eligible", "b") + eligible := f.srv.SeedIssue("not cached yet", "b", youtracktest.EligibilityTagID) + + cases := []struct { + name string + issue int64 + wantClear bool + }{ + {"moved away: cleared", moved, true}, + {"found but not eligible: cleared", untagged, true}, + {"found and eligible: kept", eligible, false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + runID := f.completedRun(t, tc.issue) + f.srv.ResetRequests() + + f.lifecycle().notifyOnce(ctx, runID, "completed") + + pending, col := f.pending(t, runID) + if pending == tc.wantClear { + t.Fatalf("marker pending = %v, want %v", pending, !tc.wantClear) + } + if col.Valid { + t.Fatalf("board_column = %q for a move that did not happen", col.String) + } + if w := f.writes(); len(w) != 0 { + t.Fatalf("an uncached issue must never be written: %+v", w) + } + }) + } +} + +// A move the issue tracker refuses (the column has no State mapped) keeps the marker and +// sends no write. Once the owner saves the mapping, reconciliation moves the card, records +// the column and clears the marker. +func TestLinkedRefusedMoveKeepsTheMarkerUntilTheMappingIsSavedLiveDB(t *testing.T) { + f := newLinkedLifecycleLive(t) + ctx := context.Background() + n := f.srv.SeedIssue("a card", "b", youtracktest.EligibilityTagID) + f.cacheIssue(t, n) + runID := f.completedRun(t, n) + f.srv.ResetRequests() + + l := f.lifecycle() + l.notifyOnce(ctx, runID, "completed") + if pending, _ := f.pending(t, runID); !pending { + t.Fatal("a refused move must leave move_pending_since") + } + if w := f.writes(); len(w) != 0 { + t.Fatalf("a refused move sent a write: %+v", w) + } + l.Reconcile(ctx) + if pending, _ := f.pending(t, runID); !pending { + t.Fatal("a reconcile with the mapping still missing must keep the marker") + } + + saveStateMapping(t, f.pool, f.repo) + l.Reconcile(ctx) + + if pending, col := f.pending(t, runID); pending || col.String != board.ColumnHumanReview { + t.Fatalf("after the mapping: pending=%v board_column=%+v, want cleared and Human Review", pending, col) + } + if got := f.srv.IssueState(n); got != "In Review" { + t.Fatalf("issue State = %q, want In Review", got) + } + var cached pgtype.Text + if err := f.pool.QueryRow(ctx, `SELECT issue_tracker_column FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2`, f.repo, n).Scan(&cached); err != nil || cached.String != board.ColumnHumanReview { + t.Fatalf("cached column = %+v (%v), want Human Review", cached, err) + } +} + +// A person resolved the issue in YouTrack and the cache still shows it open: a run +// transition writes no State and clears the marker, so the resolution stands. +func TestLinkedMoveOfAnIssueResolvedInYouTrackWritesNothingLiveDB(t *testing.T) { + f := newLinkedLifecycleLive(t) + ctx := context.Background() + saveStateMapping(t, f.pool, f.repo) + n := f.srv.SeedIssue("a card", "b", youtracktest.EligibilityTagID) + f.cacheIssue(t, n) + runID := f.completedRun(t, n) + tr, err := f.svc.IssueTrackerForRepo(ctx, f.user, f.repo) + if err != nil { + t.Fatal(err) + } + if err := tr.SetIssueState(ctx, 77, n, forge.StateClosed); err != nil { + t.Fatalf("resolve the issue in YouTrack: %v", err) + } + resolved := f.srv.IssueState(n) + f.srv.ResetRequests() + + f.lifecycle().notifyOnce(ctx, runID, "completed") + + if w := f.writes(); len(w) != 0 { + t.Fatalf("a write reached the resolved issue: %+v", w) + } + if got := f.srv.IssueState(n); got != resolved { + t.Fatalf("issue State = %q, want the resolved %q untouched", got, resolved) + } + if pending, col := f.pending(t, runID); pending || col.Valid { + t.Fatalf("pending=%v board_column=%+v, want the marker cleared and nothing recorded", pending, col) + } +} + +// A moved-away cached issue is skipped: the write is attempted (the mapping is saved) and +// answers not-found, the marker is cleared, and the cache is untouched. +func TestLinkedMovedAwayCachedIssueClearsTheMarkerLiveDB(t *testing.T) { + f := newLinkedLifecycleLive(t) + saveStateMapping(t, f.pool, f.repo) + n := f.srv.SeedIssue("a card", "b", youtracktest.EligibilityTagID) + f.cacheIssue(t, n) + runID := f.completedRun(t, n) + f.srv.MoveIssue(n, youtracktest.OtherProjectID) + + f.lifecycle().notifyOnce(context.Background(), runID, "completed") + + if pending, col := f.pending(t, runID); pending || col.Valid { + t.Fatalf("pending=%v board_column=%+v, want the marker cleared and nothing recorded", pending, col) + } + if w := f.writes(); len(w) != 0 { + t.Fatalf("a write reached the moved-away issue: %+v", w) + } + var cached pgtype.Text + if err := f.pool.QueryRow(context.Background(), `SELECT issue_tracker_column FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2`, f.repo, n).Scan(&cached); err != nil || cached.Valid { + t.Fatalf("cached column = %+v (%v), want NULL", cached, err) + } +} + +// PRD #4173 M5 U6: the autopilot terminal comment reaches the YouTrack issue through the +// resolver, the guard and the driver. A moved-away issue is logged and the lifecycle +// carries on (the move marker is still cleared). +func autopilotCompletedRun(t *testing.T, f *linkedLifecycleLive, n int64) uuid.UUID { + t.Helper() + f.cacheIssue(t, n) + runID := f.completedRun(t, n) + f.exec(t, `UPDATE runs SET auto_approve = true WHERE id = $1`, runID) + return runID +} + +func TestLinkedTerminalCommentLandsOnTheYouTrackIssueLiveDB(t *testing.T) { + f := newLinkedLifecycleLive(t) + saveStateMapping(t, f.pool, f.repo) + n := f.srv.SeedIssue("a card", "b", youtracktest.EligibilityTagID) + runID := autopilotCompletedRun(t, f, n) + + f.lifecycle().notifyOnce(context.Background(), runID, "completed") + + tr, err := f.svc.IssueTrackerForRepo(context.Background(), f.user, f.repo) + if err != nil { + t.Fatal(err) + } + cs, err := tr.ListIssueComments(context.Background(), 77, n) + if err != nil { + t.Fatal(err) + } + if len(cs) != 1 || !strings.HasPrefix(cs[0].Body, "**Autopilot opened a merge request.**") || !strings.Contains(cs[0].Body, "!13") { + t.Fatalf("YouTrack comments = %+v, want the terminal comment", cs) + } +} + +func TestLinkedTerminalCommentOnAMovedAwayIssueIsLoggedAndTheLifecycleContinuesLiveDB(t *testing.T) { + f := newLinkedLifecycleLive(t) + saveStateMapping(t, f.pool, f.repo) + n := f.srv.SeedIssue("a card", "b", youtracktest.EligibilityTagID) + runID := autopilotCompletedRun(t, f, n) + f.srv.MoveIssue(n, youtracktest.OtherProjectID) + var logs bytes.Buffer + prev := slog.Default() + slog.SetDefault(slog.New(slog.NewTextHandler(&logs, nil))) + t.Cleanup(func() { slog.SetDefault(prev) }) + + f.lifecycle().notifyOnce(context.Background(), runID, "completed") + + if w := f.writes(); len(w) != 0 { + t.Fatalf("a write reached the moved-away issue: %+v", w) + } + if !strings.Contains(logs.String(), "autopilot terminal comment skipped, the linked issue is gone or moved") { + t.Fatalf("the skipped comment was not logged: %s", logs.String()) + } + if pending, _ := f.pending(t, runID); pending { + t.Fatal("the move marker was left set") + } +} diff --git a/api/internal/runlifecycle/linked_test.go b/api/internal/runlifecycle/linked_test.go index c23baa0ca4..7f0efa8c7d 100644 --- a/api/internal/runlifecycle/linked_test.go +++ b/api/internal/runlifecycle/linked_test.go @@ -1,9 +1,12 @@ package runlifecycle import ( + "bytes" "context" "errors" "fmt" + "log/slog" + "strings" "testing" "github.com/google/uuid" @@ -12,6 +15,7 @@ import ( "github.com/vtmocanu/uzi/api/internal/forge" "github.com/vtmocanu/uzi/api/internal/forgesvc" + "github.com/vtmocanu/uzi/api/internal/issuetracker" "github.com/vtmocanu/uzi/api/internal/store" ) @@ -54,9 +58,12 @@ func linkedMoveCtx(repoID, userID uuid.UUID, status string) store.GetRunMoveCont func linkedFixture(status string, autoMoveErr error) (*fakeStore, *seenMover, *fakeSources, *Lifecycle, uuid.UUID, uuid.UUID, uuid.UUID) { runID, repoID, userID := uuid.New(), uuid.New(), uuid.New() + // A linked card sits in its cached column, never in a label (PRD #4173 M5). + issue := issueWith(repoID, 4, "opened") + issue.IssueTrackerColumn = txt("Later") fs := &fakeStore{ moveCtx: linkedMoveCtx(repoID, userID, status), - issue: issueWith(repoID, 4, "opened", "Later"), + issue: issue, columns: defaultCols(), } sm := &seenMover{fakeMover: &fakeMover{autoMoveErr: autoMoveErr}} @@ -119,6 +126,24 @@ func TestLinkedMoveSkippableWriteClearsMarker(t *testing.T) { } } +// A person resolved the issue in the issue tracker and the cache still shows it open: the +// driver refuses the write and the marker is cleared, like the cached closed-issue guard. +func TestLinkedMoveIssueResolvedInTheIssueTrackerClearsMarker(t *testing.T) { + fs, sm, _, l, runID, _, _ := linkedFixture("queued", fmt.Errorf("move: %w", issuetracker.ErrIssueResolved)) + + l.notifyOnce(context.Background(), runID, "queued") + + if len(sm.moves) != 1 { + t.Fatalf("expected one move attempt, got %v", sm.moves) + } + if len(fs.cleared) != 1 || fs.cleared[0] != runID { + t.Fatalf("a resolved issue clears the marker once, got %v", fs.cleared) + } + if len(fs.recorded) != 0 { + t.Fatal("board_column must not be recorded for a refused move") + } +} + func TestLinkedMoveOtherErrorStillRetries(t *testing.T) { fs, _, _, l, runID, _, _ := linkedFixture("queued", errors.New("youtrack unreachable")) @@ -129,14 +154,100 @@ func TestLinkedMoveOtherErrorStillRetries(t *testing.T) { } } -func TestLinkedMoveIssueNoLongerCachedClearsMarker(t *testing.T) { +// probeTracker answers the lifecycle's uncached-issue probe (PRD #4173 M5): GetIssue +// returns issue and err, and every call is counted. +type probeTracker struct { + forge.IssueTracker + issue forge.Issue + err error + reads int +} + +func (p *probeTracker) GetIssue(context.Context, int64, int64) (forge.Issue, error) { + p.reads++ + return p.issue, p.err +} + +// A linked run whose issue is not cached asks the issue tracker (PRD #4173 M5). The write +// guard marks a not-found read as a moved-away issue, so the not-found case is built with +// the guard over a cause that is a forge not-found. +func TestLinkedMoveUncachedIssueProbe(t *testing.T) { + missing := &forge.IssueNotFoundError{Err: errors.New("404")} + tests := []struct { + name string + tracker func() forge.IssueTracker + wantClear bool + noTagName bool // the link carries no eligibility tag name + }{ + {"moved away (not found) clears the marker", func() forge.IssueTracker { return forgesvc.UnavailableIssueTracker(missing) }, true, false}, + {"found without the eligibility tag clears the marker", func() forge.IssueTracker { + return &probeTracker{issue: forge.Issue{IID: 4, Labels: []string{"other"}}} + }, true, false}, + {"found and eligible keeps the marker", func() forge.IssueTracker { + return &probeTracker{issue: forge.Issue{IID: 4, Labels: []string{"other", "uzi-ready"}}} + }, false, false}, + {"found with no eligibility tag name keeps the marker", func() forge.IssueTracker { + return &probeTracker{issue: forge.Issue{IID: 4, Labels: []string{"other"}}} + }, false, true}, + {"a read error keeps the marker", func() forge.IssueTracker { + return &probeTracker{err: errors.New("youtrack unreachable")} + }, false, false}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + fs, sm, src, l, runID, _, _ := linkedFixture("queued", nil) + fs.issueErr = pgx.ErrNoRows // not in the cache + fs.moveCtx.EligibilityTagName = txt("uzi-ready") + if tc.noTagName { + fs.moveCtx.EligibilityTagName = pgtype.Text{} + } + src.tr = tc.tracker() + + l.notifyOnce(context.Background(), runID, "queued") + + if len(sm.moves) != 0 { + t.Fatalf("an uncached issue is never moved: %v", sm.moves) + } + if got := len(fs.cleared) == 1; got != tc.wantClear { + t.Fatalf("cleared = %v, want clear=%v", fs.cleared, tc.wantClear) + } + }) + } +} + +// A refused move leaves move_pending_since so reconciliation can retry it after the +// owner saves the mapping (PRD #4173 M5); a skippable write still clears it. +func TestLinkedMoveRefusedColumnKeepsMarker(t *testing.T) { + for name, err := range map[string]error{ + "column not mapped": fmt.Errorf("move: %w", issuetracker.ErrColumnNotMapped), + "mapping unset": fmt.Errorf("move: %w", issuetracker.ErrStateMappingUnset), + "state not allowed": fmt.Errorf("move: %w", issuetracker.ErrStateNotAllowed), + } { + t.Run(name, func(t *testing.T) { + fs, sm, _, l, runID, _, _ := linkedFixture("queued", err) + + l.notifyOnce(context.Background(), runID, "queued") + + if len(sm.moves) != 1 { + t.Fatalf("expected one move attempt, got %v", sm.moves) + } + if len(fs.cleared) != 0 || len(fs.recorded) != 0 { + t.Fatalf("a refused move keeps the marker and records nothing: cleared=%v recorded=%v", fs.cleared, fs.recorded) + } + }) + } +} + +// The manual-drag guard compares the cached column of a linked card (PRD #4173 M5): a +// card the lifecycle did not leave in the baseline column is a manual drag. +func TestLinkedMoveManualDragIsReadFromTheCachedColumn(t *testing.T) { fs, sm, _, l, runID, _, _ := linkedFixture("queued", nil) - fs.issueErr = pgx.ErrNoRows // the moved issue was evicted from the cache + fs.issue.IssueTrackerColumn = txt("Upcoming") l.notifyOnce(context.Background(), runID, "queued") - if len(fs.cleared) != 1 || len(sm.moves) != 0 { - t.Fatalf("an evicted linked issue is a permanent skip: cleared=%v moves=%v", fs.cleared, sm.moves) + if len(sm.moves) != 0 || len(fs.cleared) != 1 { + t.Fatalf("a card in another column is a manual drag: moves=%v cleared=%v", sm.moves, fs.cleared) } } @@ -178,22 +289,47 @@ func TestUnlinkedMoveNeverConsultsResolver(t *testing.T) { } } -func TestLinkedTerminalCommentUsesResolverAndLogsSkip(t *testing.T) { +// Converted from TestLinkedTerminalCommentUsesResolverAndLogsSkip (PRD #4173 M5 U6): the +// note is no longer refused with ErrIssueWriteNotOffered, it lands on the resolver's +// tracker for the run's issue. +func TestLinkedTerminalCommentLandsOnTheResolversTracker(t *testing.T) { fs, _, src, l, runID, _, _ := linkedFixture("failed", nil) fs.moveCtx.AutoApprove = true - note := &fakeForge{noteErr: fmt.Errorf("create issue note: %w", forgesvc.ErrIssueWriteNotOffered)} + note := &fakeForge{} src.tr = note l.notifyOnce(context.Background(), runID, "failed") - if len(note.notes) != 1 { - t.Fatalf("the comment goes to the resolver's tracker, got %d notes", len(note.notes)) + if len(note.notes) != 1 || note.notes[0].issueIID != fs.moveCtx.IssueIid.Int64 || !strings.Contains(note.notes[0].body, "Autopilot") { + t.Fatalf("the comment goes to the resolver's tracker for the run's issue, got %+v", note.notes) } if fs.claimCalls != 1 { t.Fatalf("record-then-comment is unchanged, claims=%d", fs.claimCalls) } } +// A failed linked comment (the issue tracker answers an error) is logged and the +// lifecycle carries on: one attempt, no retry. +func TestLinkedTerminalCommentFailureIsLogged(t *testing.T) { + fs, _, src, l, runID, _, _ := linkedFixture("failed", nil) + fs.moveCtx.AutoApprove = true + note := &fakeForge{noteErr: errors.New("youtrack answered 500")} + src.tr = note + var logs bytes.Buffer + prev := slog.Default() + slog.SetDefault(slog.New(slog.NewTextHandler(&logs, nil))) + t.Cleanup(func() { slog.SetDefault(prev) }) + + l.notifyOnce(context.Background(), runID, "failed") + + if len(note.notes) != 1 || fs.claimCalls != 1 { + t.Fatalf("notes=%d claims=%d", len(note.notes), fs.claimCalls) + } + if !strings.Contains(logs.String(), "post autopilot terminal comment") || !strings.Contains(logs.String(), "500") { + t.Fatalf("the failure was not logged: %s", logs.String()) + } +} + func TestLinkedTerminalCommentWithoutSourcesPostsNothing(t *testing.T) { fs, sm, _, l, runID, _, _ := linkedFixture("failed", nil) fs.moveCtx.AutoApprove = true diff --git a/api/internal/runlifecycle/moved_issue_livedb_test.go b/api/internal/runlifecycle/moved_issue_livedb_test.go index 3a682d3f7a..109c7b1aae 100644 --- a/api/internal/runlifecycle/moved_issue_livedb_test.go +++ b/api/internal/runlifecycle/moved_issue_livedb_test.go @@ -101,6 +101,11 @@ func TestMovedLinkedIssueSkipsTheWriteAndTheCodeHostSideCompletesLiveDB(t *testi if err := forgesvc.EnsureBoardColumnRows(ctx, q, repo); err != nil { t.Fatal(err) } + // Since M5 the issue tracker refuses a move onto a column with no State mapped before it + // reads the issue, and a refusal keeps the marker; the mapping is saved so the write + // reaches the moved-away issue and answers not-found. + srv.SeedState("In Review", false) + saveStateMapping(t, pool, repo) issueN := srv.SeedIssue("a youtrack card", "body", youtracktest.EligibilityTagID) exec(`INSERT INTO issues (repo_id, forge_issue_iid, title, state, labels, web_url, forge_updated_at, synced_at) VALUES ($1, $2, 't', 'opened', '[]', 'https://yt.test/i', now(), now())`, repo, issueN) diff --git a/api/internal/schedsvc/linked_removal_livedb_test.go b/api/internal/schedsvc/linked_removal_livedb_test.go new file mode 100644 index 0000000000..2c2b425dd1 --- /dev/null +++ b/api/internal/schedsvc/linked_removal_livedb_test.go @@ -0,0 +1,149 @@ +package schedsvc + +import ( + "context" + "net/http" + "slices" + "testing" + "time" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5/pgtype" + "github.com/jackc/pgx/v5/pgxpool" + + "github.com/vtmocanu/uzi/api/internal/forgesvc" + "github.com/vtmocanu/uzi/api/internal/issuetracker" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" + "github.com/vtmocanu/uzi/api/internal/issuetrackersvc" + "github.com/vtmocanu/uzi/api/internal/secretbox" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// PRD #4173 M5: remove-label-on-dispatch on a repo linked to YouTrack compares the sweep's +// selector with the stored eligibility name, not the configured uzi label, so the eligibility +// tag is never stripped, and any other selector tag is removed as on a forge. +// linkScheduleRepo links repo to a fresh YouTrack fake and returns the fake, the real +// resolver the scheduler reaches the linked issue side through, and the cache wrapper it is +// built over. +func linkScheduleRepo(t *testing.T, ctx context.Context, pool *pgxpool.Pool, q *store.Queries, box *secretbox.Box, user, repo uuid.UUID) (*youtracktest.Server, *forgesvc.Service, removalCache) { + t.Helper() + srv := youtracktest.NewServer(t) + def, ok := http.DefaultTransport.(*http.Transport) + if !ok { + t.Fatal("http.DefaultTransport is not an *http.Transport") + } + saved := def.TLSClientConfig + def.TLSClientConfig = srv.Client().Transport.(*http.Transport).TLSClientConfig.Clone() + t.Cleanup(func() { def.TLSClientConfig = saved }) + + cache := removalCache{Queries: q} + svc := forgesvc.New(cache, box, time.Second, nil) + itSvc := issuetrackersvc.New(q, box, func(u string) bool { return u == srv.URL }, issuetracker.New) + svc.SetIssueTrackerDrivers(itSvc) + yt, err := itSvc.Connect(ctx, user, "youtrack", srv.URL, srv.Token) + if err != nil { + t.Fatal(err) + } + capacityExec(t, ctx, pool, `UPDATE repos SET issue_tracker_connection_id=$2, issue_tracker_project_id=$3, issue_tracker_project_short=$4, + eligibility_tag_id=$5, eligibility_tag_name=$6, issue_tracker_linked_at=now() WHERE id=$1`, + repo, yt.ID, youtracktest.ProjectID, youtracktest.ProjectShortName, youtracktest.EligibilityTagID, youtracktest.EligibilityTagName) + return srv, svc, cache +} + +// PRD #4173 M5: the self-improvement tracking issue is created in the linked project through +// the repo's issue side, carries only the tracking label (no eligibility tag), and is reused +// by the next cycle. +func TestLinkedSelfImproveTrackingIssueIsCreatedInTheLinkedProjectLiveDB(t *testing.T) { + ctx, pool, q, box := openScheduleFireLiveDB(t) + user, repo := scheduleFireUser(ctx, t, pool) + srv, svc, _ := linkScheduleRepo(t, ctx, pool, q, box, user, repo) + tr, err := svc.IssueTrackerForRepo(ctx, user, repo) + if err != nil { + t.Fatal(err) + } + iid, err := ensureTrackingIssue(ctx, tr, 1) + if err != nil { + t.Fatal(err) + } + var names []string + for _, id := range srv.IssueTagIDs(iid) { + names = append(names, srv.TagByID(id).Name) + } + if !slices.Equal(names, []string{SelfImproveTrackingLabel}) { + t.Fatalf("tracking issue tags = %v, want only %q (no eligibility tag)", names, SelfImproveTrackingLabel) + } + again, err := ensureTrackingIssue(ctx, tr, 1) + if err != nil || again != iid { + t.Fatalf("the next cycle created issue %d (%v), want to reuse %d", again, err, iid) + } +} + +func TestLinkedSweepRemovalKeepsTheEligibilityTagLiveDB(t *testing.T) { + for _, tc := range []struct { + name string + selector string + wantRemoved bool + wantTags []string + }{ + {"selector equal to the eligibility name is kept", youtracktest.EligibilityTagName, false, []string{youtracktest.EligibilityTagName, "triage"}}, + {"another selector tag is removed", "triage", true, []string{youtracktest.EligibilityTagName}}, + } { + t.Run(tc.name, func(t *testing.T) { + ctx, pool, q, box := openScheduleFireLiveDB(t) + user, repo := scheduleFireUser(ctx, t, pool) + + srv, svc, _ := linkScheduleRepo(t, ctx, pool, q, box, user, repo) + + triage := srv.SeedOwnTag("triage") + n := srv.SeedIssue("candidate", "body", youtracktest.EligibilityTagID, triage) + capacityExec(t, ctx, pool, `INSERT INTO issues (repo_id,forge_issue_iid,title,state,labels,assignee_ids,web_url,forge_updated_at,synced_at) + VALUES ($1,$2,'candidate','opened',$3::jsonb,'[]','https://yt.test/i',now(),now())`, repo, n, `["`+youtracktest.EligibilityTagName+`","triage"]`) + + sc := capacityLiveSchedule(t, ctx, pool, q, user, repo, 4, 2, pgtype.Int4{Int32: 1, Valid: true}) + capacityExec(t, ctx, pool, `UPDATE run_schedules SET labels=$2::jsonb,remove_label_on_dispatch=true,capacity_limit=NULL,capacity_room_needed=NULL WHERE id=$1`, + sc.ID, `["`+tc.selector+`"]`) + sc, err := q.GetRunSchedule(ctx, sc.ID) + if err != nil { + t.Fatal(err) + } + + st := &removalLiveStore{capacityLiveStore: &capacityLiveStore{Queries: q, scheduleID: sc.ID}} + runs := &removalRuns{pool: pool} + settings := &removalSettings{fakeSettings: fakeSettings{uziLabel: "uzi"}} + e := New(st, runs, removalBuilder{Service: svc, f: &removalForge{}}, settings, nil, nil, time.Minute, nil) + e.SetIssueSources(svc) + srv.ResetRequests() + + out, err := e.RunNow(ctx, sc) + if err != nil { + t.Fatal(err) + } + if len(out.Started) != 1 { + t.Fatalf("out = %+v, want one started run", out) + } + started := out.Started[0] + if started.LabelRemoved != tc.wantRemoved || started.LabelRemoveFailed || started.SelectorLabel != tc.selector { + t.Fatalf("started = %+v, want LabelRemoved=%v without a failure", started, tc.wantRemoved) + } + var names []string + for _, id := range srv.IssueTagIDs(n) { + names = append(names, srv.TagByID(id).Name) + } + slices.Sort(names) + slices.Sort(tc.wantTags) + if !slices.Equal(names, tc.wantTags) { + t.Fatalf("tags on the issue = %v, want %v", names, tc.wantTags) + } + if !slices.Contains(srv.IssueTagIDs(n), youtracktest.EligibilityTagID) { + t.Fatal("the eligibility tag was stripped") + } + cached, err := q.GetIssueByIID(ctx, store.GetIssueByIIDParams{RepoID: repo, ForgeIssueIid: n}) + if err != nil { + t.Fatal(err) + } + if got := scheduleRemovalLabels(t, cached.Labels); !slices.Contains(got, youtracktest.EligibilityTagName) { + t.Fatalf("cached labels = %v, want the eligibility name kept", got) + } + }) + } +} diff --git a/api/internal/schedsvc/linked_test.go b/api/internal/schedsvc/linked_test.go index d785d22d52..79656d687f 100644 --- a/api/internal/schedsvc/linked_test.go +++ b/api/internal/schedsvc/linked_test.go @@ -111,17 +111,38 @@ func TestLinkedProposalDigestWithoutSourcesIsSkipped(t *testing.T) { } } -func TestLinkedSelfImproveIsSkippedWithoutAnyIssueWrite(t *testing.T) { - h, tr, _ := linkedHarness(true) +// PRD #4173 M5 (converted from TestLinkedSelfImproveIsSkippedWithoutAnyIssueWrite, the M4 +// refusal): the tracking issue is created through the resolver's issue side, never the +// forge, carries only the tracking label (no eligibility tag), and the cycle starts. +func TestLinkedSelfImproveFilesTheTrackingIssueThroughTheResolver(t *testing.T) { + h, tr, src := linkedHarness(true) + tr.createdIID = 501 out, err := h.sched.fireOne(context.Background(), h.selfImproveSchedule()) if err != nil { t.Fatal(err) } - if len(out.Skips) != 1 || out.Skips[0].Reason != SkipConfigNotSupported { - t.Fatalf("out = %+v, want one config_not_supported skip", out) + if len(out.Started) != 1 || len(h.runs.selfImprove) != 1 { + t.Fatalf("out = %+v, runs = %d, want one started self_improve run", out, len(h.runs.selfImprove)) + } + if tr.listCount != 1 || tr.createCount != 1 || h.fb.f.createCount != 0 || h.fb.f.listCount != 0 { + t.Fatalf("resolver list/create = %d/%d, forge list/create = %d/%d, want 1/1 and 0/0", + tr.listCount, tr.createCount, h.fb.f.listCount, h.fb.f.createCount) + } + if len(tr.createdLabels) != 1 || tr.createdLabels[0] != SelfImproveTrackingLabel { + t.Fatalf("tracking issue labels = %v, want only %q", tr.createdLabels, SelfImproveTrackingLabel) + } + if len(src.users) == 0 || src.users[0] != h.owner || src.repos[0] != h.repoID { + t.Fatalf("the resolver is bound by owner and repo row, got %v %v", src.users, src.repos) + } +} + +func TestLinkedSelfImproveWithoutSourcesFailsClosedAndNeverTouchesTheForge(t *testing.T) { + h, _, _ := linkedHarness(false) + if _, err := h.sched.fireOne(context.Background(), h.selfImproveSchedule()); err == nil { + t.Fatal("a linked repo with no resolver fired") } - if h.fb.f.createCount != 0 || h.fb.f.listCount != 0 || tr.createCount != 0 || tr.listCount != 0 || len(h.runs.selfImprove) != 0 { - t.Fatal("self_improve touched an issue tracker or started a run on a linked repo") + if h.fb.f.createCount != 0 || h.fb.f.listCount != 0 || len(h.runs.selfImprove) != 0 { + t.Fatal("self_improve fell back to the forge or started a run on an unresolved linked repo") } } diff --git a/api/internal/schedsvc/scheduler.go b/api/internal/schedsvc/scheduler.go index 4f430e6551..0b049aec19 100644 --- a/api/internal/schedsvc/scheduler.go +++ b/api/internal/schedsvc/scheduler.go @@ -881,15 +881,7 @@ func (e *Scheduler) fireSweep(ctx context.Context, sched store.RunSchedule) (Fir for i := range res.Started { res.Started[i].SelectorLabel = selectorLabel } - uziLabel := settings.DefaultUziLabel - var removeErr error - if e.settings != nil { - var live string - live, removeErr = e.settings.UziLabel(ctx) - if strings.TrimSpace(live) != "" { - uziLabel = live - } - } + uziLabel, removeErr := e.dispatchEligibilityLabel(ctx, repo) if removeErr != nil || selectorLabel != uziLabel { if removeErr == nil { var cached store.Issue @@ -1342,6 +1334,26 @@ func (e *Scheduler) resolveSweepLabels(stored []byte, defaultLabel string) ([]by return out, nil } +// dispatchEligibilityLabel is the label remove-label-on-dispatch must never remove: the +// stored eligibility tag name on a repo linked to an issue tracker (PRD #4173 D14), so a +// selector equal to it is kept; the live configured uzi label otherwise, falling back to +// settings.DefaultUziLabel when it is blank. A failed settings read is returned alongside +// the label, as before, so the caller records the removal as failed. +func (e *Scheduler) dispatchEligibilityLabel(ctx context.Context, repo store.GetRepoForUserRow) (string, error) { + if repo.IssueTrackerConnectionID.Valid { + return e.eligibilityLabel(ctx, repo) + } + uziLabel := settings.DefaultUziLabel + if e.settings == nil { + return uziLabel, nil + } + live, err := e.settings.UziLabel(ctx) + if strings.TrimSpace(live) != "" { + uziLabel = live + } + return uziLabel, err +} + // eligibilityLabel is the label a sweep treats as run eligibility for repo: the link's // stored eligibility tag name for a repo linked to an issue tracker (PRD #4173 D14), // the configured uzi label otherwise. A linked repo with no stored name cannot be diff --git a/api/internal/schedsvc/self_improve.go b/api/internal/schedsvc/self_improve.go index a6f3a9c08a..3f4a2ece74 100644 --- a/api/internal/schedsvc/self_improve.go +++ b/api/internal/schedsvc/self_improve.go @@ -91,15 +91,6 @@ func (e *Scheduler) fireSelfImprove(ctx context.Context, sched store.RunSchedule return FireOutcome{}, err } - // PRD #4173 M4: self-improve files a tracking issue on the repo's issue tracker, and a - // linked repo has none to write to until M5 (the write guard would refuse it anyway). - // Skipped as a benign, logged no-op before any forge call; never falls back to the forge. - if repo.IssueTrackerConnectionID.Valid { - e.logger.Info("scheduler: self_improve skipped, issue creation is not offered on a repo linked to an issue tracker", - "schedule", sched.ID.String(), "repo", repo.ID.String()) - return FireOutcome{Matched: 1, Skips: []Skip{{Reason: SkipConfigNotSupported}}}, nil - } - // 2. Active-run pre-check, per repo. On a DB error retry next tick (transient). If a run // is already active for this repo, benign skip — no notification (mirrors firePrompt's // active skip); the per-repo unique index is the hard guard, this just skips the forge work. @@ -132,9 +123,14 @@ func (e *Scheduler) fireSelfImprove(ctx context.Context, sched store.RunSchedule return FireOutcome{}, fmt.Errorf("build forge driver: %w", err) // transient } // One driver, two halves: MR reads go through the code host, the tracking issue - // through the issue tracker. + // through the repo's issue side: the forge driver for an unlinked repo, the linked + // issue tracker for a linked one (PRD #4173 M5), never the forge for a linked repo. + // The tracking issue carries no eligibility tag, so the next linked full sync evicts it. var ch forge.CodeHost = f - var tr forge.IssueTracker = f + tr, err := e.issueSide(ctx, repo, f) + if err != nil { + return FireOutcome{}, fmt.Errorf("resolve issue tracker: %w", err) // transient + } // 4b. Cap concurrent OPEN self-improve MRs per repo (PRD #686 D10/D12). This runs BEFORE // ensureTrackingIssue so a capped cycle does NO forge write. Open-state is resolved LIVE diff --git a/api/internal/store/forge.sql.go b/api/internal/store/forge.sql.go index 6e163eea7f..7649798d3b 100644 --- a/api/internal/store/forge.sql.go +++ b/api/internal/store/forge.sql.go @@ -241,6 +241,26 @@ func (q *Queries) ClearBoardOrderExcept(ctx context.Context, arg ClearBoardOrder return err } +const clearIssueTrackerColumnsNotIn = `-- name: ClearIssueTrackerColumnsNotIn :exec +UPDATE issues +SET issue_tracker_column = NULL +WHERE repo_id = $1 AND issue_tracker_column <> ALL($2::text[]) +` + +type ClearIssueTrackerColumnsNotInParams struct { + RepoID uuid.UUID `json:"repo_id"` + Keep []string `json:"keep"` +} + +// Clears the cached column of every issue of the repo whose column is no longer one of +// the board's columns (PRD #4173 M5), so a removed column puts its cards back in the Open +// lane instead of leaving a column no board shows. keep is never NULL (NULL would match +// nothing): an empty array clears every cached column. +func (q *Queries) ClearIssueTrackerColumnsNotIn(ctx context.Context, arg ClearIssueTrackerColumnsNotInParams) error { + _, err := q.db.Exec(ctx, clearIssueTrackerColumnsNotIn, arg.RepoID, arg.Keep) + return err +} + const clearRepoGuardrailOverride = `-- name: ClearRepoGuardrailOverride :one UPDATE repos SET guardrail_override_reason = NULL, @@ -692,7 +712,7 @@ func (q *Queries) GetForgeConnectionForUser(ctx context.Context, arg GetForgeCon } const getIssueByIID = `-- name: GetIssueByIID :one -SELECT id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2 +SELECT id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key, issue_tracker_column FROM issues WHERE repo_id = $1 AND forge_issue_iid = $2 ` type GetIssueByIIDParams struct { @@ -718,6 +738,7 @@ func (q *Queries) GetIssueByIID(ctx context.Context, arg GetIssueByIIDParams) (I &i.BoardPosition, &i.AssigneeIds, &i.IssueKey, + &i.IssueTrackerColumn, ) return i, err } @@ -1144,7 +1165,12 @@ func (q *Queries) ListBoardColumns(ctx context.Context, repoID uuid.UUID) ([]Boa } const listBoardFreeMRStateWatchCandidates = `-- name: ListBoardFreeMRStateWatchCandidates :many -SELECT id, branch, mr_iid, mr_state +SELECT id, branch, mr_iid, mr_state, issue_iid, kind, + (issue_iid IS NULL OR kind IN ('prompt', 'self_improve') + OR NOT EXISTS (SELECT 1 FROM runs nl + WHERE nl.repo_id = runs.repo_id AND nl.issue_iid = runs.issue_iid + AND nl.kind NOT IN ('prompt', 'self_improve') + AND nl.created_at > runs.created_at))::boolean AS is_latest FROM runs WHERE repo_id = $1::uuid AND (issue_iid IS NULL OR kind IN ('prompt', 'self_improve') @@ -1167,10 +1193,13 @@ LIMIT 100 ` type ListBoardFreeMRStateWatchCandidatesRow struct { - ID uuid.UUID `json:"id"` - Branch pgtype.Text `json:"branch"` - MrIid pgtype.Int8 `json:"mr_iid"` - MrState pgtype.Text `json:"mr_state"` + ID uuid.UUID `json:"id"` + Branch pgtype.Text `json:"branch"` + MrIid pgtype.Int8 `json:"mr_iid"` + MrState pgtype.Text `json:"mr_state"` + IssueIid pgtype.Int8 `json:"issue_iid"` + Kind string `json:"kind"` + IsLatest bool `json:"is_latest"` } // Board-FREE MR-state watch for every issue-less MR-bearing run — PRD #908, widened in @@ -1190,15 +1219,21 @@ type ListBoardFreeMRStateWatchCandidatesRow struct { // ListMRWatchCandidates), not a sqlc param, so zero Go signature change. A repo linked to an // external issue tracker (PRD #4173 M4) is widened IN whole: every issue-lane run of such a repo // belongs here (the board lane excludes the repo), so linked runs record mr_state and cancel -// rework with no board card; an unlinked repo matches exactly as before. On a linked repo the +// rework, and the issue's latest run moves the linked card on a close or reopen (M5); an unlinked repo matches exactly as before. On a linked repo the // lane also keeps watching an issue-lane run recorded mr_state='closed', so a reopen is recorded // back to 'opened' (the board lane's Lane A reopen-edge watch, Decision 10), bounded as Lane A is: // only the issue's LATEST run (no newer non-scheduled run of the issue, so a superseded rework // MR is not watched) and only while the cached issue exists AND is opened (an evicted or closed // issue is not polled forever). Live rows (NULL/opened/locked mr_state) sort ahead of closed // reopen-watch rows so the latter can never starve them under the LIMIT; among live rows the -// order is created_at DESC as before, so an unlinked repo's order is unchanged. Keep this rationale -// ABOVE the statement — a comment trailing after the `;` is grabbed by sqlc as the NEXT query's doc. +// order is created_at DESC as before, so an unlinked repo's order is unchanged. The output also +// carries issue_iid, kind and is_latest (PRD #4173 M5): the caller tells an issue-lane run from a +// scheduled or issue-less one by the first two, and is_latest is true iff no newer run of the same +// repo and issue_iid with kind NOT IN ('prompt','self_improve') exists, regardless of its status +// (the board lane's DISTINCT ON (issue_iid) ORDER BY created_at DESC semantics: every status +// counts), so a linked card moves only for its issue's latest run. They add no filter and change +// no ordering. Keep this rationale ABOVE the statement — a comment trailing after the `;` is +// grabbed by sqlc as the NEXT query's doc. func (q *Queries) ListBoardFreeMRStateWatchCandidates(ctx context.Context, repoID uuid.UUID) ([]ListBoardFreeMRStateWatchCandidatesRow, error) { rows, err := q.db.Query(ctx, listBoardFreeMRStateWatchCandidates, repoID) if err != nil { @@ -1213,6 +1248,9 @@ func (q *Queries) ListBoardFreeMRStateWatchCandidates(ctx context.Context, repoI &i.Branch, &i.MrIid, &i.MrState, + &i.IssueIid, + &i.Kind, + &i.IsLatest, ); err != nil { return nil, err } @@ -1509,7 +1547,7 @@ func (q *Queries) ListForgeConnectionsByUser(ctx context.Context, userID uuid.UU } const listIssuesByRepo = `-- name: ListIssuesByRepo :many -SELECT id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key FROM issues WHERE repo_id = $1 +SELECT id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key, issue_tracker_column FROM issues WHERE repo_id = $1 ORDER BY board_position ASC NULLS LAST, forge_issue_iid ASC ` @@ -1556,6 +1594,7 @@ func (q *Queries) ListIssuesByRepo(ctx context.Context, repoID uuid.UUID) ([]Iss &i.BoardPosition, &i.AssigneeIds, &i.IssueKey, + &i.IssueTrackerColumn, ); err != nil { return nil, err } @@ -1751,9 +1790,9 @@ JOIN issues i ON i.repo_id = $1 AND i.forge_issue_iid = l.issue_iid WHERE l.status = 'completed' AND l.mr_iid IS NOT NULL -- A repo linked to an external issue tracker (PRD #4173 M4) is owned by - -- ListBoardFreeMRStateWatchCandidates entirely: uzi moves no card there (writes arrive - -- in M5), so its MR state must not depend on the issue cache or a column label. Excluding - -- the repo here keeps the two lanes disjoint. + -- ListBoardFreeMRStateWatchCandidates entirely: that lane moves the card through the + -- issue tracker (PRD #4173 M5), so its MR state must not depend on the issue cache or a + -- column label. Excluding the repo here keeps the two lanes disjoint. AND NOT EXISTS (SELECT 1 FROM repos lr WHERE lr.id = $1 AND lr.issue_tracker_connection_id IS NOT NULL) AND ( -- Lane A (UNCHANGED): open-issue board-move watch — card in Human Review @@ -2047,7 +2086,7 @@ SET labels = COALESCE((SELECT jsonb_agg(value ORDER BY ordinal) FROM jsonb_array_elements(labels) WITH ORDINALITY AS item(value, ordinal) WHERE value <> to_jsonb($1::text)), '[]'::jsonb) WHERE repo_id = $2 AND forge_issue_iid = $3 -RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key +RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key, issue_tracker_column ` type RemoveCachedIssueLabelParams struct { @@ -2076,6 +2115,7 @@ func (q *Queries) RemoveCachedIssueLabel(ctx context.Context, arg RemoveCachedIs &i.BoardPosition, &i.AssigneeIds, &i.IssueKey, + &i.IssueTrackerColumn, ) return i, err } @@ -2084,7 +2124,7 @@ const reopenIssueState = `-- name: ReopenIssueState :one UPDATE issues SET state = 'opened', board_position = NULL, synced_at = now() WHERE repo_id = $1 AND forge_issue_iid = $2 -RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key +RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key, issue_tracker_column ` type ReopenIssueStateParams struct { @@ -2118,6 +2158,7 @@ func (q *Queries) ReopenIssueState(ctx context.Context, arg ReopenIssueStatePara &i.BoardPosition, &i.AssigneeIds, &i.IssueKey, + &i.IssueTrackerColumn, ) return i, err } @@ -2208,6 +2249,47 @@ func (q *Queries) SetForgeConnectionHumanUsername(ctx context.Context, arg SetFo return i, err } +const setIssueTrackerColumn = `-- name: SetIssueTrackerColumn :one +UPDATE issues +SET issue_tracker_column = NULLIF($1::text, ''), synced_at = now() +WHERE repo_id = $2 AND forge_issue_iid = $3 +RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key, issue_tracker_column +` + +type SetIssueTrackerColumnParams struct { + IssueTrackerColumn string `json:"issue_tracker_column"` + RepoID uuid.UUID `json:"repo_id"` + ForgeIssueIid int64 `json:"forge_issue_iid"` +} + +// The cached board column of a linked card after a successful move (PRD #4173 M5): +// writes ONLY issue_tracker_column (an empty column is the Open lane and is stored as +// NULL, because the table's CHECK refuses an empty string) and synced_at, never state, labels or +// board_position. Forge-first like every cache write here: the caller has already moved +// the issue in the issue tracker. RETURNING * for the single-card re-render. +func (q *Queries) SetIssueTrackerColumn(ctx context.Context, arg SetIssueTrackerColumnParams) (Issue, error) { + row := q.db.QueryRow(ctx, setIssueTrackerColumn, arg.IssueTrackerColumn, arg.RepoID, arg.ForgeIssueIid) + var i Issue + err := row.Scan( + &i.ID, + &i.RepoID, + &i.ForgeIssueIid, + &i.Title, + &i.State, + &i.Labels, + &i.WebUrl, + &i.Author, + &i.HasPrdLink, + &i.ForgeUpdatedAt, + &i.SyncedAt, + &i.BoardPosition, + &i.AssigneeIds, + &i.IssueKey, + &i.IssueTrackerColumn, + ) + return i, err +} + const setRepoAgentSourceDefault = `-- name: SetRepoAgentSourceDefault :one UPDATE repos SET agent_source_default = $1::text WHERE repos.id = $2 @@ -3242,7 +3324,7 @@ const updateIssueState = `-- name: UpdateIssueState :one UPDATE issues SET state = $3, synced_at = now() WHERE repo_id = $1 AND forge_issue_iid = $2 -RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key +RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key, issue_tracker_column ` type UpdateIssueStateParams struct { @@ -3278,6 +3360,7 @@ func (q *Queries) UpdateIssueState(ctx context.Context, arg UpdateIssueStatePara &i.BoardPosition, &i.AssigneeIds, &i.IssueKey, + &i.IssueTrackerColumn, ) return i, err } @@ -3384,7 +3467,7 @@ SET title = EXCLUDED.title, -- A forge-only writer passes NULL; it must not erase the key of a linked row. issue_key = COALESCE(EXCLUDED.issue_key, issues.issue_key), synced_at = now() -RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key +RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key, issue_tracker_column ` type UpsertIssueParams struct { @@ -3442,6 +3525,7 @@ func (q *Queries) UpsertIssue(ctx context.Context, arg UpsertIssueParams) (Issue &i.BoardPosition, &i.AssigneeIds, &i.IssueKey, + &i.IssueTrackerColumn, ) return i, err } @@ -3449,8 +3533,8 @@ func (q *Queries) UpsertIssue(ctx context.Context, arg UpsertIssueParams) (Issue const upsertIssueIfChanged = `-- name: UpsertIssueIfChanged :exec INSERT INTO issues ( repo_id, forge_issue_iid, title, state, labels, assignee_ids, web_url, author, - has_prd_link, forge_updated_at, issue_key, synced_at -) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, now()) + has_prd_link, forge_updated_at, issue_key, issue_tracker_column, synced_at +) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, now()) ON CONFLICT (repo_id, forge_issue_iid) DO UPDATE SET title = EXCLUDED.title, state = EXCLUDED.state, @@ -3461,33 +3545,38 @@ SET title = EXCLUDED.title, has_prd_link = EXCLUDED.has_prd_link, forge_updated_at = EXCLUDED.forge_updated_at, issue_key = EXCLUDED.issue_key, + issue_tracker_column = EXCLUDED.issue_tracker_column, synced_at = now() WHERE (issues.title, issues.state, issues.labels, issues.assignee_ids, issues.web_url, issues.author, - issues.has_prd_link, issues.forge_updated_at, issues.issue_key) + issues.has_prd_link, issues.forge_updated_at, issues.issue_key, issues.issue_tracker_column) IS DISTINCT FROM (EXCLUDED.title, EXCLUDED.state, EXCLUDED.labels, EXCLUDED.assignee_ids, EXCLUDED.web_url, EXCLUDED.author, - EXCLUDED.has_prd_link, EXCLUDED.forge_updated_at, EXCLUDED.issue_key) + EXCLUDED.has_prd_link, EXCLUDED.forge_updated_at, EXCLUDED.issue_key, EXCLUDED.issue_tracker_column) ` type UpsertIssueIfChangedParams struct { - RepoID uuid.UUID `json:"repo_id"` - ForgeIssueIid int64 `json:"forge_issue_iid"` - Title string `json:"title"` - State string `json:"state"` - Labels []byte `json:"labels"` - AssigneeIds []byte `json:"assignee_ids"` - WebUrl string `json:"web_url"` - Author pgtype.Text `json:"author"` - HasPrdLink bool `json:"has_prd_link"` - ForgeUpdatedAt pgtype.Timestamptz `json:"forge_updated_at"` - IssueKey pgtype.Text `json:"issue_key"` + RepoID uuid.UUID `json:"repo_id"` + ForgeIssueIid int64 `json:"forge_issue_iid"` + Title string `json:"title"` + State string `json:"state"` + Labels []byte `json:"labels"` + AssigneeIds []byte `json:"assignee_ids"` + WebUrl string `json:"web_url"` + Author pgtype.Text `json:"author"` + HasPrdLink bool `json:"has_prd_link"` + ForgeUpdatedAt pgtype.Timestamptz `json:"forge_updated_at"` + IssueKey pgtype.Text `json:"issue_key"` + IssueTrackerColumn pgtype.Text `json:"issue_tracker_column"` } // The linked issue sync's write (PRD #4173): the same columns as UpsertIssue (and the same // board_position omission, with the same reason), but a row whose cached values are all // unchanged is left untouched, so a tick that finds nothing new rewrites nothing and // issues.synced_at is not refreshed by it. Nothing reads issues.synced_at. issue_key is -// compared as written (the linked sync always passes it). +// compared as written (the linked sync always passes it). It is also the one upsert that +// writes issues.issue_tracker_column (PRD #4173 M5), the card's board column read from its +// issue tracker State: NULL is the Open lane, and the column is part of the change check so +// a changed State mapping is cached again. UpsertIssue and UpsertIssueLabels never touch it. func (q *Queries) UpsertIssueIfChanged(ctx context.Context, arg UpsertIssueIfChangedParams) error { _, err := q.db.Exec(ctx, upsertIssueIfChanged, arg.RepoID, @@ -3501,6 +3590,7 @@ func (q *Queries) UpsertIssueIfChanged(ctx context.Context, arg UpsertIssueIfCha arg.HasPrdLink, arg.ForgeUpdatedAt, arg.IssueKey, + arg.IssueTrackerColumn, ) return err } @@ -3536,7 +3626,7 @@ SET title = EXCLUDED.title, has_prd_link = EXCLUDED.has_prd_link, forge_updated_at = EXCLUDED.forge_updated_at, synced_at = now() -RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key +RETURNING id, repo_id, forge_issue_iid, title, state, labels, web_url, author, has_prd_link, forge_updated_at, synced_at, board_position, assignee_ids, issue_key, issue_tracker_column ` type UpsertIssueLabelsParams struct { @@ -3603,6 +3693,7 @@ func (q *Queries) UpsertIssueLabels(ctx context.Context, arg UpsertIssueLabelsPa &i.BoardPosition, &i.AssigneeIds, &i.IssueKey, + &i.IssueTrackerColumn, ) return i, err } diff --git a/api/internal/store/issue_tracker_links.sql.go b/api/internal/store/issue_tracker_links.sql.go index 34c2eaa3d0..8c666a6cdd 100644 --- a/api/internal/store/issue_tracker_links.sql.go +++ b/api/internal/store/issue_tracker_links.sql.go @@ -30,11 +30,15 @@ SELECT r.id, r.forge_project_id, r.awaiting_first_enable, r.issue_tracker_connection_id, r.issue_tracker_project_id, r.issue_tracker_project_short, r.eligibility_tag_id, r.eligibility_tag_name, r.issue_tracker_linked_at, c.forge_type, c.base_url, c.token_ciphertext, - itc.id AS linked_connection_id + itc.id AS linked_connection_id, + s.open_state, s.closed_state, + COALESCE((SELECT jsonb_object_agg(cs.label_name, cs.state_name) + FROM repo_issue_tracker_column_states cs WHERE cs.repo_id = r.id), '{}'::jsonb)::jsonb AS column_states FROM repos r JOIN forge_connections c ON c.id = r.connection_id LEFT JOIN issue_tracker_connections itc ON itc.id = r.issue_tracker_connection_id AND itc.user_id = c.user_id +LEFT JOIN repo_issue_tracker_states s ON s.repo_id = r.id WHERE r.id = $1 AND c.user_id = $2 ` @@ -57,6 +61,9 @@ type GetRepoIssueSourceForUserRow struct { BaseUrl string `json:"base_url"` TokenCiphertext []byte `json:"token_ciphertext"` LinkedConnectionID pgtype.UUID `json:"linked_connection_id"` + OpenState pgtype.Text `json:"open_state"` + ClosedState pgtype.Text `json:"closed_state"` + ColumnStates []byte `json:"column_states"` } // Repo to issue tracker project links (PRD #4173 M4, DL-3..DL-5). @@ -65,7 +72,9 @@ type GetRepoIssueSourceForUserRow struct { // owning user, never by forge project number. linked_connection_id is the link's // connection ONLY when it belongs to the same user as the forge connection; a linked // repo whose connection is missing for that owner reads NULL here and the resolver -// fails closed. +// fails closed. open_state and closed_state are NULL until the state mapping is saved +// (PRD #4173 M5); column_states is the board column label to State name map, '{}' when +// no column is mapped. func (q *Queries) GetRepoIssueSourceForUser(ctx context.Context, arg GetRepoIssueSourceForUserParams) (GetRepoIssueSourceForUserRow, error) { row := q.db.QueryRow(ctx, getRepoIssueSourceForUser, arg.ID, arg.UserID) var i GetRepoIssueSourceForUserRow @@ -83,6 +92,9 @@ func (q *Queries) GetRepoIssueSourceForUser(ctx context.Context, arg GetRepoIssu &i.BaseUrl, &i.TokenCiphertext, &i.LinkedConnectionID, + &i.OpenState, + &i.ClosedState, + &i.ColumnStates, ) return i, err } diff --git a/api/internal/store/issue_tracker_state_mapping_livedb_test.go b/api/internal/store/issue_tracker_state_mapping_livedb_test.go new file mode 100644 index 0000000000..5b7344d018 --- /dev/null +++ b/api/internal/store/issue_tracker_state_mapping_livedb_test.go @@ -0,0 +1,259 @@ +package store_test + +import ( + "context" + "encoding/json" + "errors" + "strings" + "testing" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5" + "github.com/jackc/pgx/v5/pgxpool" + + "github.com/vtmocanu/uzi/api/internal/store" +) + +// issue_tracker_state_mapping_livedb_test.go proves migration 00468 (PRD #4173 M5) and its +// queries against a real Postgres on an isolated database: the name and pair CHECKs, the +// one-column-per-State uniqueness, the cascades, the owner-and-linked scoping of the upsert +// and the lock, the resolver row's new columns, the prune and the issues column. +// Skipped without UZI_TEST_DATABASE_URL. + +const checkViolation = "23514" +const uniqueViolation = "23505" +const foreignKeyViolation = "23503" + +type stateMappingFixture struct { + ctx context.Context + q *store.Queries + pool *pgxpool.Pool + user, stranger uuid.UUID + linked, plain uuid.UUID +} + +func newStateMappingFixture(t *testing.T) (*stateMappingFixture, *store.Queries, func(sql string, args ...any)) { + t.Helper() + ctx, dsn := standIsolatedDB(t, "state_mapping_") + if err := store.Migrate(ctx, dsn); err != nil { + t.Fatalf("migrate: %v", err) + } + pool, err := store.OpenPool(ctx, dsn) + if err != nil { + t.Fatal(err) + } + t.Cleanup(pool.Close) + exec := func(sql string, args ...any) { + t.Helper() + itlExec(t, pool, sql, args...) + } + f := &stateMappingFixture{ctx: ctx, q: store.New(pool), pool: pool, + user: uuid.New(), stranger: uuid.New(), linked: uuid.New(), plain: uuid.New()} + forgeConn, itc := uuid.New(), uuid.New() + for _, u := range []uuid.UUID{f.user, f.stranger} { + exec(`INSERT INTO users (id,email,password_hash) VALUES ($1,$2,'x')`, u, u.String()+"@example.test") + } + exec(`INSERT INTO forge_connections (id,user_id,forge_type,base_url,bot_username,bot_forge_user_id,token_ciphertext) + VALUES ($1,$2,'github','https://example.com','bot',1,'x')`, forgeConn, f.user) + exec(`INSERT INTO issue_tracker_connections (id,user_id,kind,base_url,account_id,account_login,token_ciphertext) + VALUES ($1,$2,'youtrack','https://yt.example.com','a','al','x')`, itc, f.user) + exec(`INSERT INTO repos (id,connection_id,forge_project_id,path_with_namespace,web_url,enabled,awaiting_first_enable, + issue_tracker_connection_id,issue_tracker_project_id,issue_tracker_project_short,eligibility_tag_id,eligibility_tag_name,issue_tracker_linked_at) + VALUES ($1,$2,1,'o/linked','https://example.com/o/linked',true,false,$3,'0-1','UZI','5-1','uzi',now())`, f.linked, forgeConn, itc) + exec(`INSERT INTO repos (id,connection_id,forge_project_id,path_with_namespace,web_url,enabled) + VALUES ($1,$2,2,'o/plain','https://example.com/o/plain',true)`, f.plain, forgeConn) + return f, f.q, exec +} + +func (f *stateMappingFixture) upsert(t *testing.T, repo, user uuid.UUID, open, closed string) error { + t.Helper() + _, err := f.q.UpsertRepoIssueTrackerStates(f.ctx, store.UpsertRepoIssueTrackerStatesParams{OpenState: open, ClosedState: closed, RepoID: repo, UserID: user}) + return err +} + +func TestIssueTrackerStateMappingChecksLiveDB(t *testing.T) { + f, q, exec := newStateMappingFixture(t) + + bad := []struct{ name, open, closed string }{ + {"empty open", "", "Fixed"}, + {"empty closed", "Open", ""}, + {"too long", strings.Repeat("x", 201), "Fixed"}, + {"control character", "Op\ten", "Fixed"}, + {"newline", "Open", "Fi\nxed"}, + {"same pair", "Open", "Open"}, + } + for _, tc := range bad { + t.Run(tc.name, func(t *testing.T) { + if err := f.upsert(t, f.linked, f.user, tc.open, tc.closed); sqlState(err) != checkViolation { + t.Fatalf("err = %v (state %q), want a check violation", err, sqlState(err)) + } + }) + } + // The bounds are inclusive: 200 characters, counted as characters and not bytes. + if err := f.upsert(t, f.linked, f.user, strings.Repeat("é", 200), "Fixed"); err != nil { + t.Fatalf("a 200-character name: %v", err) + } + if err := f.upsert(t, f.linked, f.user, "Open", "Fixed"); err != nil { + t.Fatal(err) + } + + col := func(label, state string) error { + return q.InsertRepoIssueTrackerColumnState(f.ctx, store.InsertRepoIssueTrackerColumnStateParams{RepoID: f.linked, LabelName: label, StateName: state}) + } + if err := col("In Progress", "Doing"); err != nil { + t.Fatal(err) + } + if err := col("Review", "Doing"); sqlState(err) != uniqueViolation { + t.Errorf("a State mapped to two columns: %v, want a unique violation", err) + } + if err := col("In Progress", "Other"); sqlState(err) != uniqueViolation { + t.Errorf("a column mapped twice: %v, want a unique violation", err) + } + if err := col("Review", "Do\x07ing"); sqlState(err) != checkViolation { + t.Errorf("a control character in a column State: %v, want a check violation", err) + } + if err := col("Review", strings.Repeat("y", 201)); sqlState(err) != checkViolation { + t.Errorf("an overlong column State: %v, want a check violation", err) + } + if err := col("Re\tview", "Reviewing"); sqlState(err) != checkViolation { + t.Errorf("a control character in a column label: %v, want a check violation", err) + } + if err := col(strings.Repeat("y", 201), "Reviewing"); sqlState(err) != checkViolation { + t.Errorf("an overlong column label: %v, want a check violation", err) + } + if err := col("", "Reviewing"); sqlState(err) != checkViolation { + t.Errorf("an empty column label: %v, want a check violation", err) + } + // A column row needs the states row. + if err := q.InsertRepoIssueTrackerColumnState(f.ctx, store.InsertRepoIssueTrackerColumnStateParams{RepoID: uuid.New(), LabelName: "a", StateName: "b"}); sqlState(err) != foreignKeyViolation { + t.Errorf("a column row with no states row: %v, want a foreign key violation", err) + } + + // issues.issue_tracker_column is nullable and bounded. + exec(`INSERT INTO issues (repo_id, forge_issue_iid, title, state, labels, web_url, forge_updated_at, synced_at) + VALUES ($1, 1, 't', 'opened', '[]', 'https://example.com/i', now(), now())`, f.linked) + var isNull bool + if err := f.pool.QueryRow(f.ctx, `SELECT issue_tracker_column IS NULL FROM issues WHERE repo_id = $1`, f.linked).Scan(&isNull); err != nil || !isNull { + t.Fatalf("issue_tracker_column null = %v (%v), want a NULL default", isNull, err) + } +} + +func TestIssueTrackerStateMappingQueriesLiveDB(t *testing.T) { + f, q, exec := newStateMappingFixture(t) + exec(`INSERT INTO board_columns (repo_id,label_name,position) VALUES ($1,'In Progress',0),($1,'Review',1),($1,'Done',2)`, f.linked) + + read := func() store.GetRepoIssueSourceForUserRow { + t.Helper() + row, err := q.GetRepoIssueSourceForUser(f.ctx, store.GetRepoIssueSourceForUserParams{ID: f.linked, UserID: f.user}) + if err != nil { + t.Fatal(err) + } + return row + } + // No mapping yet: the optional values are NULL and the column map is empty. + if row := read(); row.OpenState.Valid || row.ClosedState.Valid || string(row.ColumnStates) != "{}" { + t.Fatalf("resolver row before a mapping = %+v", row) + } + + // The upsert is owner-scoped and needs a linked repo. + if err := f.upsert(t, f.linked, f.stranger, "Open", "Fixed"); !errors.Is(err, pgx.ErrNoRows) { + t.Errorf("upsert as a stranger: %v, want no rows", err) + } + if err := f.upsert(t, f.plain, f.user, "Open", "Fixed"); !errors.Is(err, pgx.ErrNoRows) { + t.Errorf("upsert on an unlinked repo: %v, want no rows", err) + } + if _, err := q.LockRepoForStateMappingForUser(f.ctx, store.LockRepoForStateMappingForUserParams{ID: f.linked, UserID: f.stranger}); !errors.Is(err, pgx.ErrNoRows) { + t.Errorf("lock as a stranger: %v, want no rows", err) + } + if lock, err := q.LockRepoForStateMappingForUser(f.ctx, store.LockRepoForStateMappingForUserParams{ID: f.plain, UserID: f.user}); err != nil || lock.IssueTrackerConnectionID.Valid { + t.Errorf("lock of an unlinked repo = %+v (%v), want the row, unlinked", lock, err) + } + + if err := f.upsert(t, f.linked, f.user, "Open", "Fixed"); err != nil { + t.Fatal(err) + } + for label, state := range map[string]string{"In Progress": "Doing", "Review": "In Review"} { + if err := q.InsertRepoIssueTrackerColumnState(f.ctx, store.InsertRepoIssueTrackerColumnStateParams{RepoID: f.linked, LabelName: label, StateName: state}); err != nil { + t.Fatal(err) + } + } + row := read() + var cols map[string]string + if err := json.Unmarshal(row.ColumnStates, &cols); err != nil { + t.Fatal(err) + } + if row.OpenState.String != "Open" || row.ClosedState.String != "Fixed" || len(cols) != 2 || cols["In Progress"] != "Doing" || cols["Review"] != "In Review" { + t.Fatalf("resolver row = open %q closed %q columns %v", row.OpenState.String, row.ClosedState.String, cols) + } + // The stranger still sees nothing of it. + if _, err := q.GetRepoIssueSourceForUser(f.ctx, store.GetRepoIssueSourceForUserParams{ID: f.linked, UserID: f.stranger}); !errors.Is(err, pgx.ErrNoRows) { + t.Errorf("resolver row as a stranger: %v, want no rows", err) + } + + // A second upsert replaces the pair and keeps the column rows. + if err := f.upsert(t, f.linked, f.user, "Doing", "Done"); err != nil { + t.Fatal(err) + } + if row := read(); row.OpenState.String != "Doing" || row.ClosedState.String != "Done" || !strings.Contains(string(row.ColumnStates), "Review") { + t.Fatalf("after a second upsert: %+v", row) + } + + // Prune keeps the listed labels only; an empty keep list removes every row. + if err := q.PruneRepoIssueTrackerColumnStates(f.ctx, store.PruneRepoIssueTrackerColumnStatesParams{RepoID: f.linked, Keep: []string{"Review", "Backlog"}}); err != nil { + t.Fatal(err) + } + if got := string(read().ColumnStates); got != `{"Review": "In Review"}` { + t.Fatalf("after pruning to Review: %s", got) + } + if err := q.PruneRepoIssueTrackerColumnStates(f.ctx, store.PruneRepoIssueTrackerColumnStatesParams{RepoID: f.linked, Keep: []string{}}); err != nil { + t.Fatal(err) + } + if got := string(read().ColumnStates); got != "{}" { + t.Fatalf("after pruning to nothing: %s", got) + } + + // Delete then insert is the replace the route does; the states row survives it. + if err := q.InsertRepoIssueTrackerColumnState(f.ctx, store.InsertRepoIssueTrackerColumnStateParams{RepoID: f.linked, LabelName: "Done", StateName: "Shipped"}); err != nil { + t.Fatal(err) + } + if err := q.DeleteRepoIssueTrackerColumnStates(f.ctx, f.linked); err != nil { + t.Fatal(err) + } + if row := read(); string(row.ColumnStates) != "{}" || !row.OpenState.Valid { + t.Fatalf("after deleting the column rows: %+v", row) + } +} + +func TestIssueTrackerStateMappingCascadesLiveDB(t *testing.T) { + f, q, exec := newStateMappingFixture(t) + if err := f.upsert(t, f.linked, f.user, "Open", "Fixed"); err != nil { + t.Fatal(err) + } + if err := q.InsertRepoIssueTrackerColumnState(f.ctx, store.InsertRepoIssueTrackerColumnStateParams{RepoID: f.linked, LabelName: "a", StateName: "b"}); err != nil { + t.Fatal(err) + } + count := func(table string) int { + t.Helper() + var n int + if err := f.pool.QueryRow(f.ctx, `SELECT count(*) FROM `+table).Scan(&n); err != nil { + t.Fatal(err) + } + return n + } + // Deleting the states row takes its column rows with it. + exec(`DELETE FROM repo_issue_tracker_states WHERE repo_id = $1`, f.linked) + if n := count("repo_issue_tracker_column_states"); n != 0 { + t.Fatalf("column rows after the states row is gone = %d, want 0", n) + } + // Deleting the repo takes the whole mapping. + if err := f.upsert(t, f.linked, f.user, "Open", "Fixed"); err != nil { + t.Fatal(err) + } + if err := q.InsertRepoIssueTrackerColumnState(f.ctx, store.InsertRepoIssueTrackerColumnStateParams{RepoID: f.linked, LabelName: "a", StateName: "b"}); err != nil { + t.Fatal(err) + } + exec(`DELETE FROM repos WHERE id = $1`, f.linked) + if s, c := count("repo_issue_tracker_states"), count("repo_issue_tracker_column_states"); s != 0 || c != 0 { + t.Fatalf("after the repo is deleted: %d states rows, %d column rows, want none", s, c) + } +} diff --git a/api/internal/store/issue_tracker_states.sql.go b/api/internal/store/issue_tracker_states.sql.go new file mode 100644 index 0000000000..a9960cda28 --- /dev/null +++ b/api/internal/store/issue_tracker_states.sql.go @@ -0,0 +1,131 @@ +// Code generated by sqlc. DO NOT EDIT. +// versions: +// sqlc v1.31.1 +// source: issue_tracker_states.sql + +package store + +import ( + "context" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5/pgtype" +) + +const deleteRepoIssueTrackerColumnStates = `-- name: DeleteRepoIssueTrackerColumnStates :exec +DELETE FROM repo_issue_tracker_column_states WHERE repo_id = $1 +` + +// Removes every column mapping of the repo; the mapping route calls it before it inserts the +// new set, in one transaction. +func (q *Queries) DeleteRepoIssueTrackerColumnStates(ctx context.Context, repoID uuid.UUID) error { + _, err := q.db.Exec(ctx, deleteRepoIssueTrackerColumnStates, repoID) + return err +} + +const insertRepoIssueTrackerColumnState = `-- name: InsertRepoIssueTrackerColumnState :exec +INSERT INTO repo_issue_tracker_column_states (repo_id, label_name, state_name) +VALUES ($1, $2, $3) +` + +type InsertRepoIssueTrackerColumnStateParams struct { + RepoID uuid.UUID `json:"repo_id"` + LabelName string `json:"label_name"` + StateName string `json:"state_name"` +} + +func (q *Queries) InsertRepoIssueTrackerColumnState(ctx context.Context, arg InsertRepoIssueTrackerColumnStateParams) error { + _, err := q.db.Exec(ctx, insertRepoIssueTrackerColumnState, arg.RepoID, arg.LabelName, arg.StateName) + return err +} + +const lockRepoForStateMappingForUser = `-- name: LockRepoForStateMappingForUser :one + +SELECT r.id, r.issue_tracker_connection_id, r.issue_tracker_project_id +FROM repos r +JOIN forge_connections c ON c.id = r.connection_id +WHERE r.id = $1 AND c.user_id = $2 +FOR NO KEY UPDATE OF r +` + +type LockRepoForStateMappingForUserParams struct { + ID uuid.UUID `json:"id"` + UserID uuid.UUID `json:"user_id"` +} + +type LockRepoForStateMappingForUserRow struct { + ID uuid.UUID `json:"id"` + IssueTrackerConnectionID pgtype.UUID `json:"issue_tracker_connection_id"` + IssueTrackerProjectID pgtype.Text `json:"issue_tracker_project_id"` +} + +// The board column to issue tracker State mapping of a linked repo (PRD #4173 M5). +// The mapping is read through GetRepoIssueSourceForUser; the statements here write it. +// Takes the repo row lock the state mapping route and a board column reconfigure of a linked +// repo share, scoped to the owning user, so the two serialise: a mapping is never validated +// against board columns that a concurrent reconfigure is deleting. FOR NO KEY UPDATE does not +// block the FOR KEY SHARE that inserts of rows referencing the repo take, and conflicts with +// the link's and the enable's locks. Call inside a transaction. The row is returned whether or +// not the repo is linked; the caller decides. +func (q *Queries) LockRepoForStateMappingForUser(ctx context.Context, arg LockRepoForStateMappingForUserParams) (LockRepoForStateMappingForUserRow, error) { + row := q.db.QueryRow(ctx, lockRepoForStateMappingForUser, arg.ID, arg.UserID) + var i LockRepoForStateMappingForUserRow + err := row.Scan(&i.ID, &i.IssueTrackerConnectionID, &i.IssueTrackerProjectID) + return i, err +} + +const pruneRepoIssueTrackerColumnStates = `-- name: PruneRepoIssueTrackerColumnStates :exec +DELETE FROM repo_issue_tracker_column_states +WHERE repo_id = $1 AND label_name <> ALL($2::text[]) +` + +type PruneRepoIssueTrackerColumnStatesParams struct { + RepoID uuid.UUID `json:"repo_id"` + Keep []string `json:"keep"` +} + +// Removes the column mappings whose label is not in @keep. An empty @keep removes them all, +// which is the right answer when the board has no column left (the opposite trap to a +// "keep" list that wipes everything by accident: here the empty set is the intent). +func (q *Queries) PruneRepoIssueTrackerColumnStates(ctx context.Context, arg PruneRepoIssueTrackerColumnStatesParams) error { + _, err := q.db.Exec(ctx, pruneRepoIssueTrackerColumnStates, arg.RepoID, arg.Keep) + return err +} + +const upsertRepoIssueTrackerStates = `-- name: UpsertRepoIssueTrackerStates :one +INSERT INTO repo_issue_tracker_states (repo_id, open_state, closed_state) +SELECT r.id, $1::text, $2::text +FROM repos r +JOIN forge_connections c ON c.id = r.connection_id +WHERE r.id = $3 AND c.user_id = $4 AND r.issue_tracker_connection_id IS NOT NULL +ON CONFLICT (repo_id) DO UPDATE +SET open_state = EXCLUDED.open_state, closed_state = EXCLUDED.closed_state, updated_at = now() +RETURNING repo_id, open_state, closed_state, updated_at +` + +type UpsertRepoIssueTrackerStatesParams struct { + OpenState string `json:"open_state"` + ClosedState string `json:"closed_state"` + RepoID uuid.UUID `json:"repo_id"` + UserID uuid.UUID `json:"user_id"` +} + +// Writes the open and closed State of a repo's mapping. The repo must be owned by @user_id +// through its forge connection and linked to an issue tracker: no row means refused. Call +// after LockRepoForStateMappingForUser in the same transaction. +func (q *Queries) UpsertRepoIssueTrackerStates(ctx context.Context, arg UpsertRepoIssueTrackerStatesParams) (RepoIssueTrackerState, error) { + row := q.db.QueryRow(ctx, upsertRepoIssueTrackerStates, + arg.OpenState, + arg.ClosedState, + arg.RepoID, + arg.UserID, + ) + var i RepoIssueTrackerState + err := row.Scan( + &i.RepoID, + &i.OpenState, + &i.ClosedState, + &i.UpdatedAt, + ) + return i, err +} diff --git a/api/internal/store/migrations/00468_issue_tracker_state_mapping.sql b/api/internal/store/migrations/00468_issue_tracker_state_mapping.sql new file mode 100644 index 0000000000..b0a8a8c2dc --- /dev/null +++ b/api/internal/store/migrations/00468_issue_tracker_state_mapping.sql @@ -0,0 +1,66 @@ +-- +goose Up + +-- ALTER TABLE issues ADD COLUMN takes ACCESS EXCLUSIVE on issues. The column is nullable with +-- no default, so it is a catalog change that rewrites nothing, but the lock is still queued +-- behind any long transaction and stalls every reader behind it. lock_timeout bounds only the +-- wait for a lock, not how long it is held: fail fast instead of queueing (the 00402 pattern). +SET LOCAL lock_timeout = '5s'; + +-- PRD #4173 M5: which State of the linked issue tracker project means open, which means +-- closed, and which State each board column stands for. One row per linked repo, written +-- only by the state mapping route (PUT /api/repos/{id}/issue-tracker-link/states). +-- +-- Each name is a State name as the issue tracker lists it: 1 to 200 characters +-- (maxLabelNameLen in the issuetracker package) and no control characters. The open and the +-- closed state must differ; the route also refuses a resolved open state and a closed state +-- that does not mean done, which needs the issue tracker's own listing and cannot be a CHECK. +CREATE TABLE repo_issue_tracker_states ( + repo_id uuid PRIMARY KEY REFERENCES repos(id) ON DELETE CASCADE, + open_state text NOT NULL, + closed_state text NOT NULL, + updated_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT repo_issue_tracker_states_shape CHECK ( + char_length(open_state) BETWEEN 1 AND 200 + AND char_length(closed_state) BETWEEN 1 AND 200 + AND open_state !~ '[[:cntrl:]]' + AND closed_state !~ '[[:cntrl:]]' + AND open_state <> closed_state) +); + +-- The State a board column maps to. Keyed by the column's label name, not by the +-- board_columns id: those ids change on every reconfigure, and the label name is what a +-- column is. A State maps to at most one column, so a card's State reads back as one column. +-- A column with no row here is unmapped: a move onto it is refused. Rows go with the repo's +-- mapping (the cascade) and with a column that is removed from the board (ConfigureColumns +-- prunes them in the transaction that replaces the board columns). +CREATE TABLE repo_issue_tracker_column_states ( + repo_id uuid NOT NULL REFERENCES repo_issue_tracker_states(repo_id) ON DELETE CASCADE, + label_name text NOT NULL, + state_name text NOT NULL, + PRIMARY KEY (repo_id, label_name), + CONSTRAINT repo_issue_tracker_column_states_state_unique UNIQUE (repo_id, state_name), + CONSTRAINT repo_issue_tracker_column_states_shape CHECK ( + char_length(state_name) BETWEEN 1 AND 200 + AND state_name !~ '[[:cntrl:]]' + AND char_length(label_name) BETWEEN 1 AND 200 + AND label_name !~ '[[:cntrl:]]') +); + +-- The board column a linked issue was last placed in, as uzi moved or synced it. NULL for every +-- issue of a repo that is not linked, and until a linked issue has a mapped State. Nullable and +-- additive, so an older worker that does not know it keeps reading issues unchanged. +-- +-- The CHECK is added NOT VALID, as 00415 and 00416 do: a CHECK added with the column makes +-- Postgres scan all of issues under ACCESS EXCLUSIVE, and NOT VALID skips that scan. No +-- VALIDATE step follows because every existing value is NULL, which the CHECK passes, and new +-- writes are checked either way. The Open lane is stored as NULL, never '' (the CHECK refuses +-- the empty string). +ALTER TABLE issues ADD COLUMN issue_tracker_column text; +ALTER TABLE issues ADD CONSTRAINT issues_issue_tracker_column_len + CHECK (char_length(issue_tracker_column) BETWEEN 1 AND 200) NOT VALID; + +-- +goose Down +SET LOCAL lock_timeout = '5s'; +ALTER TABLE issues DROP COLUMN issue_tracker_column; +DROP TABLE repo_issue_tracker_column_states; +DROP TABLE repo_issue_tracker_states; diff --git a/api/internal/store/models.go b/api/internal/store/models.go index b49e24e13c..d33cc2800a 100644 --- a/api/internal/store/models.go +++ b/api/internal/store/models.go @@ -579,20 +579,21 @@ type InstanceIntakePauseEvent struct { } type Issue struct { - ID uuid.UUID `json:"id"` - RepoID uuid.UUID `json:"repo_id"` - ForgeIssueIid int64 `json:"forge_issue_iid"` - Title string `json:"title"` - State string `json:"state"` - Labels []byte `json:"labels"` - WebUrl string `json:"web_url"` - Author pgtype.Text `json:"author"` - HasPrdLink bool `json:"has_prd_link"` - ForgeUpdatedAt pgtype.Timestamptz `json:"forge_updated_at"` - SyncedAt pgtype.Timestamptz `json:"synced_at"` - BoardPosition pgtype.Int8 `json:"board_position"` - AssigneeIds []byte `json:"assignee_ids"` - IssueKey pgtype.Text `json:"issue_key"` + ID uuid.UUID `json:"id"` + RepoID uuid.UUID `json:"repo_id"` + ForgeIssueIid int64 `json:"forge_issue_iid"` + Title string `json:"title"` + State string `json:"state"` + Labels []byte `json:"labels"` + WebUrl string `json:"web_url"` + Author pgtype.Text `json:"author"` + HasPrdLink bool `json:"has_prd_link"` + ForgeUpdatedAt pgtype.Timestamptz `json:"forge_updated_at"` + SyncedAt pgtype.Timestamptz `json:"synced_at"` + BoardPosition pgtype.Int8 `json:"board_position"` + AssigneeIds []byte `json:"assignee_ids"` + IssueKey pgtype.Text `json:"issue_key"` + IssueTrackerColumn pgtype.Text `json:"issue_tracker_column"` } type IssueProposal struct { @@ -1085,6 +1086,19 @@ type Repo struct { IssueTrackerLinkedAt pgtype.Timestamptz `json:"issue_tracker_linked_at"` } +type RepoIssueTrackerColumnState struct { + RepoID uuid.UUID `json:"repo_id"` + LabelName string `json:"label_name"` + StateName string `json:"state_name"` +} + +type RepoIssueTrackerState struct { + RepoID uuid.UUID `json:"repo_id"` + OpenState string `json:"open_state"` + ClosedState string `json:"closed_state"` + UpdatedAt pgtype.Timestamptz `json:"updated_at"` +} + type RepoRemovalMarker struct { UserID uuid.UUID `json:"user_id"` ForgeType string `json:"forge_type"` diff --git a/api/internal/store/queries/forge.sql b/api/internal/store/queries/forge.sql index 17a1bc20e2..7f9f3d21eb 100644 --- a/api/internal/store/queries/forge.sql +++ b/api/internal/store/queries/forge.sql @@ -552,11 +552,14 @@ RETURNING *; -- board_position omission, with the same reason), but a row whose cached values are all -- unchanged is left untouched, so a tick that finds nothing new rewrites nothing and -- issues.synced_at is not refreshed by it. Nothing reads issues.synced_at. issue_key is --- compared as written (the linked sync always passes it). +-- compared as written (the linked sync always passes it). It is also the one upsert that +-- writes issues.issue_tracker_column (PRD #4173 M5), the card's board column read from its +-- issue tracker State: NULL is the Open lane, and the column is part of the change check so +-- a changed State mapping is cached again. UpsertIssue and UpsertIssueLabels never touch it. INSERT INTO issues ( repo_id, forge_issue_iid, title, state, labels, assignee_ids, web_url, author, - has_prd_link, forge_updated_at, issue_key, synced_at -) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, now()) + has_prd_link, forge_updated_at, issue_key, issue_tracker_column, synced_at +) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, now()) ON CONFLICT (repo_id, forge_issue_iid) DO UPDATE SET title = EXCLUDED.title, state = EXCLUDED.state, @@ -567,12 +570,33 @@ SET title = EXCLUDED.title, has_prd_link = EXCLUDED.has_prd_link, forge_updated_at = EXCLUDED.forge_updated_at, issue_key = EXCLUDED.issue_key, + issue_tracker_column = EXCLUDED.issue_tracker_column, synced_at = now() WHERE (issues.title, issues.state, issues.labels, issues.assignee_ids, issues.web_url, issues.author, - issues.has_prd_link, issues.forge_updated_at, issues.issue_key) + issues.has_prd_link, issues.forge_updated_at, issues.issue_key, issues.issue_tracker_column) IS DISTINCT FROM (EXCLUDED.title, EXCLUDED.state, EXCLUDED.labels, EXCLUDED.assignee_ids, EXCLUDED.web_url, EXCLUDED.author, - EXCLUDED.has_prd_link, EXCLUDED.forge_updated_at, EXCLUDED.issue_key); + EXCLUDED.has_prd_link, EXCLUDED.forge_updated_at, EXCLUDED.issue_key, EXCLUDED.issue_tracker_column); + +-- name: SetIssueTrackerColumn :one +-- The cached board column of a linked card after a successful move (PRD #4173 M5): +-- writes ONLY issue_tracker_column (an empty column is the Open lane and is stored as +-- NULL, because the table's CHECK refuses an empty string) and synced_at, never state, labels or +-- board_position. Forge-first like every cache write here: the caller has already moved +-- the issue in the issue tracker. RETURNING * for the single-card re-render. +UPDATE issues +SET issue_tracker_column = NULLIF(@issue_tracker_column::text, ''), synced_at = now() +WHERE repo_id = @repo_id AND forge_issue_iid = @forge_issue_iid +RETURNING *; + +-- name: ClearIssueTrackerColumnsNotIn :exec +-- Clears the cached column of every issue of the repo whose column is no longer one of +-- the board's columns (PRD #4173 M5), so a removed column puts its cards back in the Open +-- lane instead of leaving a column no board shows. keep is never NULL (NULL would match +-- nothing): an empty array clears every cached column. +UPDATE issues +SET issue_tracker_column = NULL +WHERE repo_id = @repo_id AND issue_tracker_column <> ALL(@keep::text[]); -- name: UpsertIssueLabels :one -- Labels is the INSERT seed only. On conflict, remove from CURRENT issues.labels, @@ -904,9 +928,9 @@ JOIN issues i ON i.repo_id = @repo_id AND i.forge_issue_iid = l.issue_iid WHERE l.status = 'completed' AND l.mr_iid IS NOT NULL -- A repo linked to an external issue tracker (PRD #4173 M4) is owned by - -- ListBoardFreeMRStateWatchCandidates entirely: uzi moves no card there (writes arrive - -- in M5), so its MR state must not depend on the issue cache or a column label. Excluding - -- the repo here keeps the two lanes disjoint. + -- ListBoardFreeMRStateWatchCandidates entirely: that lane moves the card through the + -- issue tracker (PRD #4173 M5), so its MR state must not depend on the issue cache or a + -- column label. Excluding the repo here keeps the two lanes disjoint. AND NOT EXISTS (SELECT 1 FROM repos lr WHERE lr.id = @repo_id AND lr.issue_tracker_connection_id IS NOT NULL) AND ( -- Lane A (UNCHANGED): open-issue board-move watch — card in Human Review @@ -950,16 +974,27 @@ LIMIT 100; -- ListMRWatchCandidates), not a sqlc param, so zero Go signature change. A repo linked to an -- external issue tracker (PRD #4173 M4) is widened IN whole: every issue-lane run of such a repo -- belongs here (the board lane excludes the repo), so linked runs record mr_state and cancel --- rework with no board card; an unlinked repo matches exactly as before. On a linked repo the +-- rework, and the issue's latest run moves the linked card on a close or reopen (M5); an unlinked repo matches exactly as before. On a linked repo the -- lane also keeps watching an issue-lane run recorded mr_state='closed', so a reopen is recorded -- back to 'opened' (the board lane's Lane A reopen-edge watch, Decision 10), bounded as Lane A is: -- only the issue's LATEST run (no newer non-scheduled run of the issue, so a superseded rework -- MR is not watched) and only while the cached issue exists AND is opened (an evicted or closed -- issue is not polled forever). Live rows (NULL/opened/locked mr_state) sort ahead of closed -- reopen-watch rows so the latter can never starve them under the LIMIT; among live rows the --- order is created_at DESC as before, so an unlinked repo's order is unchanged. Keep this rationale --- ABOVE the statement — a comment trailing after the `;` is grabbed by sqlc as the NEXT query's doc. -SELECT id, branch, mr_iid, mr_state +-- order is created_at DESC as before, so an unlinked repo's order is unchanged. The output also +-- carries issue_iid, kind and is_latest (PRD #4173 M5): the caller tells an issue-lane run from a +-- scheduled or issue-less one by the first two, and is_latest is true iff no newer run of the same +-- repo and issue_iid with kind NOT IN ('prompt','self_improve') exists, regardless of its status +-- (the board lane's DISTINCT ON (issue_iid) ORDER BY created_at DESC semantics: every status +-- counts), so a linked card moves only for its issue's latest run. They add no filter and change +-- no ordering. Keep this rationale ABOVE the statement — a comment trailing after the `;` is +-- grabbed by sqlc as the NEXT query's doc. +SELECT id, branch, mr_iid, mr_state, issue_iid, kind, + (issue_iid IS NULL OR kind IN ('prompt', 'self_improve') + OR NOT EXISTS (SELECT 1 FROM runs nl + WHERE nl.repo_id = runs.repo_id AND nl.issue_iid = runs.issue_iid + AND nl.kind NOT IN ('prompt', 'self_improve') + AND nl.created_at > runs.created_at))::boolean AS is_latest FROM runs WHERE repo_id = @repo_id::uuid AND (issue_iid IS NULL OR kind IN ('prompt', 'self_improve') diff --git a/api/internal/store/queries/issue_tracker_links.sql b/api/internal/store/queries/issue_tracker_links.sql index 975f738e0b..a94a564157 100644 --- a/api/internal/store/queries/issue_tracker_links.sql +++ b/api/internal/store/queries/issue_tracker_links.sql @@ -6,16 +6,22 @@ -- owning user, never by forge project number. linked_connection_id is the link's -- connection ONLY when it belongs to the same user as the forge connection; a linked -- repo whose connection is missing for that owner reads NULL here and the resolver --- fails closed. +-- fails closed. open_state and closed_state are NULL until the state mapping is saved +-- (PRD #4173 M5); column_states is the board column label to State name map, '{}' when +-- no column is mapped. SELECT r.id, r.forge_project_id, r.awaiting_first_enable, r.issue_tracker_connection_id, r.issue_tracker_project_id, r.issue_tracker_project_short, r.eligibility_tag_id, r.eligibility_tag_name, r.issue_tracker_linked_at, c.forge_type, c.base_url, c.token_ciphertext, - itc.id AS linked_connection_id + itc.id AS linked_connection_id, + s.open_state, s.closed_state, + COALESCE((SELECT jsonb_object_agg(cs.label_name, cs.state_name) + FROM repo_issue_tracker_column_states cs WHERE cs.repo_id = r.id), '{}'::jsonb)::jsonb AS column_states FROM repos r JOIN forge_connections c ON c.id = r.connection_id LEFT JOIN issue_tracker_connections itc ON itc.id = r.issue_tracker_connection_id AND itc.user_id = c.user_id +LEFT JOIN repo_issue_tracker_states s ON s.repo_id = r.id WHERE r.id = $1 AND c.user_id = $2; -- name: LockRepoForLinkForUser :one diff --git a/api/internal/store/queries/issue_tracker_states.sql b/api/internal/store/queries/issue_tracker_states.sql new file mode 100644 index 0000000000..452422e1c0 --- /dev/null +++ b/api/internal/store/queries/issue_tracker_states.sql @@ -0,0 +1,44 @@ +-- The board column to issue tracker State mapping of a linked repo (PRD #4173 M5). +-- The mapping is read through GetRepoIssueSourceForUser; the statements here write it. + +-- name: LockRepoForStateMappingForUser :one +-- Takes the repo row lock the state mapping route and a board column reconfigure of a linked +-- repo share, scoped to the owning user, so the two serialise: a mapping is never validated +-- against board columns that a concurrent reconfigure is deleting. FOR NO KEY UPDATE does not +-- block the FOR KEY SHARE that inserts of rows referencing the repo take, and conflicts with +-- the link's and the enable's locks. Call inside a transaction. The row is returned whether or +-- not the repo is linked; the caller decides. +SELECT r.id, r.issue_tracker_connection_id, r.issue_tracker_project_id +FROM repos r +JOIN forge_connections c ON c.id = r.connection_id +WHERE r.id = $1 AND c.user_id = $2 +FOR NO KEY UPDATE OF r; + +-- name: UpsertRepoIssueTrackerStates :one +-- Writes the open and closed State of a repo's mapping. The repo must be owned by @user_id +-- through its forge connection and linked to an issue tracker: no row means refused. Call +-- after LockRepoForStateMappingForUser in the same transaction. +INSERT INTO repo_issue_tracker_states (repo_id, open_state, closed_state) +SELECT r.id, @open_state::text, @closed_state::text +FROM repos r +JOIN forge_connections c ON c.id = r.connection_id +WHERE r.id = @repo_id AND c.user_id = @user_id AND r.issue_tracker_connection_id IS NOT NULL +ON CONFLICT (repo_id) DO UPDATE +SET open_state = EXCLUDED.open_state, closed_state = EXCLUDED.closed_state, updated_at = now() +RETURNING *; + +-- name: DeleteRepoIssueTrackerColumnStates :exec +-- Removes every column mapping of the repo; the mapping route calls it before it inserts the +-- new set, in one transaction. +DELETE FROM repo_issue_tracker_column_states WHERE repo_id = $1; + +-- name: InsertRepoIssueTrackerColumnState :exec +INSERT INTO repo_issue_tracker_column_states (repo_id, label_name, state_name) +VALUES (@repo_id, @label_name, @state_name); + +-- name: PruneRepoIssueTrackerColumnStates :exec +-- Removes the column mappings whose label is not in @keep. An empty @keep removes them all, +-- which is the right answer when the board has no column left (the opposite trap to a +-- "keep" list that wipes everything by accident: here the empty set is the intent). +DELETE FROM repo_issue_tracker_column_states +WHERE repo_id = @repo_id AND label_name <> ALL(@keep::text[]); diff --git a/api/internal/store/queries/runtime.sql b/api/internal/store/queries/runtime.sql index fac7499693..fba7a93ede 100644 --- a/api/internal/store/queries/runtime.sql +++ b/api/internal/store/queries/runtime.sql @@ -11087,6 +11087,7 @@ SELECT r.status, r.issue_iid, r.repo_id, r.origin_column, r.board_column, r.move r.auto_approve, r.mr_iid, rp.forge_project_id, rp.web_url AS repo_web_url, rp.issue_tracker_connection_id, rp.awaiting_first_enable, + rp.eligibility_tag_name, -- PRD #4173 M5: the lifecycle's uncached-issue probe c.forge_type, c.base_url, c.token_ciphertext, r.user_id FROM runs r diff --git a/api/internal/store/runtime.sql.go b/api/internal/store/runtime.sql.go index 10a433620f..4356ec12e7 100644 --- a/api/internal/store/runtime.sql.go +++ b/api/internal/store/runtime.sql.go @@ -8048,6 +8048,7 @@ SELECT r.status, r.issue_iid, r.repo_id, r.origin_column, r.board_column, r.move r.auto_approve, r.mr_iid, rp.forge_project_id, rp.web_url AS repo_web_url, rp.issue_tracker_connection_id, rp.awaiting_first_enable, + rp.eligibility_tag_name, -- PRD #4173 M5: the lifecycle's uncached-issue probe c.forge_type, c.base_url, c.token_ciphertext, r.user_id FROM runs r @@ -8069,6 +8070,7 @@ type GetRunMoveContextRow struct { RepoWebUrl string `json:"repo_web_url"` IssueTrackerConnectionID pgtype.UUID `json:"issue_tracker_connection_id"` AwaitingFirstEnable bool `json:"awaiting_first_enable"` + EligibilityTagName pgtype.Text `json:"eligibility_tag_name"` ForgeType string `json:"forge_type"` BaseUrl string `json:"base_url"` TokenCiphertext []byte `json:"token_ciphertext"` @@ -8104,6 +8106,7 @@ func (q *Queries) GetRunMoveContext(ctx context.Context, runID uuid.UUID) (GetRu &i.RepoWebUrl, &i.IssueTrackerConnectionID, &i.AwaitingFirstEnable, + &i.EligibilityTagName, &i.ForgeType, &i.BaseUrl, &i.TokenCiphertext, diff --git a/api/internal/workersvc/composite.go b/api/internal/workersvc/composite.go index 3958f3d407..48b1d74950 100644 --- a/api/internal/workersvc/composite.go +++ b/api/internal/workersvc/composite.go @@ -154,14 +154,18 @@ func (s *Service) ConfirmProposalForUser(ctx context.Context, userID, runID, pro } } - // PRD #4173 DL-5: a linked repo's resolver wraps the write guard, which refuses - // CreateIssue before any request (writes arrive in M5). The claim reverts below, so - // the proposal stays pending and nothing was created. + // PRD #4173 DL-5: a linked repo creates through its resolver, never the forge. The + // labels map the uzi label to the link's stored eligibility name and drop the + // autopilot label (D12, D14); a proposal without the uzi label carries no eligibility + // tag. A failure reverts the claim, so the proposal stays pending. tr, err := issuesource.For(ctx, s.issueSources, repo.IssueTrackerConnectionID.Valid, userID, repo.ID, f) if err != nil { s.revertProposal(ctx, propID) return CreatedIssue{}, fmt.Errorf("%w: %v", ErrForgeBuild, err) } + if repo.IssueTrackerConnectionID.Valid { + labels = issuesource.LinkedLabels(labels, s.uziLabel(ctx), s.autopilotLabel(ctx), repo.EligibilityTagName.String) + } created, err := tr.CreateIssue(ctx, repo.ForgeProjectID, claim.Title, claim.Description, labels) if err != nil { s.revertProposal(ctx, propID) diff --git a/api/internal/workersvc/issue_sources_test.go b/api/internal/workersvc/issue_sources_test.go index 37a5107ce2..cdd6991f35 100644 --- a/api/internal/workersvc/issue_sources_test.go +++ b/api/internal/workersvc/issue_sources_test.go @@ -3,6 +3,7 @@ package workersvc import ( "context" "errors" + "slices" "strings" "testing" @@ -24,7 +25,6 @@ import ( // no forge user ids, as the YouTrack driver's do. type ytTracker struct { forgetest.BaseFake - createErr error } func (*ytTracker) GetIssue(_ context.Context, _, iid int64) (forge.Issue, error) { @@ -34,8 +34,22 @@ func (*ytTracker) GetIssue(_ context.Context, _, iid int64) (forge.Issue, error) func (*ytTracker) ListIssueComments(context.Context, int64, int64) ([]forge.IssueComment, error) { return []forge.IssueComment{{Body: "yt comment"}}, nil } -func (y *ytTracker) CreateIssue(context.Context, int64, string, string, []string) (forge.Issue, error) { - return forge.Issue{}, y.createErr + +// createRecorder is a linked repo's issue tracker that records the CreateIssue it receives. +type createRecorder struct { + forgetest.BaseFake + createErr error + created forge.Issue + creates int + createdTitle string + createdLabel []string +} + +func (c *createRecorder) CreateIssue(_ context.Context, _ int64, title, _ string, labels []string) (forge.Issue, error) { + c.creates++ + c.createdTitle = title + c.createdLabel = labels + return c.created, c.createErr } type fakeSources struct { @@ -160,28 +174,68 @@ func TestUnlinkedRunCaptureNeverConsultsResolver(t *testing.T) { } } -func TestLinkedConfirmProposalIsRefusedAndReverts(t *testing.T) { +// PRD #4173 M5 (converted from TestLinkedConfirmProposalIsRefusedAndReverts, which modelled the +// M4 guard's refusal): a confirmed chat proposal is created through the resolver's issue side +// with the uzi label mapped to the stored eligibility name and the autopilot label dropped. +func TestLinkedConfirmProposalCreatesThroughTheResolverWithTheEligibilityTag(t *testing.T) { svc, fs, fb := confirmFixture(t) fs.repoRow.IssueTrackerConnectionID = pgtype.UUID{Bytes: uuid.New(), Valid: true} - refused := errors.New("create issue: issue writes on a repo linked to an issue tracker are not offered yet") - src := &fakeSources{tr: &ytTracker{createErr: refused}} + fs.repoRow.EligibilityTagName = pgtype.Text{String: "ready-for-uzi", Valid: true} + fs.claimProposalRow.Labels = []byte(`["bug","uzi","autopilot"]`) + tr := &createRecorder{created: forge.Issue{IID: 9, WebURL: "https://yt.test/UZI-9", Title: "t"}} + src := &fakeSources{tr: tr} svc.SetIssueSources(src) - _, err := svc.ConfirmProposalForUser(context.Background(), uuid.New(), uuid.New(), uuid.New()) - if !errors.Is(err, ErrForgeIssueWrite) || !strings.Contains(err.Error(), "not offered") { - t.Fatalf("err = %v, want the guard's refusal behind ErrForgeIssueWrite", err) + created, err := svc.ConfirmProposalForUser(context.Background(), uuid.New(), uuid.New(), uuid.New()) + if err != nil || created.IID != 9 { + t.Fatalf("created = %+v, err = %v, want issue 9", created, err) + } + if want := []string{"bug", "ready-for-uzi"}; !slices.Equal(tr.createdLabel, want) { + t.Fatalf("labels = %v, want %v", tr.createdLabel, want) } if fb.f.createCalls != 0 { t.Fatal("the forge's CreateIssue was called for a linked repo") } - if len(fs.revertedProposals) != 1 || fs.markedProposalConfirm != nil { - t.Fatalf("the claim must revert to pending: reverts=%d", len(fs.revertedProposals)) + if len(fs.revertedProposals) != 0 || fs.markedProposalConfirm == nil || fs.markedProposalConfirm.CreatedIssueIid.Int64 != 9 { + t.Fatalf("the proposal must settle confirmed with iid 9: reverts=%d", len(fs.revertedProposals)) } if len(src.calls) != 1 { t.Fatalf("resolver calls = %d, want 1", len(src.calls)) } } +// A confirmed proposal without the uzi label carries no eligibility tag. +func TestLinkedConfirmProposalWithoutTheUziLabelCarriesNoEligibilityTag(t *testing.T) { + svc, fs, _ := confirmFixture(t) + fs.repoRow.IssueTrackerConnectionID = pgtype.UUID{Bytes: uuid.New(), Valid: true} + fs.repoRow.EligibilityTagName = pgtype.Text{String: "ready-for-uzi", Valid: true} + tr := &createRecorder{created: forge.Issue{IID: 9}} + svc.SetIssueSources(&fakeSources{tr: tr}) + + if _, err := svc.ConfirmProposalForUser(context.Background(), uuid.New(), uuid.New(), uuid.New()); err != nil { + t.Fatal(err) + } + if want := []string{"bug"}; !slices.Equal(tr.createdLabel, want) { + t.Fatalf("labels = %v, want %v", tr.createdLabel, want) + } +} + +// An issue tracker failure on a linked repo still reverts the claim to pending. +func TestLinkedConfirmProposalFailureReverts(t *testing.T) { + svc, fs, fb := confirmFixture(t) + fs.repoRow.IssueTrackerConnectionID = pgtype.UUID{Bytes: uuid.New(), Valid: true} + fs.repoRow.EligibilityTagName = pgtype.Text{String: "ready-for-uzi", Valid: true} + svc.SetIssueSources(&fakeSources{tr: &createRecorder{createErr: errors.New("youtrack unreachable")}}) + + _, err := svc.ConfirmProposalForUser(context.Background(), uuid.New(), uuid.New(), uuid.New()) + if !errors.Is(err, ErrForgeIssueWrite) { + t.Fatalf("err = %v, want ErrForgeIssueWrite", err) + } + if fb.f.createCalls != 0 || len(fs.revertedProposals) != 1 || fs.markedProposalConfirm != nil { + t.Fatalf("forge creates=%d reverts=%d: the claim must revert", fb.f.createCalls, len(fs.revertedProposals)) + } +} + func TestLinkedConfirmProposalWithoutSourcesFailsClosed(t *testing.T) { svc, fs, fb := confirmFixture(t) fs.repoRow.IssueTrackerConnectionID = pgtype.UUID{Bytes: uuid.New(), Valid: true} @@ -195,10 +249,15 @@ func TestLinkedConfirmProposalWithoutSourcesFailsClosed(t *testing.T) { } } -func TestLinkedRepoFilesNoScheduledProposal(t *testing.T) { +// PRD #4173 M5 (converted from TestLinkedRepoFilesNoScheduledProposal, the M4 skip): a scheduled +// proposal is filed through the resolver's issue side with the proposal marker only (no +// eligibility tag), and the audit row is stamped. +func TestLinkedRepoFilesTheScheduledProposalThroughTheResolver(t *testing.T) { svc, fs, ff, run := proposalFixture(t, "feature-bingo") fs.repoRow.IssueTrackerConnectionID = pgtype.UUID{Bytes: uuid.New(), Valid: true} - src := &fakeSources{tr: &ytTracker{}} + fs.repoRow.EligibilityTagName = pgtype.Text{String: "ready-for-uzi", Valid: true} + tr := &createRecorder{created: forge.Issue{IID: 11}} + src := &fakeSources{tr: tr} svc.SetIssueSources(src) svc.maybeFileProposal(context.Background(), run, aProposal()) @@ -206,8 +265,27 @@ func TestLinkedRepoFilesNoScheduledProposal(t *testing.T) { if ff.createCalls != 0 { t.Fatal("a proposal was filed on the forge for a linked repo") } - if fs.createdProposal != nil { - t.Fatal("a pending audit row was left behind for a skipped filing") + if want := []string{"proposal::feature-bingo"}; tr.creates != 1 || !slices.Equal(tr.createdLabel, want) { + t.Fatalf("creates = %d labels = %v, want one create with %v", tr.creates, tr.createdLabel, want) + } + if fs.createdProposal == nil || fs.stampedProposal == nil || fs.stampedProposal.Iid.Int64 != 11 { + t.Fatalf("the audit row must be inserted and stamped with iid 11: %+v %+v", fs.createdProposal, fs.stampedProposal) + } + if len(src.calls) != 1 || src.calls[0].user != run.UserID { + t.Fatalf("the resolver is bound by the run's owner, got %+v", src.calls) + } +} + +// With no resolver a linked repo files nothing, falls back to nothing, and leaves no +// pending audit row. +func TestLinkedRepoScheduledProposalWithoutSourcesFilesNothing(t *testing.T) { + svc, fs, ff, run := proposalFixture(t, "feature-bingo") + fs.repoRow.IssueTrackerConnectionID = pgtype.UUID{Bytes: uuid.New(), Valid: true} + + svc.maybeFileProposal(context.Background(), run, aProposal()) + + if ff.createCalls != 0 || fs.createdProposal != nil { + t.Fatalf("forge creates = %d, audit row = %v: nothing may be filed", ff.createCalls, fs.createdProposal) } } diff --git a/api/internal/workersvc/linked_origin_column_livedb_test.go b/api/internal/workersvc/linked_origin_column_livedb_test.go new file mode 100644 index 0000000000..5018c4d89b --- /dev/null +++ b/api/internal/workersvc/linked_origin_column_livedb_test.go @@ -0,0 +1,72 @@ +package workersvc + +import ( + "context" + "testing" + + "github.com/google/uuid" + + "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issueinput" +) + +// PRD #4173 M5: runs.origin_column of a run created on a linked repo is the card's cached +// issue tracker column, never a label, so the lifecycle's manual-drag guard compares like with +// like. Skipped unless UZI_TEST_DATABASE_URL points at a throwaway Postgres. + +func (f *crFix) createLinkedIssueRun(t *testing.T, iid int64, cachedLabels, cachedColumn string) string { + t.Helper() + f.env.exec(`INSERT INTO board_columns(repo_id, label_name, position) VALUES ($1, 'In Progress', 0), ($1, 'Later', 1)`, f.repo) + f.env.exec(`INSERT INTO issues (id, repo_id, forge_issue_iid, title, state, labels, web_url, has_prd_link, forge_updated_at, synced_at, issue_tracker_column) + VALUES ($1, $2, $3, 'linked card', 'opened', $4::jsonb, 'https://yt.example.com/i', false, now(), now(), NULLIF($5, ''))`, + uuid.New(), f.repo, iid, cachedLabels, cachedColumn) + // The fresh issue read a linked run needs is handed in on the context, as a schedule + // fire hands its capture; it carries the eligibility tag. + capture := &issueinput.Capture{ProjectID: 77, Issue: forge.Issue{IID: iid, Title: "linked card", Labels: []string{"uzi"}}, Thread: &issueinput.Thread{}} + ctx := issueinput.WithCapture(f.env.ctx, capture) + run, err := f.svc.CreateRun(ctx, f.user, f.repo, iid, "desc", nil, nil, false, nil, nil, nil, nil, nil, nil) + if err != nil { + t.Fatalf("CreateRun: %v", err) + } + var origin *string + if err := f.env.pool.QueryRow(context.Background(), `SELECT origin_column FROM runs WHERE id = $1`, run.ID).Scan(&origin); err != nil { + t.Fatal(err) + } + if origin == nil { + return "" + } + return *origin +} + +func TestLinkedRunOriginColumnIsTheCachedColumnLiveDB(t *testing.T) { + t.Run("a card in a mapped column", func(t *testing.T) { + f := newCRFix(t) + f.linkFixtureRepo(t) + f.env.exec(`UPDATE repos SET forge_project_id = 77 WHERE id = $1`, f.repo) + if got := f.createLinkedIssueRun(t, 93201, `["uzi"]`, "In Progress"); got != "In Progress" { + t.Fatalf("origin_column = %q, want In Progress", got) + } + }) + t.Run("a card in the Open lane ignores a stray column label", func(t *testing.T) { + f := newCRFix(t) + f.linkFixtureRepo(t) + f.env.exec(`UPDATE repos SET forge_project_id = 77 WHERE id = $1`, f.repo) + if got := f.createLinkedIssueRun(t, 93202, `["uzi","Later"]`, ""); got != "" { + t.Fatalf("origin_column = %q, want the Open lane (empty)", got) + } + }) + t.Run("an unlinked repo still reads the labels", func(t *testing.T) { + f := newCRFix(t) + f.env.exec(`INSERT INTO board_columns(repo_id, label_name, position) VALUES ($1, 'In Progress', 0), ($1, 'Later', 1)`, f.repo) + seedEligibleIssue(t, f.env, f.repo, 93203) + f.env.exec(`UPDATE issues SET labels = '["uzi","Later"]'::jsonb WHERE repo_id = $1 AND forge_issue_iid = 93203`, f.repo) + run, err := f.svc.CreateRun(f.env.ctx, f.user, f.repo, 93203, "desc", nil, nil, false, nil, nil, nil, nil, nil, nil) + if err != nil { + t.Fatal(err) + } + var origin string + if err := f.env.pool.QueryRow(context.Background(), `SELECT origin_column FROM runs WHERE id = $1`, run.ID).Scan(&origin); err != nil || origin != "Later" { + t.Fatalf("origin_column = %q (%v), want Later", origin, err) + } + }) +} diff --git a/api/internal/workersvc/linked_proposal_create_livedb_test.go b/api/internal/workersvc/linked_proposal_create_livedb_test.go new file mode 100644 index 0000000000..dc6943dc35 --- /dev/null +++ b/api/internal/workersvc/linked_proposal_create_livedb_test.go @@ -0,0 +1,171 @@ +package workersvc + +import ( + "context" + "net/http" + "slices" + "testing" + "time" + + "github.com/google/uuid" + "github.com/jackc/pgx/v5/pgtype" + + "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuetracker" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" + "github.com/vtmocanu/uzi/api/internal/issuetrackersvc" + "github.com/vtmocanu/uzi/api/internal/store" +) + +// PRD #4173 M5: the chat-proposal confirm and the scheduled-proposal filing create their +// issue in the YouTrack project a linked repo is bound to, never the forge, through the +// real issue tracker driver that issuetrackersvc builds. The resolver is a local stand-in +// for forgesvc.IssueTrackerForRepo (forgesvc imports workersvc), without its write guard; +// the handler LiveDB tests cover the production resolver. Skipped unless UZI_TEST_DATABASE_URL points +// at a throwaway Postgres. + +type linkedProposalFix struct { + env codexTestEnv + user uuid.UUID + repo uuid.UUID + svc *Service + srv *youtracktest.Server +} + +func newLinkedProposalFix(t *testing.T) *linkedProposalFix { + t.Helper() + env := setupCodexLiveDB(t) + user, _, repo := env.seedCodexInfra(t) + srv := youtracktest.NewServer(t) + def, ok := http.DefaultTransport.(*http.Transport) + if !ok { + t.Fatal("http.DefaultTransport is not an *http.Transport") + } + saved := def.TLSClientConfig + def.TLSClientConfig = srv.Client().Transport.(*http.Transport).TLSClientConfig.Clone() + t.Cleanup(func() { def.TLSClientConfig = saved }) + + itSvc := issuetrackersvc.New(env.q, env.box, func(u string) bool { return u == srv.URL }, issuetracker.New) + yt, err := itSvc.Connect(env.ctx, user, "youtrack", srv.URL, srv.Token) + if err != nil { + t.Fatal(err) + } + env.exec(`UPDATE repos SET issue_tracker_connection_id = $2, issue_tracker_project_id = $3, issue_tracker_project_short = $4, + eligibility_tag_id = $5, eligibility_tag_name = $6, issue_tracker_linked_at = now() WHERE id = $1`, + repo, yt.ID, youtracktest.ProjectID, youtracktest.ProjectShortName, youtracktest.EligibilityTagID, youtracktest.EligibilityTagName) + + svc := New(env.q, env.box, testParams()) + // The forge only has to exist: the proposal sites build it, then create through the + // resolver. The assertions read the YouTrack fake, where the issue must appear. + svc.SetForges(m2Builder{&linkedTaskForge{}}) + svc.SetIssueSources(linkedSources{q: env.q, it: itSvc}) + srv.ResetRequests() + return &linkedProposalFix{env: env, user: user, repo: repo, svc: svc, srv: srv} +} + +// linkedSources is the issue source resolver of a linked repo: the driver the stored +// binding names, built by the production issuetrackersvc. forgesvc.Service, which wires +// the same call in cmd/server, cannot be imported here (forgesvc imports workersvc), and +// the issue write guard it wraps the driver in only refuses writes this test never makes. +type linkedSources struct { + q *store.Queries + it *issuetrackersvc.Service +} + +func (s linkedSources) IssueTrackerForRepo(ctx context.Context, userID, repoID uuid.UUID) (forge.IssueTracker, error) { + src, err := s.q.GetRepoIssueSourceForUser(ctx, store.GetRepoIssueSourceForUserParams{ID: repoID, UserID: userID}) + if err != nil { + return nil, err + } + return s.it.DriverFor(ctx, userID, uuid.UUID(src.LinkedConnectionID.Bytes), issuetracker.Binding{ + Project: issuetracker.ProjectRef{ID: src.IssueTrackerProjectID.String, ShortName: src.IssueTrackerProjectShort.String}, + ProjectID: src.ForgeProjectID, + EligibilityTag: issuetracker.TagRef{ID: src.EligibilityTagID.String, Name: src.EligibilityTagName.String}, + }, issuetracker.StateMapping{}) +} + +func (f *linkedProposalFix) tagNames(n int64) []string { + var names []string + for _, id := range f.srv.IssueTagIDs(n) { + names = append(names, f.srv.TagByID(id).Name) + } + slices.Sort(names) + return names +} + +func TestLinkedChatProposalConfirmCreatesInTheLinkedProjectLiveDB(t *testing.T) { + f := newLinkedProposalFix(t) + runID, propID := uuid.New(), uuid.New() + f.env.exec(`INSERT INTO runs (id, user_id, issue_title, issue_description, status, kind) + VALUES ($1, $2, 'chat', 'd', 'completed', 'chat')`, runID, f.user) + f.env.exec(`INSERT INTO issue_proposals (id, run_id, repo_id, title, description, labels, status) + VALUES ($1, $2, $3, 'Proposed card', 'proposal body', '["bug","uzi","autopilot"]', 'pending')`, propID, runID, f.repo) + + created, err := f.svc.ConfirmProposalForUser(f.env.ctx, f.user, runID, propID) + if err != nil { + t.Fatalf("ConfirmProposalForUser: %v", err) + } + if created.IID == 0 || created.Title != "Proposed card" { + t.Fatalf("created = %+v", created) + } + if got, want := f.tagNames(created.IID), []string{"bug", youtracktest.EligibilityTagName}; !slices.Equal(got, want) { + t.Errorf("tags on the created issue = %v, want %v", got, want) + } + if !slices.Contains(f.srv.IssueTagIDs(created.IID), youtracktest.EligibilityTagID) { + t.Errorf("the stored eligibility tag id is not on the issue: %v", f.srv.IssueTagIDs(created.IID)) + } + for _, stray := range []string{"uzi", "autopilot"} { + if len(f.srv.TagsNamed(stray)) != 0 { + t.Errorf("a %q tag was created on the issue tracker", stray) + } + } + var status string + var iid pgtype.Int8 + if err := f.env.pool.QueryRow(f.env.ctx, `SELECT status, created_issue_iid FROM issue_proposals WHERE id = $1`, propID).Scan(&status, &iid); err != nil { + t.Fatal(err) + } + if status != "confirmed" || !iid.Valid || iid.Int64 != created.IID { + t.Errorf("proposal = %s iid %+v, want confirmed with iid %d", status, iid, created.IID) + } +} + +func TestLinkedScheduledProposalFilesInTheLinkedProjectWithoutTheEligibilityTagLiveDB(t *testing.T) { + f := newLinkedProposalFix(t) + sched, err := f.env.q.CreateRunSchedule(f.env.ctx, store.CreateRunScheduleParams{ + UserID: f.user, RepoID: f.repo, Target: "prompt", Prompt: pgtype.Text{String: "propose", Valid: true}, + Timing: "recurring", CronExpr: pgtype.Text{String: "0 * * * *", Valid: true}, Timezone: "UTC", + NextFireAt: pgtype.Timestamptz{Time: time.Now().Add(time.Hour), Valid: true}, Enabled: true, + OutputMode: pgtype.Text{String: "issues", Valid: true}, + }) + if err != nil { + t.Fatalf("create schedule: %v", err) + } + t.Cleanup(func() { + _, _ = f.env.pool.Exec(f.env.ctx, `DELETE FROM runs WHERE schedule_id = $1`, sched.ID) + _, _ = f.env.pool.Exec(f.env.ctx, `DELETE FROM run_schedules WHERE id = $1`, sched.ID) + }) + runID := uuid.New() + f.env.exec(`INSERT INTO runs (id, user_id, repo_id, issue_title, issue_description, status, kind, schedule_id) + VALUES ($1, $2, $3, 'scheduled idea', 'd', 'completed', 'prompt', $4)`, runID, f.user, f.repo, sched.ID) + run, err := f.env.q.GetRunByID(f.env.ctx, runID) + if err != nil { + t.Fatal(err) + } + + f.svc.maybeFileProposal(f.env.ctx, run, &ProposalPayload{Title: "Scheduled proposal", Body: "proposal body"}) + + var iid pgtype.Int8 + var status string + if err := f.env.pool.QueryRow(f.env.ctx, `SELECT status, created_issue_iid FROM issue_proposals WHERE run_id = $1`, runID).Scan(&status, &iid); err != nil { + t.Fatal(err) + } + if status != "confirmed" || !iid.Valid { + t.Fatalf("audit row = %s iid %+v, want confirmed with an iid", status, iid) + } + if got, want := f.tagNames(iid.Int64), []string{ProposalLabel("")}; !slices.Equal(got, want) { + t.Errorf("tags on the filed issue = %v, want only %v", got, want) + } + if slices.Contains(f.srv.IssueTagIDs(iid.Int64), youtracktest.EligibilityTagID) { + t.Error("a scheduled proposal carries the eligibility tag") + } +} diff --git a/api/internal/workersvc/proposal_filing.go b/api/internal/workersvc/proposal_filing.go index 0c60bdbbde..feeb1599bb 100644 --- a/api/internal/workersvc/proposal_filing.go +++ b/api/internal/workersvc/proposal_filing.go @@ -7,7 +7,7 @@ import ( "github.com/jackc/pgx/v5/pgtype" - "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuesource" "github.com/vtmocanu/uzi/api/internal/runkind" "github.com/vtmocanu/uzi/api/internal/schedtmpl" "github.com/vtmocanu/uzi/api/internal/store" @@ -156,12 +156,14 @@ func (s *Service) maybeFileProposal(ctx context.Context, run store.Run, proposal slog.Warn("file proposal: build forge", "run_id", run.ID, "error", err) return } - // PRD #4173 M4: filing a proposal creates an issue, which a linked repo does not - // offer until M5, so it is skipped before the audit row (no pending row is left - // behind) and never falls back to the forge, whose project number means nothing there. - if repo.IssueTrackerConnectionID.Valid { - slog.Warn("file proposal: skipped, issue creation is not offered on a repo linked to an issue tracker", - "run_id", run.ID, "repo_id", repo.ID) + // A linked repo files through its resolver (PRD #4173 M5), never the forge, whose + // project number means nothing there. It is resolved before the audit row so a failed + // resolution leaves no pending row behind. The labels are the proposal marker alone: + // a scheduled proposal carries no eligibility tag, so the next linked full sync evicts + // it from the cache. + tr, err := issuesource.For(ctx, s.issueSources, repo.IssueTrackerConnectionID.Valid, run.UserID, repo.ID, f) + if err != nil { + slog.Warn("file proposal: resolve issue tracker", "run_id", run.ID, "error", err) return } @@ -187,7 +189,6 @@ func (s *Service) maybeFileProposal(ctx context.Context, run store.Run, proposal return } - var tr forge.IssueTracker = f created, err := tr.CreateIssue(ctx, repo.ForgeProjectID, proposal.Title, proposal.Body, labels) if err != nil { // err is already PAT-redacted by the driver. The pending audit row stays for diff --git a/api/internal/workersvc/service.go b/api/internal/workersvc/service.go index 115e8ffbd5..d87f84949d 100644 --- a/api/internal/workersvc/service.go +++ b/api/internal/workersvc/service.go @@ -8369,7 +8369,7 @@ func (s *Service) createRun(ctx context.Context, userID, repoID uuid.UUID, issue // two plain reads are kept out of the lock-holding window rather than acquiring a second // connection inside it. (An explicit credential override is still validated in-closure, on the // tx-bound q, so it takes no second connection either.) - originColumn := s.originColumn(ctx, repoID, issue) + originColumn := s.originColumn(ctx, repoID, issue, row.IssueTrackerConnectionID.Valid) resolvedWaitOnLimit := s.resolveWaitOnLimit(ctx, userID, waitOnLimit) run, err := s.createRunResolved(ctx, userID, explicit, func(q Store, resolved resolvedHarness) (store.Run, error) { // Issue #1626: the authoritative cross-kind branch check, FIRST in the closure. The @@ -8561,6 +8561,20 @@ func (s *Service) uziLabel(ctx context.Context) string { return settings.DefaultUziLabel } +// autopilotLabel is the configured autopilot label, falling back to the compiled-in +// default when the settings reader is unwired, cannot say, or has it blank. It reads the +// label through an optional interface so a SettingsReader implementation need not carry it. +func (s *Service) autopilotLabel(ctx context.Context) string { + if r, ok := s.settings.(interface { + AutopilotLabel(ctx context.Context) (string, error) + }); ok { + if l, _ := r.AutopilotLabel(ctx); l != "" { + return l + } + } + return settings.DefaultAutopilotLabel +} + // isEligibleIssue reports whether a cached issue's labels jsonb carries ANY of the // given eligible labels (PRD #764 M1: the single-element uzi_label set). A row whose // labels cannot be decoded is NOT eligible: the gate has no basis for letting it @@ -8602,7 +8616,11 @@ func isAssignedToBot(assigneeIDsJSON []byte, botID int64) bool { // cannot be listed the origin is left NULL (unknown), NOT "" — the run is not // blocked, but a later restore must skip rather than confidently strip the card // to Open on a guess (spec invariant: NULL = unknown = never restore). -func (s *Service) originColumn(ctx context.Context, repoID uuid.UUID, issue store.Issue) pgtype.Text { +// +// linked is the repo's linkage (PRD #4173 M5): a linked card's column is its cached +// issue_tracker_column, never a label, so the origin is the card's real column and the +// lifecycle's manual-drag guard compares like with like. +func (s *Service) originColumn(ctx context.Context, repoID uuid.UUID, issue store.Issue, linked bool) pgtype.Text { cols, err := s.q.ListBoardColumns(ctx, repoID) if err != nil { return pgtype.Text{} // NULL = unknown, never guess Open @@ -8615,7 +8633,7 @@ func (s *Service) originColumn(ctx context.Context, repoID uuid.UUID, issue stor if err := json.Unmarshal(issue.Labels, &labels); err != nil { labels = []string{} } - col, _, _ := board.ResolveColumn(labels, issue.State, position) + col, _, _ := board.ResolveColumn(board.CardColumnLabels(linked, labels, issue.IssueTrackerColumn.String), issue.State, position) return pgconv.Text(col) // always valid: "" = the implicit Open column, never NULL } diff --git a/changelog.d/010f4981-939a-4a4b-95c4-69db1e7f5356.md b/changelog.d/010f4981-939a-4a4b-95c4-69db1e7f5356.md new file mode 100644 index 0000000000..81968ae3f3 --- /dev/null +++ b/changelog.d/010f4981-939a-4a4b-95c4-69db1e7f5356.md @@ -0,0 +1,4 @@ +### Added + +- **A repo linked to YouTrack now moves cards, creates issues and writes comments in YouTrack (#4173).** + Board moves write the issue's State through a per-repo column-to-State mapping set at `GET`/`PUT /api/repos/{id}/issue-tracker-link/states` (no UI yet); close never lands on a won't-fix State, a column move refuses an issue YouTrack already shows as resolved (only an explicit reopen from the Closed lane moves it; a drag to Closed still closes it), issues uzi creates carry the eligibility tag where a forge issue would carry the uzi label, and removing that tag is still refused. There is no close-on-merge, and the web and CLI settings, docs and the live YouTrack pass are still pending. diff --git a/e2e/forge-fake/issue-tracker-contracts.test.mjs b/e2e/forge-fake/issue-tracker-contracts.test.mjs index 5102916f68..dfcec97209 100644 --- a/e2e/forge-fake/issue-tracker-contracts.test.mjs +++ b/e2e/forge-fake/issue-tracker-contracts.test.mjs @@ -49,7 +49,7 @@ async function stop(child) { // seams. ProjectGuard must pass on every forge. const ALLOWED_SKIPS = new Set(["MovedIssueIsRefused", "SameNameTagUsesEligibilityTag", "RenamedEligibilityTagIsStillListed", "ListLabelsOnlyUsableTags"]); const CASES = [ - "EnsureLabelsIsIdempotent", "CreateIssueRoundTrips", "ListIssuesFilters", "UpdateLabelsAndDescription", + "EnsureLabelsIsIdempotent", "CreateIssueRoundTrips", "ListIssuesFilters", "UpdateLabelsAndDescription", "MoveToColumnMovesTheIssue", "SetIssueStateClosesAndReopens", "CommentsOldestFirst", "LabelEventsOldestFirst", "MissingIssueIsNotFound", "MovedIssueIsRefused", "ProjectGuard", "SameNameTagUsesEligibilityTag", "RenamedEligibilityTagIsStillListed", "ListLabelsOnlyUsableTags", ]; diff --git a/prds/4173-external-issue-trackers.md b/prds/4173-external-issue-trackers.md index 75afc04a6b..2691d90d0f 100644 --- a/prds/4173-external-issue-trackers.md +++ b/prds/4173-external-issue-trackers.md @@ -4,7 +4,7 @@ **Priority:** Medium -**Status:** In progress, r10 2026-11-17. M1–M4 delivered; M5 and M6 pending. r3 narrowed scope to one real issue tracker; r4 fixed a repo's issue source before first enable (D13); r4 and r5 fixed the YouTrack id, tag and author decisions before the driver; r6 records the M3 rework that follows them; r7 records the M4 preconditions from the maintainer's M3 approval (id-scoped intake, `ListLabels` limited to usable tags) and the M4 design decisions; r8 records M4's completion; r9 (2026-11-16) records the M4 follow-ups (the enable and link race, forge health on linked repos, removed and re-listed repos, the autopilot D11 guard, small items); r10 (2026-11-17) records the port of M4 and the M4 follow-ups onto the final M3 and the current upstream main. +**Status:** In progress, r12 2026-11-18. M1–M5 delivered; M6 pending. r3 narrowed scope to one real issue tracker; r4 fixed a repo's issue source before first enable (D13); r4 and r5 fixed the YouTrack id, tag and author decisions before the driver; r6 records the M3 rework that follows them; r7 records the M4 preconditions from the maintainer's M3 approval (id-scoped intake, `ListLabels` limited to usable tags) and the M4 design decisions; r8 records M4's completion; r9 (2026-11-16) records the M4 follow-ups (the enable and link race, forge health on linked repos, removed and re-listed repos, the autopilot D11 guard, small items); r10 (2026-11-17) records the port of M4 and the M4 follow-ups onto the final M3 and the current upstream main. r11 (2026-11-17) records the M5 design decisions before code. r12 (2026-11-18) records M5's delivery: the board and issue writes on YouTrack; the web and CLI surfaces, docs and the live pass remain M6. **Delivery:** one PR per milestone, each merged before the next starts. No implementation or validation step touches `.github/workflows/**`. @@ -140,7 +140,7 @@ Progress (one PR per milestone, each merged before the next starts): - [x] **M2:** every caller on its half. The type-checked inventory shows 45 issue-side calls through `IssueTracker`, 66 code-host calls through `CodeHost` and none through `forge.Forge`, and handing `privcheck` an `IssueTracker`-only value fails to compile. - [x] **M3:** issue tracker connections and the YouTrack driver. The contract suite, with the `ProjectGuard` and `SameNameTagUsesEligibilityTag` cases, runs on the real YouTrack driver and the three real forge drivers; the tag sharing settings parse is tested against a response recorded from a real YouTrack; the chart renders `issueTracker.allowedBaseURLs` and refuses a non-https entry. - [x] **M4:** issue identity, the repo link and issue intake from YouTrack. Live-DB acceptance tests with the YouTrack fake cover linked intake, D11 author mapping and eviction; the linked claim golden is read by the agent contract test; the recorded-call golden of the repo handlers changed in the 2026-11-17 port (the three `ListIssues` calls after a repo enable are gone, because upstream moved the first import from the enable request to the poller, issue #4399); every other recorded call is unchanged. -- [ ] **M5:** the board and issue writes on YouTrack +- [x] **M5:** the board and issue writes on YouTrack. The forge recorded-call goldens are byte-identical (the Recorder keeps `MoveToColumn` in a separate `Moves()` log); live-DB tests with the YouTrack fake cover a move writing the State and caching the column, an unmapped, failed, moved-away or already-resolved-in-YouTrack move leaving the card where it was, and close refusing a State renamed to Won't fix; `ClosableState` is an allowlist of done names. - [ ] **M6:** web and CLI surfaces, docs and live acceptance ### M1: the `IssueTracker` and `CodeHost` interfaces @@ -272,7 +272,7 @@ Settings → Issue trackers, the CLI commands, a setup page under `docs/`. - **2026-11-11: the link and the first enable share the repo row lock, and the database refuses an issue number on a never-enabled repo (D13).** The enable is an `UPDATE` (row lock); the link is one transaction that reads the owner-scoped repo `FOR UPDATE`, refuses unless it is awaiting and unlinked, refuses if any table stores an issue number for it, then writes the link. Under READ COMMITTED whichever commits first wins, so exactly one D13 outcome follows. One shared plpgsql guard trigger on every table that stores an issue number for a repo refuses a non-NULL number while the repo is awaiting; since the marker only goes true to false, a write that passed the guard means the repo can never be linked, which closes the race a review found in the link's `EXISTS` check (kept as defence in depth). Application gates in front of the guard return a clean 409 before any forge or issue tracker write, so no created issue is stranded: board seed and refresh, `importAfterEnable`, `createRun`, task runs, schedule create, patch, run-now and catalog enable, self-improve, and every forge-first issue creator; the guard's SQLSTATE maps to the same error. No relink and no unlink. Remaining window: `EnableRepo` seeds board labels before its `UPDATE`, so it re-reads the link first; a link landing between that read and the update can still leave forge labels on a linked repo, while the database reaches one outcome. Rejected: an advisory lock (D13 names the row lock), Go-only gates (racy). - **2026-11-11: the resolvers bind repo, owner and issue tracker connection, and fail closed (D5, D10).** `IssueTrackerForRepo(ctx, userID, repoID)` and `CodeHostForRepo(ctx, userID, repoID)` read one owner-scoped row joining the repo, its forge connection and its issue tracker connection for the same owner, keyed by repo id and owner, never by forge project number. A linked repo gets the YouTrack driver bound to the link's project, the repo's forge project id and the eligibility tag; a link whose connection is missing for that owner is an error, never a forge fallback. `CodeHostForRepo` never reads the link (S2). Services left on their own factories by M2 get the resolver by setter, and the repo rows they already read carry the link id; without a resolver they may use the forge factory only for an unlinked row, so a missing wire fails closed with no boot check and existing fakes stay unchanged. Rejected: a method on every `ForgeBuilder` (about 28 fake edits), resolving by `(connection_id, forge_project_id)` (the collision itself), a boot check (unneeded under fail closed). - **2026-11-11: a renamed eligibility tag is handled in the driver (D14), with an id-only fallback; the mechanism matches the upstream M3 driver helpers (owner guidance, 2026-11-11).** Asked for the eligibility label, `ListIssues` reads the tag by id (`GET tags/{id}`, fields `id,name`) and queries with its live name; the client-side filter keeps only the stored id. A tag that cannot be read is an error and nothing is listed: the sync fails, logged, nothing is evicted. A live name that `labelNameProblem` refuses gets no tag term at all: the query is the project term only, plus the existing client-side id filter, under the existing listing cap. Upstream names the helpers `queryTagNames` and `eligibilityLiveName` in `youtrack_labels.go` and `issueQuery(shortName, tagNames, opts)`; M4 keeps its change to those driver functions minimal so the upstream merge brings no second design. Callers keep naming the stored name, and the stored name is not rewritten in M4. Rejected: always id-only (pages the whole project every tick), refusing a renamed tag at sync (with no relink the repo would break for good), a poller-side rename (more moving parts). The forge subjects skip the new contract case, as forge labels have no owners. -- **2026-11-11: on a linked repo M4 writes nothing to the issue tracker and skips the forge-only features.** The resolver wraps the driver in a write guard: label, description, state, create and comment calls return `ErrIssueWriteNotOffered` without a request (writes are M5). That error, and a not-found the linked wrapper marks (a moved issue, D10), are a permanent skip, so the run lifecycle, the MR state record, the PRD link patch and comments never wait on the issue side. Autopilot (D12), GitHub Projects v2 sync (its numbers are forge issue numbers; a project link on a linked repo is refused) and self-improve (it creates an issue) are skipped. A linked run's MR-state watch moves to the board-free lane, which records the MR edge and cancels rework without the issue cache or a card label; the board lane excludes linked repos, so the lanes stay disjoint and unlinked repos match as before. +- **2026-11-11: on a linked repo M4 writes nothing to the issue tracker and skips the forge-only features.** The resolver wraps the driver in a write guard: label, description, state, create and comment calls return `ErrIssueWriteNotOffered` without a request (writes are M5). That error, and a not-found the linked wrapper marks (a moved issue, D10), are a permanent skip, so the run lifecycle, the MR state record, the PRD link patch and comments never wait on the issue side (writes superseded 2026-11-17 by M5, which lifts them one operation at a time; see the write guard entry). Autopilot (D12), GitHub Projects v2 sync (its numbers are forge issue numbers; a project link on a linked repo is refused) and self-improve (it creates an issue) are skipped. A linked run's MR-state watch moves to the board-free lane, which records the MR edge and cancels rework without the issue cache or a card label; the board lane excludes linked repos, so the lanes stay disjoint and unlinked repos match as before. - **2026-11-11: migration `00425_issue_tracker_repo_link`** is the next free number on this branch; it is renumbered at upstream landing (it landed as `00466_issue_tracker_repo_link`, see the 2026-11-17 entry). - **2026-11-11: issue tracker text goes through `issueinput` unchanged (S4, owner guidance, 2026-11-11).** YouTrack titles, descriptions and comments reach a run through the same `issueinput` bounds and fencing as forge issue text; tests compare an oversized YouTrack description and comment with their forge equivalents. - **2026-11-11: a reconnect cannot change the account of a connection with a linked repo (D11, D14, owner guidance, 2026-11-11).** A reconnect (`Connect` upsert on the same user, kind and base URL) that would change the connection's account id is refused while any repo links to it, so mapping (D11) and the owner of the tags uzi creates (D14) never change silently. The link binds to the account its tag check ran against: the link transaction reads the connection row `FOR SHARE` and requires its account id to equal the checked account. The reconnect locks the connection row `FOR UPDATE` and checks for linked repos in a separate statement in the same transaction, so the two cannot race. Tested. @@ -282,12 +282,12 @@ Settings → Issue trackers, the CLI commands, a setup page under `docs/`. - **2026-11-12: offset paging and the linked sync's marks.** The id-scoped listing pages by offset over a set that can change during the walk (an issue tagged, untagged or updated mid-walk shifts later entries), so a walk can miss an issue that was eligible throughout. Because the listing has no server-side date filter, a narrower incremental request saves nothing, so the linked sync never passes `UpdatedAfter`: the incremental form walks the same listing as the full form, upserts every issue it returns and evicts nothing; only the full form evicts. So no mark can hide a missed issue: every later sync tick, incremental or full, walks the whole listing for it again, and it is cached by the first walk that does not miss it, normally the next tick (a walk misses it only if the set shifts again across it during that walk). The marks are kept for the poller's bookkeeping but never filter the linked listing. If a full walk missed it, that full sync evicts it and the first later walk that sees it caches it again. Stated exactly (operator point, 2026-11-12): an issue a walk missed is caught by the next linked sync of either form, not only the next full sync, because no form passes `UpdatedAfter` and the marks the poller keeps are advanced only by the newest `updated` of what a walk returned and are never read back by the listing; only a full walk evicts. Tested at both levels: `TestSyncIssuesDispatchesOnTheLink` (an incremental sync with marks later than the issue's `updated` upserts it and never evicts) and `TestLinkedRepoIncrementalSyncCachesBelowTheMarksAndNeverEvictsLiveDB`. - **2026-11-12: `ListLabels` returns only what the driver can use (D14; the maintainer's M3-approval precondition).** The stored eligibility name, plus every tag the connection's account owns (by the `users/me` account, never a stored id), excluding the eligibility tag's id, once per name; a failed `users/me` is an error. Before, it returned every visible tag name, so `CheckRepoLabels` could report present a label the driver can neither list issues by nor remove. It supersedes "`ListLabels` lists the tags the token can see, once per name" in the 2026-11-11 entry "YouTrack label and id details that still hold". The contract case `ListLabelsOnlyUsableTags` (to be added in M4) seeds a foreign tag; the forge subjects skip it, because forge labels have no owners. - **2026-11-12: reads report the eligibility tag under its stored name (D14).** Issue reads and label events report the tag whose id is the stored eligibility id under the stored eligibility name, so the cache, the run eligibility gate and the sweep selector compare one stable name; a same-named tag with another id is still dropped. This replaces "reads report each tag … under its live YouTrack name" in the 2026-11-11 entry "YouTrack tag names resolve to ids in one place in the driver", for the eligibility tag only; other tags keep their live names. -- **2026-11-12: the linked sync, eligibility gate and sweep selector (D10, D14).** The linked sync caches only issues carrying the eligibility tag (no open-issue or finding fetch), writes `issue_key` as the link's project short name plus `-` plus `numberInProject`, reads the link through the owner-scoped repo row (the sync entry points take the owner), and replaces the `ErrLinkedSyncNotReady` refusal. On a linked repo, run eligibility and a sweep's default selector use the link's stored eligibility name instead of the configured uzi label; the run's fresh issue read must also carry it, so an issue untagged since the last full sync cannot start a run. Bot assignment never counts on YouTrack (assignees are empty). Remove-label-on-dispatch keeps comparing with the configured uzi label; on a linked repo that removal is a write, refused as skippable until M5 (an M5 item). +- **2026-11-12: the linked sync, eligibility gate and sweep selector (D10, D14).** The linked sync caches only issues carrying the eligibility tag (no open-issue or finding fetch), writes `issue_key` as the link's project short name plus `-` plus `numberInProject`, reads the link through the owner-scoped repo row (the sync entry points take the owner), and replaces the `ErrLinkedSyncNotReady` refusal. On a linked repo, run eligibility and a sweep's default selector use the link's stored eligibility name instead of the configured uzi label; the run's fresh issue read must also carry it, so an issue untagged since the last full sync cannot start a run. Bot assignment never counts on YouTrack (assignees are empty). Remove-label-on-dispatch keeps comparing with the configured uzi label; on a linked repo that removal is a write, refused as skippable until M5 (an M5 item; superseded 2026-11-17 by M5, which compares with the eligibility label). - **2026-11-12: D11 wiring.** `issueinput` defines `AuthorMapper` (implemented by the YouTrack driver as `MapsAuthor`, forwarded by the linked write guard). When the issue side maps authors, the forge's `RepositoryAuthorEligibility` is never called: an author is eligible only when its login is non-empty and equals the `users/me` login of the connection; anything else, including a failed lookup, is `permission_unknown`. A guard over an issue tracker that cannot be built maps nobody. The run capture and the worker get-issue route assess the same way. - **2026-11-12: D11 maps the issue text by who wrote it.** The reporter is who created an issue, not who wrote its current title and body: another user can edit a description the connection's account reported. The issue text is mapped only when the reporter maps and every recorded change to the title and body (`IssueCreatedCategory`, `SummaryCategory`, `DescriptionCategory` activities, creation included) was made by the connection's own account, and the creation activity exists and is by the account, so an empty history is not mapped (`issueinput.IssueTextMapper`, driver `IssueTextMapped`, forwarded by the write guard). An activity with no author, a different login, a missing creation activity, an unreadable activity list or the listing ceiling is `permission_unknown`; the capture and the worker get-issue route read it right after the issue, before the comments, and never cache it under the login the comment mapping uses. Comments have the same flaw: the driver also reads `CommentTextCategory` activities and reports a comment whose text was changed by a login other than its own author with an empty author, so it is withheld as unknown; when those activities cannot be read it reports every comment with an empty author (all withheld) and the run goes on. Tested end to end: an issue the account reported and tagged, with its description edited by another user, is created by a scheduled start without auto-approval and with `permission_unknown`; without the edit it is auto-approved. - **2026-11-12: a linked run's claim is non-closing by the repo link, not the issue row.** The claim's `issue_key` is built from the repo's stored project short name and the run's issue number, so an evicted issue row can never turn a linked claim into a closing one. The agent folds "linked" into its single non-closing-delivery predicate; it renders `Related to ABC-12.`, never a closing keyword, and still refuses commits that would close `#` on the code host, an unrelated issue. This supersedes "and no closing gate" in the 2026-11-11 entry "a linked run's issue references use the issue key": the commit-message gate stays, because `#` on the code host is an unrelated issue that must never be closed. The API also records a linked run's source publication as non-closing, whatever the worker reports. - **2026-11-12: agent tests use `node:test`.** The agent package runs its unit tests with `node:test` through `tsx` and has no Vitest; the M4 agent tests follow it. The board-card change in `web/` uses Vitest. -- **2026-11-12: a linked run's board-free MR watch keeps the reopen edge (M4 review).** The board lane excludes a linked repo, so its board-free lane also watches a linked issue run's closed MR while that run is the issue's latest and the cached issue is open (the board lane's Lane A bound), and records a reopen as `opened`; live rows sort ahead of closed ones under the lane's `LIMIT 100`, so a closed watch never starves a live MR. An issue evicted from the cache (moved, untagged) drops the watch. Left for M5, where issue writes become real: a linked run whose issue is not cached yet has its pending card move cleared as moved (harmless while writes are refused), and the PRD link patch reads the issue before its refused write settles the marker. +- **2026-11-12: a linked run's board-free MR watch keeps the reopen edge (M4 review).** The board lane excludes a linked repo, so its board-free lane also watches a linked issue run's closed MR while that run is the issue's latest and the cached issue is open (the board lane's Lane A bound), and records a reopen as `opened`; live rows sort ahead of closed ones under the lane's `LIMIT 100`, so a closed watch never starves a live MR. An issue evicted from the cache (moved, untagged) drops the watch. Left for M5, where issue writes become real: a linked run whose issue is not cached yet has its pending card move cleared as moved (harmless while writes are refused), and the PRD link patch reads the issue before its refused write settles the marker. (Resolved 2026-11-17 by M5: see the uncached-run probe and comments entries.) - **2026-11-12: the id-scoped walk fails closed at the listing ceilings, which other projects' issues count toward (D14, M4 security review).** The tag listing returns the tag's issues from every project, resolved ones included, and the item, page and byte ceilings count them before the client keeps only the bound project. So whoever can apply the eligibility tag, or its slow accumulation on resolved issues, can push the walk past a ceiling (about 50,000 tagged issues by items, fewer by bytes because descriptions are read); then every listing fails and intake for the bound project stops, logged, evicting nothing, until the tag is taken off issues elsewhere. Memory stays bounded. The people who can apply the tag are the ones the admin confirmed at link time should start work (D14), so this is a fail-closed availability limit, not an escalation. Operator remedy: remove the eligibility tag from issues outside the linked project, or from resolved ones; the documented project-walk fallback would avoid the cross-project part. Tested in the driver: one eligible issue plus more than a lowered item ceiling in another project fails the listing rather than returning part of it. - **2026-11-12: a worker must advertise `issue_key_v1` to claim a run on a linked repo (M4 review).** A worker image from before M4 ignores `issue_key` and would write `Closes #` into its first merge request body, closing an unrelated code-host issue; the server stamping the publication non-closing does not stop it. uzi already allows the API and the workers to run different versions (each protocol change has its own claim capability), so "ship together" is not a guarantee. `ClaimRun` gets a standalone clause, outside `fn_worker_can_claim`, `required_capabilities` and the kill-switch: a run whose repo has an issue tracker link is claimable only by a worker whose stored protocol capabilities contain `issue_key_v1`. The clause is mirrored at every site of the `ci_config_guard_v1` clause and pinned by a test, and the ephemeral placement listing counts such a run as demand. Without such a worker the run waits, with a health reason naming the capability. Roll the API out before the workers; a newer worker against an older API is unaffected. Rejected: a documented ship-together boundary (nothing enforces it), and refusing at claim time (the run would have to be released and requeued). - **2026-11-13: M4 is complete; three gaps go to M6.** The final M4 run added live-DB acceptance tests for linked intake and D11 (comments and unmapped author, manual and scheduled start), incremental marks, a deleted-row claim keeping its `issue_key`, and a board with no `issue_key` for an unlinked repo; and a web board card with an `issue_key` counts as uzi's and shows its key. Deferred to M6: the issue detail page and its API payload do not carry `issue_key` yet, so on a linked repo the detail page decides eligibility by labels and shows `#iid`, and can offer Promote, whose write is refused until M5; board search does not match the key; screen-reader labels still name `#iid`. Correction to the 2026-11-11 entry "M3 layout": M4 wired the connection and repo link routes into `cmd/server` (`PUT /api/repos/{id}/issue-tracker-link`, session-only); the web and CLI screens remain M6, and a connection still needs the operator's `ISSUE_TRACKER_ALLOWED_BASE_URLS`. S4's comment bound is tested for the connection's own account (truncated to the forge bound) and for an unmapped author (withheld). @@ -305,3 +305,31 @@ Settings → Issue trackers, the CLI commands, a setup page under `docs/`. - *Every test the port adapted.* `TestListIssuesFollowsARenamedEligibilityTag`, `TestEligibilityNameAloneMakesNoRequest` (allows the by-id listing request and requires at least one) and `TestIssueQuery`, as above; the handler recorded-call golden (the three `ListIssues` calls after a repo enable are gone, the first import is the poller's); the `youtrack_issues_test.go` label-filter order, now `[6 1 5]`; the `youtrack_labels_test.go` `ListLabels` test, which returns the eligibility name plus `dup`; the health-check count tests, 27 to 28; and the live-DB package count in `e2e/run-store-it.test.sh`, 8 to 9. New in the port: `poller/linked_repo_initial_import_livedb_test.go`, which pins an owed first import on a linked repo (claimed in one tick, listed through the issue tracker and not the forge, settled `succeeded`, or `failed` with the issue tracker's error class). - *Migrations* land as `00466_issue_tracker_repo_link` (was `00425`) and `00467_repo_removal_markers` (was `00427`). The Postgres row-type fix and its migration from the fork are not brought over: upstream fixed the same defect independently in `00436_intake_pause_string_body`. - **2026-11-17: a re-listed linked repo is refused wherever issues are read, imported or written or a run starts (D13, review finding).** The 2026-11-16 removal-marker entry refused only the enable and the link. The re-list trigger clears `awaiting_first_enable` for a project with a marker, so the never-enabled gates passed the new row (disabled, unlinked, `was_linked`), and board open, sync, issue detail and run creation could read or import forge issues numbered unlike the issue tracker's, then start work on the same `agent/issue-` branches. Fix: `GetRepoForUser` and `GetRepoByID` return `issue_source_locked`, the expression `LockRepoForEnableForUser` already uses for the enable refusal (unlinked and a `was_linked` marker); `store.RefuseNeverEnabled` takes it beside `awaiting_first_enable` at every existing never-enabled gate and answers `store.ErrRepoSourceLocked`, which matches `ErrRepoNeverEnabled` under `errors.Is` (so every existing 409 mapping fires) with the message of the real reason. Five handlers gained the gate (`GetIssueDetail`, `MoveIssue`, `PromoteIssue`, `SetBoardOrder`, `EnsureRepoLabels`), and all five now also refuse a never-enabled repo; before, only a direct issue-detail URL reached such a repo (the board already refused), and `EnsureRepoLabels` no longer writes forge labels before the refusal. No new column, migration or marker semantics. The poller never reaches such a row, because it lists enabled repos only and a source-locked row cannot be enabled. Accepted gaps: the Slack proposal-confirm path (`ErrChatProposalRepoNeverEnabled`) still answers the never-enabled text for this refusal, because a distinct message needs a new slacksvc sentinel (the Slack start-run answer, `startRunCardMessage`, was fixed to name the removed issue tracker link), and the database issue-number guard still reads only `awaiting_first_enable`, so the refusal is the application gates, not a trigger. Also fixed in the same review: the manual run-now scheduler built in `handler.New` is wired with the issue sources, as the background scheduler is, so a manual fire of an issue or sweep schedule on a linked repo no longer answers 502 (`issuesource.ErrNotWired`). +- **2026-11-17: a card move is `MoveToColumn(ctx, projectID, issueIID, forge.ColumnMove{Column, Ensure, Add, Remove})` on `IssueTracker` (D9, M5).** `Column` is the target column's label name (`""` is the implicit Open column); `Ensure`, `Add` and `Remove` are exactly what AutoMove plans today with `board.PlanLabelMove` and `ColumnColor`. The three forge drivers implement it through one shared helper, `forge.MoveByLabels`, which makes today's calls unchanged: `EnsureLabels` only when `Ensure` is non-empty, then one `UpdateIssueLabels` with the slices passed through (nil versus empty is preserved). AutoMove goes through `MoveToColumn`; `SetIssueLabel` stays a label write and is never a column move. The YouTrack driver maps the column to a State and ignores the label fields, because columns are not tags (D14). `forgetest.Recorder` appends `MoveToColumn` to a separate `Moves()` log and runs `MoveByLabels` over itself, so its `EnsureLabels` and `UpdateIssueLabels` lines are recorded and delegated as before; it never hands the move to `inner.MoveToColumn`. Why a separate log: the issue asks both that the Recorder records `MoveToColumn` and that the forge goldens stay unchanged, so `Calls()` and the five goldens holding card moves (`runlifecycle` cardmove and terminalcomment, `forgesvc` mrstates, `handler` repo_handlers, `poller` tick) stay byte-identical and a new test asserts `Moves()`. Rejected: recording `MoveToColumn` in `Calls()` (it changes every golden) and a move that drops `EnsureLabels` (it changes the forge request sequence). +- **2026-11-17: the column-to-State mapping lives in side tables keyed by label name (D9, M5).** Migration `00468_issue_tracker_state_mapping` (the next free number on this branch; renumbered at upstream landing) adds `repo_issue_tracker_states` (one row per repo: `open_state`, `closed_state`, which must differ), `repo_issue_tracker_column_states` keyed by `(repo_id, label_name)` with `UNIQUE (repo_id, state_name)`, and a nullable `issues.issue_tracker_column`, the cached board column of a linked card. Every State name is 1–200 characters with no control characters, no looser than the driver's `maxLabelNameLen` (200 bytes), so a name the driver accepts always fits. Keyed by label name because `ConfigureColumns` replaces `board_columns` wholesale and its ids change on every reconfigure. Rejected: a foreign key on `board_columns` ids with a cascade (every reconfigure would silently delete the whole mapping). `DriverFor` takes an `issuetracker.StateMapping` (field, open, closed, columns) in place of its three state strings, and the resolver passes the stored values. Rolling the migration back drops the mapping and the cached columns. +- **2026-11-17: the mapping is set through `GET` and `PUT /api/repos/{id}/issue-tracker-link/states` (M5).** Session-only (the `requireAuth` group), owner-scoped through the repo lookup, behind the forge limiter, like the link route. PUT replaces the whole mapping in one transaction under the repo row lock; the repo must be linked (409), every column must be a current board column (400), and every value must be a state `ListStates` returns for the linked project and pass the "never won't fix" rules below (400). `ConfigureColumns` on a linked repo takes the same row lock, writes the column rows only, and prunes mapping rows whose column is gone; it never calls `EnsureLabels`, because columns are not tags (D14). GET returns the stored mapping and the project's states (name, resolved, closable). No UI: that is M6. +- **2026-11-17: an unmapped column is never guessed (M5).** The YouTrack driver returns `ErrColumnNotMapped` before any request. A board drag onto it returns 409 and the card snaps back (the web board moves nothing optimistically). The lifecycle treats a refused move like a failed one: it keeps `move_pending_since`, and reconciliation retries inside the existing 30-minute window, so a mapping saved in time lets the move land (`specs/human.md`, Feature #12: failed forge column moves are retried via reconciliation). The MR watch counts a refused move as `moveSkipped` and records the MR state with no move: an accepted exception to that line, because `mr_state` is the lane's only marker and keeping it would stall the watch, its rework cancel and the `mr_rework` gate. `IssueWriteSkippable` is not widened; a separate classifier names the refusals. +- **2026-11-17: "never won't fix" is an allowlist, checked at save and before every State write (M5).** YouTrack's state bundle has no won't-fix flag, and `isResolved` is true for `Won't fix`, so resolved alone is not enough. `issuetracker.ClosableState` is true only for a resolved, not archived state whose normalised `name` (lower-cased, trimmed, whitespace, `-` and `_` collapsed) is one of: fixed, done, resolved, verified, completed, complete, closed, implemented, shipped, released, delivered, merged, finished. Closability is decided on the State's `name` alone. `localizedName` is display only, because the localized name is free text set by the same bundle admin, and an earlier "name or localizedName" rule let a won't-fix name relabelled "Fixed" through (found in review). `ClosableState` itself rejects names that fail the state-name bounds, which now also reject default-ignorable fillers. Anything else fails closed, including `Won't fix`, `Duplicate`, `Obsolete` and custom names. The open state and every column state must be unresolved and not archived, and all mapped states must be distinct. The rules apply when the mapping is saved and again from the live bundle before every YouTrack State write, so a bundle edited after the save (a state renamed to `Won't fix`, or made resolved) cannot be landed on; a failure returns `ErrStateNotAllowed`, which a board drag maps to 409 (the card snaps back), like `ErrStateMappingUnset`; a YouTrack failure on a drag returns 502 and a moved-away issue 404. `ListStates` drops state names that fail the tag-name bounds, so such a state cannot be mapped. Rejected: a denylist of won't-fix names (an unknown name would be accepted, so "never" would not hold). Limitations: a done state named only in another language is refused; whether a non-admin account can read the `customFields` endpoint the check uses, and the window between the check and the write (as in D10), go to M6. +- **2026-11-17: close and reopen use the stored explicit values (M5; continues the 2026-11-11 entry).** Close writes the stored closed state through `SetIssueState`; an unset value is `ErrStateMappingUnset`, a 409. A linked reopen calls `MoveToColumn` directly with no separate open-state write, so the column and bundle checks run before the single State write, and a reopen onto an unmapped column changes neither YouTrack nor the cache. The reopen sets `ColumnMove.Reopen`; see the 2026-11-18 resolved-issue entry. The board handler (`MoveIssue`) is the only caller of `CloseIssue` and `ReopenIssue`. **No close-on-merge in M5.** A linked MR body is non-closing since M4, so where a forge relies on `Closes #N` the linked issue stays in its column (Human Review) after the merge; the board-free lane records `merged`, and a person closes the issue from the board or in YouTrack. +- **2026-11-17: a linked card's board column comes from its State (D9, D10, M5).** The YouTrack driver reads a bounded `State` custom field with each issue (a field list over the cap fails the listing) and returns it through `ListIssuesWithState`, which yields `issuesource.StatedIssue{Issue, State}`; `forge.Issue` is unchanged (D10). The linked sync maps the State back to a column through the stored mapping and caches it in `issues.issue_tracker_column`; unmapped states, the open state and resolved states map to the Open lane, and closed is decided by the issue state. The column has three writers: the linked `UpsertIssueIfChanged` (the column joins its change check, so a changed mapping is cached again); `SetIssueTrackerColumn`, which writes only that column after a successful move; and `ClearIssueTrackerColumnsNotIn`, which `ConfigureColumns` on a linked repo runs in the same transaction that prunes the mapping, so a removed column is not left cached (the card falls to the Open lane until the next sync). A sync that read the mapping before a concurrent reconfigure can briefly re-cache a removed column; `ResolveColumn` ignores a label that is not a column, so the card shows in the Open lane and the next sync heals it. A linked run started before M5 recorded its `origin_column` from labels; once the cache holds a mapped column, its next lifecycle step reads the card as dragged by hand and clears the marker without moving it, which is accepted. A sync whose listing began before a concurrent move can briefly write back the old column; the next sync heals it. One helper, `board.CardColumnLabels(linked, labels, column)`, feeds `ResolveColumn` in `assembleCards`, `issueToCard`, the lifecycle `apply`, `guardedMRMove` and `originColumn`; it is keyed on the repo being linked, not on `issue_key` (newly created rows have none), and on an unlinked repo returns the labels unchanged. `buildIssueDetail` works from a live read with no column and stays under M6's detail-page deferral (2026-11-13 entry). +- **2026-11-17: linked runs stay in the board-free MR-watch lane; the two lanes stay disjoint (M5).** The board-free lane gains card moves for a linked issue-lane run on the close and reopen edges only (an MR closed unmerged moves Human Review to In Progress; a reopen moves it back), and only when the run is the issue's latest issue-lane run, the board lane's semantics; a superseded run records its MR state with no move. It reuses `syncOneMRState`'s edge logic and `guardedMRMove` through a small adapter, not a third copy, and a differential test pins both lanes to the same edges. A missing cached row counts as `moveSkipped` on this lane (the lane's live rows are not tied to the issue cache; only its closed reopen-watch rows check `issues`), so an evicted issue's MR is never polled forever. Rejected: moving linked repos into the board lane. Its prefilter is label-based, and with no close-on-merge a merged linked run would be polled forever. +- **2026-11-17: issues uzi creates on a linked repo, and their labels (D12, D14, M5).** `issuesource.LinkedLabels` replaces the configured uzi label with the link's stored eligibility name and drops the autopilot label (D12). All eight create sites (the board's new issue, findings, finding groups, the judge's admin issue, review issues, the self-improve tracking issue, chat proposals and scheduled proposals) create in the linked project; the M4 skips in self-improve and proposal filing are lifted, and both resolve the issue side, never falling back to the forge. The eligibility tag is carried by the board's new issue, judge and review issues, findings and finding groups filed with send-to-uzi, and confirmed chat proposals carrying the uzi label. The rest (findings without send-to-uzi, the self-improve tracking issue, scheduled proposals) are created in the linked project and evicted from the cache by the next full sync, which caches only eligible issues. Parity difference: on a forge the self-improve tracking issue still shows on the board as a non-uzi issue; on a linked repo it does not. Promote uses the stored eligibility name. Remove-label-on-dispatch compares the selector with the eligibility label on a linked repo (resolving the 2026-11-12 sweep item), so a selector equal to the eligibility name is never removed, and the write guard refuses (`ErrIssueWriteNotOffered`, skippable) any label write that removes the eligibility name, so the eligibility tag is never stripped. `EnsureRepoLabels` gets an explicit linked 409 before any request, because the eligibility tag is chosen at link time and column and autopilot tags are not wanted. Autopilot and GitHub Projects v2 sync are not offered. On a linked repo a failed finding-group create is marked uncertain (202, manual release), because the YouTrack create returns no definitive rejection. YouTrack's create has no rollback: if the POST lands but attaching a tag or the read-back fails, the create site reports failure and reverts its claim; the untagged issue stays in YouTrack (not cached, since it is not eligible) and a retry can file a duplicate. This is accepted for M5 and listed for M6's live pass. +- **2026-11-17: the write guard after M5 (D10, M5).** The guard keeps wrapping the driver and lifts writes one operation per unit, each in the unit that adds its rules: moves and State first; then create and labels (with the label rewrite, the eligibility-removal refusal and the `EnsureRepoLabels` check); then comments and the description. Until its unit a write is refused as in M4. A lifted write passes through wrapped in `markNotFound`, so a write that reaches a moved-away issue stays skippable (D10) and the code-host side continues. `unavailableIssueTracker` writes return their cause and are not skippable, so a failed repo-source read retries in the board-free lane (its linked issue-lane candidates are skipped that tick); any other unavailable cause records the MR state without a move. The refusal message is reworded to "this issue write is not offered on a repo linked to an issue tracker"; after M5 only the eligibility removal and `EnsureRepoLabels` reach it. A board drag onto a moved-away issue returns 404 and leaves the cache unchanged; the next full sync evicts it. +- **2026-11-17: the lifecycle probes a linked run's uncached issue (resolves the 2026-11-12 MR-watch entry's M5 item).** M4 cleared the pending card move of a linked run whose issue was not cached, which was harmless only while writes were refused. `GetRunMoveContext` gains the link's `eligibility_tag_name`, and the lifecycle reads the issue with `GetIssue` through the guard: not found (moved away) clears the marker; found but not carrying the eligibility tag (it will never be cached, such as the self-improve tracking issue) clears it; eligible, or a read error, keeps the marker so reconciliation moves the card once the next sync caches it, as on an unlinked repo. +- **2026-11-17: comments reach YouTrack (M5).** The CI autofix start and halt comments, the MR rework halt comment and the terminal comment are written to the linked issue. No retry and no outbox: a failed comment is logged and the run continues, as on a forge. `IssueNote.ID` stays 0. The autopilot refusal comment is unreachable on a linked repo (D12). The PRD link patch keeps reading the issue before it writes, so a moved-away issue settles on the read; the description write now lands. +- **2026-11-17: the M5 test-change rule.** Existing tests change only as named findings, each listed by name in a later entry when it changes: mechanical `MoveToColumn` one-liners (`return forge.MoveByLabels(...)`) on fakes that reach AutoMove without a Recorder (so far `forgesvc` `fakeForge`), since embedding cannot dispatch a base fake's move to an outer fake's label overrides; the `BaseFake` method-count pins in `forgetest` and `issuetrackertest`; `DriverFor` call sites taking `StateMapping`; and the M4 refusal tests that M5 turns into acceptance tests, subtest by subtest in the unit that lifts each write. Goldens do not change. +- **2026-11-18: the M5 U5 tests changed as "M4 refusal tests turned into acceptance tests".** `forgesvc` `TestGuardPassesMovesAndStateWritesThroughAndMarksNotFound` and `TestIssueTrackerForRepoLinkedBuildsTheBoundDriverBehindTheGuard`; `handler` `TestLinkedRepoFilingRoutesAreRefusedAndReleaseTheirClaimLiveDB` (renamed `TestLinkedRepoConfigureColumnsTouchesNoIssueTrackerLiveDB`) and `TestLinkedRepoIssueWritesAreRefusedWith409LiveDB` (renamed `TestLinkedRepoEnsureLabelsIsRefusedWith409LiveDB`); `schedsvc` `TestLinkedSelfImproveIsSkippedWithoutAnyIssueWrite`; `workersvc` `TestLinkedConfirmProposalIsRefusedAndReverts` and `TestLinkedRepoFilesNoScheduledProposal`. +- **2026-11-18: the M5 U6 tests changed as "M4 refusal tests turned into acceptance tests".** `forgesvc` `TestGuardPassesMovesAndStateWritesThroughAndMarksNotFound` (its subtest "description and note writes are still refused without reaching the driver" became "description and note writes reach the driver", with two new subtests for a moved-away issue and for the unavailable tracker), `TestIssueTrackerForRepoLinkedBuildsTheBoundDriverBehindTheGuard` (all seven writes are accepted and reach the driver; only the eligibility removal is still refused), `TestPRDLinkPatchSettlesWhenWritesAreNotOffered` (renamed `TestPRDLinkPatchWritesTheDescriptionOnALinkedRepo`: the description write reaches the tracker) and `TestUnavailableIssueTrackerReadsFailAndWritesAreRefused` (renamed `TestUnavailableIssueTrackerReadsAndWritesFailWithTheCause`: the write returns the cause and is not skippable); `runlifecycle` `TestLinkedTerminalCommentUsesResolverAndLogsSkip` (renamed `TestLinkedTerminalCommentLandsOnTheResolversTracker`: the note no longer fails with `ErrIssueWriteNotOffered`). The `runlifecycle` log line for a skippable terminal comment now says the linked issue is gone or moved. +- **2026-11-18: a column move never writes over an issue resolved in YouTrack unless it is the explicit reopen (M5 review finding).** The closed guards in the run lifecycle and the MR watch read only the cache, so an automatic move could reopen an issue a person resolved in YouTrack between two syncs. `forge.ColumnMove` gains `Reopen`, set only by `reopenLinkedIssue` (the explicit reopen of a closed card); label-backed forges ignore it. The YouTrack driver's `MoveToColumn` reads the issue's `resolved` and, unless `Reopen` is set, returns `issuetracker.ErrIssueResolved` with no write. `forgesvc.LinkedIssueResolved` names that refusal and is in neither `IssueWriteSkippable` nor `ColumnMoveRefused`: retrying cannot help, and the person's resolution wins until the next sync caches it. The run lifecycle clears the pending marker (as for the cached closed guard); the MR watch counts it as `moveSkipped` and records the MR state; a board drag to a column of a card the cache shows open answers 409 ("reopen it from the Closed lane after the next sync") and leaves the cache unchanged; a drag to Closed is a close (`SetIssueState`), not a column move, and is not refused. *Why a field, not a method:* the rejected alternative, a type-asserted `ReopenToColumn` forwarded by the write guard, changes the linked-reopen fakes and adds a guard seam for no behaviour gain, while a zero-value field costs the forge drivers nothing. *Accepted limits:* the check reads and then writes in two requests, so a resolution landing between them is still overwritten (the same window as `checkTargetState`); and a card the cache still shows open refuses drags to a column until the next sync. Tests: `issuetracker` `TestMoveToColumnRefusesAnIssueResolvedInTheIssueTrackerWithoutAReopen`, `TestMoveToColumnWithReopenWritesOverAResolvedIssue`; `forgesvc` `TestLinkedIssueResolvedNamesOnlyTheResolvedRefusal`, `TestGuardedMRMoveOnAnIssueResolvedInTheIssueTrackerRecordsAndSkips`; `runlifecycle` `TestLinkedMoveIssueResolvedInTheIssueTrackerClearsMarker`, `TestLinkedMoveOfAnIssueResolvedInYouTrackWritesNothingLiveDB`; `handler` `TestLinkedBoardDragOfAnIssueResolvedInYouTrackIs409LiveDB`. +- **2026-11-18: the M5 U2–U4 tests changed (the 2026-11-17 test-change rule; written after the fact, in review).** Every test function or test helper that existed before a U2–U4 commit and whose body that commit changed or deleted; new test functions are not listed. *Pre-M5 tests:* + - `forgetest`: `BaseFake` gains a not-stubbed `MoveToColumn`, so `actionMethods` and `TestBaseFakeActionMethodsNotStubbed` count 38, not 37 (U2). The `Recorder` gains `MoveToColumn`, logged in a separate `Moves()` so `Calls()` and the goldens are unchanged; `record` splits its formatting into `formatCall` (U2). + - `issuetrackertest`: `BaseFake` gains `MoveToColumn` and `TestBaseFakeMethodsNotStubbed` counts 16, not 15 (U2). The contract `Subject` gains `Columns`, `PlanMove` and `ColumnOf`; `RunContract` gains the `MoveToColumnMovesTheIssue` case; `caseMoved` and `caseProjectGuard` gain a `MoveToColumn` row (the guard table counts 12, not 11) (U2); `caseProjectGuard` also asserts a refused move leaves the column unchanged (U2). + - `issuetracker`: `TestYouTrackContract` (maps two columns to seeded States and supplies the contract seams; later its `ColumnOf` reports an unmapped State as a failure rather than the Open column) and `TestIssueTrackerForgeFakeContracts` (supplies the seams through `board.PlanLabelMove`) (U2). + - `e2e/forge-fake/issue-tracker-contracts.test.mjs`: `CASES` gains `MoveToColumnMovesTheIssue` (U2). + - `forgesvc`: `fakeForge` gains the mechanical `MoveToColumn` one-liner (U2) and `ListIssuesWithState` with a scripted `states` map (U4a); `fakeStore` gains `changedColumns`, `columnWrites` and `SetIssueTrackerColumn`, and its `UpsertIssueIfChanged` records the params field by field because they gained the column (U4a); `fakeDrivers.DriverFor` takes `StateMapping` (U3); `stubDriver` gains `MoveToColumn` and `ListIssuesWithState`, and `writeAll` gains a move (U4a); `TestIssueTrackerForRepoLinkedBuildsTheBoundDriverBehindTheGuard` (U3: asserts the zero mapping instead of empty strings; U4a: two writes accepted, the move and the State write, five refused); `TestIssueTrackerForRepoUnlinkedWritesReachTheForge` (U4a: seven recorder calls and one move, the move reaching the forge as its label call); `TestLinkedSyncWritesOnlyChangedRowsLiveDB` (U4a: `upsertIssues` takes the column map, passed nil). + - `handler`: `issueTrackerRoutes` and `wantRouteMounts` gain the two `states` routes (U3); `linkedDrivers.DriverFor` takes `StateMapping` and counts builds, and `linkedHandlerFixture` and `newLinkedHandlerFixture` carry the issue tracker service (U3); `TestLinkedRepoFilingRoutesAreRefusedAndReleaseTheirClaimLiveDB` (U3: `ConfigureColumns` leaves the refused table for an acceptance subtest that touches no driver and no forge; U4a: the refusal message is reworded); `TestLinkedRepoIssueWritesAreRefusedWith409LiveDB` (U4a: the "move issue" row is removed, the board move being offered, and the message is reworded); `assembleCards` and `issueToCard` take a linked flag, passed false, in `TestAccountCooldownReadersLiveDB`, `TestAccountCooldownBoardReaders`, `TestAccountCooldownBoardPrecedenceAndExpiry`, `TestAssembleCards`, `TestAssembleCardsCarriesAutofixHalt` and `TestCardLabelsNeverMarshalAsNull` (U4a). + - `issuetrackersvc`: `fakeRepo` and `fakeQuerier.GetRepoIssueSourceForUser` carry the State mapping (U3); `TestDriverForOpensTokenAndPassesOptions`, `TestDriverForFailsClosedWhenAllowlistEmptied`, `TestDriverForRefusesTamperedRow`, `TestDriverForRefusesNonCanonicalStoredURL` and `TestDriverForRefusesPartialOrMalformedBinding` pass a `StateMapping` (U3). + - `poller`: `refusingDrivers.DriverFor` takes `StateMapping` (U3). + - `runlifecycle`: `linkedFixture` seeds the card in the cached column (`IssueTrackerColumn`) instead of the label `Later` (U4a); `TestLinkedMoveIssueNoLongerCachedClearsMarker` is deleted (M4 cleared the marker of an uncached linked issue) and replaced by `TestLinkedMoveUncachedIssueProbe` (U4a, per the 2026-11-17 uncached-issue entry); `TestMovedLinkedIssueSkipsTheWriteAndTheCodeHostSideCompletesLiveDB` seeds the `In Review` State and saves a mapping so the write is attempted (U4a). + - *Tests added earlier in M5 and revised by a later U2–U4 commit:* `issuetracker` `TestMoveToColumnUnmappedSendsNothing`, `TestMoveToColumnWritesTheMappedState`, `TestMoveToColumnEmptyUsesTheOpenState`, `TestMoveToColumnRefusesAResolvedColumnState`, `TestMoveToColumnRefusesABundleEditedAfterMapping`, `TestMoveToColumnOnAMovedIssueIsNotFound`, `TestMoveToColumnProjectMismatchSendsNothing`, `TestNewSentinelsRedactTheToken`, `TestListIssuesWithStateReturnsTheStateNames`, `TestStateChecksRunBeforeTheIssueIsRead` and `TestOnlyTheListingAsksForCustomFields` (U1 tests; U2 passes a `ColumnMove` instead of a column string); `issuetrackersvc` `TestSetStateMappingRefusals` (U3 fixes add the zero-width, bidi, filler and variation-selector rows); `store` `TestIssueTrackerStateMappingChecksLiveDB` (U3 fix adds the column-label checks); `handler` `TestConfigureColumnsOnALinkedRepoRefusesBadLabelsLiveDB` (U3 fix adds the filler rows), `TestLinkedBoardMoveWritesTheStateAndCachesTheColumnLiveDB` (U4a fix: labels and position survive a move) and `TestLinkedBoardCloseWithNoMappingIs409LiveDB` (U4 fix: asserts the unset-States message); `forgesvc` `TestLinkedSyncCachesTheColumnOfEachStatesMapping` (U4a fix: a Human Review row); `runlifecycle` `TestLinkedMoveUncachedIssueProbe` (U4a fix: a link with no eligibility tag name keeps the marker). Goldens did not change. +- **2026-11-18: the review rework's tests changed.** `forgetest` `TestRecorderMoveToColumnRecordsTheLabelBackedMove` (`wantMoves` gains ` Reopen:false`, because the Recorder formats every `ColumnMove` field); `forgesvc` `TestReopenIssueOnALinkedRepoIsOneMoveThenTheCache` (asserts `Reopen` is set); `handler` `TestLinkedRepoRunNowResolvesTheIssueTrackerLiveDB` (from the `prd/ws-2` merge: its sweep selector and card labels use the link's eligibility tag, because M5's linked sweep filters on it); bare "tracker" becomes "issue tracker" in the failure messages, comments and error values of `forgesvc` `TestPRDLinkPatchWritesTheDescriptionOnALinkedRepo` and `TestUnavailableIssueTrackerReadsAndWritesFailWithTheCause` (`issue_write_guard_test.go`) and `TestLinkedMRCloseIsRecordedWhenTheIssueSideIsUnavailableLiveDB` and `TestLinkedMRCloseIsDeferredOnATransientRepoSourceReadFailureLiveDB` (`linked_mr_watch_livedb_test.go`). New tests closing spec gaps: `handler` `TestLinkedRepoUntaggedCreateIsNotOnTheBoardAfterTheFullSyncLiveDB` (an issue created without the eligibility tag is not on the board after a full sync); `workersvc` `TestLinkedChatProposalConfirmCreatesInTheLinkedProjectLiveDB` and `TestLinkedScheduledProposalFilesInTheLinkedProjectWithoutTheEligibilityTagLiveDB` (on `youtracktest`, through a local stand-in resolver, because `forgesvc` imports `workersvc`).