diff --git a/Taskfile.yml b/Taskfile.yml index f4537bd457..3fd2c813b7 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -1222,7 +1222,7 @@ tasks: cmds: # Same shape as test:forge-fake-contract: the launcher starts the fake with # FORGE_FAKE_ISSUE_CONTRACT=1 and a self-signed cert, then runs this exact Go - # argv from api. It allows only the moved-issue subtest to skip (the forge subjects supply no MoveIssue seam). + # argv from api. It allows only two subtests to skip on the forge subjects: the moved-issue one (no MoveIssue seam) and SameNameTagUsesEligibilityTag (forge labels have no owners). The forge subjects also run the ProjectGuard case. - env GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null node e2e/forge-fake/issue-tracker-contracts.test.mjs -- go test -buildvcs=false -count=1 -v -timeout=60s -run='^TestIssueTrackerForgeFakeContracts$' ./internal/issuetracker test:create-gitlab-bot: @@ -1685,6 +1685,9 @@ tasks: # ceiling guards, and the controller's single get-only StorageClass ClusterRole), the # same no-workflow-edit wiring as the DB capacity check above. - ./scripts/assert-pvc-expansion-render.sh + # Also runs the issue tracker allowlist render check (PRD #4173 S3: the separate + # ISSUE_TRACKER_ALLOWED_BASE_URLS, never worker egress), same no-workflow-edit wiring. + - ./scripts/assert-issue-tracker-allowlist-render.sh render:pvc-expansion-check: desc: assert the chart renders the PVC expansion surface (DinD data-size env, ceiling guards, the get-only StorageClass ClusterRole) @@ -1702,6 +1705,14 @@ tasks: # render:openshift-check. - ./scripts/assert-db-capacity-render.sh + render:issue-tracker-allowlist-check: + desc: assert the chart renders the api's ISSUE_TRACKER_ALLOWED_BASE_URLS from issueTracker.allowedBaseURLs and keeps it out of worker egress + cmds: + # Standalone, NOT wired into gate:repo/gate: the lint job lacks helm. Offline + # (CNPG subchart stripped), non-vacuous, needs helm + yq. Also run by + # render:openshift-check. + - ./scripts/assert-issue-tracker-allowlist-render.sh + render:drain-knobs-check: desc: assert the controller Deployment renders the drain-deadline + force-roll knobs (PRD 422 M5) cmds: diff --git a/adr/4173-issue-tracker-seam.md b/adr/4173-issue-tracker-seam.md new file mode 100644 index 0000000000..cd3424ff98 --- /dev/null +++ b/adr/4173-issue-tracker-seam.md @@ -0,0 +1,70 @@ +# ADR-4173: The issue tracker seam + +**Status**: Accepted (PRD #4173, M3 landed; M4–M6 pending) +**Date**: 2026-11-11 +**Deciders**: the maintainer + agent team +**PRD**: [prds/4173-external-issue-trackers.md](../prds/4173-external-issue-trackers.md) — the PRD carries the milestones and the Decision Log; this ADR carries the durable shape of what M3 built and the invariants a later change must keep. + +## Decision (summary) + +Issue trackers other than the repo's forge sit behind a `Driver` in their own package, `api/internal/issuetracker`. A driver is `forge.IssueTracker` plus four connection operations. Connections live apart from forge connections (`issuetrackersvc`, table `issue_tracker_connections`) with their own base URL allowlist. The YouTrack driver adapts YouTrack's string ids inside itself, is built for exactly one project, and treats tags as user-owned objects chosen by id. Through M3 no repo can link to an issue tracker, so no behaviour changes for forges. + +Not shipped yet: the repo link, `IssueTrackerForRepo` returning an issue tracker (M4), the board and run behaviour on YouTrack (M5), and the live pass against a real YouTrack with docs (M6). + +## Context + +`forge.IssueTracker` was written for forge issues: `projectID int64`, `issueIID int64`, `IssueNote.ID int64`. YouTrack ids for projects, comments and accounts are strings, and a YouTrack tag is not a project label: it belongs to a user, is private unless shared, and can share its name with another user's tag. The three forge drivers and every caller depend on the neutral types, so the seam had to take YouTrack without changing them. + +## Decisions + +### D6. The package and the `Driver` seam + +- `Driver` is `forge.IssueTracker` plus `VerifyToken`, `ListProjects`, `ListTags` and `ListStates(project)`. It never embeds `forge.CodeHost`, so passing a driver where a code host is required is a compile error (S1). +- A new issue tracker is one `Kind` constant, one entry in the `constructors` map in `registry.go` and one driver. There is no init-time registration. +- The package ships one shared in-memory fake, `issuetrackertest.BaseFake` (loud not-stubbed defaults, for the tests of callers), and each driver ships its HTTP fake (YouTrack: `youtracktest`, an in-process HTTPS server). +- `issuetrackertest.RunContract` is the one contract suite. It runs on the real YouTrack driver against its HTTPS fake, and on the three real forge drivers against `e2e/forge-fake` in contract mode. Passing it on an in-memory fake is not evidence of parity. + +### D8. Separate connections + +- The `issuetrackersvc` package and the `issue_tracker_connections` table hold connections per user, kind and base URL, apart from forge connections. +- The token is sealed with `secretbox` using the user, kind and base URL as AAD. The connection type the service returns has no token or ciphertext field, and there is no reveal endpoint. +- The base URL is https, scheme and host only (`strictBaseURL`). +- The allowlist `ISSUE_TRACKER_ALLOWED_BASE_URLS` is separate from `FORGE_ALLOWED_BASE_URLS` and empty by default, so no connection can be created until an operator sets it. It is wired through the Helm value `issueTracker.allowedBaseURLs` (setting both the value and `api.config.ISSUE_TRACKER_ALLOWED_BASE_URLS` fails the render, and a non-https, comma or whitespace entry is refused) and through a pass-through in `docker-compose.yml`. It never feeds worker egress, because the worker never calls an issue tracker (S5). +- `DriverFor` re-checks the strict URL form and the allowlist against the stored row before opening the token, so removing a host stops use of existing rows. +- Redirects are refused (`ErrRedirectRefused`), response bodies and listings are bounded, and every error passes the redactor. + +### D10. The id adapter + +- The driver is built for one project through `issuetracker.Binding{Project, ProjectID, EligibilityTag}`: all three set or none (`ValidBinding`). A driver built with none serves only the connection operations. +- Every `IssueTracker` method refuses a `projectID` other than the bound one with `ErrProjectMismatch` before any request. This catches a mismatched number only. Two connections can expose the same forge project number, so the owner-scoped repo resolver in M4 must still bind the right repo and connection, and M4 tests that collision. +- `issueIID` is YouTrack's `numberInProject`. Each per-issue call checks the issue reached is in the bound project and carries the number, so a moved issue reads as not found. YouTrack has no conditional write, so a move between that check and the write is not caught. +- `IssueNote.ID` is 0, not derived from YouTrack's string comment id. Account ids are 0, with the login kept as text. +- `ListStates` takes the project it reads, because states belong to a project. +- The nullable `issue_key` column is M4's. + +### D14. The tag rules + +- The eligibility tag is chosen by id. Its name resolves to the stored id whoever owns the tag, for reads and for writes. +- Any other name resolves only to a tag owned by the connection's account, which the driver learns from `users/me`. A missing one is created, owned by that account, with the eligibility tag's sharing settings. The eligibility tag is never created, and a drifted name never resolves to the eligibility id. +- Reads drop a tag that only shares the eligibility name. +- `ListTags` returns the tags the token can see, with owner and read/use sharing settings. `AccountCanApply` and `OthersCanApply` judge the use settings, and the owner counts as covered. A named group counts for others only with `UsersCount >= 2`; the anonymous response recorded from a real YouTrack (`testdata/youtrack_tags_recorded.json`) omits `usersCount`, so on such a response only the All Users group counts. This fails closed. Using these to refuse a link is M4. + +## Invariants a future change must keep + +- **S1. An issue tracker never gains code access.** `Driver` embeds `forge.IssueTracker`, not `forge.CodeHost`; a driver passed as a code host does not compile. +- **S2. Code-host checks stay on the forge.** The privilege check, the protected-branch check and the worker-held forge credential act on the repo's forge connection and fail closed. Nothing in M3 touches them, and no repo is linked yet to test the claim further. +- **S3. Credentials follow the forge rules.** Sealed token with AAD and no reveal path, https-only base URL on the separate empty-by-default allowlist re-checked in `DriverFor`, redirects refused, errors redacted. The redirect, oversized-body and token-in-error cases each have a test in `api/internal/issuetracker`. +- **S4. Issue tracker text is untrusted input.** The driver bounds what it reads (response bytes, listing pages and items, nested items, label name length). Fencing of titles, descriptions and comments is done by the existing callers once a repo can link in M4; it is not exercised on YouTrack text yet. +- **S5. The worker never holds an issue tracker token.** The allowlist does not feed worker egress, and the token is opened only inside `DriverFor` in the API. The worker read routes are M4. + +## Open questions for M6's live pass + +- Whether YouTrack accepts `readSharingSettings` and `tagSharingSettings` in the tag-create body. The fake accepts them; no real server has been asked. +- Whether an authenticated token receives `usersCount` on a tag's sharing groups (the anonymous recording has none). +- Whether the server query `tag: {name}` resolves every tag of that name, including ones owned by others. The client side filters by id, so a wrong answer here would drop issues, not mis-tag them. + +## Consequences + +- Forge drivers, the neutral types and every caller are unchanged. A second tracker (Asana) is the real test of the seam: YouTrack specifics (the id adapter, the tag rules) must stay inside its driver. +- Operators get one new empty setting. Until it is set no issue tracker connection can be created, and uzi boots as before. +- The `projectID` guard is one half of a binding. M4 owns the other half, the resolver. diff --git a/api/internal/issuetracker/driver.go b/api/internal/issuetracker/driver.go index 0356fa1bc7..dee371ff44 100644 --- a/api/internal/issuetracker/driver.go +++ b/api/internal/issuetracker/driver.go @@ -31,13 +31,16 @@ type State struct { // Driver is an external issue tracker connection. It embeds forge.IssueTracker // only, never forge.CodeHost (PRD #4173 S1). The extra methods serve the -// connection flow: verify the token, then discover projects and states. +// connection flow: verify the token, then discover projects, tags and states. type Driver interface { forge.IssueTracker // VerifyToken confirms the token works and returns the account it belongs to. VerifyToken(ctx context.Context) (Account, error) // ListProjects returns the projects the token can see. ListProjects(ctx context.Context) ([]Project, error) + // ListTags returns the tags the token can see, with their owners and + // sharing settings. + ListTags(ctx context.Context) ([]Tag, error) // ListStates returns the issue states available in a project. ListStates(ctx context.Context, projectID string) ([]State, error) } diff --git a/api/internal/issuetracker/errors.go b/api/internal/issuetracker/errors.go index e5fc387f12..33e5a75da5 100644 --- a/api/internal/issuetracker/errors.go +++ b/api/internal/issuetracker/errors.go @@ -8,6 +8,9 @@ var ( // ErrProjectNotBound is returned by an IssueTracker method on a driver built // with no bound project (Options.Project empty). ErrProjectNotBound = errors.New("issuetracker: project not bound to this connection") + // ErrProjectMismatch is returned by an IssueTracker method called with a + // project id other than the one the driver is bound to. + ErrProjectMismatch = errors.New("issuetracker: project id does not match the one this connection is bound to") // ErrStateMappingUnset is returned when open/closed state mapping is needed // but the connection has not configured it. ErrStateMappingUnset = errors.New("issuetracker: state mapping not configured") @@ -20,4 +23,4 @@ var ( // packageSentinels lists every sentinel above. redactor.sentinel keeps only // these in an error chain, so a new sentinel must be added here too. -var packageSentinels = []error{ErrUnsupportedKind, ErrProjectNotBound, ErrStateMappingUnset, ErrInvalidLabelName, ErrRedirectRefused} +var packageSentinels = []error{ErrUnsupportedKind, ErrProjectNotBound, ErrProjectMismatch, ErrStateMappingUnset, ErrInvalidLabelName, ErrRedirectRefused} diff --git a/api/internal/issuetracker/forgefake_contract_test.go b/api/internal/issuetracker/forgefake_contract_test.go index e4fb9b8b60..dc2e0e987c 100644 --- a/api/internal/issuetracker/forgefake_contract_test.go +++ b/api/internal/issuetracker/forgefake_contract_test.go @@ -17,6 +17,11 @@ import ( // forgeFakeProjectID is the project id of the Node fake's first project. const forgeFakeProjectID = 1 +// forgeFakeOtherProjectID is a project id the Node fake does not have. Under +// FORGE_FAKE_ISSUE_CONTRACT=1 the fake answers 404 for it instead of falling +// back to its first project, which is what the ProjectGuard case relies on. +const forgeFakeOtherProjectID = 4242 + // TestIssueTrackerForgeFakeContracts runs the shared issue tracker contract // against the real GitLab, GitHub and Forgejo drivers over the Node HTTPS fake // (e2e/forge-fake/forge-fake.mjs). The launcher @@ -44,8 +49,9 @@ func TestIssueTrackerForgeFakeContracts(t *testing.T) { t.Fatalf("forge.New(%s): %v", dialect, err) } return issuetrackertest.Subject{ - IssueTracker: driver, - ProjectID: forgeFakeProjectID, + IssueTracker: driver, + ProjectID: forgeFakeProjectID, + OtherProjectID: forgeFakeOtherProjectID, SeedHumanComment: func(t *testing.T, iid int64, username, body string) { postFake(t, endpoint, fmt.Sprintf("/_e2e/issues/%d/comments", iid), map[string]any{"username": username, "body": body}) diff --git a/api/internal/issuetracker/issuetrackertest/basefake.go b/api/internal/issuetracker/issuetrackertest/basefake.go index c22d8f3fd7..90edee40f8 100644 --- a/api/internal/issuetracker/issuetrackertest/basefake.go +++ b/api/internal/issuetracker/issuetrackertest/basefake.go @@ -90,6 +90,11 @@ func (*BaseFake) ListProjects(context.Context) ([]issuetracker.Project, error) { return nil, notStubbed("ListProjects") } +// ListTags implements issuetracker.Driver. +func (*BaseFake) ListTags(context.Context) ([]issuetracker.Tag, error) { + return nil, notStubbed("ListTags") +} + // ListStates implements issuetracker.Driver. func (*BaseFake) ListStates(context.Context, string) ([]issuetracker.State, error) { return nil, notStubbed("ListStates") diff --git a/api/internal/issuetracker/issuetrackertest/basefake_test.go b/api/internal/issuetracker/issuetrackertest/basefake_test.go index 4b9a0a3659..80867571f4 100644 --- a/api/internal/issuetracker/issuetrackertest/basefake_test.go +++ b/api/internal/issuetracker/issuetrackertest/basefake_test.go @@ -33,10 +33,11 @@ func TestBaseFakeMethodsNotStubbed(t *testing.T) { {"CreateIssueNote", func(b *BaseFake) error { _, err := b.CreateIssueNote(ctx, 1, 2, "b"); return err }}, {"VerifyToken", func(b *BaseFake) error { _, err := b.VerifyToken(ctx); return err }}, {"ListProjects", func(b *BaseFake) error { _, err := b.ListProjects(ctx); return err }}, + {"ListTags", func(b *BaseFake) error { _, err := b.ListTags(ctx); return err }}, {"ListStates", func(b *BaseFake) error { _, err := b.ListStates(ctx, "p"); return err }}, } - if len(methods) != 14 { - t.Fatalf("expected 14 methods, got %d", len(methods)) + if len(methods) != 15 { + t.Fatalf("expected 15 methods, got %d", len(methods)) } b := &BaseFake{} for _, m := range methods { diff --git a/api/internal/issuetracker/issuetrackertest/contract.go b/api/internal/issuetracker/issuetrackertest/contract.go index abdc47c590..2c6a182c0a 100644 --- a/api/internal/issuetracker/issuetrackertest/contract.go +++ b/api/internal/issuetracker/issuetrackertest/contract.go @@ -29,6 +29,48 @@ type Subject struct { // MoveIssue moves issue iid to another project. Nil means issues do not // move on this implementation. MoveIssue func(t *testing.T, iid int64) + // OtherProjectID is a project id the subject is not bound to; the + // ProjectGuard case calls every method with it. Required. + OtherProjectID int64 + // WantProjectGuardErr, when set, is a sentinel every guarded call must + // match with errors.Is; nil only requires some error. + WantProjectGuardErr error + // NoteIDUnset says CreateIssueNote returns ID 0, because the issue tracker's + // comment ids have no numeric form. When false the case requires a non-zero ID. + NoteIDUnset bool + // SameName carries the seams of the SameNameTagUsesEligibilityTag case. Nil + // means labels on this implementation have no owners, so the case is skipped. + SameName *SameNameSeams +} + +// SameNameSeams put server-side state in a subject whose labels are tags with +// owners, and read it back. The subject is bound to an eligibility tag owned by +// another user. Besides it, the connection's own account owns a tag with the +// same name. +type SameNameSeams struct { + // EligibilityName is the eligibility tag's label name. + EligibilityName string + // EligibilityTagID is the id of the eligibility tag the subject is bound to. + EligibilityTagID string + // SameNameTagID is the id of the account's own tag that has the eligibility name. + SameNameTagID string + // SeedIssue creates an issue carrying the tags with the given ids, bypassing + // the IssueTracker under test, and returns its number. + SeedIssue func(t *testing.T, tagIDs ...string) int64 + // IssueTagIDs returns the ids of the tags on issue iid, server side. + IssueTagIDs func(t *testing.T, iid int64) []string + // TagsNamed returns every tag with the name, server side. + TagsNamed func(t *testing.T, name string) []TagView +} + +// TagView is one server-side tag, as far as the SameName case judges it. +type TagView struct { + ID string + // OwnedByAccount is true when the connection's own account owns the tag. + OwnedByAccount bool + // SharedLikeEligibility is true when the tag's read and tag sharing settings + // equal the eligibility tag's. + SharedLikeEligibility bool } // uniqueSeq makes every name the suite creates unique within the process, so @@ -57,6 +99,8 @@ func RunContract(t *testing.T, newSubject func(t *testing.T) Subject) { {"LabelEventsOldestFirst", caseLabelEvents}, {"MissingIssueIsNotFound", caseMissing}, {"MovedIssueIsRefused", caseMoved}, + {"ProjectGuard", caseProjectGuard}, + {"SameNameTagUsesEligibilityTag", caseSameNameTag}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { @@ -261,8 +305,14 @@ func caseComments(t *testing.T, c *contract) { if err != nil { t.Fatalf("CreateIssueNote: %v", err) } - if created.ID == 0 || created.Body != note { - t.Errorf("CreateIssueNote = %+v, want a non-zero ID and body %q", created, note) + if created.Body != note { + t.Errorf("CreateIssueNote body = %q, want %q", created.Body, note) + } + if c.NoteIDUnset && created.ID != 0 { + t.Errorf("CreateIssueNote ID = %d, want 0 (no numeric comment id)", created.ID) + } + if !c.NoteIDUnset && created.ID == 0 { + t.Errorf("CreateIssueNote ID = 0, want a non-zero ID") } comments, err := c.IssueTracker.ListIssueComments(c.ctx, c.ProjectID, issue.IID) if err != nil { @@ -364,3 +414,113 @@ func caseMoved(t *testing.T, c *contract) { } } } + +// caseProjectGuard calls every method with a project id the subject is not bound +// to. Each must fail, and none may have changed the issue made under the right id. +func caseProjectGuard(t *testing.T, c *contract) { + if c.OtherProjectID == 0 || c.OtherProjectID == c.ProjectID { + t.Fatalf("Subject.OtherProjectID = %d must differ from ProjectID %d and be set", c.OtherProjectID, c.ProjectID) + } + label := unique("contract-label") + c.ensure(t, label) + issue := c.create(t, "guarded", label) + it, pid, iid := c.IssueTracker, c.OtherProjectID, issue.IID + calls := map[string]func() error{ + "ListLabels": func() error { _, err := it.ListLabels(c.ctx, pid); return err }, + "EnsureLabels": func() error { return it.EnsureLabels(c.ctx, pid, []forge.Label{{Name: unique("contract-label")}}) }, + "ListIssues": func() error { _, err := it.ListIssues(c.ctx, pid, forge.ListIssuesOptions{}); return err }, + "GetIssue": func() error { _, err := it.GetIssue(c.ctx, pid, iid); return err }, + "CreateIssue": func() error { + _, err := it.CreateIssue(c.ctx, pid, unique("contract guarded"), "x", []string{label}) + return err + }, + "UpdateIssueLabels": func() error { return it.UpdateIssueLabels(c.ctx, pid, iid, nil, []string{label}) }, + "UpdateIssueDescription": func() error { return it.UpdateIssueDescription(c.ctx, pid, iid, "changed") }, + "SetIssueState": func() error { return it.SetIssueState(c.ctx, pid, iid, forge.StateClosed) }, + "ListIssueLabelEvents": func() error { _, err := it.ListIssueLabelEvents(c.ctx, pid, iid); return err }, + "ListIssueComments": func() error { _, err := it.ListIssueComments(c.ctx, pid, iid); return err }, + "CreateIssueNote": func() error { _, err := it.CreateIssueNote(c.ctx, pid, iid, "x"); return err }, + } + if len(calls) != 11 { + t.Fatalf("table covers %d methods, want 11", len(calls)) + } + for name, call := range calls { + err := call() + if err == nil { + t.Errorf("%s with another project id succeeded", name) + } else if c.WantProjectGuardErr != nil && !errors.Is(err, c.WantProjectGuardErr) { + t.Errorf("%s with another project id = %v, want %v", name, err, c.WantProjectGuardErr) + } + } + got := c.get(t, iid) + if got.Description != "guarded" || got.State != string(forge.StateOpened) || !sameSet(got.Labels, []string{label}) { + t.Errorf("a guarded call changed the issue: %+v", got) + } + comments, err := it.ListIssueComments(c.ctx, c.ProjectID, iid) + if err != nil { + t.Fatalf("ListIssueComments: %v", err) + } + if len(comments) != 0 { + t.Errorf("a guarded call left comments: %+v", comments) + } +} + +// caseSameNameTag shows that the eligibility label name always means the +// eligibility tag, though the connection's account owns another tag of that +// name, and that other names only ever mean tags the account owns. +func caseSameNameTag(t *testing.T, c *contract) { + s := c.SameName + if s == nil { + t.Skip("labels have no owners on this implementation") + } + elig := s.EligibilityName + fresh := unique("contract-label") + + // EnsureLabels: no eligibility tag is created, the account's same-named tag + // is left alone, and a new tag is the account's, shared like the eligibility tag. + c.ensure(t, elig, fresh) + c.ensure(t, elig, fresh) + var ids []string + for _, tv := range s.TagsNamed(t, elig) { + ids = append(ids, tv.ID) + } + if !sameSet(ids, []string{s.EligibilityTagID, s.SameNameTagID}) { + t.Errorf("tags named %q = %v after EnsureLabels, want only the eligibility tag and the account's own", elig, ids) + } + made := s.TagsNamed(t, fresh) + if len(made) != 1 || !made[0].OwnedByAccount || !made[0].SharedLikeEligibility { + t.Errorf("tags named %q = %+v, want one owned by the account and shared like the eligibility tag", fresh, made) + } + + // CreateIssue attaches the eligibility tag, not the account's tag. + created := c.create(t, "eligible", elig) + if got := s.IssueTagIDs(t, created.IID); !slices.Equal(got, []string{s.EligibilityTagID}) { + t.Errorf("created issue carries tags %v, want only the eligibility tag %s", got, s.EligibilityTagID) + } + if got := c.get(t, created.IID).Labels; !slices.Equal(got, []string{elig}) { + t.Errorf("created issue labels = %v, want [%s]", got, elig) + } + + // ListIssues by the eligibility name finds the eligibility issue and not + // one that carries only the same-named tag; GetIssue of that one does not + // report the eligibility name. + lookalike := s.SeedIssue(t, s.SameNameTagID) + wantIIDs(t, c.listIIDs(t, forge.ListIssuesOptions{Labels: []string{elig}}), created.IID) + if got := c.get(t, lookalike).Labels; slices.Contains(got, elig) { + t.Errorf("an issue with only a same-named tag reports labels %v", got) + } + + // UpdateIssueLabels touches only the eligibility id. + if err := c.IssueTracker.UpdateIssueLabels(c.ctx, c.ProjectID, lookalike, []string{elig}, nil); err != nil { + t.Fatalf("add: %v", err) + } + if got := s.IssueTagIDs(t, lookalike); !sameSet(got, []string{s.SameNameTagID, s.EligibilityTagID}) { + t.Errorf("after add the issue carries %v, want the same-named tag and the eligibility tag", got) + } + if err := c.IssueTracker.UpdateIssueLabels(c.ctx, c.ProjectID, lookalike, nil, []string{elig}); err != nil { + t.Fatalf("remove: %v", err) + } + if got := s.IssueTagIDs(t, lookalike); !slices.Equal(got, []string{s.SameNameTagID}) { + t.Errorf("after remove the issue carries %v, want only the same-named tag", got) + } +} diff --git a/api/internal/issuetracker/options.go b/api/internal/issuetracker/options.go index 57dbc27604..75013c9010 100644 --- a/api/internal/issuetracker/options.go +++ b/api/internal/issuetracker/options.go @@ -8,6 +8,22 @@ type ProjectRef struct { ShortName string } +// TagRef identifies a tag by its server-side id and its name. +type TagRef struct { + ID string + Name string +} + +// Binding is what ties a driver to one repository's project: the issue +// tracker project, the repository's forge project id the driver answers to, and +// the eligibility tag. Either all three are set or none is; a connection-level +// driver has none (see ValidBinding). +type Binding struct { + Project ProjectRef + ProjectID int64 + EligibilityTag TagRef +} + // Options configures a driver. There is deliberately no TLS field: a driver // trusts the system roots and nothing else (PRD #4173 S3). type Options struct { @@ -17,6 +33,12 @@ type Options struct { Token string // Project is the project the connection is bound to. Project ProjectRef + // ProjectID is the repository's forge project id. Every forge.IssueTracker + // method refuses a different id with ErrProjectMismatch (PRD #4173 D10). + ProjectID int64 + // EligibilityTag is the tag that marks an issue as eligible. Its label name + // always resolves to this id, whoever owns the tag (PRD #4173 D14). + EligibilityTag TagRef // StateField, OpenState and ClosedState map neutral open/closed onto the // issue tracker's own state field and values. StateField string diff --git a/api/internal/issuetracker/tag.go b/api/internal/issuetracker/tag.go new file mode 100644 index 0000000000..8c4f1bc1d2 --- /dev/null +++ b/api/internal/issuetracker/tag.go @@ -0,0 +1,84 @@ +package issuetracker + +// Tag is a YouTrack tag as ListTags reports it: only tags the token can see +// come back, so a Tag in hand is one the account can see. +type Tag struct { + ID string + Name string + Owner Account + // Read says who can see the tag; Use says who can apply it to an issue. + Read SharingSettings + Use SharingSettings +} + +// SharingSettings is who a tag is shared with: user groups and single users. +type SharingSettings struct { + Groups []Group + Users []Account +} + +// Group is a YouTrack user group. UsersCount is 0 when the response did not +// carry a count, which the rules below read as "not known to have members". +type Group struct { + ID string + Name string + AllUsers bool // the built-in group of every user + UsersCount int +} + +// AccountCanApply reports whether account a can apply tag t to an issue. It is +// true when a owns the tag, is named in the tag's use settings, or a use group +// covers all users. A tag's settings do not list the members of a named group, +// so membership in one cannot be read from them: a named group alone does not +// count, and the answer fails closed. +// +// The tag must come from ListTags (or a read of the tag by id), which return +// only tags the token can see. This function judges the apply half only; it does +// not know whether a can see t. +func AccountCanApply(t Tag, a Account) bool { + if a.ID == "" { + return false + } + if t.Owner.ID == a.ID { + return true + } + for _, u := range t.Use.Users { + if u.ID == a.ID { + return true + } + } + for _, g := range t.Use.Groups { + if g.AllUsers { + return true + } + } + return false +} + +// OthersCanApply reports whether someone other than account a can apply tag t. +// It is true when the tag has another owner (an owner can always apply their own +// tag), when a user other than a is named in the use settings, or when a use +// group covers all users or has at least two members. A named group of one +// member does not count, because that member may be a. An unknown owner or +// account never counts, so the answer fails closed. +// +// The tag must come from ListTags, as for AccountCanApply. +func OthersCanApply(t Tag, a Account) bool { + if a.ID == "" { + return false + } + if t.Owner.ID != "" && t.Owner.ID != a.ID { + return true + } + for _, u := range t.Use.Users { + if u.ID != "" && u.ID != a.ID { + return true + } + } + for _, g := range t.Use.Groups { + if g.AllUsers || g.UsersCount >= 2 { + return true + } + } + return false +} diff --git a/api/internal/issuetracker/tag_test.go b/api/internal/issuetracker/tag_test.go new file mode 100644 index 0000000000..7ad33a1075 --- /dev/null +++ b/api/internal/issuetracker/tag_test.go @@ -0,0 +1,269 @@ +package issuetracker + +import ( + "context" + "net/http" + "net/http/httptest" + "os" + "strings" + "testing" + + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" +) + +func acct(id string) Account { return Account{ID: id, Login: "login-" + id} } + +func use(groups []Group, users ...string) SharingSettings { + s := SharingSettings{Groups: groups} + for _, id := range users { + s.Users = append(s.Users, acct(id)) + } + return s +} + +var ( + allUsers = Group{ID: "675-0", Name: "All Users", AllUsers: true} + team2 = Group{ID: "673-1", Name: "Team", UsersCount: 2} + solo = Group{ID: "673-2", Name: "Solo", UsersCount: 1} + noCount = Group{ID: "673-3", Name: "Unknown size"} +) + +func TestAccountCanApply(t *testing.T) { + me := acct("1-1") + for _, tc := range []struct { + name string + tag Tag + who Account + want bool + }{ + {"owner", Tag{Owner: me}, me, true}, + {"named in use users", Tag{Owner: acct("1-2"), Use: use(nil, "1-3", "1-1")}, me, true}, + {"all users group", Tag{Owner: acct("1-2"), Use: use([]Group{allUsers})}, me, true}, + {"named group alone does not count", Tag{Owner: acct("1-2"), Use: use([]Group{team2})}, me, false}, + {"named group of one does not count", Tag{Owner: acct("1-2"), Use: use([]Group{solo})}, me, false}, + {"someone else only", Tag{Owner: acct("1-2"), Use: use(nil, "1-3")}, me, false}, + {"read settings do not grant use", Tag{Owner: acct("1-2"), Read: use([]Group{allUsers}, "1-1")}, me, false}, + {"unshared", Tag{Owner: acct("1-2")}, me, false}, + {"no account never matches an unknown owner", Tag{}, Account{}, false}, + {"no account never matches an empty user", Tag{Owner: acct("1-2"), Use: SharingSettings{Users: []Account{{}}}}, Account{}, false}, + } { + t.Run(tc.name, func(t *testing.T) { + if got := AccountCanApply(tc.tag, tc.who); got != tc.want { + t.Errorf("AccountCanApply = %v, want %v", got, tc.want) + } + }) + } +} + +func TestOthersCanApply(t *testing.T) { + me := acct("1-1") + for _, tc := range []struct { + name string + tag Tag + want bool + }{ + {"owned by another user", Tag{Owner: acct("1-2")}, true}, + {"owned by me and unshared", Tag{Owner: me}, false}, + {"owned by me, shared with another user", Tag{Owner: me, Use: use(nil, "1-2")}, true}, + {"owned by me, use users are only me", Tag{Owner: me, Use: use(nil, "1-1")}, false}, + {"owned by me, shared with all users", Tag{Owner: me, Use: use([]Group{allUsers})}, true}, + {"owned by me, group of two", Tag{Owner: me, Use: use([]Group{team2})}, true}, + {"owned by me, group of one", Tag{Owner: me, Use: use([]Group{solo})}, false}, + {"owned by me, group with no count", Tag{Owner: me, Use: use([]Group{noCount})}, false}, + {"owned by me, only read settings", Tag{Owner: me, Read: use([]Group{allUsers}, "1-2")}, false}, + {"unknown owner is not another user", Tag{}, false}, + {"empty user entry is not another user", Tag{Owner: me, Use: SharingSettings{Users: []Account{{}}}}, false}, + } { + t.Run(tc.name, func(t *testing.T) { + if got := OthersCanApply(tc.tag, me); got != tc.want { + t.Errorf("OthersCanApply = %v, want %v", got, tc.want) + } + }) + } + if OthersCanApply(Tag{Owner: acct("1-2")}, Account{}) { + t.Error("OthersCanApply with an unknown account must fail closed") + } +} + +// unboundDriver builds a driver with no project over a raw handler. +func unboundDriver(t *testing.T, h http.HandlerFunc) (*youTrack, *requestLog) { + t.Helper() + log := &requestLog{} + srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + log.record(r) + h(w, r) + })) + t.Cleanup(srv.Close) + y, err := newYouTrackOver(Options{BaseURL: srv.URL, Token: credential()}, srv.Client().Transport) + if err != nil { + t.Fatal(err) + } + return y, log +} + +// TestListTagsParsesARecordedResponse parses testdata/youtrack_tags_recorded.json +// through the driver's real decoding path. The file is a response of GET +// /api/tags with the fields ListTags asks for, recorded anonymously from the +// public YouTrack instance at youtrack.jetbrains.com. Every login, tag name +// and user or tag id in it is a placeholder; the structure, the $type members, +// the booleans and the absence of usersCount (the server did not return one) are +// as recorded. +func TestListTagsParsesARecordedResponse(t *testing.T) { + raw, err := os.ReadFile("testdata/youtrack_tags_recorded.json") + if err != nil { + t.Fatal(err) + } + y, log := unboundDriver(t, func(w http.ResponseWriter, _ *http.Request) { writeRaw(w, string(raw)) }) + tags, err := y.ListTags(context.Background()) + if err != nil { + t.Fatal(err) + } + if got := log.all(); len(got) != 1 || got[0] != "GET /api/tags" { + t.Errorf("requests = %v, want one GET /api/tags", got) + } + if len(tags) != 5 { + t.Fatalf("got %d tags, want 5", len(tags)) + } + first := tags[0] + if first.ID != "68-1001" || first.Name != "tag-1" || first.Owner != (Account{ID: "11-1001", Login: "user-1"}) { + t.Errorf("first tag = %+v", first) + } + if len(first.Read.Groups) != 1 || first.Read.Groups[0] != (Group{ID: "675-0", Name: "All Users", AllUsers: true}) { + t.Errorf("read groups = %+v, want the All Users group", first.Read.Groups) + } + if len(first.Read.Users) != 2 || len(first.Use.Users) != 2 || len(first.Use.Groups) != 0 { + t.Errorf("sharing = read %+v, use %+v", first.Read, first.Use) + } + // One tag is also readable by a named group, which is not an all-users group. + var named *Group + for _, tg := range tags { + for i, g := range tg.Read.Groups { + if !g.AllUsers { + named = &tg.Read.Groups[i] + } + } + } + if named == nil || named.ID == "" || named.Name == "" || named.UsersCount != 0 { + t.Errorf("named group = %+v, want one with a name and no count", named) + } + // The rules read the parsed settings: a user named in the use settings can + // apply the tag, a stranger cannot, and nobody is judged from the read settings. + if !AccountCanApply(first, first.Use.Users[0]) || AccountCanApply(first, acct("11-9999")) { + t.Error("AccountCanApply on the recorded tag does not follow its use settings") + } + if !OthersCanApply(first, first.Owner) { + t.Error("OthersCanApply on the recorded tag: its use users are not its owner") + } +} + +// TestListTagsParsesARecordedPrivateInstanceResponse parses +// testdata/youtrack_tags_recorded_private.json, a response of the same request +// recorded with a permanent token on a private YouTrack instance. Every login, +// tag name, id and the member count in it is a placeholder; the structure, the +// $type members and the booleans are as recorded. Unlike the public recording it +// carries usersCount, an all-users group in the use settings, a tag only its +// owner can apply and a tag only its owner can see. +func TestListTagsParsesARecordedPrivateInstanceResponse(t *testing.T) { + raw, err := os.ReadFile("testdata/youtrack_tags_recorded_private.json") + if err != nil { + t.Fatal(err) + } + y, _ := unboundDriver(t, func(w http.ResponseWriter, _ *http.Request) { writeRaw(w, string(raw)) }) + tags, err := y.ListTags(context.Background()) + if err != nil { + t.Fatal(err) + } + if len(tags) != 3 { + t.Fatalf("got %d tags, want 3", len(tags)) + } + shared, ownerOnly, private := tags[0], tags[1], tags[2] + stranger := acct("25-9999") + + allUsers := Group{ID: "579-1", Name: "All Users", AllUsers: true, UsersCount: 12} + if len(shared.Use.Groups) != 1 || shared.Use.Groups[0] != allUsers || len(shared.Read.Groups) != 1 || shared.Read.Groups[0] != allUsers { + t.Errorf("shared tag sharing = read %+v, use %+v, want the All Users group with its count in both", shared.Read, shared.Use) + } + if !AccountCanApply(shared, stranger) || !OthersCanApply(shared, shared.Owner) { + t.Error("a tag whose use settings cover all users must be applicable by any account and by others") + } + + if len(ownerOnly.Read.Groups) != 1 || !ownerOnly.Read.Groups[0].AllUsers || len(ownerOnly.Use.Groups) != 0 || len(ownerOnly.Use.Users) != 0 { + t.Errorf("owner-only tag sharing = read %+v, use %+v, want visible to all and usable by nobody listed", ownerOnly.Read, ownerOnly.Use) + } + if AccountCanApply(ownerOnly, stranger) || !AccountCanApply(ownerOnly, ownerOnly.Owner) || OthersCanApply(ownerOnly, ownerOnly.Owner) { + t.Error("a tag everyone can see but only its owner can apply must not count as applicable by anyone else") + } + + if len(private.Read.Groups) != 0 || len(private.Read.Users) != 0 || len(private.Use.Groups) != 0 || len(private.Use.Users) != 0 { + t.Errorf("private tag sharing = read %+v, use %+v, want empty", private.Read, private.Use) + } + if AccountCanApply(private, stranger) || OthersCanApply(private, private.Owner) { + t.Error("a private tag must not count as applicable by anyone but its owner") + } +} + +func TestListTagsNeedsNoProject(t *testing.T) { + srv := youtracktest.NewServer(t) + own := srv.SeedOwnTag("mine") + unbound, err := NewYouTrackOver(Options{BaseURL: srv.URL, Token: srv.Token}, srv.Client().Transport) + if err != nil { + t.Fatal(err) + } + tags, err := unbound.ListTags(context.Background()) + if err != nil { + t.Fatal(err) + } + byID := map[string]Tag{} + for _, tg := range tags { + byID[tg.ID] = tg + } + elig := byID[fakeEligibility.ID] + if elig.Name != fakeEligibility.Name || elig.Owner.Login != "maintainer" || len(elig.Use.Groups) != 1 || !elig.Use.Groups[0].AllUsers || elig.Use.Groups[0].UsersCount != 2 { + t.Errorf("eligibility tag = %+v", elig) + } + if mine := byID[own]; mine.Owner.ID != "1-1" || len(mine.Read.Groups)+len(mine.Use.Groups)+len(mine.Read.Users) != 0 { + t.Errorf("own tag = %+v", mine) + } + // VerifyToken and ListProjects work on the unbound driver too. + if _, err := unbound.VerifyToken(context.Background()); err != nil { + t.Error(err) + } + if _, err := unbound.ListProjects(context.Background()); err != nil { + t.Error(err) + } +} + +func TestListTagsRefusesMalformedIDsAndOversizedSharing(t *testing.T) { + user := func(id string) string { return `{"id":"` + id + `","login":"u"}` } + group := func(id string) string { return `{"id":"` + id + `","name":"g","allUsersGroup":false,"usersCount":3}` } + tag := func(id, owner, readGroups, readUsers string) string { + return `[{"id":"` + id + `","name":"t","owner":` + owner + `,"readSharingSettings":{"permittedGroups":[` + readGroups + + `],"permittedUsers":[` + readUsers + `]},"tagSharingSettings":{"permittedGroups":[],"permittedUsers":[]}}]` + } + for name, body := range map[string]string{ + "tag id": tag("..", user("1-1"), "", ""), + "owner id": tag("5-1", user("../x"), "", ""), + "group id": tag("5-1", user("1-1"), group("a/b"), ""), + "sharer id": tag("5-1", user("1-1"), "", user("")), + "too many": tag("5-1", user("1-1"), strings.TrimSuffix(strings.Repeat(group("673-1")+",", maxNestedItems+1), ","), ""), + "not objects": `[{"id":"5-1","name":"t","readSharingSettings":{"permittedUsers":[1]}}]`, + } { + t.Run(name, func(t *testing.T) { + y, _ := unboundDriver(t, func(w http.ResponseWriter, _ *http.Request) { writeRaw(w, body) }) + if got, err := y.ListTags(context.Background()); err == nil { + t.Errorf("ListTags accepted %s: %+v", name, got) + } + }) + } +} + +func TestListTagsNestedSharingCountsTowardTheItemCeiling(t *testing.T) { + lowerMaxItems(t, 5) + // Two tags with three sharers each: two items, eight counting the sharers. + tag := `{"id":"5-1","name":"t","owner":{"id":"1-1","login":"u"},"readSharingSettings":{"permittedGroups":[],"permittedUsers":[` + + `{"id":"1-2","login":"a"},{"id":"1-3","login":"b"},{"id":"1-4","login":"c"}]}}` + y, _ := unboundDriver(t, func(w http.ResponseWriter, _ *http.Request) { writeRaw(w, "["+tag+","+tag+"]") }) + if _, err := y.ListTags(context.Background()); err == nil { + t.Fatal("nested sharing entries did not count toward the item ceiling") + } +} diff --git a/api/internal/issuetracker/testdata/youtrack_tags_recorded.json b/api/internal/issuetracker/testdata/youtrack_tags_recorded.json new file mode 100644 index 0000000000..96c5cf6ee2 --- /dev/null +++ b/api/internal/issuetracker/testdata/youtrack_tags_recorded.json @@ -0,0 +1,263 @@ +[ + { + "$type": "Tag", + "id": "68-1001", + "name": "tag-1", + "owner": { + "$type": "User", + "id": "11-1001", + "login": "user-1" + }, + "readSharingSettings": { + "$type": "WatchFolderSharingSettings", + "permittedGroups": [ + { + "$type": "AllUsersGroup", + "allUsersGroup": true, + "id": "675-0", + "name": "All Users" + } + ], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + }, + { + "$type": "User", + "id": "11-1003", + "login": "user-3" + } + ] + }, + "tagSharingSettings": { + "$type": "TagSharingSettings", + "permittedGroups": [], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + }, + { + "$type": "User", + "id": "11-1003", + "login": "user-3" + } + ] + } + }, + { + "$type": "Tag", + "id": "68-1002", + "name": "tag-2", + "owner": { + "$type": "User", + "id": "11-1004", + "login": "user-4" + }, + "readSharingSettings": { + "$type": "WatchFolderSharingSettings", + "permittedGroups": [ + { + "$type": "AllUsersGroup", + "allUsersGroup": true, + "id": "675-0", + "name": "All Users" + } + ], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + }, + { + "$type": "User", + "id": "11-1003", + "login": "user-3" + } + ] + }, + "tagSharingSettings": { + "$type": "TagSharingSettings", + "permittedGroups": [], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + }, + { + "$type": "User", + "id": "11-1003", + "login": "user-3" + } + ] + } + }, + { + "$type": "Tag", + "id": "68-1003", + "name": "tag-3", + "owner": { + "$type": "User", + "id": "11-1005", + "login": "user-5" + }, + "readSharingSettings": { + "$type": "WatchFolderSharingSettings", + "permittedGroups": [ + { + "$type": "AllUsersGroup", + "allUsersGroup": true, + "id": "675-0", + "name": "All Users" + } + ], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1006", + "login": "user-6" + }, + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + }, + { + "$type": "User", + "id": "11-1007", + "login": "user-7" + }, + { + "$type": "User", + "id": "11-1003", + "login": "user-3" + } + ] + }, + "tagSharingSettings": { + "$type": "TagSharingSettings", + "permittedGroups": [], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1006", + "login": "user-6" + }, + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + }, + { + "$type": "User", + "id": "11-1007", + "login": "user-7" + }, + { + "$type": "User", + "id": "11-1003", + "login": "user-3" + } + ] + } + }, + { + "$type": "Tag", + "id": "68-1004", + "name": "tag-4", + "owner": { + "$type": "User", + "id": "11-1008", + "login": "user-8" + }, + "readSharingSettings": { + "$type": "WatchFolderSharingSettings", + "permittedGroups": [ + { + "$type": "ProjectTeam", + "allUsersGroup": false, + "id": "673-1", + "name": "Named Team 1" + }, + { + "$type": "AllUsersGroup", + "allUsersGroup": true, + "id": "675-0", + "name": "All Users" + } + ], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + } + ] + }, + "tagSharingSettings": { + "$type": "TagSharingSettings", + "permittedGroups": [], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + } + ] + } + }, + { + "$type": "Tag", + "id": "68-1005", + "name": "tag-5", + "owner": { + "$type": "User", + "id": "11-1009", + "login": "user-9" + }, + "readSharingSettings": { + "$type": "WatchFolderSharingSettings", + "permittedGroups": [ + { + "$type": "AllUsersGroup", + "allUsersGroup": true, + "id": "675-0", + "name": "All Users" + } + ], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + }, + { + "$type": "User", + "id": "11-1003", + "login": "user-3" + } + ] + }, + "tagSharingSettings": { + "$type": "TagSharingSettings", + "permittedGroups": [], + "permittedUsers": [ + { + "$type": "User", + "id": "11-1002", + "login": "user-2" + }, + { + "$type": "User", + "id": "11-1003", + "login": "user-3" + } + ] + } + } +] diff --git a/api/internal/issuetracker/testdata/youtrack_tags_recorded_private.json b/api/internal/issuetracker/testdata/youtrack_tags_recorded_private.json new file mode 100644 index 0000000000..622f94ede1 --- /dev/null +++ b/api/internal/issuetracker/testdata/youtrack_tags_recorded_private.json @@ -0,0 +1,86 @@ +[ + { + "tagSharingSettings": { + "permittedGroups": [ + { + "allUsersGroup": true, + "usersCount": 12, + "name": "All Users", + "id": "579-1", + "$type": "AllUsersGroup" + } + ], + "permittedUsers": [], + "$type": "TagSharingSettings" + }, + "owner": { + "login": "user-1", + "id": "25-1", + "$type": "User" + }, + "readSharingSettings": { + "permittedGroups": [ + { + "allUsersGroup": true, + "usersCount": 12, + "name": "All Users", + "id": "579-1", + "$type": "AllUsersGroup" + } + ], + "permittedUsers": [], + "$type": "WatchFolderSharingSettings" + }, + "name": "tag-1", + "id": "6-1", + "$type": "Tag" + }, + { + "tagSharingSettings": { + "permittedGroups": [], + "permittedUsers": [], + "$type": "TagSharingSettings" + }, + "owner": { + "login": "user-1", + "id": "25-1", + "$type": "User" + }, + "readSharingSettings": { + "permittedGroups": [ + { + "allUsersGroup": true, + "usersCount": 12, + "name": "All Users", + "id": "579-1", + "$type": "AllUsersGroup" + } + ], + "permittedUsers": [], + "$type": "WatchFolderSharingSettings" + }, + "name": "tag-2", + "id": "6-2", + "$type": "Tag" + }, + { + "tagSharingSettings": { + "permittedGroups": [], + "permittedUsers": [], + "$type": "TagSharingSettings" + }, + "owner": { + "login": "user-2", + "id": "25-2", + "$type": "User" + }, + "readSharingSettings": { + "permittedGroups": [], + "permittedUsers": [], + "$type": "WatchFolderSharingSettings" + }, + "name": "tag-3", + "id": "6-3", + "$type": "Tag" + } +] diff --git a/api/internal/issuetracker/validate.go b/api/internal/issuetracker/validate.go index 361d71a4b4..b39b531ddc 100644 --- a/api/internal/issuetracker/validate.go +++ b/api/internal/issuetracker/validate.go @@ -2,6 +2,7 @@ package issuetracker import ( "errors" + "slices" "unicode" ) @@ -35,6 +36,38 @@ func InvisibleRune(c rune) bool { // "0-1". func ValidProjectID(id string) bool { return entityIDRE.MatchString(id) } +// ValidTagRef requires a well-formed tag id and a name a label may carry. +func ValidTagRef(ref TagRef) error { + if !entityIDRE.MatchString(ref.ID) { + return errors.New("issuetracker: invalid eligibility tag id") + } + if labelNameProblem(ref.Name) { + return errors.New("issuetracker: invalid eligibility tag name") + } + return nil +} + +// ValidBinding accepts the empty Binding (a connection-level driver) and a +// Binding with all of its parts set and well formed; a partial one is refused, +// so a driver can never answer for a project without knowing its eligibility +// tag. The message never includes the input. +func ValidBinding(b Binding) error { + set := []bool{b.Project != (ProjectRef{}), b.ProjectID != 0, b.EligibilityTag != (TagRef{})} + if !slices.Contains(set, true) { + return nil + } + if slices.Contains(set, false) { + return errors.New("issuetracker: project, project id and eligibility tag must all be set or all be empty") + } + if b.ProjectID < 0 { + return errors.New("issuetracker: invalid project id") + } + if err := ValidProjectRef(b.Project); err != nil { + return err + } + return ValidTagRef(b.EligibilityTag) +} + // ValidProjectRef accepts the empty ProjectRef (no project bound) and // otherwise requires a well-formed id and short name. The message never // includes the input. diff --git a/api/internal/issuetracker/youtrack.go b/api/internal/issuetracker/youtrack.go index aafa89de7e..e1006fb2ae 100644 --- a/api/internal/issuetracker/youtrack.go +++ b/api/internal/issuetracker/youtrack.go @@ -6,6 +6,7 @@ import ( "net/url" "regexp" "strings" + "sync" "unicode" ) @@ -20,18 +21,27 @@ var ( ) // youTrack is the JetBrains YouTrack driver. It is bound to one project when -// built (see ProjectRef): the projectID argument of every forge.IssueTracker -// method is ignored, and a driver with no project bound refuses them with -// ErrProjectNotBound. VerifyToken, ListProjects and ListStates need no binding. +// built (see Binding). Every forge.IssueTracker method checks its projectID +// argument against the bound id before any request: a driver with no project +// bound refuses them with ErrProjectNotBound, and a different id gets +// ErrProjectMismatch. VerifyToken, ListProjects, ListTags and ListStates need +// no binding. type youTrack struct { baseURL string // scheme and host, no trailing slash token string project ProjectRef + projectID int64 // the forge project id the driver answers to + eligibility TagRef // the eligibility tag; its name always resolves to its id stateField string openState string closedState string client *http.Client red redactor + + // accountMu guards account, the token's own account. It is read from + // users/me the first time tag ownership matters, then kept (see connAccount). + accountMu sync.Mutex + account *Account } var _ Driver = (*youTrack)(nil) @@ -51,7 +61,7 @@ func buildYouTrack(opts Options, base http.RoundTripper) (*youTrack, error) { if err := ValidateToken(opts.Token); err != nil { return nil, err } - if err := ValidProjectRef(opts.Project); err != nil { + if err := ValidBinding(Binding{Project: opts.Project, ProjectID: opts.ProjectID, EligibilityTag: opts.EligibilityTag}); err != nil { return nil, err } stateField := opts.StateField @@ -62,6 +72,8 @@ func buildYouTrack(opts Options, base http.RoundTripper) (*youTrack, error) { baseURL: baseURL, token: opts.Token, project: opts.Project, + projectID: opts.ProjectID, + eligibility: opts.EligibilityTag, stateField: stateField, openState: opts.OpenState, closedState: opts.ClosedState, @@ -105,11 +117,16 @@ func strictBaseURL(raw string) (string, error) { return "https://" + u.Host, nil } -// requireProject returns the bound project or the ErrProjectNotBound sentinel. -func (y *youTrack) requireProject() (ProjectRef, error) { +// requireProject is the guard every forge.IssueTracker method starts with, +// before any request: it returns the bound project, ErrProjectNotBound for an +// unbound driver, or ErrProjectMismatch when projectID is not the bound one. +func (y *youTrack) requireProject(projectID int64) (ProjectRef, error) { if y.project == (ProjectRef{}) { return ProjectRef{}, y.red.sentinel(ErrProjectNotBound, 0, "") } + if projectID != y.projectID { + return ProjectRef{}, y.red.sentinel(ErrProjectMismatch, 0, "") + } return y.project, nil } diff --git a/api/internal/issuetracker/youtrack_auth.go b/api/internal/issuetracker/youtrack_auth.go index 0fceedd73b..c934f61b6f 100644 --- a/api/internal/issuetracker/youtrack_auth.go +++ b/api/internal/issuetracker/youtrack_auth.go @@ -10,6 +10,30 @@ import ( // VerifyToken implements Driver. It asks YouTrack who the token belongs to. func (y *youTrack) VerifyToken(ctx context.Context) (Account, error) { + return y.fetchAccount(ctx) +} + +// connAccount returns the account the token belongs to, asking YouTrack once +// and keeping the answer for the driver's lifetime. Tag ownership is judged +// against this account, never against a stored id. A failed lookup is not kept. +func (y *youTrack) connAccount(ctx context.Context) (Account, error) { + y.accountMu.Lock() + defer y.accountMu.Unlock() + if y.account != nil { + return *y.account, nil + } + a, err := y.fetchAccount(ctx) + if err != nil { + return Account{}, err + } + if !entityIDRE.MatchString(a.ID) { + return Account{}, y.fail("youtrack: account with a malformed id in the response") + } + y.account = &a + return a, nil +} + +func (y *youTrack) fetchAccount(ctx context.Context) (Account, error) { var u ytUser _, _, err := y.do(ctx, ytCall{ method: http.MethodGet, diff --git a/api/internal/issuetracker/youtrack_comments.go b/api/internal/issuetracker/youtrack_comments.go index c2723c3793..35ba8b8d1c 100644 --- a/api/internal/issuetracker/youtrack_comments.go +++ b/api/internal/issuetracker/youtrack_comments.go @@ -13,8 +13,8 @@ import ( // ListIssueComments implements forge.IssueTracker: deleted comments are // dropped and the rest come back oldest first. Author ids are 0 (unresolved), // because YouTrack account ids are strings. -func (y *youTrack) ListIssueComments(ctx context.Context, _, issueIID int64) ([]forge.IssueComment, error) { - w, err := y.resolveIssue(ctx, issueIID) +func (y *youTrack) ListIssueComments(ctx context.Context, projectID, issueIID int64) ([]forge.IssueComment, error) { + w, err := y.resolveIssue(ctx, projectID, issueIID) if err != nil { return nil, err } @@ -45,10 +45,11 @@ func (y *youTrack) ListIssueComments(ctx context.Context, _, issueIID int64) ([] return out, nil } -// CreateIssueNote implements forge.IssueTracker. ID is the numeric suffix of -// the comment's entity id. -func (y *youTrack) CreateIssueNote(ctx context.Context, _, issueIID int64, body string) (forge.IssueNote, error) { - w, err := y.resolveIssue(ctx, issueIID) +// CreateIssueNote implements forge.IssueTracker. ID is always 0: a YouTrack +// comment id is a string such as "4-17" and has no numeric form that is unique +// across issues, so none is invented (PRD #4173 D10). +func (y *youTrack) CreateIssueNote(ctx context.Context, projectID, issueIID int64, body string) (forge.IssueNote, error) { + w, err := y.resolveIssue(ctx, projectID, issueIID) if err != nil { return forge.IssueNote{}, err } @@ -64,9 +65,5 @@ func (y *youTrack) CreateIssueNote(ctx context.Context, _, issueIID int64, body if err != nil { return forge.IssueNote{}, err } - id, err := noteID(c.ID) - if err != nil { - return forge.IssueNote{}, y.red.error(err) - } - return forge.IssueNote{ID: id, Body: c.Text}, nil + return forge.IssueNote{Body: c.Text}, nil } diff --git a/api/internal/issuetracker/youtrack_comments_test.go b/api/internal/issuetracker/youtrack_comments_test.go index 65e3946714..868960710a 100644 --- a/api/internal/issuetracker/youtrack_comments_test.go +++ b/api/internal/issuetracker/youtrack_comments_test.go @@ -21,15 +21,26 @@ func TestDeletedCommentsAreDropped(t *testing.T) { } } -func TestCreateIssueNoteRefusesAMalformedCommentID(t *testing.T) { - y, _ := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { - if r.Method == http.MethodPost { - writeRaw(w, `{"id":"../x","text":"n"}`) - return - } - writeRaw(w, rawIssue("2-1", "UZI-1", 1, "0-1", "")) - }) - if _, err := y.CreateIssueNote(context.Background(), 1, 1, "n"); err == nil { - t.Fatal("CreateIssueNote accepted a malformed comment id") +// TestCreateIssueNoteHasNoNumericID pins PRD #4173 D10: a YouTrack comment id +// such as "4-17" has no numeric form that is unique across issues, so the note's +// ID is 0 whatever id the response carries. +func TestCreateIssueNoteHasNoNumericID(t *testing.T) { + for _, id := range []string{"4-17", "9-123456", "4-0"} { + t.Run(id, func(t *testing.T) { + y, _ := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { + if r.Method == http.MethodPost { + writeRaw(w, `{"id":"`+id+`","text":"n"}`) + return + } + writeRaw(w, rawIssue("2-1", "UZI-1", 1, "0-1", "")) + }) + note, err := y.CreateIssueNote(context.Background(), testProjectID, 1, "n") + if err != nil { + t.Fatal(err) + } + if note.ID != 0 || note.Body != "n" { + t.Errorf("CreateIssueNote = %+v, want ID 0 and body n", note) + } + }) } } diff --git a/api/internal/issuetracker/youtrack_contract_test.go b/api/internal/issuetracker/youtrack_contract_test.go index 4b3eb58ef9..9dac3cadd3 100644 --- a/api/internal/issuetracker/youtrack_contract_test.go +++ b/api/internal/issuetracker/youtrack_contract_test.go @@ -1,6 +1,7 @@ package issuetracker_test import ( + "reflect" "testing" "github.com/vtmocanu/uzi/api/internal/issuetracker" @@ -8,6 +9,10 @@ import ( "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" ) +// contractProjectID is the forge project id the contract's YouTrack driver is +// bound to; it is deliberately not 1, the id the forge fake's first project has. +const contractProjectID = 4173 + // TestYouTrackContract runs the shared issue tracker contract against the // YouTrack driver over the in-process fake. func TestYouTrackContract(t *testing.T) { @@ -17,23 +22,38 @@ func TestYouTrackContract(t *testing.T) { BaseURL: srv.URL, Token: srv.Token, Project: issuetracker.ProjectRef{ID: youtracktest.ProjectID, ShortName: youtracktest.ProjectShortName}, + ProjectID: contractProjectID, OpenState: "Open", ClosedState: "Fixed", + EligibilityTag: issuetracker.TagRef{ + ID: youtracktest.EligibilityTagID, Name: youtracktest.EligibilityTagName, + }, }, srv.Client().Transport) if err != nil { t.Fatal(err) } return issuetrackertest.Subject{ IssueTracker: y, - ProjectID: 1, + ProjectID: contractProjectID, SeedHumanComment: func(t *testing.T, iid int64, username, body string) { srv.SeedComment(iid, username, body) }, SeedLabelEvent: func(t *testing.T, iid int64, action, label, username string) { + // The contract made the label through EnsureLabels, so it is the + // tag the bot owns. + var id string + for _, tag := range srv.TagsNamed(label) { + if tag.OwnerLogin == youtracktest.BotLogin { + id = tag.ID + } + } + if id == "" { + t.Fatalf("no tag %q owned by the bot", label) + } if action == "add" { - srv.SeedTagActivity(iid, username, []string{label}, nil) + srv.SeedTagActivity(iid, username, []string{id}, nil) } else { - srv.SeedTagActivity(iid, username, nil, []string{label}) + srv.SeedTagActivity(iid, username, nil, []string{id}) } }, SeedDeletedComment: func(t *testing.T, iid int64, username, body string) { @@ -42,6 +62,30 @@ func TestYouTrackContract(t *testing.T) { MoveIssue: func(t *testing.T, iid int64) { srv.MoveIssue(iid, youtracktest.OtherProjectID) }, + OtherProjectID: contractProjectID + 1, + WantProjectGuardErr: issuetracker.ErrProjectMismatch, + NoteIDUnset: true, + SameName: &issuetrackertest.SameNameSeams{ + EligibilityName: youtracktest.EligibilityTagName, + EligibilityTagID: youtracktest.EligibilityTagID, + SameNameTagID: youtracktest.SameNameTagID, + SeedIssue: func(t *testing.T, tagIDs ...string) int64 { + return srv.SeedIssue("seeded", "", tagIDs...) + }, + IssueTagIDs: func(t *testing.T, iid int64) []string { return srv.IssueTagIDs(iid) }, + TagsNamed: func(t *testing.T, name string) []issuetrackertest.TagView { + eligibility := srv.TagByID(youtracktest.EligibilityTagID) + var out []issuetrackertest.TagView + for _, tag := range srv.TagsNamed(name) { + out = append(out, issuetrackertest.TagView{ + ID: tag.ID, + OwnedByAccount: tag.OwnerLogin == youtracktest.BotLogin, + SharedLikeEligibility: reflect.DeepEqual(tag.Read, eligibility.Read) && reflect.DeepEqual(tag.Use, eligibility.Use), + }) + } + return out + }, + }, } }) } diff --git a/api/internal/issuetracker/youtrack_events.go b/api/internal/issuetracker/youtrack_events.go index 603edd03b8..ec8ef81b3c 100644 --- a/api/internal/issuetracker/youtrack_events.go +++ b/api/internal/issuetracker/youtrack_events.go @@ -11,9 +11,10 @@ import ( // ListIssueLabelEvents implements forge.IssueTracker from the issue's tag // activities. Events are oldest first, one per added or removed tag; ID is the -// 1-based position in that order. -func (y *youTrack) ListIssueLabelEvents(ctx context.Context, _, issueIID int64) ([]forge.LabelEvent, error) { - w, err := y.resolveIssue(ctx, issueIID) +// 1-based position in that order. A tag that has the eligibility name but is not +// the eligibility tag is left out. +func (y *youTrack) ListIssueLabelEvents(ctx context.Context, projectID, issueIID int64) ([]forge.LabelEvent, error) { + w, err := y.resolveIssue(ctx, projectID, issueIID) if err != nil { return nil, err } @@ -34,6 +35,9 @@ func (y *youTrack) ListIssueLabelEvents(ctx context.Context, _, issueIID int64) out := []forge.LabelEvent{} emit := func(a ytActivity, action string, tags []ytTag) { for _, t := range tags { + if y.foreignEligibility(t) { + continue + } out = append(out, forge.LabelEvent{ ID: int64(len(out) + 1), Action: action, diff --git a/api/internal/issuetracker/youtrack_events_test.go b/api/internal/issuetracker/youtrack_events_test.go index dc587f75a3..c3a24183ff 100644 --- a/api/internal/issuetracker/youtrack_events_test.go +++ b/api/internal/issuetracker/youtrack_events_test.go @@ -10,8 +10,9 @@ import ( func TestLabelEventsOrdinalAndExpansion(t *testing.T) { y, srv := newFakeDriver(t) n := srv.SeedIssue("t", "") - srv.SeedTagActivity(n, "alice", []string{"a", "b"}, []string{"c"}) - srv.SeedTagActivity(n, "bob", []string{"d"}, nil) + a, b, c, d := srv.SeedOwnTag("a"), srv.SeedOwnTag("b"), srv.SeedOwnTag("c"), srv.SeedOwnTag("d") + srv.SeedTagActivity(n, "alice", []string{a, b}, []string{c}) + srv.SeedTagActivity(n, "bob", []string{d}, nil) got, err := y.ListIssueLabelEvents(context.Background(), 1, n) if err != nil { t.Fatal(err) diff --git a/api/internal/issuetracker/youtrack_guards_test.go b/api/internal/issuetracker/youtrack_guards_test.go index e37332ac14..864904ae34 100644 --- a/api/internal/issuetracker/youtrack_guards_test.go +++ b/api/internal/issuetracker/youtrack_guards_test.go @@ -9,6 +9,7 @@ import ( "time" "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" ) func TestSetIssueStateRefusesAStateThatIsNeitherOpenedNorClosed(t *testing.T) { @@ -97,9 +98,11 @@ func TestCreateIssueRefusesAMalformedCreatedID(t *testing.T) { } writeRaw(w, rawIssue("2-1", "UZI-1", 1, "0-1", "")) }) - if _, err := y.CreateIssue(context.Background(), 1, "t", "d", []string{"a"}); err == nil { + if _, err := y.CreateIssue(context.Background(), 1, "t", "d", []string{fakeEligibility.Name}); err == nil { t.Fatal("CreateIssue accepted a malformed created id") } + // The eligibility label needs no lookup, so the POST is the only request: + // the malformed id stops the attach and the read-back. if got := log.all(); len(got) != 1 { t.Errorf("requests = %v, want only the POST", got) } @@ -140,17 +143,38 @@ func TestUpdateIssueLabelsDeduplicates(t *testing.T) { } } -func TestUpdateIssueLabelsSkipsTheTagListingForAPresentLabel(t *testing.T) { - y, srv := newFakeDriver(t) - n := srv.SeedIssue("t", "", "x") - srv.ResetRequests() - if err := y.UpdateIssueLabels(context.Background(), 1, n, []string{"x"}, nil); err != nil { - t.Fatal(err) - } - for _, r := range srv.Requests() { - if strings.HasPrefix(r.Path, "/api/tags") || r.Method != http.MethodGet { - t.Errorf("unexpected request for an already-present label: %v", r) - } +// TestUpdateIssueLabelsAddOfAPresentTagWritesNothing: "already on the issue" is +// decided by tag id. The eligibility name resolves to its stored id with no +// request at all; another name needs the account and its tags, and then still +// writes nothing. +func TestUpdateIssueLabelsAddOfAPresentTagWritesNothing(t *testing.T) { + for _, tc := range []struct { + name string + seed func(*youtracktest.Server) string // returns the tag id put on the issue + label string + wantRead bool // the tag list and users/me are read + }{ + {"eligibility tag", func(*youtracktest.Server) string { return youtracktest.EligibilityTagID }, youtracktest.EligibilityTagName, false}, + {"account tag", func(s *youtracktest.Server) string { return s.SeedOwnTag("x") }, "x", true}, + } { + t.Run(tc.name, func(t *testing.T) { + y, srv := newFakeDriver(t) + n := srv.SeedIssue("t", "", tc.seed(srv)) + srv.ResetRequests() + if err := y.UpdateIssueLabels(context.Background(), testProjectID, n, []string{tc.label}, nil); err != nil { + t.Fatal(err) + } + readsTags := false + for _, r := range srv.Requests() { + if r.Method != http.MethodGet { + t.Errorf("wrote for an already-present label: %v", r) + } + readsTags = readsTags || strings.HasPrefix(r.Path, "/api/tags") || r.Path == "/api/users/me" + } + if readsTags != tc.wantRead { + t.Errorf("read tags or the account = %v, want %v", readsTags, tc.wantRead) + } + }) } } diff --git a/api/internal/issuetracker/youtrack_issue_resolve.go b/api/internal/issuetracker/youtrack_issue_resolve.go index 11204d61a9..b34fcb25b4 100644 --- a/api/internal/issuetracker/youtrack_issue_resolve.go +++ b/api/internal/issuetracker/youtrack_issue_resolve.go @@ -9,7 +9,8 @@ import ( "github.com/vtmocanu/uzi/api/internal/forge" ) -// resolveIssue is the one step every per-issue method starts with. issueIID is +// resolveIssue is the one step every per-issue method starts with. It checks +// projectID against the bound project before any request. issueIID is // YouTrack's numberInProject. The issue is read as SHORT-n and must sit in the // bound project with that number; otherwise it is reported as // *forge.IssueNotFoundError. YouTrack keeps answering an old SHORT-n after the @@ -17,8 +18,8 @@ import ( // read as gone. Follow-up calls address the issue by the returned entity id. // The check is a read: YouTrack has no conditional write, so an issue moved // after resolveIssue returns can still be written by the follow-up call. -func (y *youTrack) resolveIssue(ctx context.Context, issueIID int64) (ytIssue, error) { - p, err := y.requireProject() +func (y *youTrack) resolveIssue(ctx context.Context, projectID, issueIID int64) (ytIssue, error) { + p, err := y.requireProject(projectID) if err != nil { return ytIssue{}, err } diff --git a/api/internal/issuetracker/youtrack_issues.go b/api/internal/issuetracker/youtrack_issues.go index 78f117b9ba..a504b4f6df 100644 --- a/api/internal/issuetracker/youtrack_issues.go +++ b/api/internal/issuetracker/youtrack_issues.go @@ -9,23 +9,33 @@ import ( ) // ListIssues implements forge.IssueTracker. The query narrows by project, -// labels, state and update date; the driver then filters again on exact tag -// names, the exact update instant and the bound project, and honours -// opts.Limit after that filter. -func (y *youTrack) ListIssues(ctx context.Context, _ int64, opts forge.ListIssuesOptions) ([]forge.Issue, error) { - p, err := y.requireProject() +// labels, state and update date; the driver then filters again on the resolved +// tag ids (see resolveTagIDs), the exact update instant and the bound project, +// and honours opts.Limit after that filter. A label whose tag the token's +// account does not own yields no issue. +func (y *youTrack) ListIssues(ctx context.Context, projectID int64, opts forge.ListIssuesOptions) ([]forge.Issue, error) { + p, err := y.requireProject(projectID) if err != nil { return nil, err } if err := y.checkLabelNames(opts.Labels); err != nil { return nil, err } + tagIDs, err := y.resolveTagIDs(ctx, opts.Labels, false) + if err != nil { + return nil, err + } + for _, l := range opts.Labels { + if _, ok := tagIDs[l]; !ok { + return []forge.Issue{}, nil + } + } query := url.Values{"query": {issueQuery(p.ShortName, opts)}, "fields": {issueFields}} out := []forge.Issue{} var mapErr error err = ytPages(ctx, y, []string{"issues"}, query, func(page []ytIssue) bool { for _, w := range page { - if !y.matchesIssue(w, opts) { + if !y.matchesIssue(w, opts, tagIDs) { continue } issue, err := y.toIssue(w) @@ -50,27 +60,34 @@ func (y *youTrack) ListIssues(ctx context.Context, _ int64, opts forge.ListIssue } // GetIssue implements forge.IssueTracker. issueIID is the number in the project. -func (y *youTrack) GetIssue(ctx context.Context, _, issueIID int64) (forge.Issue, error) { - w, err := y.resolveIssue(ctx, issueIID) +func (y *youTrack) GetIssue(ctx context.Context, projectID, issueIID int64) (forge.Issue, error) { + w, err := y.resolveIssue(ctx, projectID, issueIID) if err != nil { return forge.Issue{}, err } return y.toIssue(w) } -// CreateIssue implements forge.IssueTracker. The issue is created first, then -// each label is attached as a tag (created when missing), and the result is +// CreateIssue implements forge.IssueTracker. The labels resolve to tag ids first +// (see resolveTagIDs; created when missing), so a failure there leaves no issue +// behind. Then the issue is created, each tag is attached, and the result is // read back, so it has the shape GetIssue returns. There is no rollback: when // attaching a tag or the read-back fails, the error is returned (redacted) but // the issue stays created. -func (y *youTrack) CreateIssue(ctx context.Context, _ int64, title, description string, labels []string) (forge.Issue, error) { - p, err := y.requireProject() +func (y *youTrack) CreateIssue(ctx context.Context, projectID int64, title, description string, labels []string) (forge.Issue, error) { + p, err := y.requireProject(projectID) if err != nil { return forge.Issue{}, err } if err := y.checkLabelNames(labels); err != nil { return forge.Issue{}, err } + var ids map[string]string + if len(labels) > 0 { + if ids, err = y.resolveTagIDs(ctx, labels, true); err != nil { + return forge.Issue{}, err + } + } var created ytIssue _, _, err = y.do(ctx, ytCall{ method: http.MethodPost, @@ -90,20 +107,14 @@ func (y *youTrack) CreateIssue(ctx context.Context, _ int64, title, description if !entityIDRE.MatchString(created.ID) { return forge.Issue{}, y.fail("youtrack: issue with a malformed id in the response") } - if len(labels) > 0 { - ids, err := y.tagIDsFor(ctx, labels) - if err != nil { - return forge.Issue{}, err + attached := map[string]bool{} + for _, n := range labels { + if attached[n] { + continue } - attached := map[string]bool{} - for _, n := range labels { - if attached[n] { - continue - } - attached[n] = true - if err := y.attachTag(ctx, created.ID, ids[n]); err != nil { - return forge.Issue{}, err - } + attached[n] = true + if err := y.attachTag(ctx, created.ID, ids[n]); err != nil { + return forge.Issue{}, err } } w, err := y.issueByEntity(ctx, created.ID) @@ -114,8 +125,8 @@ func (y *youTrack) CreateIssue(ctx context.Context, _ int64, title, description } // UpdateIssueDescription implements forge.IssueTracker. -func (y *youTrack) UpdateIssueDescription(ctx context.Context, _, issueIID int64, description string) error { - w, err := y.resolveIssue(ctx, issueIID) +func (y *youTrack) UpdateIssueDescription(ctx context.Context, projectID, issueIID int64, description string) error { + w, err := y.resolveIssue(ctx, projectID, issueIID) if err != nil { return err } diff --git a/api/internal/issuetracker/youtrack_issues_test.go b/api/internal/issuetracker/youtrack_issues_test.go index 05c51b6779..e1eb99757a 100644 --- a/api/internal/issuetracker/youtrack_issues_test.go +++ b/api/internal/issuetracker/youtrack_issues_test.go @@ -83,7 +83,17 @@ func TestListIssuesClientSideFilters(t *testing.T) { resolved, old, } - y, _ := loggedDriver(t, rawListHandler(issues...)) + // "bug" is a tag the account owns; the other tags exist only on the issues. + y, _ := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { + switch r.URL.Path { + case "/api/users/me": + writeRaw(w, rawMe) + case "/api/tags": + writeRaw(w, `[`+strings.Replace(bug, `}`, `,"owner":{"id":"`+rawMeID+`"}}`, 1)+`]`) + default: + rawListHandler(issues...)(w, r) + } + }) ctx := context.Background() iids := func(opts forge.ListIssuesOptions) []int64 { t.Helper() diff --git a/api/internal/issuetracker/youtrack_labels.go b/api/internal/issuetracker/youtrack_labels.go index 57eeb218ec..bc4eadc95e 100644 --- a/api/internal/issuetracker/youtrack_labels.go +++ b/api/internal/issuetracker/youtrack_labels.go @@ -8,34 +8,95 @@ import ( "github.com/vtmocanu/uzi/api/internal/forge" ) -// listTags returns the tags the token can see, keyed by name. -func (y *youTrack) listTags(ctx context.Context) (map[string]ytTag, []ytTag, error) { - tags, err := ytList[ytTag](ctx, y, []string{"tags"}, url.Values{"fields": {tagFields}}) +// listTags returns the tags the token can see, in the server's order. +func (y *youTrack) listTags(ctx context.Context, fields string) ([]ytTag, error) { + tags, err := ytList[ytTag](ctx, y, []string{"tags"}, url.Values{"fields": {fields}}) if err != nil { - return nil, nil, err + return nil, err } - byName := make(map[string]ytTag, len(tags)) for _, t := range tags { if !entityIDRE.MatchString(t.ID) { - return nil, nil, y.fail("youtrack: tag with a malformed id in the response") + return nil, y.fail("youtrack: tag with a malformed id in the response") + } + } + return tags, nil +} + +// sharingCopy is the sharing settings of the eligibility tag by id, the form +// POST /api/tags takes. +type sharingCopy struct { + Read ytSharingIDs `json:"readSharingSettings"` + Use ytSharingIDs `json:"tagSharingSettings"` +} + +type ytSharingIDs struct { + Groups []ytIDRef `json:"permittedGroups"` + Users []ytIDRef `json:"permittedUsers"` +} + +type ytIDRef struct { + ID string `json:"id"` +} + +// eligibilitySharing reads the eligibility tag's read and tag sharing settings +// as ids. A tag the driver creates gets the same ones, so whoever can see and +// apply the eligibility tag can see and apply the tags the driver makes. +func (y *youTrack) eligibilitySharing(ctx context.Context) (sharingCopy, error) { + const spec = "permittedGroups(id),permittedUsers(id)" + var t struct { + Read *ytSharing `json:"readSharingSettings"` + Use *ytSharing `json:"tagSharingSettings"` + } + _, _, err := y.do(ctx, ytCall{ + method: http.MethodGet, + path: []string{"tags", y.eligibility.ID}, + query: url.Values{"fields": {"readSharingSettings(" + spec + "),tagSharingSettings(" + spec + ")"}}, + limit: objectBytes, + prov: forge.ResponseSuccess, + }, &t) + if err != nil { + return sharingCopy{}, err + } + var c sharingCopy + for _, p := range []struct { + from *ytSharing + to *ytSharingIDs + }{{t.Read, &c.Read}, {t.Use, &c.Use}} { + p.to.Groups, p.to.Users = []ytIDRef{}, []ytIDRef{} + if p.from == nil { + continue + } + for _, g := range p.from.Groups { + if !entityIDRE.MatchString(g.ID) { + return sharingCopy{}, y.fail("youtrack: sharing settings with a malformed id in the response") + } + p.to.Groups = append(p.to.Groups, ytIDRef{ID: g.ID}) } - if _, dup := byName[t.Name]; !dup { - byName[t.Name] = t + for _, u := range p.from.Users { + if !entityIDRE.MatchString(u.ID) { + return sharingCopy{}, y.fail("youtrack: sharing settings with a malformed id in the response") + } + p.to.Users = append(p.to.Users, ytIDRef{ID: u.ID}) } } - return byName, tags, nil + return c, nil } -// createTag creates a tag by name and returns it. -func (y *youTrack) createTag(ctx context.Context, name string) (ytTag, error) { +// createTag creates a tag owned by the token's account, with the sharing +// settings in share, and returns it. +func (y *youTrack) createTag(ctx context.Context, name string, share sharingCopy) (ytTag, error) { var t ytTag _, _, err := y.do(ctx, ytCall{ method: http.MethodPost, path: []string{"tags"}, - query: url.Values{"fields": {tagFields}}, - body: map[string]string{"name": name}, - limit: objectBytes, - prov: forge.ResponseSuccess, + query: url.Values{"fields": {ownedTagFields}}, + body: map[string]any{ + "name": name, + "readSharingSettings": share.Read, + "tagSharingSettings": share.Use, + }, + limit: objectBytes, + prov: forge.ResponseSuccess, }, &t) if err != nil { return ytTag{}, err @@ -46,40 +107,93 @@ func (y *youTrack) createTag(ctx context.Context, name string) (ytTag, error) { return t, nil } -// tagIDsFor maps names to the ids of their tags, creating a tag that does not -// exist yet. Duplicate names are created once. -func (y *youTrack) tagIDsFor(ctx context.Context, names []string) (map[string]string, error) { - byName, _, err := y.listTags(ctx) +// resolveTagIDs maps label names to tag ids, the one place a name becomes a tag. +// - The eligibility name is the stored eligibility tag's id, whoever owns that +// tag and whatever other tag carries the name. It is never created. +// - Any other name is the visible tag the token's own account owns. With create +// set, a missing one is created with the eligibility tag's sharing settings; +// without it, the name is left out of the result. +// +// The eligibility tag is never a candidate for another name, even when its +// YouTrack name drifted to one: name matching skips the eligibility tag's id, +// so such a name gets an account-owned tag of its own. +// +// A tag of the same name that someone else owns is never used for such a name, +// because its owner may change what it means or who can see it. It makes no +// request when every name is the eligibility name. +func (y *youTrack) resolveTagIDs(ctx context.Context, names []string, create bool) (map[string]string, error) { + ids := make(map[string]string, len(names)) + var others []string + for _, n := range names { + if _, done := ids[n]; done { + continue + } + if n == y.eligibility.Name { + ids[n] = y.eligibility.ID + continue + } + ids[n] = "" + others = append(others, n) + } + if len(others) == 0 { + return ids, nil + } + account, err := y.connAccount(ctx) if err != nil { return nil, err } - ids := make(map[string]string, len(names)) - for _, n := range names { - t, ok := byName[n] - if !ok { - if t, err = y.createTag(ctx, n); err != nil { + tags, err := y.listTags(ctx, ownedTagFields) + if err != nil { + return nil, err + } + owned := make(map[string]string, len(tags)) + for _, t := range tags { + if t.ID == y.eligibility.ID || t.Owner == nil || t.Owner.ID != account.ID { + continue + } + if _, dup := owned[t.Name]; !dup { + owned[t.Name] = t.ID + } + } + var share *sharingCopy + for _, n := range others { + id, ok := owned[n] + switch { + case ok: + ids[n] = id + case !create: + delete(ids, n) + default: + if share == nil { + c, err := y.eligibilitySharing(ctx) + if err != nil { + return nil, err + } + share = &c + } + t, err := y.createTag(ctx, n, *share) + if err != nil { return nil, err } - byName[n] = t + ids[n] = t.ID } - ids[n] = t.ID } return ids, nil } // ListLabels implements forge.IssueTracker. Labels are YouTrack tags, global -// to the account; projectID is ignored. A name carried by several tags is -// listed once. -func (y *youTrack) ListLabels(ctx context.Context, _ int64) ([]forge.Label, error) { - if _, err := y.requireProject(); err != nil { +// to the account; the result is every visible tag name, whoever owns the tag. A +// name carried by several tags is listed once. +func (y *youTrack) ListLabels(ctx context.Context, projectID int64) ([]forge.Label, error) { + if _, err := y.requireProject(projectID); err != nil { return nil, err } - byName, tags, err := y.listTags(ctx) + tags, err := y.listTags(ctx, tagFields) if err != nil { return nil, err } - out := make([]forge.Label, 0, len(byName)) - seen := make(map[string]bool, len(byName)) + out := make([]forge.Label, 0, len(tags)) + seen := make(map[string]bool, len(tags)) for _, t := range tags { if seen[t.Name] { continue @@ -90,11 +204,12 @@ func (y *youTrack) ListLabels(ctx context.Context, _ int64) ([]forge.Label, erro return out, nil } -// EnsureLabels implements forge.IssueTracker: it creates the tags that do not -// exist by name. Label.Color is ignored, because YouTrack colours are palette +// EnsureLabels implements forge.IssueTracker: it creates the tags the token's +// account does not own yet, by name (see resolveTagIDs). It never creates the +// eligibility tag. Label.Color is ignored, because YouTrack colours are palette // ids. -func (y *youTrack) EnsureLabels(ctx context.Context, _ int64, labels []forge.Label) error { - if _, err := y.requireProject(); err != nil { +func (y *youTrack) EnsureLabels(ctx context.Context, projectID int64, labels []forge.Label) error { + if _, err := y.requireProject(projectID); err != nil { return err } names := make([]string, 0, len(labels)) @@ -104,59 +219,58 @@ func (y *youTrack) EnsureLabels(ctx context.Context, _ int64, labels []forge.Lab if err := y.checkLabelNames(names); err != nil { return err } - _, err := y.tagIDsFor(ctx, names) + _, err := y.resolveTagIDs(ctx, names, true) return err } // UpdateIssueLabels implements forge.IssueTracker. It makes one REST call per // tag, adds first and then removes, so it is not atomic. The commands API is -// not used, so a label name never enters a command query. A tag to add that -// does not exist is created; one to remove that is not on the issue is skipped. -func (y *youTrack) UpdateIssueLabels(ctx context.Context, _, issueIID int64, add, remove []string) error { - if _, err := y.requireProject(); err != nil { +// not used, so a label name never enters a command query. Names resolve to tag +// ids (see resolveTagIDs) and "already on the issue" is decided by id. An add +// whose tag does not exist is created; a remove whose tag does not exist or is +// not on the issue is skipped. +func (y *youTrack) UpdateIssueLabels(ctx context.Context, projectID, issueIID int64, add, remove []string) error { + if _, err := y.requireProject(projectID); err != nil { return err } if err := y.checkLabelNames(add, remove); err != nil { return err } - w, err := y.resolveIssue(ctx, issueIID) + w, err := y.resolveIssue(ctx, projectID, issueIID) if err != nil { return err } - present := make(map[string]string, len(w.Tags)) + present := make(map[string]bool, len(w.Tags)) for _, t := range w.Tags { - present[t.Name] = t.ID - } - var missing []string - for _, n := range add { - if _, ok := present[n]; !ok { - missing = append(missing, n) - } + present[t.ID] = true } - var ids map[string]string - if len(missing) > 0 { - if ids, err = y.tagIDsFor(ctx, missing); err != nil { + if len(add) > 0 { + ids, err := y.resolveTagIDs(ctx, add, true) + if err != nil { return err } - } - for _, n := range missing { - if _, ok := present[n]; ok { - continue - } - if err := y.attachTag(ctx, w.ID, ids[n]); err != nil { - return err + for _, n := range add { + if id := ids[n]; !present[id] { + if err := y.attachTag(ctx, w.ID, id); err != nil { + return err + } + present[id] = true + } } - present[n] = ids[n] } - for _, n := range remove { - id, ok := present[n] - if !ok { - continue - } - if err := y.detachTag(ctx, w.ID, id); err != nil { + if len(remove) > 0 { + ids, err := y.resolveTagIDs(ctx, remove, false) + if err != nil { return err } - delete(present, n) + for _, n := range remove { + if id, ok := ids[n]; ok && present[id] { + if err := y.detachTag(ctx, w.ID, id); err != nil { + return err + } + delete(present, id) + } + } } return nil } diff --git a/api/internal/issuetracker/youtrack_labels_test.go b/api/internal/issuetracker/youtrack_labels_test.go index 23e3d687ae..94c38e29a4 100644 --- a/api/internal/issuetracker/youtrack_labels_test.go +++ b/api/internal/issuetracker/youtrack_labels_test.go @@ -9,6 +9,7 @@ import ( "testing" "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" ) // TestLabelGuardOnEveryGuardedMethod sends a brace, and a name one byte over @@ -40,7 +41,8 @@ func TestLabelGuardOnEveryGuardedMethod(t *testing.T) { func TestUpdateIssueLabelsSkipsAbsentTagRemoval(t *testing.T) { y, srv := newFakeDriver(t) - n := srv.SeedIssue("t", "", "keep", "drop") + keep, drop := srv.SeedOwnTag("keep"), srv.SeedOwnTag("drop") + n := srv.SeedIssue("t", "", keep, drop) srv.ResetRequests() if err := y.UpdateIssueLabels(context.Background(), 1, n, nil, []string{"never-there", "drop"}); err != nil { t.Fatal(err) @@ -54,18 +56,22 @@ func TestUpdateIssueLabelsSkipsAbsentTagRemoval(t *testing.T) { if deletes != 1 { t.Errorf("sent %d DELETE requests, want 1 (only the tag on the issue)", deletes) } - if got := srv.IssueTags(n); !slices.Equal(got, []string{"keep"}) { - t.Errorf("tags = %v, want [keep]", got) + if got := srv.IssueTagIDs(n); !slices.Equal(got, []string{keep}) { + t.Errorf("tags = %v, want [%s]", got, keep) } } -func TestTagIDsForTakesTheFirstTagOfADuplicatedName(t *testing.T) { - y, _ := loggedDriver(t, func(w http.ResponseWriter, _ *http.Request) { - writeRaw(w, `[{"id":"5-1","name":"dup"},{"id":"5-2","name":"dup"}]`) +func TestResolveTagIDsTakesTheFirstTagTheAccountOwns(t *testing.T) { + y, _ := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/api/users/me" { + writeRaw(w, rawMe) + return + } + writeRaw(w, `[{"id":"5-1","name":"dup","owner":{"id":"1-2"}},{"id":"5-2","name":"dup","owner":{"id":"`+rawMeID+`"}},{"id":"5-3","name":"dup","owner":{"id":"`+rawMeID+`"}}]`) }) - ids, err := y.tagIDsFor(context.Background(), []string{"dup"}) - if err != nil || ids["dup"] != "5-1" { - t.Fatalf("tagIDsFor = %v, %v; want the first tag, 5-1", ids, err) + ids, err := y.resolveTagIDs(context.Background(), []string{"dup"}, false) + if err != nil || ids["dup"] != "5-2" { + t.Fatalf("resolveTagIDs = %v, %v; want the first tag the account owns, 5-2", ids, err) } labels, err := y.ListLabels(context.Background(), 1) if err != nil || len(labels) != 1 || labels[0].Name != "dup" { @@ -106,11 +112,14 @@ func TestMalformedTagIDsNeverReachAPath(t *testing.T) { }) t.Run("in the tag list", func(t *testing.T) { y, log := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/api/tags" { - writeRaw(w, `[{"id":"`+bad+`","name":"x"}]`) - return + switch r.URL.Path { + case "/api/users/me": + writeRaw(w, rawMe) + case "/api/tags": + writeRaw(w, `[{"id":"`+bad+`","name":"x","owner":{"id":"`+rawMeID+`"}}]`) + default: + writeRaw(w, rawIssue("2-1", "UZI-1", 1, "0-1", "")) } - writeRaw(w, rawIssue("2-1", "UZI-1", 1, "0-1", "")) }) ctx := context.Background() if _, err := y.ListLabels(ctx, 1); err == nil { @@ -125,15 +134,18 @@ func TestMalformedTagIDsNeverReachAPath(t *testing.T) { }) t.Run("in a created tag", func(t *testing.T) { y, log := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path == "/api/tags" && r.Method == http.MethodGet { + switch { + case r.URL.Path == "/api/users/me": + writeRaw(w, rawMe) + case r.URL.Path == "/api/tags" && r.Method == http.MethodGet: writeRaw(w, `[]`) - return - } - if r.URL.Path == "/api/tags" { + case r.URL.Path == "/api/tags": writeRaw(w, `{"id":"`+bad+`","name":"x"}`) - return + case strings.HasPrefix(r.URL.Path, "/api/tags/"): + writeRaw(w, rawNoSharing) + default: + writeRaw(w, rawIssue("2-1", "UZI-1", 1, "0-1", "")) } - writeRaw(w, rawIssue("2-1", "UZI-1", 1, "0-1", "")) }) if err := y.UpdateIssueLabels(context.Background(), 1, 1, []string{"x"}, nil); err == nil { t.Error("UpdateIssueLabels attached a malformed created tag id") @@ -145,3 +157,59 @@ func TestMalformedTagIDsNeverReachAPath(t *testing.T) { } }) } + +// driftedEligibilityDriver builds a driver whose stored eligibility tag is an +// account-owned tag that YouTrack now calls "uzi-finding", while the stored +// name, which resolves the eligibility label on writes, is still "uzi-ready". +func driftedEligibilityDriver(t *testing.T) (*youTrack, *youtracktest.Server, string) { + t.Helper() + srv := youtracktest.NewServer(t) + eligID := srv.SeedOwnTag("uzi-finding") + opts := fakeOptions(srv) + opts.EligibilityTag = TagRef{ID: eligID, Name: "uzi-ready"} + y, err := newYouTrackOver(opts, srv.Client().Transport) + if err != nil { + t.Fatal(err) + } + return y, srv, eligID +} + +// TestDriftedEligibilityTagNameNeverResolvesToTheEligibilityTag pins that a +// uzi write of a label equal to the eligibility tag's current YouTrack name +// does not attach the eligibility tag: a finding issue must not become +// eligible. YouTrack refuses a second tag of that name for the same owner, so +// the write fails (409) rather than succeeding with the wrong tag. +func TestDriftedEligibilityTagNameNeverResolvesToTheEligibilityTag(t *testing.T) { + ctx := context.Background() + t.Run("UpdateIssueLabels", func(t *testing.T) { + y, srv, eligID := driftedEligibilityDriver(t) + n := srv.SeedIssue("t", "") + if err := y.UpdateIssueLabels(ctx, 1, n, []string{"uzi-finding"}, nil); err == nil { + t.Error("UpdateIssueLabels succeeded; want the tag creation to fail") + } + if got := srv.IssueTagIDs(n); slices.Contains(got, eligID) { + t.Errorf("issue tags = %v, include the eligibility tag %s", got, eligID) + } + }) + t.Run("CreateIssue", func(t *testing.T) { + y, srv, eligID := driftedEligibilityDriver(t) + srv.ResetRequests() + if _, err := y.CreateIssue(ctx, 1, "t", "d", []string{"uzi-finding"}); err == nil { + t.Error("CreateIssue succeeded; want the tag creation to fail") + } + for _, r := range srv.Requests() { + if r.Method == http.MethodPost && strings.HasSuffix(r.Path, "/api/issues") { + t.Errorf("an issue was created although its label could not be resolved (eligibility tag %s)", eligID) + } + } + }) + t.Run("EnsureLabels", func(t *testing.T) { + y, srv, _ := driftedEligibilityDriver(t) + if err := y.EnsureLabels(ctx, 1, []forge.Label{{Name: "uzi-finding"}}); err == nil { + t.Error("EnsureLabels succeeded; want the tag creation to fail") + } + if got := srv.TagsNamed("uzi-finding"); len(got) != 1 { + t.Errorf("tags named uzi-finding = %d, want 1 (the eligibility tag alone)", len(got)) + } + }) +} diff --git a/api/internal/issuetracker/youtrack_projectguard_test.go b/api/internal/issuetracker/youtrack_projectguard_test.go new file mode 100644 index 0000000000..21d839299d --- /dev/null +++ b/api/internal/issuetracker/youtrack_projectguard_test.go @@ -0,0 +1,70 @@ +package issuetracker + +import ( + "errors" + "net/http" + "sync/atomic" + "testing" +) + +// countingTripper counts the requests that reach the transport and fails each +// one, so a call that gets past a guard shows up as a count. +type countingTripper struct{ n atomic.Int64 } + +func (c *countingTripper) RoundTrip(*http.Request) (*http.Response, error) { + c.n.Add(1) + return nil, errors.New("transport reached") +} + +// TestProjectGuardRefusesAMismatchedProjectBeforeAnyRequest is PRD #4173 D10: +// every forge.IssueTracker method answers only for the project id the driver is +// bound to. A different id fails with ErrProjectMismatch and sends no request, +// where before the argument was ignored. +func TestProjectGuardRefusesAMismatchedProjectBeforeAnyRequest(t *testing.T) { + for _, pid := range []int64{0, testProjectID + 1, -testProjectID, 4242} { + rt := &countingTripper{} + y, err := newYouTrackOver(Options{ + BaseURL: "https://yt.example.com", Token: credential(), + Project: ProjectRef{ID: "0-1", ShortName: "UZI"}, ProjectID: testProjectID, EligibilityTag: fakeEligibility, + OpenState: "Open", ClosedState: "Fixed", + }, rt) + if err != nil { + t.Fatal(err) + } + calls := everyMethodWith(y, pid) + if len(calls) != 11 { + t.Fatalf("table covers %d methods, want 11", len(calls)) + } + for name, call := range calls { + if err := call(); !errors.Is(err, ErrProjectMismatch) { + t.Errorf("%s with project id %d = %v, want ErrProjectMismatch", name, pid, err) + } + } + if n := rt.n.Load(); n != 0 { + t.Errorf("project id %d: %d requests were sent, want none", pid, n) + } + } +} + +// TestProjectGuardLetsTheBoundProjectThrough is the control for the test above: +// with the bound id every method gets as far as the transport. +func TestProjectGuardLetsTheBoundProjectThrough(t *testing.T) { + for name := range everyMethod(&youTrack{}) { + rt := &countingTripper{} + y, err := newYouTrackOver(Options{ + BaseURL: "https://yt.example.com", Token: credential(), + Project: ProjectRef{ID: "0-1", ShortName: "UZI"}, ProjectID: testProjectID, EligibilityTag: fakeEligibility, + OpenState: "Open", ClosedState: "Fixed", + }, rt) + if err != nil { + t.Fatal(err) + } + err = everyMethod(y)[name]() + if errors.Is(err, ErrProjectMismatch) || errors.Is(err, ErrProjectNotBound) { + t.Errorf("%s with the bound id = %v", name, err) + } + if rt.n.Load() == 0 { + t.Errorf("%s with the bound id sent no request", name) + } + } +} diff --git a/api/internal/issuetracker/youtrack_query.go b/api/internal/issuetracker/youtrack_query.go index 6c685f0cff..dc22db0f12 100644 --- a/api/internal/issuetracker/youtrack_query.go +++ b/api/internal/issuetracker/youtrack_query.go @@ -14,16 +14,21 @@ const maxLabelNameLen = 200 // that carries invisible characters: braces (the query's quoting), control and // format characters, and an empty name. The error never echoes the name. func (y *youTrack) checkLabelName(name string) error { + if labelNameProblem(name) { + return y.red.sentinel(ErrInvalidLabelName, 0, "name is empty, too long, or has a brace or control character") + } + return nil +} + +// labelNameProblem reports whether name is one checkLabelName refuses. +func labelNameProblem(name string) bool { bad := strings.TrimSpace(name) == "" || len(name) > maxLabelNameLen for _, c := range name { if c == '{' || c == '}' || InvisibleRune(c) { bad = true } } - if bad { - return y.red.sentinel(ErrInvalidLabelName, 0, "name is empty, too long, or has a brace or control character") - } - return nil + return bad } func (y *youTrack) checkLabelNames(names ...[]string) error { @@ -59,16 +64,18 @@ func issueQuery(shortName string, opts forge.ListIssuesOptions) string { return strings.Join(parts, " ") } -// matchesIssue is the client-side filter that backs the query: exact tag -// names, exact state, the exact update instant, and the bound project. -func (y *youTrack) matchesIssue(w ytIssue, opts forge.ListIssuesOptions) bool { +// matchesIssue is the client-side filter that backs the query: the resolved tag +// id of every label (tagIDs, from resolveTagIDs, so a same-named tag of someone +// else does not match), exact state, the exact update instant, and the bound +// project. +func (y *youTrack) matchesIssue(w ytIssue, opts forge.ListIssuesOptions, tagIDs map[string]string) bool { if !y.inBoundProject(w) { return false } for _, l := range opts.Labels { found := false for _, t := range w.Tags { - if t.Name == l { + if t.ID == tagIDs[l] { found = true break } diff --git a/api/internal/issuetracker/youtrack_security_test.go b/api/internal/issuetracker/youtrack_security_test.go index 0c7ba2afe5..df564f7285 100644 --- a/api/internal/issuetracker/youtrack_security_test.go +++ b/api/internal/issuetracker/youtrack_security_test.go @@ -30,11 +30,13 @@ func tlsDriverToken(t *testing.T, h http.Handler, token string) (*youTrack, *htt srv := httptest.NewTLSServer(h) t.Cleanup(srv.Close) y, err := newYouTrackOver(Options{ - BaseURL: srv.URL, - Token: token, - Project: ProjectRef{ID: "0-1", ShortName: "UZI"}, - OpenState: "Open", - ClosedState: "Fixed", + BaseURL: srv.URL, + Token: token, + Project: ProjectRef{ID: "0-1", ShortName: "UZI"}, + ProjectID: testProjectID, + EligibilityTag: fakeEligibility, + OpenState: "Open", + ClosedState: "Fixed", }, srv.Client().Transport) if err != nil { t.Fatal(err) diff --git a/api/internal/issuetracker/youtrack_state.go b/api/internal/issuetracker/youtrack_state.go index 5b81f9de75..68504bce7c 100644 --- a/api/internal/issuetracker/youtrack_state.go +++ b/api/internal/issuetracker/youtrack_state.go @@ -11,8 +11,8 @@ import ( // SetIssueState implements forge.IssueTracker. It writes the explicit state // value Options names for the requested state and never guesses one; with that // value unset it returns ErrStateMappingUnset. -func (y *youTrack) SetIssueState(ctx context.Context, _, issueIID int64, state forge.IssueState) error { - if _, err := y.requireProject(); err != nil { +func (y *youTrack) SetIssueState(ctx context.Context, projectID, issueIID int64, state forge.IssueState) error { + if _, err := y.requireProject(projectID); err != nil { return err } var value string @@ -27,7 +27,7 @@ func (y *youTrack) SetIssueState(ctx context.Context, _, issueIID int64, state f if value == "" { return y.red.sentinel(ErrStateMappingUnset, 0, "no value configured for "+string(state)) } - w, err := y.resolveIssue(ctx, issueIID) + w, err := y.resolveIssue(ctx, projectID, issueIID) if err != nil { return err } diff --git a/api/internal/issuetracker/youtrack_tagresolve_test.go b/api/internal/issuetracker/youtrack_tagresolve_test.go new file mode 100644 index 0000000000..9a13207d82 --- /dev/null +++ b/api/internal/issuetracker/youtrack_tagresolve_test.go @@ -0,0 +1,309 @@ +package issuetracker + +import ( + "context" + "net/http" + "reflect" + "slices" + "strconv" + "strings" + "sync" + "sync/atomic" + "testing" + + "github.com/vtmocanu/uzi/api/internal/forge" + "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" +) + +// These tests pin PRD #4173 D14: the eligibility label name is the stored +// eligibility tag by id, whoever owns it, and any other name is only ever a tag +// the connection's own account owns. The fake seeds an eligibility tag owned by +// another user and shared with All Users, and a tag with the same name owned by +// the account. + +func labels(names ...string) []forge.Label { + out := make([]forge.Label, 0, len(names)) + for _, n := range names { + out = append(out, forge.Label{Name: n}) + } + return out +} + +func ids(tags []youtracktest.TagInfo) []string { + var out []string + for _, t := range tags { + out = append(out, t.ID) + } + return out +} + +func TestEnsureLabelsNeverCreatesTheEligibilityTag(t *testing.T) { + y, srv := newFakeDriver(t) + ctx := context.Background() + for range 2 { + if err := y.EnsureLabels(ctx, testProjectID, labels(youtracktest.EligibilityTagName, "fresh")); err != nil { + t.Fatal(err) + } + } + got := srv.TagsNamed(youtracktest.EligibilityTagName) + if want := []string{youtracktest.EligibilityTagID, youtracktest.SameNameTagID}; !sameSet(ids(got), want) { + t.Fatalf("tags named %q = %v, want the seeded %v and no new one", youtracktest.EligibilityTagName, ids(got), want) + } + fresh := srv.TagsNamed("fresh") + if len(fresh) != 1 { + t.Fatalf("tags named fresh = %+v, want one", fresh) + } + elig := srv.TagByID(youtracktest.EligibilityTagID) + if fresh[0].OwnerLogin != youtracktest.BotLogin { + t.Errorf("fresh is owned by %q, want the account %q", fresh[0].OwnerLogin, youtracktest.BotLogin) + } + if !reflect.DeepEqual(fresh[0].Read, elig.Read) || !reflect.DeepEqual(fresh[0].Use, elig.Use) { + t.Errorf("fresh sharing = read %+v use %+v, want the eligibility tag's read %+v use %+v", fresh[0].Read, fresh[0].Use, elig.Read, elig.Use) + } + if len(elig.Use.Groups) != 1 || elig.Use.Groups[0] != youtracktest.AllUsersGroupID { + t.Fatalf("fixture error: the seeded eligibility tag is not shared with All Users: %+v", elig) + } +} + +func TestEnsureLabelsDoesNotAdoptSomeoneElsesTagOfTheSameName(t *testing.T) { + y, srv := newFakeDriver(t) + theirs := srv.SeedTag("theirs", youtracktest.OtherLogin, + youtracktest.Sharing{Groups: []string{youtracktest.AllUsersGroupID}}, youtracktest.Sharing{Groups: []string{youtracktest.AllUsersGroupID}}) + if err := y.EnsureLabels(context.Background(), testProjectID, labels("theirs")); err != nil { + t.Fatal(err) + } + got := srv.TagsNamed("theirs") + if len(got) != 2 || got[0].ID != theirs || got[1].OwnerLogin != youtracktest.BotLogin { + t.Errorf("tags named theirs = %+v, want theirs plus one the account owns", got) + } +} + +func TestCreateIssueAttachesTheEligibilityTagNotTheSameNamedOne(t *testing.T) { + y, srv := newFakeDriver(t) + issue, err := y.CreateIssue(context.Background(), testProjectID, "t", "d", []string{youtracktest.EligibilityTagName, "other"}) + if err != nil { + t.Fatal(err) + } + got := srv.IssueTagIDs(issue.IID) + if len(got) != 2 || got[0] != youtracktest.EligibilityTagID || got[1] == youtracktest.SameNameTagID { + t.Errorf("created issue carries %v, want the eligibility tag then the account's own %q", got, "other") + } + if !sameSet(issue.Labels, []string{youtracktest.EligibilityTagName, "other"}) { + t.Errorf("labels = %v", issue.Labels) + } +} + +func sameSet(a, b []string) bool { + a, b = slices.Clone(a), slices.Clone(b) + slices.Sort(a) + slices.Sort(b) + return slices.Equal(a, b) +} + +func TestListIssuesMatchesByResolvedTagID(t *testing.T) { + y, srv := newFakeDriver(t) + ctx := context.Background() + eligible := srv.SeedIssue("eligible", "", youtracktest.EligibilityTagID) + lookalike := srv.SeedIssue("lookalike", "", youtracktest.SameNameTagID) + theirs := srv.SeedTag("shared-name", youtracktest.OtherLogin, + youtracktest.Sharing{Groups: []string{youtracktest.AllUsersGroupID}}, youtracktest.Sharing{Groups: []string{youtracktest.AllUsersGroupID}}) + onTheirs := srv.SeedIssue("on their tag", "", theirs) + + iids := func(labels ...string) []int64 { + t.Helper() + got, err := y.ListIssues(ctx, testProjectID, forge.ListIssuesOptions{Labels: labels}) + if err != nil { + t.Fatal(err) + } + out := []int64{} + for _, i := range got { + out = append(out, i.IID) + } + return out + } + if got := iids(youtracktest.EligibilityTagName); !slices.Equal(got, []int64{eligible}) { + t.Errorf("eligibility name finds %v, want only %d (not the lookalike %d)", got, eligible, lookalike) + } + // The lookalike's labels never show the eligibility name. + if got, err := y.GetIssue(ctx, testProjectID, lookalike); err != nil || len(got.Labels) != 0 { + t.Errorf("lookalike = %+v, %v; want no label", got.Labels, err) + } + all, err := y.ListIssues(ctx, testProjectID, forge.ListIssuesOptions{}) + if err != nil { + t.Fatal(err) + } + for _, i := range all { + if i.IID == lookalike && len(i.Labels) != 0 { + t.Errorf("ListIssues reports labels %v for the lookalike", i.Labels) + } + } + // A name the account owns no tag of finds nothing, though someone else's + // tag has the name, and no tag is created by the read. + if got := iids("shared-name"); len(got) != 0 { + t.Errorf("a name only someone else owns finds %v (their issue is %d)", got, onTheirs) + } + if n := len(srv.TagsNamed("shared-name")); n != 1 { + t.Errorf("ListIssues created a tag: %d tags named shared-name", n) + } + // Once the account owns a tag of that name, only issues with that tag match. + if err := y.EnsureLabels(ctx, testProjectID, labels("shared-name")); err != nil { + t.Fatal(err) + } + if got := iids("shared-name"); len(got) != 0 { + t.Errorf("after EnsureLabels finds %v, want none: no issue carries the account's tag", got) + } + named := srv.TagsNamed("shared-name") + if len(named) != 2 { + t.Fatalf("tags named shared-name = %+v, want their tag and one the account owns", named) + } + mine := named[1].ID + own := srv.SeedIssue("on my tag", "", mine) + if got := iids("shared-name"); !slices.Equal(got, []int64{own}) { + t.Errorf("finds %v, want only %d", got, own) + } +} + +func TestUpdateIssueLabelsOfTheEligibilityNameTouchesOnlyTheEligibilityTag(t *testing.T) { + y, srv := newFakeDriver(t) + ctx := context.Background() + n := srv.SeedIssue("t", "", youtracktest.SameNameTagID) + if err := y.UpdateIssueLabels(ctx, testProjectID, n, []string{youtracktest.EligibilityTagName}, nil); err != nil { + t.Fatal(err) + } + if got := srv.IssueTagIDs(n); !sameSet(got, []string{youtracktest.SameNameTagID, youtracktest.EligibilityTagID}) { + t.Errorf("after add: %v, want the account's tag and the eligibility tag", got) + } + if err := y.UpdateIssueLabels(ctx, testProjectID, n, nil, []string{youtracktest.EligibilityTagName}); err != nil { + t.Fatal(err) + } + if got := srv.IssueTagIDs(n); !slices.Equal(got, []string{youtracktest.SameNameTagID}) { + t.Errorf("after remove: %v, want only the account's tag", got) + } + // With no eligibility tag on the issue, a remove removes nothing, though the + // issue carries a tag of that name. + srv.ResetRequests() + if err := y.UpdateIssueLabels(ctx, testProjectID, n, nil, []string{youtracktest.EligibilityTagName}); err != nil { + t.Fatal(err) + } + for _, r := range srv.Requests() { + if r.Method != http.MethodGet { + t.Errorf("removing an absent eligibility tag wrote: %v", r) + } + } +} + +func TestUpdateIssueLabelsRemoveNeverTouchesSomeoneElsesTag(t *testing.T) { + y, srv := newFakeDriver(t) + theirs := srv.SeedTag("theirs", youtracktest.OtherLogin, + youtracktest.Sharing{Groups: []string{youtracktest.AllUsersGroupID}}, youtracktest.Sharing{Groups: []string{youtracktest.AllUsersGroupID}}) + n := srv.SeedIssue("t", "", theirs) + srv.ResetRequests() + if err := y.UpdateIssueLabels(context.Background(), testProjectID, n, nil, []string{"theirs"}); err != nil { + t.Fatal(err) + } + if got := srv.IssueTagIDs(n); !slices.Equal(got, []string{theirs}) { + t.Errorf("tags = %v, want their tag left on the issue", got) + } + for _, r := range srv.Requests() { + if r.Method != http.MethodGet { + t.Errorf("a remove with no tag of the account's wrote: %v", r) + } + } + if len(srv.TagsNamed("theirs")) != 1 { + t.Error("a remove created a tag") + } +} + +func TestLabelEventsDropATagThatOnlySharesTheEligibilityName(t *testing.T) { + y, srv := newFakeDriver(t) + n := srv.SeedIssue("t", "") + srv.SeedTagActivity(n, "alice", []string{youtracktest.SameNameTagID, youtracktest.EligibilityTagID}, []string{youtracktest.SameNameTagID}) + got, err := y.ListIssueLabelEvents(context.Background(), testProjectID, n) + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Action != "add" || got[0].LabelName != youtracktest.EligibilityTagName || got[0].ID != 1 { + t.Errorf("events = %+v, want one add of the eligibility tag", got) + } +} + +// TestAccountIsReadOnceAndKept: the token's account comes from users/me the +// first time ownership matters, once for the driver's lifetime, even when +// calls run in parallel; a failed lookup is not kept. +func TestAccountIsReadOnceAndKept(t *testing.T) { + var meCalls atomic.Int64 + var failMe atomic.Bool + y, _ := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { + switch { + case r.URL.Path == "/api/users/me": + meCalls.Add(1) + if failMe.Load() { + w.WriteHeader(http.StatusInternalServerError) + return + } + writeRaw(w, rawMe) + case r.URL.Path == "/api/tags" && r.Method == http.MethodGet: + writeRaw(w, `[]`) + case strings.HasPrefix(r.URL.Path, "/api/tags/"): + writeRaw(w, rawNoSharing) + case r.URL.Path == "/api/tags": + writeRaw(w, `{"id":"5-7","name":"x"}`) + } + }) + ctx := context.Background() + failMe.Store(true) + if err := y.EnsureLabels(ctx, testProjectID, labels("a")); err == nil { + t.Fatal("EnsureLabels succeeded with users/me failing") + } + failMe.Store(false) + var wg sync.WaitGroup + for i := range 8 { + wg.Go(func() { + if err := y.EnsureLabels(ctx, testProjectID, labels("tag-"+strconv.Itoa(i))); err != nil { + t.Error(err) + } + }) + } + wg.Wait() + if err := y.EnsureLabels(ctx, testProjectID, labels("b")); err != nil { + t.Fatal(err) + } + // One failed read, one that was kept. + if got := meCalls.Load(); got != 2 { + t.Errorf("users/me was read %d times, want 2 (the failure, then once for good)", got) + } +} + +func TestEnsureLabelsStopsWhenTheEligibilityTagCannotBeRead(t *testing.T) { + y, log := loggedDriver(t, func(w http.ResponseWriter, r *http.Request) { + switch { + case r.URL.Path == "/api/users/me": + writeRaw(w, rawMe) + case r.URL.Path == "/api/tags" && r.Method == http.MethodGet: + writeRaw(w, `[]`) + default: + w.WriteHeader(http.StatusNotFound) + } + }) + if err := y.EnsureLabels(context.Background(), testProjectID, labels("a")); err == nil { + t.Fatal("EnsureLabels created a tag without the eligibility tag's sharing settings") + } + for _, w := range log.writes() { + t.Errorf("sent %v", w) + } +} + +func TestEligibilityNameAloneMakesNoRequest(t *testing.T) { + y, log := loggedDriver(t, func(w http.ResponseWriter, _ *http.Request) { writeRaw(w, "[]") }) + if err := y.EnsureLabels(context.Background(), testProjectID, labels(youtracktest.EligibilityTagName)); err != nil { + t.Fatal(err) + } + if _, err := y.ListIssues(context.Background(), testProjectID, forge.ListIssuesOptions{Labels: []string{youtracktest.EligibilityTagName}}); err != nil { + t.Fatal(err) + } + for _, c := range log.all() { + if strings.Contains(c, "/tags") || strings.Contains(c, "users/me") { + t.Errorf("sent %v for the eligibility name alone", c) + } + } +} diff --git a/api/internal/issuetracker/youtrack_tags.go b/api/internal/issuetracker/youtrack_tags.go new file mode 100644 index 0000000000..7f1a25c80c --- /dev/null +++ b/api/internal/issuetracker/youtrack_tags.go @@ -0,0 +1,64 @@ +package issuetracker + +import "context" + +// ListTags implements Driver. It lists the tags the token can see with their +// owners and sharing settings; it needs no bound project. Every id taken from +// the response is checked, and one tag's users and groups are capped at +// maxNestedItems and count toward the listing ceiling. +func (y *youTrack) ListTags(ctx context.Context) ([]Tag, error) { + wire, err := y.listTags(ctx, sharedTagFields) + if err != nil { + return nil, err + } + out := make([]Tag, 0, len(wire)) + for _, w := range wire { + if w.nestedCount() > maxNestedItems { + return nil, y.red.error(errNestedCeiling) + } + t := Tag{ID: w.ID, Name: w.Name} + if w.Owner != nil { + if t.Owner, err = y.accountOf(*w.Owner); err != nil { + return nil, err + } + } + if t.Read, err = y.sharingOf(w.Read); err != nil { + return nil, err + } + if t.Use, err = y.sharingOf(w.Use); err != nil { + return nil, err + } + out = append(out, t) + } + return out, nil +} + +// accountOf maps a wire user, refusing a malformed id. +func (y *youTrack) accountOf(u ytUser) (Account, error) { + if !entityIDRE.MatchString(u.ID) { + return Account{}, y.fail("youtrack: user with a malformed id in the response") + } + return Account(u), nil +} + +// sharingOf maps wire sharing settings; absent settings are empty. +func (y *youTrack) sharingOf(s *ytSharing) (SharingSettings, error) { + var out SharingSettings + if s == nil { + return out, nil + } + for _, g := range s.Groups { + if !entityIDRE.MatchString(g.ID) { + return SharingSettings{}, y.fail("youtrack: group with a malformed id in the response") + } + out.Groups = append(out.Groups, Group(g)) + } + for _, u := range s.Users { + a, err := y.accountOf(u) + if err != nil { + return SharingSettings{}, err + } + out.Users = append(out.Users, a) + } + return out, nil +} diff --git a/api/internal/issuetracker/youtrack_test.go b/api/internal/issuetracker/youtrack_test.go index 1fdb354bfb..60fbcaf315 100644 --- a/api/internal/issuetracker/youtrack_test.go +++ b/api/internal/issuetracker/youtrack_test.go @@ -15,15 +15,25 @@ import ( "github.com/vtmocanu/uzi/api/internal/issuetracker/youtracktest" ) +// testProjectID is the forge project id the test drivers are bound to; the +// tests pass it as the projectID argument. +const testProjectID = 1 + +// fakeEligibility is the eligibility tag the fake seeds: owned by another user +// and shared with All Users. +var fakeEligibility = TagRef{ID: youtracktest.EligibilityTagID, Name: youtracktest.EligibilityTagName} + // fakeOptions are the Options that bind a driver to the fake's main project -// with the state mapping its State bundle supports. +// and eligibility tag with the state mapping its State bundle supports. func fakeOptions(srv *youtracktest.Server) Options { return Options{ - BaseURL: srv.URL, - Token: srv.Token, - Project: ProjectRef{ID: youtracktest.ProjectID, ShortName: youtracktest.ProjectShortName}, - OpenState: "Open", - ClosedState: "Fixed", + BaseURL: srv.URL, + Token: srv.Token, + Project: ProjectRef{ID: youtracktest.ProjectID, ShortName: youtracktest.ProjectShortName}, + ProjectID: testProjectID, + EligibilityTag: fakeEligibility, + OpenState: "Open", + ClosedState: "Fixed", } } @@ -63,12 +73,15 @@ func newFakeDriverWithField(t *testing.T, field string) (*youtracktest.Server, * } func TestConstructorValidation(t *testing.T) { - good := Options{BaseURL: "https://yt.example.com", Token: credential(), Project: ProjectRef{ID: "0-1", ShortName: "UZI"}} + good := Options{ + BaseURL: "https://yt.example.com", Token: credential(), Project: ProjectRef{ID: "0-1", ShortName: "UZI"}, + ProjectID: 7, EligibilityTag: TagRef{ID: "5-3", Name: "uzi"}, + } mutate := func(f func(*Options)) Options { o := good; f(&o); return o } if _, err := New(KindYouTrack, good); err != nil { t.Fatalf("good options: %v", err) } - if _, err := New(KindYouTrack, mutate(func(o *Options) { o.Project = ProjectRef{} })); err != nil { + if _, err := New(KindYouTrack, mutate(func(o *Options) { o.Project, o.ProjectID, o.EligibilityTag = ProjectRef{}, 0, TagRef{} })); err != nil { t.Fatalf("unbound options: %v", err) } bad := map[string]Options{ @@ -87,6 +100,15 @@ func TestConstructorValidation(t *testing.T) { "bad short": mutate(func(o *Options) { o.Project.ShortName = "1X" }), "bad id": mutate(func(o *Options) { o.Project.ID = "abc" }), "half bound": mutate(func(o *Options) { o.Project.ID = "" }), + // A binding is all or nothing (PRD #4173 D10, D14). + "no project id": mutate(func(o *Options) { o.ProjectID = 0 }), + "negative project id": mutate(func(o *Options) { o.ProjectID = -3 }), + "no eligibility": mutate(func(o *Options) { o.EligibilityTag = TagRef{} }), + "project id only": mutate(func(o *Options) { o.Project, o.EligibilityTag = ProjectRef{}, TagRef{} }), + "eligibility only": mutate(func(o *Options) { o.Project, o.ProjectID = ProjectRef{}, 0 }), + "bad tag id": mutate(func(o *Options) { o.EligibilityTag.ID = "../x" }), + "bad tag name": mutate(func(o *Options) { o.EligibilityTag.Name = "a}b" }), + "empty tag name": mutate(func(o *Options) { o.EligibilityTag.Name = "" }), } for name, o := range bad { if _, err := New(KindYouTrack, o); err == nil { @@ -144,26 +166,37 @@ func rawIssue(id, readable string, number int64, projectID, tags string) string `"project":{"id":%q,"shortName":"UZI"},"tags":[%s]}`, id, readable, number, projectID, tags) } +// rawMe is the users/me answer of the account that raw handlers' tags belong to. +const ( + rawMeID = "1-1" + rawMe = `{"id":"` + rawMeID + `","login":"u"}` + rawNoSharing = `{"readSharingSettings":{"permittedGroups":[],"permittedUsers":[]},"tagSharingSettings":{"permittedGroups":[],"permittedUsers":[]}}` +) + func writeRaw(w http.ResponseWriter, body string) { w.Header().Set("Content-Type", "application/json") _, _ = w.Write([]byte(body)) } -// everyMethod lists the eleven forge.IssueTracker methods as closures over y. -func everyMethod(y *youTrack) map[string]func() error { +// everyMethod lists the eleven forge.IssueTracker methods as closures over y, +// each called with the project id the test drivers are bound to. +func everyMethod(y *youTrack) map[string]func() error { return everyMethodWith(y, testProjectID) } + +// everyMethodWith is everyMethod with a chosen projectID argument. +func everyMethodWith(y *youTrack, pid int64) map[string]func() error { ctx := context.Background() return map[string]func() error{ - "ListLabels": func() error { _, err := y.ListLabels(ctx, 1); return err }, - "EnsureLabels": func() error { return y.EnsureLabels(ctx, 1, []forge.Label{{Name: "a"}}) }, - "ListIssues": func() error { _, err := y.ListIssues(ctx, 1, forge.ListIssuesOptions{}); return err }, - "GetIssue": func() error { _, err := y.GetIssue(ctx, 1, 1); return err }, - "CreateIssue": func() error { _, err := y.CreateIssue(ctx, 1, "t", "d", []string{"a"}); return err }, - "UpdateIssueLabels": func() error { return y.UpdateIssueLabels(ctx, 1, 1, []string{"a"}, []string{"b"}) }, - "UpdateIssueDescription": func() error { return y.UpdateIssueDescription(ctx, 1, 1, "d") }, - "SetIssueState": func() error { return y.SetIssueState(ctx, 1, 1, forge.StateClosed) }, - "ListIssueLabelEvents": func() error { _, err := y.ListIssueLabelEvents(ctx, 1, 1); return err }, - "ListIssueComments": func() error { _, err := y.ListIssueComments(ctx, 1, 1); return err }, - "CreateIssueNote": func() error { _, err := y.CreateIssueNote(ctx, 1, 1, "n"); return err }, + "ListLabels": func() error { _, err := y.ListLabels(ctx, pid); return err }, + "EnsureLabels": func() error { return y.EnsureLabels(ctx, pid, []forge.Label{{Name: "a"}}) }, + "ListIssues": func() error { _, err := y.ListIssues(ctx, pid, forge.ListIssuesOptions{}); return err }, + "GetIssue": func() error { _, err := y.GetIssue(ctx, pid, 1); return err }, + "CreateIssue": func() error { _, err := y.CreateIssue(ctx, pid, "t", "d", []string{"a"}); return err }, + "UpdateIssueLabels": func() error { return y.UpdateIssueLabels(ctx, pid, 1, []string{"a"}, []string{"b"}) }, + "UpdateIssueDescription": func() error { return y.UpdateIssueDescription(ctx, pid, 1, "d") }, + "SetIssueState": func() error { return y.SetIssueState(ctx, pid, 1, forge.StateClosed) }, + "ListIssueLabelEvents": func() error { _, err := y.ListIssueLabelEvents(ctx, pid, 1); return err }, + "ListIssueComments": func() error { _, err := y.ListIssueComments(ctx, pid, 1); return err }, + "CreateIssueNote": func() error { _, err := y.CreateIssueNote(ctx, pid, 1, "n"); return err }, } } diff --git a/api/internal/issuetracker/youtrack_wire.go b/api/internal/issuetracker/youtrack_wire.go index ea813e10a4..b9f906b7a8 100644 --- a/api/internal/issuetracker/youtrack_wire.go +++ b/api/internal/issuetracker/youtrack_wire.go @@ -1,10 +1,7 @@ package issuetracker import ( - "fmt" "regexp" - "strconv" - "strings" "time" "github.com/vtmocanu/uzi/api/internal/forge" @@ -12,10 +9,13 @@ import ( // fields parameters. YouTrack returns only what is asked for. const ( - issueFields = "id,idReadable,numberInProject,summary,description,created,updated,resolved,reporter(id,login),tags(id,name),project(id,shortName)" - tagFields = "id,name" - commentFields = "id,text,created,deleted,author(id,login)" - activityFields = "id,timestamp,author(id,login),added(id,name),removed(id,name)" + issueFields = "id,idReadable,numberInProject,summary,description,created,updated,resolved,reporter(id,login),tags(id,name),project(id,shortName)" + tagFields = "id,name" + ownedTagFields = "id,name,owner(id)" + sharingFields = "permittedGroups(id,name,allUsersGroup,usersCount),permittedUsers(id,login)" + sharedTagFields = "id,name,owner(id,login),readSharingSettings(" + sharingFields + "),tagSharingSettings(" + sharingFields + ")" + commentFields = "id,text,created,deleted,author(id,login)" + activityFields = "id,timestamp,author(id,login),added(id,name),removed(id,name)" ) // idReadableRE matches an id-readable such as "UZI-15"; it ends up in WebURL. @@ -26,9 +26,39 @@ type ytUser struct { Login string `json:"login"` } +// ytTag is a wire tag. Owner and the sharing settings are present only where +// the request asked for them (see ownedTagFields and sharedTagFields). type ytTag struct { - ID string `json:"id"` - Name string `json:"name"` + ID string `json:"id"` + Name string `json:"name"` + Owner *ytUser `json:"owner"` + Read *ytSharing `json:"readSharingSettings"` + Use *ytSharing `json:"tagSharingSettings"` +} + +// ytSharing is YouTrack's TagSharingSettings (and the read settings, which have +// the same members). +type ytSharing struct { + Groups []ytGroup `json:"permittedGroups"` + Users []ytUser `json:"permittedUsers"` +} + +type ytGroup struct { + ID string `json:"id"` + Name string `json:"name"` + AllUsers bool `json:"allUsersGroup"` + UsersCount int `json:"usersCount"` +} + +// nestedCount is the number of users and groups in the tag's sharing settings. +func (t ytTag) nestedCount() int { + n := 0 + for _, s := range []*ytSharing{t.Read, t.Use} { + if s != nil { + n += len(s.Groups) + len(s.Users) + } + } + return n } type ytProject struct { @@ -109,28 +139,26 @@ func (y *youTrack) toIssue(w ytIssue) (forge.Issue, error) { issue.Author = w.Reporter.Login } for _, t := range w.Tags { + if y.foreignEligibility(t) { + continue + } issue.Labels = append(issue.Labels, t.Name) } return issue, nil } +// foreignEligibility reports whether t carries the eligibility name without +// being the eligibility tag. Reads leave it out, so the eligibility name in a +// result always means the stored tag. +func (y *youTrack) foreignEligibility(t ytTag) bool { + return y.eligibility.Name != "" && t.Name == y.eligibility.Name && t.ID != y.eligibility.ID +} + // inBoundProject reports whether the issue sits in the bound project. func (y *youTrack) inBoundProject(w ytIssue) bool { return w.Project != nil && w.Project.ID == y.project.ID && w.NumberInProject > 0 } -// noteID is the numeric suffix of a comment entity id such as "4-17". -func noteID(id string) (int64, error) { - if !entityIDRE.MatchString(id) { - return 0, fmt.Errorf("youtrack: comment with a malformed id in the response") - } - n, err := strconv.ParseInt(id[strings.IndexByte(id, '-')+1:], 10, 64) - if err != nil { - return 0, fmt.Errorf("youtrack: comment with a malformed id in the response") - } - return n, nil -} - func login(u *ytUser) string { if u == nil { return "" diff --git a/api/internal/issuetracker/youtrack_wire_test.go b/api/internal/issuetracker/youtrack_wire_test.go index df31fbac4b..499e65478d 100644 --- a/api/internal/issuetracker/youtrack_wire_test.go +++ b/api/internal/issuetracker/youtrack_wire_test.go @@ -7,8 +7,8 @@ import ( func TestIssueMapping(t *testing.T) { y, srv := newFakeDriver(t) - n := srv.SeedIssue("title", "body", "x", "y") - got, err := y.GetIssue(context.Background(), 99, n) + n := srv.SeedIssue("title", "body", srv.SeedOwnTag("x"), srv.SeedOwnTag("y")) + got, err := y.GetIssue(context.Background(), testProjectID, n) if err != nil { t.Fatal(err) } diff --git a/api/internal/issuetracker/youtracktest/routes.go b/api/internal/issuetracker/youtracktest/routes.go index 3cf5010001..313d418157 100644 --- a/api/internal/issuetracker/youtracktest/routes.go +++ b/api/internal/issuetracker/youtracktest/routes.go @@ -31,6 +31,7 @@ func (s *Server) routes() *http.ServeMux { route("/api/admin/projects/{pid}/customFields", map[string]http.HandlerFunc{http.MethodGet: s.listProjectFields}) route("/api/admin/projects/{pid}/customFields/{cf}/bundle/values", map[string]http.HandlerFunc{http.MethodGet: s.listBundleValues}) route("/api/tags", map[string]http.HandlerFunc{http.MethodGet: s.listTags, http.MethodPost: s.createTag}) + route("/api/tags/{id}", map[string]http.HandlerFunc{http.MethodGet: s.getTag}) route("/api/issues", map[string]http.HandlerFunc{http.MethodGet: s.listIssues, http.MethodPost: s.createIssue}) route("/api/issues/{id}", map[string]http.HandlerFunc{http.MethodGet: s.getIssue, http.MethodPost: s.updateIssue}) route("/api/issues/{id}/tags", map[string]http.HandlerFunc{http.MethodPost: s.addIssueTag}) @@ -163,24 +164,71 @@ func (s *Server) listTags(w http.ResponseWriter, r *http.Request) { if !ok { return } - writePage(w, r, spec, s.m.tagsJSON(tagIDs(s.m.tags))) + items := []map[string]any{} + for _, t := range s.m.visibleTags() { + items = append(items, s.m.tagJSON(t)) + } + writePage(w, r, spec, items) +} + +// getTag answers 404 for a tag the bot cannot see, as for one that is not there. +func (s *Server) getTag(w http.ResponseWriter, r *http.Request) { + spec, ok := fields(w, r) + if !ok { + return + } + t := s.m.tagByID(r.PathValue("id")) + if t == nil || !t.visibleTo(BotID) { + notFound(w, r.PathValue("id")) + return + } + writeJSON(w, http.StatusOK, spec.project(s.m.tagJSON(t))) } -func tagIDs(tags []*tag) []string { - ids := make([]string, 0, len(tags)) - for _, t := range tags { - ids = append(ids, t.id) +// sharingBody is the sharing settings member of a create-tag body: ids only. +type sharingBody struct { + Groups []struct { + ID string `json:"id"` + } `json:"permittedGroups"` + Users []struct { + ID string `json:"id"` + } `json:"permittedUsers"` +} + +// toSharing resolves a body's ids against the fake's groups and users; it +// reports false for an id the fake does not know. +func (b *sharingBody) toSharing() (sharing, bool) { + var out sharing + if b == nil { + return out, true + } + for _, g := range b.Groups { + if !slices.ContainsFunc(fakeGroups, func(k group) bool { return k.id == g.ID }) { + return sharing{}, false + } + out.groups = append(out.groups, g.ID) } - return ids + for _, u := range b.Users { + if !slices.ContainsFunc(fakeUsers, func(k user) bool { return k.id == u.ID }) { + return sharing{}, false + } + out.users = append(out.users, u.ID) + } + return out, true } +// createTag makes a tag owned by the bot, with the sharing settings of the +// body. Only a duplicate of a tag the bot already owns is a conflict: another +// user's tag of the same name does not block it. func (s *Server) createTag(w http.ResponseWriter, r *http.Request) { spec, ok := fields(w, r) if !ok { return } var body struct { - Name string `json:"name"` + Name string `json:"name"` + Read *sharingBody `json:"readSharingSettings"` + Use *sharingBody `json:"tagSharingSettings"` } if !decodeBody(w, r, &body) { return @@ -189,14 +237,21 @@ func (s *Server) createTag(w http.ResponseWriter, r *http.Request) { writeError(w, http.StatusBadRequest, "Bad Request", "tag name is required") return } + read, readOK := body.Read.toSharing() + use, useOK := body.Use.toSharing() + if !readOK || !useOK { + writeError(w, http.StatusBadRequest, "Bad Request", "unknown group or user in the sharing settings") + return + } for _, t := range s.m.tags { - if t.name == body.Name { + if t.owner == BotID && t.name == body.Name { writeError(w, http.StatusConflict, "Conflict", "tag already exists") return } } - t := s.m.ensureTag(body.Name) - writeJSON(w, http.StatusOK, spec.project(s.m.tagsJSON([]string{t.id})[0])) + t := &tag{id: s.m.nextID(5), name: body.Name, owner: BotID, read: read, use: use} + s.m.tags = append(s.m.tags, t) + writeJSON(w, http.StatusOK, spec.project(s.m.tagJSON(t))) } var ( @@ -260,7 +315,7 @@ func (s *Server) listIssues(w http.ResponseWriter, r *http.Request) { } // hasAllTags matches tag names case-insensitively, as YouTrack's query does; -// the driver filters again on the exact name. +// the driver filters again by resolved tag id. func (s *Server) hasAllTags(i *issue, names []string) bool { for _, name := range names { found := false @@ -382,16 +437,20 @@ func (s *Server) addIssueTag(w http.ResponseWriter, r *http.Request) { return } t := s.m.tagByID(body.ID) - if t == nil { + if t == nil || !t.visibleTo(BotID) { writeError(w, http.StatusBadRequest, "Bad Request", "unknown tag") return } + if !t.usableBy(BotID) { + writeError(w, http.StatusForbidden, "Forbidden", "the tag is not shared with you for use") + return + } if i.hasTag(t.id) { writeError(w, http.StatusConflict, "Conflict", "tag already on the issue") return } s.m.addTag(i, t, BotLogin) - writeJSON(w, http.StatusOK, spec.project(s.m.tagsJSON([]string{t.id})[0])) + writeJSON(w, http.StatusOK, spec.project(s.m.tagJSON(t))) } func (s *Server) removeIssueTag(w http.ResponseWriter, r *http.Request) { diff --git a/api/internal/issuetracker/youtracktest/seed.go b/api/internal/issuetracker/youtracktest/seed.go index 4573240c0a..1e51771599 100644 --- a/api/internal/issuetracker/youtracktest/seed.go +++ b/api/internal/issuetracker/youtracktest/seed.go @@ -1,13 +1,98 @@ package youtracktest -// SeedIssue creates an issue in the main project and returns its number. Tags -// are created on demand and added without recording an activity. -func (s *Server) SeedIssue(summary, description string, tags ...string) int64 { +import "slices" + +// Sharing says who a tag is shared with, by group id and user id. +type Sharing struct { + Groups []string + Users []string +} + +// TagInfo is a tag as the fake holds it, for seeding and assertions. +type TagInfo struct { + ID string + Name string + OwnerLogin string + Read Sharing + Use Sharing +} + +// SeedTag adds a tag owned by the user with login ownerLogin and returns its +// id. Group and user ids must be ones the fake knows. +func (s *Server) SeedTag(name, ownerLogin string, read, use Sharing) string { + s.t.Helper() + s.mu.Lock() + defer s.mu.Unlock() + var owner string + for _, u := range fakeUsers { + if u.login == ownerLogin { + owner = u.id + } + } + if owner == "" { + s.t.Fatalf("youtracktest: no user %q", ownerLogin) + } + t := &tag{id: s.m.nextID(5), name: name, owner: owner, + read: sharing{groups: read.Groups, users: read.Users}, use: sharing{groups: use.Groups, users: use.Users}} + s.m.tags = append(s.m.tags, t) + return t.id +} + +// SeedOwnTag adds a tag owned by the bot and shared with nobody, and returns its id. +func (s *Server) SeedOwnTag(name string) string { + s.t.Helper() + return s.SeedTag(name, BotLogin, Sharing{}, Sharing{}) +} + +// TagsNamed returns every tag called name, in creation order, whoever owns it +// and whether or not the bot can see it. +func (s *Server) TagsNamed(name string) []TagInfo { + s.mu.Lock() + defer s.mu.Unlock() + var out []TagInfo + for _, t := range s.m.tags { + if t.name == name { + out = append(out, s.m.tagInfo(t)) + } + } + return out +} + +// TagByID returns the tag with id, failing the test when there is none. +func (s *Server) TagByID(id string) TagInfo { + s.t.Helper() + s.mu.Lock() + defer s.mu.Unlock() + t := s.m.tagByID(id) + if t == nil { + s.t.Fatalf("youtracktest: no tag %q", id) + } + return s.m.tagInfo(t) +} + +func (m *model) tagInfo(t *tag) TagInfo { + info := TagInfo{ID: t.id, Name: t.name, + Read: Sharing{Groups: t.read.groups, Users: t.read.users}, Use: Sharing{Groups: t.use.groups, Users: t.use.users}} + for _, u := range fakeUsers { + if u.id == t.owner { + info.OwnerLogin = u.login + } + } + return info +} + +// SeedIssue creates an issue in the main project and returns its number. The +// tags, given by id, are added without recording an activity. +func (s *Server) SeedIssue(summary, description string, tagIDs ...string) int64 { + s.t.Helper() s.mu.Lock() defer s.mu.Unlock() i := s.m.newIssue(s.m.projectByID(ProjectID), summary, description) - for _, name := range tags { - i.tags = append(i.tags, s.m.ensureTag(name).id) + for _, id := range tagIDs { + if s.m.tagByID(id) == nil { + s.t.Fatalf("youtracktest: no tag %q", id) + } + i.tags = append(i.tags, id) } return i.number } @@ -35,9 +120,9 @@ func (s *Server) seedComment(n int64, login, text string, deleted bool) { i.comments = append(i.comments, &comment{id: s.m.nextID(4), text: text, created: ts, deleted: deleted, author: login}) } -// SeedTagActivity records that login added and removed the named tags on issue -// number n. Tags are created on demand; the issue's own tag list is left -// alone, so a test can seed history that no longer matches the present. +// SeedTagActivity records that login added and removed the tags with the given +// ids on issue number n. The issue's own tag list is left alone, so a test can +// seed history that no longer matches the present. func (s *Server) SeedTagActivity(n int64, login string, added, removed []string) { s.t.Helper() s.mu.Lock() @@ -47,12 +132,12 @@ func (s *Server) SeedTagActivity(n int64, login string, added, removed []string) s.t.Fatalf("youtracktest: no issue %d", n) } a := &activity{id: s.m.nextID(3), timestamp: s.m.now(), author: login} - for _, name := range added { - a.added = append(a.added, s.m.ensureTag(name).id) - } - for _, name := range removed { - a.removed = append(a.removed, s.m.ensureTag(name).id) + for _, id := range append(slices.Clone(added), removed...) { + if s.m.tagByID(id) == nil { + s.t.Fatalf("youtracktest: no tag %q", id) + } } + a.added, a.removed = added, removed i.activities = append(i.activities, a) } @@ -75,9 +160,9 @@ func (s *Server) MoveIssue(n int64, toProjectID string) string { return i.readable() } -// IssueTags returns the names of the tags currently on issue number n in the -// main project, for assertions on server-side state. -func (s *Server) IssueTags(n int64) []string { +// IssueTagIDs returns the ids of the tags currently on issue number n in the +// main project, oldest first, for assertions on server-side state. +func (s *Server) IssueTagIDs(n int64) []string { s.t.Helper() s.mu.Lock() defer s.mu.Unlock() @@ -85,9 +170,5 @@ func (s *Server) IssueTags(n int64) []string { if i == nil { s.t.Fatalf("youtracktest: no issue %d", n) } - var names []string - for _, id := range i.tags { - names = append(names, s.m.tagByID(id).name) - } - return names + return slices.Clone(i.tags) } diff --git a/api/internal/issuetracker/youtracktest/server_test.go b/api/internal/issuetracker/youtracktest/server_test.go index cd5492aa0d..95a6637545 100644 --- a/api/internal/issuetracker/youtracktest/server_test.go +++ b/api/internal/issuetracker/youtracktest/server_test.go @@ -2,7 +2,10 @@ package youtracktest import ( "fmt" + "io" "net/http" + "strconv" + "strings" "testing" ) @@ -109,3 +112,103 @@ func TestWrongBearerTokenIsRefused(t *testing.T) { } } } + +// call sends one authorised request to the fake and returns the status and body. +func call(t *testing.T, srv *Server, method, path, body string) (int, string) { + t.Helper() + req, err := http.NewRequest(method, srv.URL+path, strings.NewReader(body)) + if err != nil { + t.Fatal(err) + } + req.Header.Set("Authorization", "Bearer "+srv.Token) + req.Header.Set("Content-Type", "application/json") + resp, err := srv.Client().Do(req) + if err != nil { + t.Fatal(err) + } + defer func() { _ = resp.Body.Close() }() + b, err := io.ReadAll(resp.Body) + if err != nil { + t.Fatal(err) + } + return resp.StatusCode, string(b) +} + +func TestDefaultSeedHasAnEligibilityTagAndASameNamedTag(t *testing.T) { + srv := NewServer(t) + got := srv.TagsNamed(EligibilityTagName) + if len(got) != 2 { + t.Fatalf("tags named %q = %+v, want two", EligibilityTagName, got) + } + elig, same := srv.TagByID(EligibilityTagID), srv.TagByID(SameNameTagID) + if elig.OwnerLogin != OtherLogin || len(elig.Use.Groups) != 1 || elig.Use.Groups[0] != AllUsersGroupID { + t.Errorf("eligibility tag = %+v, want owned by the other user and shared with All Users", elig) + } + if same.OwnerLogin != BotLogin || same.Name != EligibilityTagName || len(same.Use.Groups)+len(same.Read.Groups) != 0 { + t.Errorf("same-named tag = %+v, want owned by the bot and private", same) + } +} + +func TestCreateTagIsOwnedByTheBotAndConflictsOnlyWithItsOwn(t *testing.T) { + srv := NewServer(t) + srv.SeedTag("theirs", OtherLogin, Sharing{}, Sharing{}) + body := `{"name":"theirs","readSharingSettings":{"permittedGroups":[{"id":"` + AllUsersGroupID + `"}],"permittedUsers":[{"id":"` + OtherUserID + `"}]},"tagSharingSettings":{"permittedGroups":[],"permittedUsers":[{"id":"` + OtherUserID + `"}]}}` + status, _ := call(t, srv, http.MethodPost, "/api/tags?fields=id", body) + if status != http.StatusOK { + t.Fatalf("create = %d, want 200: another user's tag of the name must not block it", status) + } + made := srv.TagsNamed("theirs") + if len(made) != 2 || made[1].OwnerLogin != BotLogin { + t.Fatalf("tags = %+v", made) + } + if made[1].Read.Groups[0] != AllUsersGroupID || made[1].Read.Users[0] != OtherUserID || len(made[1].Use.Groups) != 0 || made[1].Use.Users[0] != OtherUserID { + t.Errorf("stored sharing = read %+v use %+v", made[1].Read, made[1].Use) + } + if status, _ := call(t, srv, http.MethodPost, "/api/tags?fields=id", `{"name":"theirs"}`); status != http.StatusConflict { + t.Errorf("duplicate of the bot's own tag = %d, want 409", status) + } + if status, _ := call(t, srv, http.MethodPost, "/api/tags?fields=id", `{"name":"x","tagSharingSettings":{"permittedGroups":[{"id":"675-99"}]}}`); status != http.StatusBadRequest { + t.Errorf("unknown group = %d, want 400", status) + } +} + +func TestGetTagAnswers404ForATagTheBotCannotSee(t *testing.T) { + srv := NewServer(t) + hidden := srv.SeedTag("hidden", OtherLogin, Sharing{}, Sharing{}) + if status, _ := call(t, srv, http.MethodGet, "/api/tags/"+hidden+"?fields=id", ""); status != http.StatusNotFound { + t.Errorf("hidden tag = %d, want 404", status) + } + status, body := call(t, srv, http.MethodGet, "/api/tags/"+EligibilityTagID+"?fields=id,owner(login)", "") + if status != http.StatusOK || !strings.Contains(body, OtherLogin) { + t.Errorf("eligibility tag = %d %s", status, body) + } + if status, body := call(t, srv, http.MethodGet, "/api/tags?fields=id", ""); status != http.StatusOK || strings.Contains(body, hidden) { + t.Errorf("tag list = %d %s, want it to omit the hidden tag", status, body) + } +} + +func TestAttachRefusesATagTheBotCannotUse(t *testing.T) { + srv := NewServer(t) + readOnly := srv.SeedTag("read-only", OtherLogin, Sharing{Groups: []string{AllUsersGroupID}}, Sharing{}) + viaTeam := srv.SeedTag("via-team", OtherLogin, Sharing{Groups: []string{AllUsersGroupID}}, Sharing{Groups: []string{TeamGroupID}}) + solo := srv.SeedTag("solo", OtherLogin, Sharing{Groups: []string{AllUsersGroupID}}, Sharing{Groups: []string{SoloGroupID}}) + n := srv.SeedIssue("t", "") + attach := func(id string) int { + status, _ := call(t, srv, http.MethodPost, "/api/issues/UZI-"+strconv.FormatInt(n, 10)+"/tags?fields=id", `{"id":"`+id+`"}`) + return status + } + for name, tc := range map[string]struct { + id string + want int + }{ + "read only": {readOnly, http.StatusForbidden}, + "group without me": {solo, http.StatusForbidden}, + "group with me": {viaTeam, http.StatusOK}, + "all users": {EligibilityTagID, http.StatusOK}, + "own tag": {SameNameTagID, http.StatusOK}, + } { + if got := attach(tc.id); got != tc.want { + t.Errorf("%s: attach = %d, want %d", name, got, tc.want) + } + } +} diff --git a/api/internal/issuetracker/youtracktest/state.go b/api/internal/issuetracker/youtracktest/state.go index 2042dbd360..0de03aa12e 100644 --- a/api/internal/issuetracker/youtracktest/state.go +++ b/api/internal/issuetracker/youtracktest/state.go @@ -2,6 +2,7 @@ package youtracktest import ( "fmt" + "slices" "strings" "time" ) @@ -15,9 +16,25 @@ const ( // target of MoveIssue. OtherProjectID = "0-2" OtherProjectShortName = "OTH" - // BotLogin is the account the bearer token authenticates as. + // BotLogin and BotID are the account the bearer token authenticates as. BotLogin = "uzi-bot" - botID = "1-1" + BotID = "1-1" + // OtherLogin and OtherUserID are a second user, who owns the default + // eligibility tag. + OtherLogin = "maintainer" + OtherUserID = "1-2" + // The fake's user groups. AllUsersGroupID covers every user; TeamGroupID is + // a named group of both users; SoloGroupID is a named group of the other + // user only. + AllUsersGroupID = "675-0" + TeamGroupID = "675-1" + SoloGroupID = "675-2" + // The default seed has an eligibility tag owned by the other user and shared + // with All Users, and a tag of the same name owned by the bot, shared with + // nobody. The ids are high so the ids the fake hands out never reach them. + EligibilityTagID = "5-9001" + EligibilityTagName = "uzi-eligible" + SameNameTagID = "5-9002" // DefaultStateField is the name of the State custom field. DefaultStateField = "State" ) @@ -27,8 +44,33 @@ type project struct { next int64 // last number handed out in this project } +type user struct{ id, login string } + +// group is a user group; members are user ids. An all-users group has every user. +type group struct { + id, name string + allUsers bool + members []string +} + +// fakeUsers and fakeGroups are the fixed identities of the fake. +var ( + fakeUsers = []user{{BotID, BotLogin}, {OtherUserID, OtherLogin}} + fakeGroups = []group{ + {id: AllUsersGroupID, name: "All Users", allUsers: true, members: []string{BotID, OtherUserID}}, + {id: TeamGroupID, name: "Team", members: []string{BotID, OtherUserID}}, + {id: SoloGroupID, name: "Solo", members: []string{OtherUserID}}, + } +) + +// sharing is who a tag is shared with, by group id and user id. +type sharing struct{ groups, users []string } + type tag struct { id, name string + owner string // user id + read sharing + use sharing } type comment struct { @@ -88,6 +130,13 @@ func newModel() *model { {id: ProjectID, shortName: ProjectShortName, name: "Uzi"}, {id: OtherProjectID, shortName: OtherProjectShortName, name: "Other"}, }, + // The bot's own tag comes first, so a driver that takes the first tag + // of a name picks it rather than the eligibility tag. + tags: []*tag{ + {id: SameNameTagID, name: EligibilityTagName, owner: BotID}, + {id: EligibilityTagID, name: EligibilityTagName, owner: OtherUserID, + read: sharing{groups: []string{AllUsersGroupID}}, use: sharing{groups: []string{AllUsersGroupID}}}, + }, aliases: map[string]*issue{}, states: []stateValue{ {id: "10-1", name: "Open", resolved: false, ordinal: 0}, @@ -138,16 +187,37 @@ func (m *model) tagByID(id string) *tag { return nil } -// tagByName returns the tag called name, creating it when missing. -func (m *model) ensureTag(name string) *tag { +// covers reports whether s reaches user id: the user is named, or is in a +// named group, or a group covers all users. +func covers(s sharing, userID string) bool { + if slices.Contains(s.users, userID) { + return true + } + for _, gid := range s.groups { + for _, g := range fakeGroups { + if g.id == gid && (g.allUsers || slices.Contains(g.members, userID)) { + return true + } + } + } + return false +} + +// visibleTo reports whether the user can see the tag. +func (t *tag) visibleTo(userID string) bool { return t.owner == userID || covers(t.read, userID) } + +// usableBy reports whether the user can apply the tag to an issue. +func (t *tag) usableBy(userID string) bool { return t.owner == userID || covers(t.use, userID) } + +// visibleTags returns the tags the bot can see. +func (m *model) visibleTags() []*tag { + var out []*tag for _, t := range m.tags { - if t.name == name { - return t + if t.visibleTo(BotID) { + out = append(out, t) } } - t := &tag{id: m.nextID(5), name: name} - m.tags = append(m.tags, t) - return t + return out } func (m *model) newIssue(p *project, summary, description string) *issue { @@ -227,7 +297,7 @@ func (m *model) issueJSON(i *issue) map[string]any { tags := make([]map[string]any, 0, len(i.tags)) for _, id := range i.tags { if t := m.tagByID(id); t != nil { - tags = append(tags, map[string]any{"id": t.id, "name": t.name, "$type": "Tag"}) + tags = append(tags, m.tagJSON(t)) } } var resolved any @@ -250,15 +320,61 @@ func (m *model) issueJSON(i *issue) map[string]any { } } +// userJSON is the wire user for a login. A login the fake does not know is +// given the bot's id, as comment authors such as "alice" always were. func userJSON(login string) map[string]any { - return map[string]any{"$type": "User", "id": botID, "login": login} + for _, u := range fakeUsers { + if u.login == login { + return map[string]any{"$type": "User", "id": u.id, "login": u.login} + } + } + return map[string]any{"$type": "User", "id": BotID, "login": login} +} + +func userJSONByID(id string) map[string]any { + for _, u := range fakeUsers { + if u.id == id { + return userJSON(u.login) + } + } + return map[string]any{"$type": "User", "id": id, "login": id} +} + +func (m *model) sharingJSON(s sharing, typ string) map[string]any { + groups := []map[string]any{} + for _, gid := range s.groups { + for _, g := range fakeGroups { + if g.id != gid { + continue + } + gt := "UserGroup" + if g.allUsers { + gt = "AllUsersGroup" + } + groups = append(groups, map[string]any{"$type": gt, "id": g.id, "name": g.name, "allUsersGroup": g.allUsers, "usersCount": len(g.members)}) + } + } + users := []map[string]any{} + for _, uid := range s.users { + users = append(users, userJSONByID(uid)) + } + return map[string]any{"$type": typ, "permittedGroups": groups, "permittedUsers": users} +} + +// tagJSON is the full wire object of a tag; callers project it through a fields spec. +func (m *model) tagJSON(t *tag) map[string]any { + return map[string]any{ + "$type": "Tag", "id": t.id, "name": t.name, "owner": userJSONByID(t.owner), + "readSharingSettings": m.sharingJSON(t.read, "WatchFolderSharingSettings"), + "tagSharingSettings": m.sharingJSON(t.use, "TagSharingSettings"), + } } func (m *model) tagsJSON(ids []string) []map[string]any { out := make([]map[string]any, 0, len(ids)) for _, id := range ids { if t := m.tagByID(id); t != nil { - out = append(out, map[string]any{"id": t.id, "name": t.name, "$type": "Tag"}) + out = append(out, m.tagJSON(t)) } } return out diff --git a/api/internal/issuetrackersvc/driver.go b/api/internal/issuetrackersvc/driver.go index df6ec6d8fd..d25aad5213 100644 --- a/api/internal/issuetrackersvc/driver.go +++ b/api/internal/issuetrackersvc/driver.go @@ -12,10 +12,13 @@ import ( // base URL form and the allowlist against the stored row BEFORE opening the // token, so removing a host from the allowlist stops use of existing rows // (fail closed). The token is opened with the row's user, kind and base URL as -// AAD, used only to build the driver, and never returned or logged. -func (s *Service) DriverFor(ctx context.Context, userID, connID uuid.UUID, project issuetracker.ProjectRef, stateField, open, closed string) (issuetracker.Driver, error) { - if issuetracker.ValidProjectRef(project) != nil { - return nil, &Error{Kind: KindBadRequest, Message: "invalid project reference"} +// AAD, used only to build the driver, and never returned or logged. binding is +// the empty issuetracker.Binding for a connection-level driver, or all of the +// project, the repository's forge project id and the eligibility tag; a partial +// or malformed one is refused before anything is read. +func (s *Service) DriverFor(ctx context.Context, userID, connID uuid.UUID, binding issuetracker.Binding, stateField, open, closed string) (issuetracker.Driver, error) { + if issuetracker.ValidBinding(binding) != nil { + return nil, &Error{Kind: KindBadRequest, Message: "invalid project binding"} } row, err := s.getRow(ctx, userID, connID) if err != nil { @@ -39,12 +42,14 @@ func (s *Service) DriverFor(ctx context.Context, userID, connID uuid.UUID, proje return nil, internal() } d, err := s.newDriver(kind, issuetracker.Options{ - BaseURL: norm, - Token: token, - Project: project, - StateField: stateField, - OpenState: open, - ClosedState: closed, + BaseURL: norm, + Token: token, + Project: binding.Project, + ProjectID: binding.ProjectID, + EligibilityTag: binding.EligibilityTag, + StateField: stateField, + OpenState: open, + ClosedState: closed, }) if err != nil { s.log.Error("build issue tracker driver", "kind", row.Kind, "error", err) @@ -55,7 +60,7 @@ func (s *Service) DriverFor(ctx context.Context, userID, connID uuid.UUID, proje // ListProjects lists the projects the connection's token can see. func (s *Service) ListProjects(ctx context.Context, userID, connID uuid.UUID) ([]issuetracker.Project, error) { - d, err := s.DriverFor(ctx, userID, connID, issuetracker.ProjectRef{}, "", "", "") + d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, "", "", "") if err != nil { return nil, err } @@ -71,7 +76,7 @@ func (s *Service) ListStates(ctx context.Context, userID, connID uuid.UUID, proj if !issuetracker.ValidProjectID(projectID) { return nil, &Error{Kind: KindBadRequest, Message: "invalid project id"} } - d, err := s.DriverFor(ctx, userID, connID, issuetracker.ProjectRef{}, "", "", "") + d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, "", "", "") if err != nil { return nil, err } @@ -81,3 +86,17 @@ func (s *Service) ListStates(ctx context.Context, userID, connID uuid.UUID, proj } return states, nil } + +// ListTags lists the tags the connection's token can see, with their owners and +// sharing settings. +func (s *Service) ListTags(ctx context.Context, userID, connID uuid.UUID) ([]issuetracker.Tag, error) { + d, err := s.DriverFor(ctx, userID, connID, issuetracker.Binding{}, "", "", "") + if err != nil { + return nil, err + } + tags, err := d.ListTags(ctx) + if err != nil { + return nil, &Error{Kind: KindBadGateway, Message: "could not list tags: " + err.Error()} + } + return tags, nil +} diff --git a/api/internal/issuetrackersvc/service_test.go b/api/internal/issuetrackersvc/service_test.go index b4a67e492e..dd3e8c9d98 100644 --- a/api/internal/issuetrackersvc/service_test.go +++ b/api/internal/issuetrackersvc/service_test.go @@ -92,6 +92,12 @@ type fakeDriver struct { verifyErr error projects []issuetracker.Project projectErr error + tags []issuetracker.Tag + tagErr error +} + +func (d *fakeDriver) ListTags(context.Context) ([]issuetracker.Tag, error) { + return d.tags, d.tagErr } func (d *fakeDriver) VerifyToken(context.Context) (issuetracker.Account, error) { @@ -420,15 +426,22 @@ func TestDriverForOpensTokenAndPassesOptions(t *testing.T) { t.Fatal(err) } h.built = nil - ref := issuetracker.ProjectRef{ID: "0-1", ShortName: "UZI"} - if _, err := h.svc.DriverFor(ctx, user, c.ID, ref, "State", "Open", "Done"); err != nil { + binding := issuetracker.Binding{ + Project: issuetracker.ProjectRef{ID: "0-1", ShortName: "UZI"}, + ProjectID: 4173, + EligibilityTag: issuetracker.TagRef{ID: "5-3", Name: "uzi"}, + } + if _, err := h.svc.DriverFor(ctx, user, c.ID, binding, "State", "Open", "Done"); err != nil { t.Fatal(err) } - want := issuetracker.Options{BaseURL: testHost, Token: tok, Project: ref, StateField: "State", OpenState: "Open", ClosedState: "Done"} + want := issuetracker.Options{ + BaseURL: testHost, Token: tok, Project: binding.Project, ProjectID: 4173, EligibilityTag: binding.EligibilityTag, + StateField: "State", OpenState: "Open", ClosedState: "Done", + } if len(h.built) != 1 || h.built[0] != want { t.Errorf("built = %+v, want %+v", h.built, want) } - _, err = h.svc.DriverFor(ctx, uuid.New(), c.ID, ref, "", "", "") + _, err = h.svc.DriverFor(ctx, uuid.New(), c.ID, binding, "", "", "") _ = wantKind(t, err, KindNotFound) } @@ -442,7 +455,7 @@ func TestDriverForFailsClosedWhenAllowlistEmptied(t *testing.T) { } h.built = nil h.allow = map[string]bool{} - _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.ProjectRef{}, "", "", "") + _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, "", "", "") _ = wantKind(t, err, KindBadRequest) if len(h.built) != 0 { t.Error("driver built after the allowlist was emptied") @@ -464,7 +477,7 @@ func TestDriverForRefusesTamperedRow(t *testing.T) { // A row whose base URL was rewritten no longer opens: the AAD differs. h.allow["https://evil.example.com"] = true h.q.rows[0].BaseUrl = "https://evil.example.com" - _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.ProjectRef{}, "", "", "") + _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, "", "", "") e := wantKind(t, err, KindInternal) if e.Message != "internal error" || len(h.built) != 0 { t.Errorf("message = %q, built = %d", e.Message, len(h.built)) @@ -472,7 +485,7 @@ func TestDriverForRefusesTamperedRow(t *testing.T) { // A row with a non-strict URL is refused before opening. h.allow["https://evil.example.com/x"] = true h.q.rows[0].BaseUrl = "https://evil.example.com/x" - _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.ProjectRef{}, "", "", "") + _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, "", "", "") _ = wantKind(t, err, KindBadRequest) } @@ -488,14 +501,14 @@ func TestDriverForRefusesNonCanonicalStoredURL(t *testing.T) { // Strict-valid and allowlisted, but not the canonical stored form. h.allow["https://ISSUES.example.com"] = true h.q.rows[0].BaseUrl = "https://ISSUES.example.com" - _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.ProjectRef{}, "", "", "") + _, err = h.svc.DriverFor(ctx, user, c.ID, issuetracker.Binding{}, "", "", "") _ = wantKind(t, err, KindBadRequest) if len(h.built) != 0 { t.Error("driver built for a non-canonical stored URL") } } -func TestDriverForRefusesMalformedProjectRef(t *testing.T) { +func TestDriverForRefusesPartialOrMalformedBinding(t *testing.T) { h := newHarness(t) ctx := context.Background() user := uuid.New() @@ -504,12 +517,33 @@ func TestDriverForRefusesMalformedProjectRef(t *testing.T) { t.Fatal(err) } h.built = nil - for _, ref := range []issuetracker.ProjectRef{{ID: "0-1"}, {ShortName: "UZI"}, {ID: "x", ShortName: "UZI"}, {ID: "0-1", ShortName: "1bad"}} { - _, err = h.svc.DriverFor(ctx, user, c.ID, ref, "", "", "") - _ = wantKind(t, err, KindBadRequest) + good := issuetracker.Binding{ + Project: issuetracker.ProjectRef{ID: "0-1", ShortName: "UZI"}, + ProjectID: 7, + EligibilityTag: issuetracker.TagRef{ID: "5-3", Name: "uzi"}, + } + mutate := func(f func(*issuetracker.Binding)) issuetracker.Binding { b := good; f(&b); return b } + for name, b := range map[string]issuetracker.Binding{ + "project id only": {ProjectID: 7}, + "eligibility only": {EligibilityTag: good.EligibilityTag}, + "project only": {Project: good.Project}, + "no project id": mutate(func(b *issuetracker.Binding) { b.ProjectID = 0 }), + "no eligibility": mutate(func(b *issuetracker.Binding) { b.EligibilityTag = issuetracker.TagRef{} }), + "no project": mutate(func(b *issuetracker.Binding) { b.Project = issuetracker.ProjectRef{} }), + "negative project id": mutate(func(b *issuetracker.Binding) { b.ProjectID = -1 }), + "half project": mutate(func(b *issuetracker.Binding) { b.Project.ID = "" }), + "bad project id": mutate(func(b *issuetracker.Binding) { b.Project.ID = "x" }), + "bad short name": mutate(func(b *issuetracker.Binding) { b.Project.ShortName = "1bad" }), + "bad tag id": mutate(func(b *issuetracker.Binding) { b.EligibilityTag.ID = "x" }), + "bad tag name": mutate(func(b *issuetracker.Binding) { b.EligibilityTag.Name = "a}b" }), + } { + t.Run(name, func(t *testing.T) { + _, err := h.svc.DriverFor(ctx, user, c.ID, b, "", "", "") + _ = wantKind(t, err, KindBadRequest) + }) } if len(h.built) != 0 { - t.Error("driver built for a malformed project reference") + t.Error("driver built for a partial or malformed binding") } } @@ -560,6 +594,30 @@ func TestListProjectsAndStates(t *testing.T) { _ = wantKind(t, err, KindNotFound) } +func TestListTags(t *testing.T) { + h := newHarness(t) + ctx := context.Background() + user := uuid.New() + c, err := h.svc.Connect(ctx, user, "youtrack", testHost, testToken("t")) + if err != nil { + t.Fatal(err) + } + h.built = nil + h.template.tags = []issuetracker.Tag{{ID: "5-1", Name: "uzi", Owner: issuetracker.Account{ID: "1-1", Login: "bot"}}} + got, err := h.svc.ListTags(ctx, user, c.ID) + if err != nil || len(got) != 1 || got[0].Name != "uzi" { + t.Fatalf("ListTags = %v, %v", got, err) + } + if len(h.built) != 1 || h.built[0].Project != (issuetracker.ProjectRef{}) || h.built[0].ProjectID != 0 || h.built[0].EligibilityTag != (issuetracker.TagRef{}) { + t.Errorf("ListTags built %+v, want one unbound driver", h.built) + } + h.template.tagErr = errors.New("boom") + _, err = h.svc.ListTags(ctx, user, c.ID) + _ = wantKind(t, err, KindBadGateway) + _, err = h.svc.ListTags(ctx, uuid.New(), c.ID) + _ = wantKind(t, err, KindNotFound) +} + func TestNewDefaultsToRegistry(t *testing.T) { s := New(&fakeQuerier{}, nil, nil, nil) if s.newDriver == nil { diff --git a/changelog.d/e1553f58-a876-4756-9a9a-fc8e104d4618.md b/changelog.d/e1553f58-a876-4756-9a9a-fc8e104d4618.md new file mode 100644 index 0000000000..a765715dd9 --- /dev/null +++ b/changelog.d/e1553f58-a876-4756-9a9a-fc8e104d4618.md @@ -0,0 +1,4 @@ +### Added + +- **An issue tracker allowlist in the Helm chart and compose (#4173).** + The Helm value `issueTracker.allowedBaseURLs`, with the compose `ISSUE_TRACKER_ALLOWED_BASE_URLS` pass-through, is empty by default, https-only and never used for worker egress; behaviour is unchanged for forges. diff --git a/deploy/chart/templates/_helpers.tpl b/deploy/chart/templates/_helpers.tpl index 24a133ed04..ead6f714b1 100644 --- a/deploy/chart/templates/_helpers.tpl +++ b/deploy/chart/templates/_helpers.tpl @@ -396,6 +396,30 @@ true {{- toJson $out }} {{- end }} +{{- /* + uzi.issueTrackerAllowedBaseURLs: issueTracker.allowedBaseURLs (PRD #4173 S3) as a JSON + list, validated fail-CLOSED like uzi.forgeEgressHosts: an entry must be an absolute https + URL with a host, mirroring what the api refuses at boot. Unlike the forge list there is + no IP-literal check, because nothing here derives an FQDN egress rule from it (a worker + never calls an issue tracker). + Consumers: `fromJsonArray (include "uzi.issueTrackerAllowedBaseURLs" .)`. +*/}} +{{- define "uzi.issueTrackerAllowedBaseURLs" -}} +{{- $out := list }} +{{- range (.Values.issueTracker).allowedBaseURLs }} +{{- /* Checked before urlParse: the list is joined with "," and the api splits on ",", so an entry holding one would smuggle in a second URL past the https check. */}} +{{- if regexMatch "[,\\s]" (toString .) }} +{{- fail (printf "issueTracker.allowedBaseURLs entry %q must not contain a comma or whitespace (the entries are joined with \",\" into ISSUE_TRACKER_ALLOWED_BASE_URLS; list each URL as its own entry)." .) }} +{{- end }} +{{- $u := urlParse . }} +{{- if or (not $u.hostname) (ne $u.scheme "https") }} +{{- fail (printf "issueTracker.allowedBaseURLs entry %q must be an absolute https URL with a host, e.g. https://tracker.example.com (it feeds ISSUE_TRACKER_ALLOWED_BASE_URLS; the api rejects the same input at boot)." .) }} +{{- end }} +{{- $out = append $out . }} +{{- end }} +{{- toJson $out }} +{{- end }} + {{- /* uzi.validateFQDNEgress: the shared preconditions for every FQDN-egress provider. An empty forge list would leave the api on its built-in default forge while the egress @@ -847,7 +871,7 @@ true = true. */ -}} {{- define "uzi.apiConfigMapEnabled" -}} -{{- if and .Values.api.enabled (or .Values.api.config .Values.forge.allowedBaseURLs .Values.releaseCheck (include "uzi.apiHostingEnabled" .) (include "uzi.dbStorageCapacityBytes" .)) -}} +{{- if and .Values.api.enabled (or .Values.api.config .Values.forge.allowedBaseURLs (.Values.issueTracker).allowedBaseURLs .Values.releaseCheck (include "uzi.apiHostingEnabled" .) (include "uzi.dbStorageCapacityBytes" .)) -}} true {{- end -}} {{- end -}} diff --git a/deploy/chart/templates/api-configmap.yaml b/deploy/chart/templates/api-configmap.yaml index ffd639f597..d8fc14e2fb 100644 --- a/deploy/chart/templates/api-configmap.yaml +++ b/deploy/chart/templates/api-configmap.yaml @@ -36,6 +36,25 @@ data: {{- end }} FORGE_ALLOWED_BASE_URLS: {{ join "," .Values.forge.allowedBaseURLs | quote }} {{- end }} + {{- /* + ISSUE_TRACKER_ALLOWED_BASE_URLS rendered EXPLICITLY from the first-class + issueTracker.allowedBaseURLs value (PRD #4173 S3): the issue tracker SSRF allowlist, + SEPARATE from the forge one and never an input to the worker egress FQDN list (a worker + never calls an issue tracker). Every entry is validated by + uzi.issueTrackerAllowedBaseURLs (https, with a host) before it is rendered. Empty (the + default) renders no key, so no issue tracker connection can be created. + + Setting BOTH issueTracker.allowedBaseURLs AND api.config.ISSUE_TRACKER_ALLOWED_BASE_URLS + would emit a DUPLICATE ConfigMap key (the range above already emitted the api.config + one), so that combination is a hard render failure, like FORGE_ALLOWED_BASE_URLS. + */ -}} + {{- $issueTrackerURLs := fromJsonArray (include "uzi.issueTrackerAllowedBaseURLs" .) }} + {{- if $issueTrackerURLs }} + {{- if hasKey .Values.api.config "ISSUE_TRACKER_ALLOWED_BASE_URLS" }} + {{- fail "Set the issue tracker allowlist in EITHER issueTracker.allowedBaseURLs OR api.config.ISSUE_TRACKER_ALLOWED_BASE_URLS, not both: both set would emit a duplicate ISSUE_TRACKER_ALLOWED_BASE_URLS ConfigMap key. Prefer issueTracker.allowedBaseURLs." }} + {{- end }} + ISSUE_TRACKER_ALLOWED_BASE_URLS: {{ join "," $issueTrackerURLs | quote }} + {{- end }} {{- /* UZI_RELEASE_CHECK_* rendered EXPLICITLY from the first-class releaseCheck values (PRD #836), NOT left to the api.config freeform map — so the chart's seed defaults diff --git a/deploy/chart/values.yaml b/deploy/chart/values.yaml index b952e27ada..e86f61e2f2 100644 --- a/deploy/chart/values.yaml +++ b/deploy/chart/values.yaml @@ -83,6 +83,17 @@ imageVerification: forge: allowedBaseURLs: [] +# --- issueTracker: the API's issue tracker SSRF allowlist (PRD #4173 S3) ----------- +# +# Renders into the api's ISSUE_TRACKER_ALLOWED_BASE_URLS env. It is SEPARATE from the +# forge allowlist above: https base URLs only, and empty (the default) means no issue +# tracker connection can be created. It never feeds the worker egress lists (forge.* does +# that), because the worker never calls an issue tracker (S5). Set this per-cluster instead +# of api.config.ISSUE_TRACKER_ALLOWED_BASE_URLS (setting BOTH fails the render — see +# api-configmap.yaml). +issueTracker: + allowedBaseURLs: [] + # --- releaseCheck: the periodic GitHub release-availability check (PRD #836) ------- # # The api periodically polls GitHub's releases API for vtmocanu/uzi and surfaces an diff --git a/docker-compose.yml b/docker-compose.yml index 49c2bd7ddc..340f2c723d 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -68,6 +68,9 @@ services: # Forge integration (PRD #2). Base URLs are an SSRF allowlist; the sync # engine polls each enabled repo on FORGE_POLL_INTERVAL. FORGE_ALLOWED_BASE_URLS: ${FORGE_ALLOWED_BASE_URLS:-https://gitlab.example.com} + # Issue tracker SSRF allowlist (PRD #4173 S3): separate from the forge one, https + # only, empty by default (no issue tracker connection can be created). Not worker egress. + ISSUE_TRACKER_ALLOWED_BASE_URLS: ${ISSUE_TRACKER_ALLOWED_BASE_URLS:-} FORGE_POLL_INTERVAL: ${FORGE_POLL_INTERVAL:-60s} FORGE_RECONCILE_EVERY: ${FORGE_RECONCILE_EVERY:-10} FORGE_HTTP_TIMEOUT: ${FORGE_HTTP_TIMEOUT:-15s} diff --git a/e2e/forge-fake/forge-fake.mjs b/e2e/forge-fake/forge-fake.mjs index e77bfe1492..bcd9c20af6 100644 --- a/e2e/forge-fake/forge-fake.mjs +++ b/e2e/forge-fake/forge-fake.mjs @@ -96,10 +96,21 @@ const EXPECT_PAT = process.env.FORGE_FAKE_EXPECT_PAT || ""; // FORGE_FAKE_ISSUE_CONTRACT=1 turns on the issue-tracker contract fidelity the // default e2e fake deliberately lacks: an in-memory label catalog on GitLab and // GitHub, issue state/description writes on all three dialects, GitHub's per-label -// add/remove routes, and the /_e2e/issues/{iid}/comments mutator. Unset, every +// add/remove routes, the /_e2e/issues/{iid}/comments mutator, and a 404 for a +// project id the fake does not serve (see missingContractProject). Unset, every // answer is byte-identical to before (e2e/phases/60-schedules-sweep.sh relies on // GitLab GET /labels answering [] and POST not persisting). const ISSUE_CONTRACT = process.env.FORGE_FAKE_ISSUE_CONTRACT === "1"; +// missingContractProject is true under ISSUE_CONTRACT for a project id the fake +// does not serve, so the issue tracker contract's ProjectGuard case sees a +// driver that names another project fail instead of landing on PROJECTS[0], the +// way resolveProject() maps an unknown id for every other caller. Without the +// flag it is always false, so the default behavior is unchanged. +function missingContractProject(idParam) { + if (!ISSUE_CONTRACT) return false; + const raw = decodeURIComponent(String(idParam || "")); + return !PROJECTS.some((p) => String(p.id) === raw || p.path_with_namespace === raw); +} // name -> {id, name, color}; only read or written when ISSUE_CONTRACT is on. const labelCatalog = /** @type {Record} */ ({}); let nextCatalogLabelId = 300; @@ -1169,6 +1180,7 @@ const server = https.createServer( const fjRepoById = path.match(/^\/api\/v1\/repositories\/(\d+)$/); if (method === "GET" && fjRepoById) { // GetRepoByID (the driver resolves a numeric project id -> owner/repo slug). + if (missingContractProject(fjRepoById[1])) return send(res, 404, { message: "The target couldn't be found." }); const project = resolveProject(fjRepoById[1]); const [owner, ...rest] = project.path_with_namespace.split("/"); return send(res, 200, { @@ -1520,6 +1532,7 @@ const server = https.createServer( // member=true (compliant, least-privilege). const ghRepoById = g.match(/^\/repositories\/(\d+)$/); if (method === "GET" && ghRepoById) { + if (missingContractProject(ghRepoById[1])) return send(res, 404, { message: "Not Found" }); const project = resolveProject(ghRepoById[1]); const [owner, ...rest] = project.path_with_namespace.split("/"); return send(res, 200, { @@ -1876,6 +1889,7 @@ const server = https.createServer( // the single-project behavior otherwise. const proj = path.match(/^\/api\/v4\/projects\/([^/]+)(\/.*)?$/); if (proj) { + if (missingContractProject(proj[1])) return send(res, 404, { message: "404 Project Not Found" }); const project = resolveProject(proj[1]); const rest = proj[2] || ""; diff --git a/e2e/forge-fake/issue-tracker-contracts.test.mjs b/e2e/forge-fake/issue-tracker-contracts.test.mjs index fe70dc15e4..b1874c2fb9 100644 --- a/e2e/forge-fake/issue-tracker-contracts.test.mjs +++ b/e2e/forge-fake/issue-tracker-contracts.test.mjs @@ -42,12 +42,15 @@ async function stop(child) { try { await closed; } finally { clearTimeout(timer); } } -// The only subtest that may skip: the forge subjects supply no MoveIssue seam, -// so the moved-issue case runs on YouTrack only. -const ALLOWED_SKIPS = new Set(["MovedIssueIsRefused"]); +// The only subtests that may skip: the forge subjects supply no MoveIssue seam, +// so the moved-issue case runs on YouTrack only, and forge labels have no +// owners, so the same-name tag case (Subject.SameName) does too. ProjectGuard +// must pass on every forge. +const ALLOWED_SKIPS = new Set(["MovedIssueIsRefused", "SameNameTagUsesEligibilityTag"]); const CASES = [ "EnsureLabelsIsIdempotent", "CreateIssueRoundTrips", "ListIssuesFilters", "UpdateLabelsAndDescription", "SetIssueStateClosesAndReopens", "CommentsOldestFirst", "LabelEventsOldestFirst", "MissingIssueIsNotFound", "MovedIssueIsRefused", + "ProjectGuard", "SameNameTagUsesEligibilityTag", ]; const DIALECTS = ["gitlab", "github", "forgejo"]; diff --git a/prds/4173-external-issue-trackers.md b/prds/4173-external-issue-trackers.md index 30a7d3d4ee..65090fef46 100644 --- a/prds/4173-external-issue-trackers.md +++ b/prds/4173-external-issue-trackers.md @@ -4,7 +4,7 @@ **Priority:** Medium -**Status:** In progress, r5 2026-11-11. M1 and M2 delivered. r3 followed the maintainer's scope review: one real issue tracker instead of abstract identity work, and the comment, command and withdrawal behaviours leave this PRD. r4 follows the maintainer's review of the M1 PR: the YouTrack id, tag and author decisions are made before the driver, and the issue source of a repo no longer changes. +**Status:** In progress, r6 2026-11-11. M1, M2 and M3 delivered. r3 narrowed scope to one real issue tracker; r4 fixed a repo's issue source before first enable (D13); r4 and r5 fixed the YouTrack id, tag and author decisions before the driver; r6 records the M3 rework that follows them. **Delivery:** one PR per milestone, each merged before the next starts. No implementation or validation step touches `.github/workflows/**`. @@ -136,7 +136,7 @@ Progress (one PR per milestone, each merged before the next starts): - [x] **M1:** the `IssueTracker` and `CodeHost` interfaces - [x] **M2:** every caller on its half. The type-checked inventory shows 45 issue-side calls through `IssueTracker`, 66 code-host calls through `CodeHost` and none through `forge.Forge`, and handing `privcheck` an `IssueTracker`-only value fails to compile. -- [ ] **M3:** issue tracker connections and the YouTrack driver +- [x] **M3:** issue tracker connections and the YouTrack driver. The contract suite, with the `ProjectGuard` and `SameNameTagUsesEligibilityTag` cases, runs on the real YouTrack driver and the three real forge drivers; the tag sharing settings parse is tested against a response recorded from a real YouTrack; the chart renders `issueTracker.allowedBaseURLs` and refuses a non-https entry. - [ ] **M4:** issue identity, the repo link and issue intake from YouTrack - [ ] **M5:** the board and issue writes on YouTrack - [ ] **M6:** web and CLI surfaces, docs and live acceptance @@ -223,14 +223,24 @@ Settings → Issue trackers, the CLI commands, a setup page under `docs/`. - **2026-11-11: the contract suite runs on the real drivers (D7), the allowlist is wired through the chart and compose in M3, and the ADR moves from M6 to M3.** - **2026-11-11: second review pass.** Every repo that exists at the M4 migration counts as previously enabled (D13). The driver refuses a `projectID` other than its own, and listing states is project-scoped (D10). The eligibility tag is checked through its see and use settings, read-only (D14). No number is derived from a comment id (D10). - **2026-11-11: the `projectID` guard is not the whole binding (D10).** `repos` is unique on `(connection_id, forge_project_id)`, so two connections can expose the same forge project number. The guard catches a mismatched number; the owner-scoped repo resolver binds the driver to the right repo and issue tracker connection, and M4 tests the collision. -- **2026-11-11: M3 was first implemented before the maintainer's review of this PRD.** The entries below are the M3 decisions that still hold. The driver does not yet follow D10 (project guard, no derived comment id) and D14 (eligibility tag by id), and the chart wiring and the ADR are missing; M3 stays unticked until that rework lands. -- **2026-11-11: M3 layout.** Drivers in `api/internal/issuetracker` (`Driver` = `forge.IssueTracker` plus `VerifyToken`, `ListProjects`, `ListStates`; one kind constant and one registry entry per issue tracker); the loud fake `issuetrackertest.BaseFake`, the contract suite `issuetrackertest.RunContract` and the in-process HTTPS YouTrack fake `youtracktest`; connections in `api/internal/issuetrackersvc`, not wired into `cmd/server` until a handler exists (M6). -- **2026-11-11: "the three forge fakes" are the Node fake `e2e/forge-fake/forge-fake.mjs`** (owner answer). The suite runs the real forge drivers against it in the `gate:repo` task `test:issue-tracker-forge-fake-contract`; the fake's extra issue routes answer only under `FORGE_FAKE_ISSUE_CONTRACT=1`. The moved-issue case skips there, because the forge subjects supply no way to move an issue; every other case must pass. +- **2026-11-11: the M3 rework landed.** M3 was first implemented before the maintainer's review of this PRD; the rework brings the driver in line with D10 and D14 and adds the S3 deployment wiring and the ADR (`adr/4173-issue-tracker-seam.md`). It replaces three earlier choices: a driver that ignored the `projectID` argument now refuses any other one; a comment id is no longer turned into a number (`CreateIssueNote` returns ID 0); and tags are no longer found or created by name alone, but resolved to ids as below. +- **2026-11-11: M3 layout.** Drivers in `api/internal/issuetracker`. `Driver` = `forge.IssueTracker` plus `VerifyToken`, `ListProjects`, `ListTags` (owner and read/use sharing settings per tag) and `ListStates`; one kind constant and one registry entry per issue tracker. A driver is bound by `issuetracker.Binding{Project, ProjectID, EligibilityTag}`: all three set, or none for a connection-level driver. Every `IssueTracker` method refuses another `projectID` with `ErrProjectMismatch` before any request (D10). The loud fake `issuetrackertest.BaseFake`, the contract suite `issuetrackertest.RunContract` and the in-process HTTPS YouTrack fake `youtracktest` ship alongside. Connections live in `api/internal/issuetrackersvc` (which also exposes `ListTags` for the M4 link check), not wired into `cmd/server` until a handler exists (M6). +- **2026-11-11: YouTrack label and id details that still hold from the first M3 implementation.** Labels are YouTrack tags; `ListLabels` lists the tags the token can see, once per name, and `Label.Color` is ignored. `UpdateIssueLabels` makes one REST call per tag (adds, then removes), so it is not atomic; the commands API is not used, so no label name enters a command query. YouTrack account ids are strings, so the numeric author ids are 0 ("unresolved") and the login is kept as text. `LabelEvent.ID` is the oldest-first ordinal of the tag events. +- **2026-11-11: "the three forge fakes" are the Node fake `e2e/forge-fake/forge-fake.mjs`** (owner answer). The suite runs the real forge drivers against it in the `gate:repo` task `test:issue-tracker-forge-fake-contract`; the fake's extra issue routes, and a 404 for a project it does not serve (which the `ProjectGuard` case relies on), answer only under `FORGE_FAKE_ISSUE_CONTRACT=1`. Two cases may skip there: `MovedIssueIsRefused`, because the forge subjects supply no way to move an issue, and `SameNameTagUsesEligibilityTag`, because forge labels have no owners. Every other case must pass. +- **2026-11-11: YouTrack tag names resolve to ids in one place in the driver (D14).** The eligibility name always resolves to the stored eligibility tag id, whoever owns the tag, and that tag is never created. Any other name resolves only to a tag owned by the connection's account; a missing one is created with `POST /api/tags`, carrying the eligibility tag's read and use sharing settings as read from `GET /api/tags/{id}`, so whoever can see and apply the eligibility tag can see and apply uzi's other tags. The eligibility tag is never a candidate for another name, even when its live YouTrack name has drifted to that name, because name matching skips the eligibility tag's id; when the account itself owns the eligibility tag, YouTrack refuses the duplicate name (409) and the write fails instead of tagging the issue eligible. `UpdateIssueLabels` decides "already on the issue" by id, and a removal resolves without creating. `CreateIssue` resolves its tags before creating the issue, so a resolution failure leaves no issue behind; a failure while attaching a tag or reading the issue back after the create still leaves the issue created, with no rollback. The stored eligibility name is what a caller names to mean the eligibility tag, a narrowing of D14's "name for display only": on a write the id decides which tag is used. If the eligibility tag is renamed in YouTrack, two things follow. Reads report each tag they return under its live YouTrack name, so an eligible issue reads back under the new name. And `ListIssues` with the stored name can miss eligible issues: the server-side `tag: {name}` narrowing matches live names (ignoring case), and the id filter only sees what the server returned, so an eligible issue comes back only if one of its tags still has a live name equal to the stored name. M4 must match eligibility by id, never by a read label name, and must decide whether to drop the eligibility name from the server query (filter by id only) or refuse a renamed tag at sync. +- **2026-11-11: tag ownership is judged against the account from `users/me`,** read once per driver and kept for its lifetime (a failed lookup is not kept), never against a stored id: the token decides who uzi acts as (D14). +- **2026-11-11: reads drop a tag that only shares the eligibility name (D14 "Matching").** Issue reads and label events leave out a tag whose name equals the eligibility name but whose id is not the eligibility tag's, so a same-named tag of another user never looks like eligibility to a caller. +- **2026-11-11: the D14 link-time rule is two pure functions, `AccountCanApply` and `OthersCanApply`, over a tag from `ListTags`.** Taking a listed tag supplies the "can see" half, because the token lists only tags it can see. Readings made within D14: + - *The owner counts for others too.* D14 says the account is covered when "it owns the tag"; YouTrack always lets an owner apply their own tag, so an owner other than the account counts as someone else who can apply it. + - *A named group does not count for the account,* because a tag's settings do not list the group's members, so the account's membership is unknown (fail closed). + - *A named group counts for others only with `usersCount` of at least 2,* since a group of one may be the account itself. The anonymous response recorded from the public YouTrack carries no `usersCount`; the one recorded with a permanent token does carry it, on the All Users group. Whether a named group carries it too is not recorded yet; if it does not, a tag shared only with a named group is refused at link time (an M6 live-pass question). The M4 link check and the M6 link screen should tell the admin to share the tag with named users or All Users. +- **2026-11-11: the sharing settings parse is tested against a real recording.** `api/internal/issuetracker/testdata/youtrack_tags_recorded.json` is an anonymous `GET /api/tags` from the public youtrack.jetbrains.com, anonymised (logins, names and ids replaced) with its structure, `$type` members, booleans and the absent `usersCount` kept as recorded (M3 acceptance). `youtrack_tags_recorded_private.json` is the same request recorded with a permanent token on a private instance and anonymised the same way (the member count too); it adds the shapes the public one lacks: `usersCount`, an all-users group in the use settings, a tag only its owner can apply and a tag only its owner can see. +- **2026-11-11: the issue tracker allowlist is the chart value `issueTracker.allowedBaseURLs` (S3).** It renders `ISSUE_TRACKER_ALLOWED_BASE_URLS` into the API ConfigMap, is empty by default (no key rendered), and never feeds the worker egress lists (S5). The render refuses an entry that is not https, has no host, or contains a comma or whitespace (entries are comma-joined), and refuses the value set both there and in `api.config`. The check is `scripts/assert-issue-tracker-allowlist-render.sh`, task `render:issue-tracker-allowlist-check`, also run by `render:openshift-check`. `docker-compose.yml` passes the variable through, empty by default; the VPS compose file `deploy/compose/compose.yaml` does not yet, and is left for M6's setup docs. - **2026-11-11: the issue number is YouTrack's `numberInProject` (D10's candidate), and every per-issue call first checks the issue is still in the bound project.** YouTrack still resolves an old `SHORT-n` after a move, so a moved issue reads as not found and is never written to or read. YouTrack has no conditional write, so a move between that check and the write is not caught. M4 still owns the identity storage. -- **2026-11-11: how `ListIssues` queries YouTrack.** The query is `project: {SHORT}`, one `tag: {name}` per label (AND), `#Unresolved` or `#Resolved` for the state, and for `UpdatedAfter` an `updated:` range from the UTC date one day early, sorted by `updated asc`. The driver then re-filters client-side on exact tag names, the exact `updated` time and the bound project. A label name holding `{`, `}` or a control or invisible character is refused with `ErrInvalidLabelName`, so no name can break out of the query. +- **2026-11-11: how `ListIssues` queries YouTrack.** The query is `project: {SHORT}`, one `tag: {name}` per label (AND), `#Unresolved` or `#Resolved` for the state, and for `UpdatedAfter` an `updated:` range from the UTC date one day early, sorted by `updated asc`. The driver first resolves each label to a tag id without creating (see the resolver entry above); a non-eligibility name with no account-owned tag returns no issues. It then re-filters client-side by those resolved tag ids, not by names, so a same-named tag of another user does not match; also on the exact `updated` time and the bound project. A label name holding `{`, `}` or a control or invisible character is refused with `ErrInvalidLabelName`, so no name can break out of the query. - **2026-11-11: closing and reopening use explicit State values from the driver options;** unset, `SetIssueState` returns `ErrStateMappingUnset` and never guesses. M5 owns the column and state mapping and the "never won't fix" rule. - **2026-11-11: issue tracker base URLs are scheme and host only,** https, ASCII host, no path other than `/`, no userinfo, query or fragment (refused, not stripped), so a YouTrack under a context path is not supported yet. The allowlist is `ISSUE_TRACKER_ALLOWED_BASE_URLS`, empty by default; it is documented with the M6 setup page. `Connect` checks the strict form before the allowlist, so the allowlist is always compared against the canonical value that is stored. - **2026-11-11: the token is sealed with AAD `uzi/issue-tracker-token/v1|||`,** so a ciphertext copied to another row does not open; the canonical base URL is frozen at v1 for that reason. The token must be 16 to 4096 bytes with no control or invisible character. There is no least-privilege check like the forge's: YouTrack permanent tokens expose no scopes. A credential reset does not delete issue tracker tokens, as for forge connections; recorded as a residual risk. `DriverFor` re-checks the allowlist before opening a stored token. - **2026-11-11: a hostile or compromised allowlisted YouTrack cannot make the driver allocate far beyond what it sent.** Responses are capped (8 MiB a list page, 2 MiB an object, 4 KiB an error body, 64 MiB a listing); a page holds at most the requested 100 items; nested tags are capped at 1000 per item and count toward the listing's item ceiling; a byte scan bounds the JSON values before decoding; list pages decode one element at a time and stop at the first error. The worst case found is about 34 MiB for an 8 MiB page, against about 13 MiB for the largest legitimate one (measured in the security review). -- **2026-11-11: left for M6's live pass** (taken from YouTrack's documentation, checked only against the fake): the `TagsCategory` activity category, the `updated:` query date format, that tag names can repeat across owners, and whether YouTrack accepts tags on issue create (the driver adds them after create, so it does not depend on it). +- **2026-11-11: left for M6's live pass** (taken from YouTrack's documentation, checked only against the fake): the `TagsCategory` activity category, the `updated:` query date format, that tag names can repeat across owners, whether YouTrack accepts tags on issue create (the driver adds them after create, so it does not depend on it), whether YouTrack accepts sharing settings in the `POST /api/tags` body, whether an authenticated token receives `usersCount` on a tag's sharing groups, and whether `tag: {name}` in a query matches every visible tag of that name (the client-side id filter depends on the server returning a superset). - **2026-11-11: migration `00424_issue_tracker_connections`** is the next free number on `prd/ws-2`; it is renumbered at upstream landing. Rolling it back deletes every stored issue tracker token. diff --git a/scripts/assert-issue-tracker-allowlist-render.sh b/scripts/assert-issue-tracker-allowlist-render.sh new file mode 100755 index 0000000000..e4351be54c --- /dev/null +++ b/scripts/assert-issue-tracker-allowlist-render.sh @@ -0,0 +1,151 @@ +#!/bin/sh +# Assert the chart renders the api's ISSUE_TRACKER_ALLOWED_BASE_URLS (the issue tracker SSRF +# allowlist, PRD #4173 S3) from the first-class issueTracker.allowedBaseURLs value. +# +# usage: scripts/assert-issue-tracker-allowlist-render.sh [chart-dir] +# +# WHY THIS EXISTS. The issue tracker allowlist is SEPARATE from the forge one: it is https +# only, empty by default (no issue tracker connection can be created), and must NEVER feed the +# worker egress lists, because a worker never calls an issue tracker (PRD #4173 S5). A +# guard that stopped firing, a lost key or an issue tracker host leaking into the FQDN egress +# rules would render and lint clean, so every property is asserted on the parsed render. +# +# OFFLINE, AND NON-VACUOUS. `helm template` against a temp copy of the chart with the CNPG +# subchart dependency stripped. A render that must succeed but fails, or a failing-render +# case that fails for a different reason than the one asserted, is a BROKEN INSTRUMENT +# (exit 2), never a pass. +# +# EXIT CODES (this script deliberately diverges from assert-db-capacity-render.sh: a +# failing-render case that fails with the WRONG message is exit 2 here, not 1, because it +# means the guard under test was not the thing that fired, so the instrument cannot +# vouch for it; a property that simply does not hold is exit 1): +# 2 = the instrument is broken (helm/yq/chart missing, a render that must succeed +# failed, an expected key absent, or a render failed for an unrelated reason) +# 1 = a property does not hold +# 0 = every property holds +set -eu + +CDPATH='' +SCRIPT_DIR=$(cd -- "$(dirname -- "$0")" && pwd) +CHART_DIR="${1:-$SCRIPT_DIR/../deploy/chart}" + +command -v helm >/dev/null 2>&1 || { echo "BROKEN: helm not on PATH" >&2; exit 2; } +command -v yq >/dev/null 2>&1 || { echo "BROKEN: yq (mikefarah v4) not on PATH" >&2; exit 2; } +[ -f "$CHART_DIR/Chart.yaml" ] || { echo "BROKEN: no Chart.yaml under $CHART_DIR" >&2; exit 2; } + +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT INT TERM + +CHART="$WORK/chart" +cp -R "$CHART_DIR" "$CHART" +rm -f "$CHART/Chart.lock" +rm -rf "$CHART/charts" +awk ' + BEGIN { skip = 0 } + /^dependencies:/ { skip = 1; next } + skip && /^[^[:space:]-]/ { skip = 0 } + skip { next } + { print } +' "$CHART_DIR/Chart.yaml" > "$CHART/Chart.yaml" + +fail=0 +ok() { echo "OK: $1"; } +bad() { echo "FAIL: $1" >&2; fail=1; } + +# render