Skip to content

Commit 737a9cc

Browse files
committed
docs: address large tag repositories in version plan
1 parent 9be4547 commit 737a9cc

1 file changed

Lines changed: 128 additions & 2 deletions

File tree

docs/plans/2026-05-16-semantic-versioning-script.md

Lines changed: 128 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,62 @@ python3 semantic-version/semantic_version_bumper.py \
8383

8484
Default remains `'{module}/v{version}'` so the behavior is predictable for monorepos.
8585

86+
### Large tag repository strategy
87+
88+
A repository with hundreds of thousands or millions of tags must not run `git fetch --tags` in every pipeline. The script should support a GitLab API tag source so CI can query only relevant tag names instead of downloading all tag refs and tag objects.
89+
90+
Recommended source priority:
91+
92+
1. `gitlab-api` in GitLab CI for large repositories.
93+
2. `ls-remote` for generic remote-only tag discovery when the GitLab API is not available.
94+
3. `git` local tags only for small repositories or local developer runs.
95+
96+
GitLab API query shape:
97+
98+
```text
99+
GET /projects/:id/repository/tags?search=^<module>/v&order_by=version&sort=desc&per_page=100
100+
```
101+
102+
Important notes:
103+
104+
- URL-encode the project id/path and the `search` value.
105+
- GitLab's tag API supports `search` with `^term` for prefix matching and supports `order_by=version` for semantic-version ordering.
106+
- Still parse and validate versions client-side because tags may include invalid names, non-SemVer tags, or tags from older conventions.
107+
- For RC generation, query the narrowest prefix possible, for example `^token-rotate/v1.5.0-rc.`.
108+
- If a module has more than one page of matching tags, follow pagination headers or keyset pagination. Do not assume the first page is sufficient unless the query prefix is narrow enough to prove it.
109+
110+
GitLab CI should keep checkout shallow and prevent runner auto-fetching tags:
111+
112+
```yaml
113+
variables:
114+
GIT_DEPTH: "1"
115+
GIT_FETCH_EXTRA_FLAGS: "--no-tags"
116+
```
117+
118+
Then calculate versions from the GitLab API:
119+
120+
```bash
121+
python3 semantic-version/semantic_version_bumper.py \
122+
--module token-rotate \
123+
--tag-source gitlab-api \
124+
--branch "$CI_COMMIT_REF_NAME" \
125+
--write-env version.env
126+
```
127+
128+
Fallback if API access is not available and Git CLI must be used:
129+
130+
```bash
131+
git fetch --no-tags origin '+refs/tags/token-rotate/v*:refs/tags/token-rotate/v*'
132+
```
133+
134+
This fetches only the module-scoped tag namespace. For RC branches, fetch an even narrower namespace:
135+
136+
```bash
137+
git fetch --no-tags origin '+refs/tags/token-rotate/v1.5.0-rc.*:refs/tags/token-rotate/v1.5.0-rc.*'
138+
```
139+
140+
Avoid branch reachability checks such as `git tag --merged` in the first version because they require commit history and defeat shallow-checkout performance. Tags are global refs, not branch-local refs. If the business rule needs branch-specific versions, encode the release line into the tag prefix or branch name instead of trying to infer it from commit reachability.
141+
86142
### Branch/mode strategy
87143

88144
Support both auto-detection and explicit mode:
@@ -183,7 +239,15 @@ Options:
183239
Write CI dotenv output.
184240
185241
--fetch-tags
186-
Run git fetch --tags before reading tags.
242+
Run git fetch --tags before reading tags. Intended only for small repos.
243+
244+
--tag-source git|gitlab-api|ls-remote
245+
Tag discovery source. Default: git for local runs, gitlab-api when GitLab CI
246+
variables are present. Large repositories should use gitlab-api.
247+
248+
--gitlab-token-var NAME
249+
CI/CD variable that contains a GitLab API token. Default: CI_JOB_TOKEN,
250+
falling back to GITLAB_TOKEN. Never print this value.
187251
188252
--repo PATH
189253
Repository path. Default: current working directory.
@@ -583,7 +647,69 @@ git commit -m "feat: discover semantic version tags from git"
583647

584648
---
585649

586-
### Task 5: Implement patch version calculation
650+
### Task 5: Add GitLab API tag discovery for large repositories
651+
652+
**Objective:** Query only module-relevant tags from GitLab without fetching all tags into the job workspace.
653+
654+
**Files:**
655+
- Modify: `semantic-version/semantic_version_bumper.py`
656+
- Modify: `semantic-version/test_semantic_version_bumper.py`
657+
658+
**Step 1: Add failing tests with mocked HTTP responses**
659+
660+
Use `unittest.mock` to patch `urllib.request.urlopen` and verify:
661+
662+
- the request URL contains `/projects/<id>/repository/tags`
663+
- the query string includes `search=%5Etoken-rotate%2Fv`
664+
- the query string includes `order_by=version`, `sort=desc`, and `per_page=100`
665+
- returned tag names are extracted from JSON
666+
- pagination is followed when GitLab returns a `Link` or `X-Next-Page` header
667+
668+
**Step 2: Implement `list_gitlab_tags(api_url, project_id, token, search_prefix)`**
669+
670+
Use only standard-library modules:
671+
672+
- `urllib.request`
673+
- `urllib.parse`
674+
- `json`
675+
676+
Authentication behavior:
677+
678+
- Prefer `JOB-TOKEN` when using `CI_JOB_TOKEN`.
679+
- Use `PRIVATE-TOKEN` when using a project/group/personal access token variable such as `GITLAB_TOKEN`.
680+
- Never print the token or response bodies that could include sensitive data.
681+
682+
**Step 3: Add source selection**
683+
684+
Implement `--tag-source git|gitlab-api|ls-remote`.
685+
686+
Default logic:
687+
688+
```text
689+
if CI_API_V4_URL and CI_PROJECT_ID are present:
690+
tag_source = gitlab-api
691+
else:
692+
tag_source = git
693+
```
694+
695+
**Step 4: Verify**
696+
697+
```bash
698+
python -m unittest semantic-version/test_semantic_version_bumper.py -v
699+
```
700+
701+
Expected: PASS.
702+
703+
**Step 5: Commit**
704+
705+
```bash
706+
git add semantic-version/semantic_version_bumper.py semantic-version/test_semantic_version_bumper.py
707+
git commit -m "feat: query module tags through GitLab API"
708+
```
709+
710+
---
711+
712+
### Task 6: Implement patch version calculation
587713

588714
**Objective:** Generate the next patch version for the selected module.
589715

0 commit comments

Comments
 (0)