Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/secret-scan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:

- name: TruffleHog OSS
id: trufflehog
uses: trufflesecurity/trufflehog@v3.97.1
uses: trufflesecurity/trufflehog@v3.97.4
with:
path: ./
base: ${{ steps.scan_range.outputs.base }}
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
## 0.9.5 - Unreleased

- Stop review-thread GraphQL pagination when GitHub returns an empty or repeated endCursor. Thanks @SebTardif.
- Raise the source-build minimum to Go 1.27.1 for CrawlKit v0.14.8 and align the Docker builder; newly built macOS binaries require macOS 13 Ventura or newer.
- Update SQLite to v1.58.0, compatible Go dependency patches, the Dockerfile frontend to 1.27, and TruffleHog to v3.97.4; retain SQLite's required libc v1.75.6.

## 0.9.4 - 2026-08-30

Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:1.26
# syntax=docker/dockerfile:1.27

ARG GO_VERSION=1.26.7
ARG GO_VERSION=1.27.1
ARG ALPINE_VERSION=3.24

FROM golang:${GO_VERSION}-alpine AS build
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Homebrew is the smallest install on macOS and Linux:
brew install openclaw/tap/gitcrawl
```

Prebuilt archives for macOS, Linux, and Windows are available from [GitHub Releases](https://github.com/openclaw/gitcrawl/releases/latest). The [installation guide](docs/installation.md) also covers source builds and update checks; source builds require Go 1.26.7 or newer. A [Docker source build](docs/docker.md) keeps its runtime state under one mounted directory.
Prebuilt archives for macOS, Linux, and Windows are available from [GitHub Releases](https://github.com/openclaw/gitcrawl/releases/latest). The [installation guide](docs/installation.md) also covers source builds and update checks; source builds require Go 1.27.1 or newer for CrawlKit v0.14.8. Binaries built with this toolchain require macOS 13 Ventura or newer. A [Docker source build](docs/docker.md) keeps its runtime state under one mounted directory.

Sync needs a GitHub token from `GITHUB_TOKEN` or `gh auth token`. Generating summaries and embeddings also needs `OPENAI_API_KEY`; semantic search and clustering use those stored embeddings, while ordinary sync and keyword search do not.

Expand Down
8 changes: 6 additions & 2 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,17 @@ permalink: /installation/

## Requirements

- **Go 1.26.7+** if building from source
- **Go 1.27.1+** if building from source
- **Git** for cloning the repository (and for portable stores)
- **A GitHub token** for any command that talks to GitHub (`sync`, `refresh`)
- **An OpenAI API key** for `summarize`, `embed`, and the embedding stage of `refresh`
- **`gh` CLI** optional, used by gitcrawl only as a token source via `gh auth token`

gitcrawl runs on macOS and Linux. Windows is not actively tested.
gitcrawl runs on macOS 13 Ventura or newer and Linux. Windows is not actively tested.

CrawlKit v0.14.8 requires Go 1.27, so source and Docker builds use Go 1.27.1
or newer. Go 1.27 also raises the minimum macOS version for newly built binaries
to macOS 13; the previous Go 1.26 build baseline no longer applies.

## Install from Homebrew

Expand Down
13 changes: 7 additions & 6 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/openclaw/gitcrawl

go 1.26.7
go 1.27.1

require (
github.com/alecthomas/kong v1.16.1
Expand All @@ -9,10 +9,10 @@ require (
github.com/charmbracelet/lipgloss v1.1.0
github.com/charmbracelet/x/ansi v0.11.8
github.com/mattn/go-isatty v0.0.24
github.com/openclaw/crawlkit v0.14.7
github.com/openclaw/crawlkit v0.14.8
github.com/zalando/go-keyring v0.2.8
golang.org/x/sys v0.47.0
modernc.org/sqlite v1.57.0
modernc.org/sqlite v1.58.0
)

require (
Expand All @@ -21,19 +21,19 @@ require (
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/charmbracelet/colorprofile v0.4.3 // indirect
github.com/charmbracelet/x/cellbuf v0.0.15 // indirect
github.com/charmbracelet/x/exp/golden v0.0.0-20260823001701-96af6d2cb5f6 // indirect
github.com/charmbracelet/x/exp/golden v0.0.0-20260902165432-6f6ad8b37b0a // indirect
github.com/charmbracelet/x/term v0.2.2 // indirect
github.com/clipperhouse/displaywidth v0.11.0 // indirect
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
github.com/danieljoos/wincred v1.2.3 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/godbus/dbus/v5 v5.2.2 // indirect
github.com/google/pprof v0.0.0-20260825171938-4d453200e7d9 // indirect
github.com/google/pprof v0.0.0-20260903180319-d6c3cb2f37ec // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/lucasb-eyer/go-colorful v1.4.1 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.28 // indirect
github.com/mattn/go-runewidth v0.0.29 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.16.0 // indirect
Expand All @@ -47,6 +47,7 @@ require (
golang.org/x/mod v0.40.0 // indirect
golang.org/x/text v0.41.0 // indirect
golang.org/x/tools v0.49.0 // indirect
// Match the exact libc version required by modernc.org/sqlite's generated runtime.
modernc.org/libc v1.75.6 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
Expand Down
20 changes: 10 additions & 10 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ github.com/charmbracelet/x/ansi v0.11.8 h1:JMFwp0CgDC2+jcOB162HH5k7I3FVbgFSMMYg7
github.com/charmbracelet/x/ansi v0.11.8/go.mod h1:ZNN+3mXny/516oTQPLMPIBeSINvNJJQ8uQXDgbeJxY0=
github.com/charmbracelet/x/cellbuf v0.0.15 h1:ur3pZy0o6z/R7EylET877CBxaiE1Sp1GMxoFPAIztPI=
github.com/charmbracelet/x/cellbuf v0.0.15/go.mod h1:J1YVbR7MUuEGIFPCaaZ96KDl5NoS0DAWkskup+mOY+Q=
github.com/charmbracelet/x/exp/golden v0.0.0-20260823001701-96af6d2cb5f6 h1:PiWBtjCI6AGfSsy3X+PfGtR9hEl2C9xCOU6Yxr49mAU=
github.com/charmbracelet/x/exp/golden v0.0.0-20260823001701-96af6d2cb5f6/go.mod h1:6fMpcW6iwN/kX+xJ52eqVWsDiBTe0UJD24JLoHFe+P0=
github.com/charmbracelet/x/exp/golden v0.0.0-20260902165432-6f6ad8b37b0a h1:CjHY9AFdfVrowqAcObH3sj7pSZmfyi6Sa8VfVt4TGv8=
github.com/charmbracelet/x/exp/golden v0.0.0-20260902165432-6f6ad8b37b0a/go.mod h1:6fMpcW6iwN/kX+xJ52eqVWsDiBTe0UJD24JLoHFe+P0=
github.com/charmbracelet/x/term v0.2.2 h1:xVRT/S2ZcKdhhOuSP4t5cLi5o+JxklsoEObBSgfgZRk=
github.com/charmbracelet/x/term v0.2.2/go.mod h1:kF8CY5RddLWrsgVwpw4kAa6TESp6EB5y3uxGLeCqzAI=
github.com/clipperhouse/displaywidth v0.11.0 h1:lBc6kY44VFw+TDx4I8opi/EtL9m20WSEFgwIwO+UVM8=
Expand All @@ -38,8 +38,8 @@ github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=
github.com/godbus/dbus/v5 v5.2.2 h1:TUR3TgtSVDmjiXOgAAyaZbYmIeP3DPkld3jgKGV8mXQ=
github.com/godbus/dbus/v5 v5.2.2/go.mod h1:3AAv2+hPq5rdnr5txxxRwiGjPXamgoIHgz9FPBfOp3c=
github.com/google/pprof v0.0.0-20260825171938-4d453200e7d9 h1:dl4UZiszMU+NKHirOiCKTC+hRuNAQ0moHPxSg6WcU1o=
github.com/google/pprof v0.0.0-20260825171938-4d453200e7d9/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/pprof v0.0.0-20260903180319-d6c3cb2f37ec h1:ZhR4jlKRUmzPEPd2RLiJRUi25rUrdKBrhQNVIbqpTNE=
github.com/google/pprof v0.0.0-20260903180319-d6c3cb2f37ec/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
Expand All @@ -52,8 +52,8 @@ github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsRe
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2JC/oIi4=
github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88=
github.com/mattn/go-runewidth v0.0.28 h1:rPyg2ybwEKPebvpzVWe1gKBkH8EQFkxO4Y0hjBeLaBU=
github.com/mattn/go-runewidth v0.0.28/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8=
github.com/mattn/go-runewidth v0.0.29 h1:3oGF3R/S2N9DQ3ptftzVIvg2eicmojCzlwBEmqEPDfQ=
github.com/mattn/go-runewidth v0.0.29/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 h1:ZK8zHtRHOkbHy6Mmr5D264iyp3TiX5OmNcI5cIARiQI=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6/go.mod h1:CJlz5H+gyd6CUWT45Oy4q24RdLyn7Md9Vj2/ldJBSIo=
github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA=
Expand All @@ -62,8 +62,8 @@ github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc
github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/openclaw/crawlkit v0.14.7 h1:Kvceob5WyEtrGt4YkwKRcepU77g7FX/HPTRIiCe+ArQ=
github.com/openclaw/crawlkit v0.14.7/go.mod h1:iDnctBAFtqB5l2sHREfpyjQwifToeebDVdVJvZWVLn0=
github.com/openclaw/crawlkit v0.14.8 h1:JdmvIcfznzGGDNczACtktebd0ueLWaTrCwW8ZPtm7Fg=
github.com/openclaw/crawlkit v0.14.8/go.mod h1:7QxbJgGm6ZQAiBXyWSR+U6Eyt8p4pMcC7zjY0h3GKjY=
github.com/pelletier/go-toml/v2 v2.4.3 h1:GTRvJQutkOSftxIFD5xw9aepkYNuPWmVJpffdDPYVpY=
github.com/pelletier/go-toml/v2 v2.4.3/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
Expand Down Expand Up @@ -115,8 +115,8 @@ modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.57.0 h1:qNQP6xnx5M0ISNtlnxoOX0+cD5bJ0/gr9aMmndFczzg=
modernc.org/sqlite v1.57.0/go.mod h1:yCJ2cmAaIkHQ25oXWrF8H4O1lIfPYPR26yCEDj2P3pQ=
modernc.org/sqlite v1.58.0 h1:38u40/bwkfM7f0Myhosl+SEMltSDxnGdQf8o6Kjmys0=
modernc.org/sqlite v1.58.0/go.mod h1:rsD2CckafgObKC4DhBlGBf+RiHxkc3hINGt1Xw32tVY=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
Expand Down
20 changes: 10 additions & 10 deletions internal/cli/cloud_commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -715,16 +715,16 @@ func sendSnapshotIngestRows(
if !fits {
if end == start {
return ingestProgress{
RowsAccepted: total,
MutationToken: mutationToken,
Result: lastResult,
}, fmt.Errorf(
"ingest table %s row %d encoded request is %d bytes, limit %d",
table,
start,
encodedBytes,
gitcrawlCloudIngestRequestMaxBytes,
)
RowsAccepted: total,
MutationToken: mutationToken,
Result: lastResult,
}, fmt.Errorf(
"ingest table %s row %d encoded request is %d bytes, limit %d",
table,
start,
encodedBytes,
gitcrawlCloudIngestRequestMaxBytes,
)
}
break
}
Expand Down
Loading