Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/build-verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ on:
- 'settings-advisor/**'
- 'go-lsp/**'
- 'worktree-manager/**'
- 'test-runner/**'
- 'build.sh'
- 'build.ps1'
- '.go-version'
Expand All @@ -29,6 +30,7 @@ on:
- 'settings-advisor/**'
- 'go-lsp/**'
- 'worktree-manager/**'
- 'test-runner/**'
- 'build.sh'
- 'build.ps1'
- '.go-version'
Expand Down Expand Up @@ -244,3 +246,36 @@ jobs:
exit 1
fi
working-directory: .

test-runner:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
defaults:
run:
working-directory: test-runner/src
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: .go-version
- run: go vet ./...
- run: go test -count=1 ./...
- name: govulncheck
if: runner.os == 'Linux'
run: |
go install golang.org/x/vuln/cmd/govulncheck@latest
"$GITHUB_WORKSPACE/.github/scripts/govulncheck-gate.sh" "$GITHUB_WORKSPACE/.go-version"
- name: Deterministic rebuild
if: runner.os == 'Linux'
run: ./build.sh test-runner
working-directory: .
- name: Verify committed binaries are up to date
if: runner.os == 'Linux'
run: git diff --exit-code -- test-runner/bin/test-runner-linux-amd64 test-runner/bin/test-runner-darwin-arm64 test-runner/bin/test-runner.exe
working-directory: .
- name: MCP smoke test through the distributed dispatcher
if: runner.os == 'Linux'
run: TEST_RUNNER_BINARY="$GITHUB_WORKSPACE/test-runner/bin/test-runner" go test -count=1 -run '^TestMCPStdio$' ./...
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
- `retro-status/`: リポジトリ情報をレトロゲーム風に表示する Go 製 MCP サーバー
- `settings-advisor/`: ワークスペースに適した agy 設定を提案する Go 製 MCP サーバー
- `worktree-manager/`: Git worktree の作成・削除・一覧・prune を行う Go 製 MCP サーバー
- `test-runner/`: Go テストを実行し、失敗ログと再実行引数を返す Go 製 MCP サーバー
- `agy-plugin-kit/`: プラグイン作成用の command、skill、template、および Go 製 validator

Go 製プラグインは原則として `<plugin>/src/` に独立した Go モジュール、`<plugin>/bin/` に配布用バイナリを持ちます。validator のみ `agy-plugin-kit/validator/{src,bin}/` 配下です。
Expand Down Expand Up @@ -71,12 +72,13 @@ go-lsp/src
retro-status/src
settings-advisor/src
worktree-manager/src
test-runner/src
```

Go ソースを変更したら、リポジトリルートで対象を決定論的に再ビルドします。

```sh
./build.sh <github|validator|ast-grep|go-lsp|retro-status|settings-advisor|worktree-manager>
./build.sh <github|validator|ast-grep|go-lsp|retro-status|settings-advisor|worktree-manager|test-runner>
```

Windows PowerShell では同じ target を `./build.ps1` に渡します。決定論ビルドが前提とする Go のパッチ版は `.go-version` に定義され、`build.sh`/`build.ps1` が `GOTOOLCHAIN` で強制します(未取得なら Go が自動ダウンロード)。素の `go build` で焼いたバイナリはローカル Go 版によって別物になるため、必ずビルドスクリプト経由で再ビルドしてください。`.go-version` を独断で書き換えないこと。引き上げ条件は CLAUDE.md「Go バージョンの引き上げ方針」に定義され、条件1(固定中のマイナー内のパッチで直る stdlib 脆弱性)は CI の govulncheck が fail で通知します。
Expand Down
5 changes: 5 additions & 0 deletions LESSONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -458,3 +458,8 @@ response = json.loads(proc.stdout.readline())

#### 判断 14: bot 誤指摘(`master`→`main`)の却下
- Codex GitHub bot が「既定ブランチは `main`」と P2 指摘したが、`gh repo view`=`master`・`main` ref 不在・PR #7 が master マージ済みで**前提が誤り**と検証。鵜呑みにすると実ブランチでゲートが無効化される。却下根拠を PR コメントとコミットに残した(#28)。

## 2026-09-19: test-runner はログの保持上限と結果イベントの集計を分離する
- 却下した案: `CombinedOutput` でテスト出力全体を保持してから解析する、またはログ上限に達した時点で出力の読み取りを止める。
- 決め手: 回帰テストでは 64 KiB のログを40回(2.5 MiB)出力した後に成功・失敗イベントを送る。逐次解析とログ保持上限を分けることで、保持1 MiB以内・返却ログ64 KiB以内を守りながら末尾の成功1件・失敗1件を集計できた。読み取りを止めると末尾の結果を失い、子プロセスのパイプも詰まる。
- 覆す条件: 完全なログをディスク上の成果物として保存・取得する機能が必要になった場合。応答サイズ制限と結果集計は引き続き独立させる。
7 changes: 7 additions & 0 deletions PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,3 +209,10 @@ settings-advisor のスキャン精度(CI/CD、言語、除外ディレクト
- [x] ステップ1: スキャン検知機能と除外ロジックの改善・テスト追加
- [x] ステップ2: models.json 更新と traits 駆動モデル選定ロジックの実装・テスト追加
- [x] ステップ3: ドキュメント同期(README.md / README_ja.md)とビルド・品質検証

## フェーズ13: Go テストランナー (test-runner) の追加

現在地: 実装・Linuxローカル検証完了。macOS/Windowsのネイティブ実行はCI確認待ち。次フェーズには進まない。

- [x] MCP実行・結果集計・失敗テストの再実行引数が実装され、境界とキャンセルを検証済み。
- [x] 日英ドキュメント・skill・3環境の配布物・CIが揃い、ローカル検証が完了。
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ A collection of Model Context Protocol (MCP) plugins designed for AI assistants,
| **[retro-status](./retro-status/README.md)** | MCP server scanning repositories to render RPG-style retro status ASCII art |
| **[settings-advisor](./settings-advisor/README.md)** | MCP server analyzing workspace scale, languages, and settings to recommend optimal models and sandbox configurations |
| **[worktree-manager](./worktree-manager/README.md)** | MCP server providing safe git worktree operations (list, add, remove, prune) for isolated parallel workflows |
| **[test-runner](./test-runner/README.md)** | MCP server running Go tests with bounded failure logs and targeted rerun arguments |

## Installation

Expand Down Expand Up @@ -41,6 +42,9 @@ agy plugin install https://github.com/kwrkb/agy-plugins/settings-advisor

# worktree-manager plugin
agy plugin install https://github.com/kwrkb/agy-plugins/worktree-manager

# test-runner plugin
agy plugin install https://github.com/kwrkb/agy-plugins/test-runner
```

Please refer to the README in each plugin directory for specific prerequisites (required CLI binaries on PATH / authentication).
Expand All @@ -61,12 +65,13 @@ Each plugin includes an agent guide in `skills/<name>/SKILL.md` (loaded upon too
| retro-status | `git` (recommended), `rg` (optional) | None |
| settings-advisor | None (prebuilt binaries included) | None |
| worktree-manager | `git` (on PATH) | None |
| test-runner | `go` >= 1.24 (on PATH) | None |

### Bundled Platforms & Self-Build

✅ **Windows native execution verified across all plugins**
Existing plugins have been verified on native Windows. `test-runner` includes CI tests for Linux, macOS, and Windows; local verification is on Linux.

Go-based plugins (`github`, `ast-grep`, `go-lsp`, `retro-status`, `settings-advisor`, `worktree-manager`, and `agy-plugin-kit` validator) include native binaries for **`linux/amd64`**, **`darwin/arm64` (Apple Silicon)**, and **`windows/amd64`** in `bin/`. The extensionless `bin/<name>` dispatcher script detects the platform via `uname` and executes `<name>-<goos>-<goarch>` (on Windows, agy directly invokes `<name>.exe`).
Go-based plugins (`github`, `ast-grep`, `go-lsp`, `retro-status`, `settings-advisor`, `worktree-manager`, `test-runner`, and `agy-plugin-kit` validator) include native binaries for **`linux/amd64`**, **`darwin/arm64` (Apple Silicon)**, and **`windows/amd64`** in `bin/`. The extensionless `bin/<name>` dispatcher script detects the platform via `uname` and executes `<name>-<goos>-<goarch>` (on Windows, agy directly invokes `<name>.exe`).

**Other platforms (e.g. `linux/arm64` / `darwin/amd64`) are not bundled by default**, but can be built on the target machine without modifying any scripts:

Expand All @@ -89,5 +94,6 @@ Each plugin acts as a wrapper for existing tools/servers and complies with their
| settings-advisor | Custom Go implementation | MIT |
| agy-plugin-kit | Custom Go implementation | MIT |
| worktree-manager | `git` CLI | MIT |
| test-runner | `go test` (Go toolchain) | BSD-3-Clause |

Plugins delegate execution to user-installed binaries/CLIs on PATH (no redistribution of external binaries).
10 changes: 8 additions & 2 deletions README_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
| **[retro-status](./retro-status/README_ja.md)** | リポジトリをスキャンし、RPGのレトロステータス画面風のAA(アスキーアート)で出力する MCP サーバー |
| **[settings-advisor](./settings-advisor/README_ja.md)** | ワークスペースの規模・言語・機密/CI/本番設定を解析し、最適なモデル・サンドボックス・許可モードを控えめに提案する MCP サーバー |
| **[worktree-manager](./worktree-manager/README_ja.md)** | `git worktree` の安全な操作(一覧・作成・削除・クリーンアップ)を提供し、ブランチ隔離や並行作業を支援する MCP サーバー |
| **[test-runner](./test-runner/README_ja.md)** | Goテストの実行・失敗ログの整理・再実行引数の生成を行う MCP サーバー |

## インストール方法

Expand Down Expand Up @@ -41,6 +42,9 @@ agy plugin install https://github.com/kwrkb/agy-plugins/settings-advisor

# worktree-manager プラグイン
agy plugin install https://github.com/kwrkb/agy-plugins/worktree-manager

# test-runner プラグイン
agy plugin install https://github.com/kwrkb/agy-plugins/test-runner
```

各プラグインの前提条件(PATH に入れるバイナリ / 認証設定)については、各ディレクトリの README を参照してください。
Expand All @@ -61,12 +65,13 @@ agy plugin install https://github.com/kwrkb/agy-plugins/worktree-manager
| retro-status | `git`(推奨)、`rg`(任意) | 不要 |
| settings-advisor | 不要(バイナリ同梱) | 不要 |
| worktree-manager | `git`(PATH 上) | 不要 |
| test-runner | `go` >= 1.24 (PATH 上) | 不要 |

### 同梱プラットフォームと self-build

✅ **全プラグインで Windows ネイティブ動作確認済み**
既存プラグインは Windows ネイティブ動作確認済み。`test-runner` は Linux・macOS・Windows のCIテストを追加し、ローカルではLinuxで検証しています。

Go 製プラグイン(`github`、`ast-grep`、`go-lsp`、`retro-status`、`settings-advisor`、`worktree-manager`、および `agy-plugin-kit` の validator)は、**`linux/amd64`**、**`darwin/arm64`(Apple Silicon)**、**`windows/amd64`** のネイティブバイナリを `bin/` に同梱しています。拡張子なしの `bin/<name>` ディスパッチャスクリプトが `uname` で実行環境を判別し、`<name>-<goos>-<goarch>` を起動します(Windows では agy が `<name>.exe` を直接起動)。
Go 製プラグイン(`github`、`ast-grep`、`go-lsp`、`retro-status`、`settings-advisor`、`worktree-manager`、`test-runner`、および `agy-plugin-kit` の validator)は、**`linux/amd64`**、**`darwin/arm64`(Apple Silicon)**、**`windows/amd64`** のネイティブバイナリを `bin/` に同梱しています。拡張子なしの `bin/<name>` ディスパッチャスクリプトが `uname` で実行環境を判別し、`<name>-<goos>-<goarch>` を起動します(Windows では agy が `<name>.exe` を直接起動)。

**それ以外の環境(`linux/arm64` や `darwin/amd64` など)は標準同梱していません**が、スクリプトの書き換えなしで手元でビルドして動かせます:

Expand All @@ -89,5 +94,6 @@ cd <plugin>/src && CGO_ENABLED=0 go build -o "../bin/<name>-$(go env GOOS)-$(go
| settings-advisor | 独自 Go 実装 | MIT |
| agy-plugin-kit | 独自 Go 実装 | MIT |
| worktree-manager | `git` CLI | MIT |
| test-runner | `go test` (Go toolchain) | BSD-3-Clause |

各プラグインはユーザーの PATH 上にあるインストール済みバイナリ/CLI に処理を委譲します(外部バイナリの再配布は行っていません)。
4 changes: 3 additions & 1 deletion build.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ switch ($Target) {
'settings-advisor' { Build 'settings-advisor' 'settings-advisor' }
'go-lsp' { Build 'go-lsp' 'go-lsp' }
'worktree-manager' { Build 'worktree-manager' 'worktree-manager' }
'test-runner' { Build 'test-runner' 'test-runner' }
'all' {
Build 'github' 'github'
Build 'agy-plugin-kit/validator' 'validator'
Expand All @@ -83,9 +84,10 @@ switch ($Target) {
Build 'settings-advisor' 'settings-advisor'
Build 'go-lsp' 'go-lsp'
Build 'worktree-manager' 'worktree-manager'
Build 'test-runner' 'test-runner'
}
default {
Write-Error "unknown target: $Target (expected: github | validator | ast-grep | retro-status | settings-advisor | go-lsp | worktree-manager | all)"
Write-Error "unknown target: $Target (expected: github | validator | ast-grep | retro-status | settings-advisor | go-lsp | worktree-manager | test-runner | all)"
exit 2
}
}
4 changes: 3 additions & 1 deletion build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ case "$target" in
settings-advisor) build settings-advisor settings-advisor ;;
go-lsp) build go-lsp go-lsp ;;
worktree-manager) build worktree-manager worktree-manager ;;
test-runner) build test-runner test-runner ;;
all)
build github github
build agy-plugin-kit/validator validator
Expand All @@ -65,9 +66,10 @@ case "$target" in
build settings-advisor settings-advisor
build go-lsp go-lsp
build worktree-manager worktree-manager
build test-runner test-runner
;;
*)
echo "unknown target: $target (expected: github | validator | ast-grep | retro-status | settings-advisor | go-lsp | worktree-manager | all)" >&2
echo "unknown target: $target (expected: github | validator | ast-grep | retro-status | settings-advisor | go-lsp | worktree-manager | test-runner | all)" >&2
exit 2
;;
esac
57 changes: 57 additions & 0 deletions test-runner/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# test-runner — Go Test Runner

A Go-only MCP plugin for agy. Runs one module at a time, summarizes test/build failures, and returns arguments for explicitly rerunning failed tests. Calls wait for completion; there is no background job, persistent history, or automatic retry.

## Requirements and installation

- Go 1.24 or later on `PATH`, with the toolchain/dependencies required by the target module available.
- Prebuilt servers: Linux amd64, macOS arm64, Windows amd64. Windows uses the OS-provided `taskkill.exe` to stop a process tree.

```sh
agy plugin install https://github.com/kwrkb/agy-plugins/test-runner
```

## Tool: `go_test`

| Argument | Meaning |
| :-- | :-- |
| `module_path` | Required directory containing `go.mod`. Prefer an absolute path: relative paths use the MCP server's working directory. |
| `packages` | Nonempty string array of Go package patterns; default `["./..."]`. Local paths and import paths are accepted only when all resolved packages belong to the selected module and are physically inside it. |
| `run` | Optional Go `-run` expression, including slash-separated subtest expressions. |
| `timeout_seconds` | Integer 1–300, default 60. Covers discovery, compilation, and tests. Process cleanup can take a few additional seconds. |

Example arguments:

```json
{"module_path":"/code/agy-plugins/test-runner/src","packages":["./..."],"timeout_seconds":60}
```

The server resolves packages with `go list -e -json`, then runs `go test -json -count=1 -timeout=…` using an argument array, without a shell. It always disables successful-test caching. It inherits the Go environment (including `go.work`, `GOFLAGS`, build settings, and toolchain selection); it does not recursively discover other modules. A workspace containing several modules can be tested through separate calls with explicit module directories.

No arbitrary command, extra flags, `.go` file list, race/coverage option, fuzzing session, or benchmark option is exposed. Standard `go test` behavior, including examples and fuzz seed cases, remains in effect. Tests **execute project code** with the server's permissions and may change files or access the network. Go may use/download caches, dependencies, or a toolchain according to the inherited environment. This plugin is not an execution sandbox; use the host's execution permissions.

## Results and retry

Each MCP result contains a JSON text object:

- `status`: `passed`, `failed`, `no_tests`, `timeout`, `cancelled`, or `error`. Test/build failures are ordinary results; execution errors, timeouts, and cancellation set MCP `isError`.
- `exit_code`: Go process exit code, or `null` when no process exit code is available. Discovery errors can report the discovery process's exit code.
- `elapsed_seconds`: Total elapsed time; `packages[].elapsed_seconds` is Go's per-package duration.
- `tests`: `passed`, `failed`, `skipped` terminal-event counts, **including parent tests and subtests**. Package events are not counted as tests. All-skipped tests are distinguished by these counts; `no_tests` means no test terminal events and no failures.
- `packages`: Sorted package results with Go status `pass`, `fail`, `skip`, or `incomplete`; failing tests include their names, logs, and `rerun` arguments. Package-level failures such as a panic or `TestMain` exit may have no named failure; inspect the package log.
- `build_failures`: Build package IDs and diagnostics, separate from failed test counts. IDs may include a test variant suffix.
- `stderr`, `error`: Diagnostics and execution errors, when present.
- `logs_truncated`: Some log bytes were omitted. Retained event logs and stderr together are bounded to 1 MiB; returned log text is bounded to 64 KiB, prioritizing named failures. JSON escaping can make the serialized result larger.
- `incomplete`: Partial output, interrupted execution, or missing terminal events. Never interpret this as complete validation.

Use a failure's `rerun` object as the next `go_test` arguments. It retains the absolute module path and timeout, selects one package, and anchors/escapes each subtest-name segment. Go still runs parent setup to discover a selected subtest. Parent failures can overlap child failures; choose the parent to rerun a group or the child to narrow the retry. There is no server-side retry state.

Malformed/oversized JSON events (over 1 MiB), over 100,000 result records, and discovery output over 8 MiB produce explicit execution errors. Output continues to drain after a parsing error to avoid blocking the child; the overall timeout still applies. Timeout/cancellation stops the Unix process group or the Windows process tree and bounds pipe shutdown. Programs that deliberately detach from the process tree are outside this mechanism.

## Development and validation

In `test-runner/src`, run `gofmt`, `go vet ./...`, and `go test -count=1 ./...`. Tests use temporary modules without third-party dependencies and verify the real Go JSON stream, exact subtest retries, child-process termination, and MCP stdio calls.

From the repository root, run `./build.sh test-runner` (or `./build.ps1 test-runner`). These use the pinned `.go-version`; do not replace distributed binaries with an unpinned build. CI runs tests on Linux, macOS, and Windows and checks vulnerability policy and deterministic artifacts on Linux.

The bundled agent guide is [test-runner-gemini](skills/test-runner-gemini/SKILL.md).
Loading
Loading