Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,7 @@ jobs:
- uses: powerman/.github/.github/actions/cache-go-and-tools@main

- run: mise run scripts:test
- run: mise run toc --check
- run: mise run lint:links
- run: mise run lint:docs

# Testing on available native platforms.
# Go scripts don't need mise: they are built and run by Go alone.
Expand All @@ -66,6 +65,7 @@ jobs:
runs-on: ${{ matrix.os }}
timeout-minutes: 10
env:
SCRIPTS_BIN: ${{ github.workspace }}/.cache/scripts
KONGUSAGE_TEST_SKIP_USAGE: 1 # Tool usage of mise.toml is not installed here.
defaults:
run:
Expand All @@ -84,9 +84,8 @@ jobs:
- run: go -C scripts test -timeout=60s ./...
if: matrix.os == 'windows-11-arm'

- run: go -C scripts build -o ../.cache/bin/ ./...
- run: .cache/bin/toc --check
- run: .cache/bin/lint-links
- run: go -C scripts build -o "$SCRIPTS_BIN/" ./...
- run: '"$SCRIPTS_BIN/lint-docs"'

# Testing on available docker QEMU platforms (Linux only).
qemu:
Expand Down Expand Up @@ -124,13 +123,13 @@ jobs:
-v "$HOME/.cache/go-build:/root/.cache/go-build" \
-v "$HOME/go:/go" \
-e KONGUSAGE_TEST_SKIP_USAGE=1 \
-e SCRIPTS_BIN=/workspace/.cache/scripts \
"golang:${V}-alpine" sh -e -c '
apk add --no-cache git >/dev/null
git config --global --add safe.directory /workspace
go -C scripts test -timeout=60s ./...
go -C scripts build -o ../.cache/bin/ ./...
.cache/bin/toc --check
.cache/bin/lint-links
go -C scripts build -o "$SCRIPTS_BIN/" ./...
"$SCRIPTS_BIN/lint-docs"
'

# The same as qemu, but by task test-qemu, which can also be run locally.
Expand Down
69 changes: 51 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ copy the parts you need and adapt them.

The scripts here are real tasks of this repository:
[toc](scripts/toc/) maintains the table of contents below,
[lint:links](scripts/lint-links/) checks links in markdown files on CI,
[lint:docs](scripts/lint-docs/) checks it and links in markdown files
by `toc --check` and [lint-links](scripts/lint-links/) on CI,
on Linux, macOS and Windows,
and [test-qemu](scripts/test-qemu/) tests scripts in docker on platforms emulated by QEMU.

Expand All @@ -32,6 +33,7 @@ and [test-qemu](scripts/test-qemu/) tests scripts in docker on platforms emulate
- [Why mise](#why-mise)
- [A separate Go module](#a-separate-go-module)
- [Build on demand](#build-on-demand)
- [Running binaries](#running-binaries)
- [Completion](#completion)
- [Code of a script](#code-of-a-script)
- [Tests: Fake for World, Mock for App](#tests-fake-for-world-mock-for-app)
Expand Down Expand Up @@ -85,10 +87,12 @@ so a Go script is as convenient to run as a shell script.

## What this example shows

- Two scripts which process files, [toc](scripts/toc/) and [lint:links](scripts/lint-links/),
- Two scripts which process files, [toc](scripts/toc/) and [lint-links](scripts/lint-links/),
with code shared in [scripts/internal/markdown](scripts/internal/markdown/).
- A script which orchestrates external commands, [test-qemu](scripts/test-qemu/):
it has little logic and a `World` which runs docker.
- A script which runs binaries of other scripts, [lint:docs](scripts/lint-docs/):
it finds them by `$SCRIPTS_BIN`, like tasks do, without a nested `mise run`.
- Scripts as mise tasks: a fast rebuild on demand, completion of flags and arguments,
tasks to build, generate, format, lint, test and cover scripts.
- A layout of a script which makes it testable:
Expand All @@ -99,7 +103,7 @@ so a Go script is as convenient to run as a shell script.
and [kongusage](https://github.com/powerman/kongusage) for its completion in mise.
- [bitfield/script](https://github.com/bitfield/script) to run commands,
so an adapter looks almost like a shell command:
[toc](scripts/toc/internal/world.go), [lint:links](scripts/lint-links/internal/world.go),
[toc](scripts/toc/internal/world.go), [lint-links](scripts/lint-links/internal/world.go),
[test-qemu](scripts/test-qemu/internal/world.go).
- Scripts used by tasks of the project ([fmt and lint](mise.toml)) and run on CI.
- Scripts run on [CI](.github/workflows/test.yml) on Linux, macOS and Windows:
Expand All @@ -121,7 +125,8 @@ so a Go script is as convenient to run as a shell script.
├── .mockery/ Template of fakes.
├── internal/
│ └── markdown/ Code shared by scripts.
├── lint-links/ Program of task lint:links.
├── lint-docs/ Program of task lint:docs.
├── lint-links/ Program without a task, run by lint-docs.
├── test-qemu/ Program of task test-qemu.
└── toc/ Program of task toc.
├── main.go Parses the CLI and runs it.
Expand All @@ -147,16 +152,17 @@ mise install
```sh
mise run toc --help
mise run toc --check README.md
mise run lint:links
mise run lint:docs
mise run test-qemu linux/arm/v7 linux/riscv64
```

Arguments are passed to the script as is, `--help` and invalid flags included.
Flags and arguments of a task are completed by Tab
when [completion of mise](https://mise.jdx.dev/cli/completion.html) is enabled.

A program is in `.cache/bin/` and may be run directly,
but it may be outdated: run `mise run scripts:build` first.
Binaries of scripts are built into `.cache/scripts/`,
but run them by tasks:
a task rebuilds a binary before the run and completes its arguments.

### Develop scripts

Expand Down Expand Up @@ -220,11 +226,11 @@ description = 'Update tables of contents in markdown files'
depends = ['scripts:build']
usage = 'mount run="mise run -q toc --usage-spec"'
raw_args = true
run = '.cache/bin/toc'
run_windows = '.cache\bin\toc'
run = '"${SCRIPTS_BIN:?}/toc"'
run_windows = '"%SCRIPTS_BIN%\toc"'
```

- `scripts:build` rebuilds all programs into `.cache/bin/`
- `scripts:build` rebuilds binaries of all programs into `$SCRIPTS_BIN`
only when files of `scripts/` change (by `sources` and `outputs`),
so a start of a task takes a few milliseconds.
`go run` isn't used: its own check of changes takes much longer.
Expand All @@ -234,6 +240,32 @@ run_windows = '.cache\bin\toc'
so an argument `--` needs `mise run toc -- -- …`.
- A program of task `<name>` is `scripts/<name>/`, with `:` in the name replaced by `-`.
The binary is named by its directory, so `kong.Name` sets the task name for help.
- A binary which only tasks and other binaries run has no task:
a task is needed only to run a binary by hand.

### Running binaries

[mise.toml](mise.toml) gives every task and binary the directory of binaries:

```toml
[env]
SCRIPTS_BIN = '{{config_root}}/.cache/scripts' # Go scripts built by scripts:build.
```

Tasks and binaries find binaries only by this variable,
so it works from any directory, and the path is written in one place.

- A task which runs a binary depends on `scripts:build` and runs `"${SCRIPTS_BIN:?}/<name>"`.
- A binary runs another binary by `filepath.Join(os.Getenv("SCRIPTS_BIN"), "<name>")`
([lint-docs](scripts/lint-docs/internal/world.go)),
without a build: the task which started it has built all binaries.
- A binary isn't run by a nested `mise run`:
mise resolves tools and env for each task it runs,
so a nested `mise run` takes about 100 ms against about 3 ms of a direct start,
and mise adds `ERROR task failed` to the output on any non-zero exit status.
- `$SCRIPTS_BIN` isn't added to `PATH`:
binaries have short common names and would shadow commands of the system
in tasks and in the shell activated by mise.

### Completion

Expand All @@ -252,7 +284,7 @@ type CLI struct {
mise does it only on Tab, so the spec is never stale,
and a broken spec breaks completion of only this task.
A test checks the spec by `kongusage.CheckSpec`,
and `scripts:lint:usage` lints specs of all programs.
and `scripts:lint:usage` lints specs of all binaries.

### Code of a script

Expand Down Expand Up @@ -323,17 +355,18 @@ With bitfield/script:

Go scripts work on Windows, but mise tasks around them need care:

- mise runs inline scripts by `cmd /c` on Windows, which can't run `.cache/bin/toc`,
so a task which runs a program has `run_windows = '.cache\bin\toc'`.
- mise runs inline scripts by `cmd /c` on Windows, which doesn't expand `$SCRIPTS_BIN`,
so a task which runs a binary has `run_windows = '"%SCRIPTS_BIN%\toc"'`.
- Tasks needed to run scripts (`scripts:build`, `scripts:build:clean`, `scripts:test`)
work in cmd too, with PowerShell in `run_windows` where POSIX sh is needed.
work in cmd too, with `run_windows` where they use sh:
`%SCRIPTS_BIN%` instead of `$SCRIPTS_BIN`, PowerShell instead of a loop.
Other tasks to develop scripts need POSIX sh (e.g. Git Bash).
- Some tools of `mise.toml` may be unavailable on Windows,
so [CI](.github/workflows/test.yml) installs only tools of scripts
(`install_args` of `jdx/mise-action`)
and sets `MISE_AUTO_INSTALL=false`, so `mise run` doesn't try to install others.
- Git returns paths with `/`, while arguments may use `\`:
[lint:links](scripts/lint-links/internal/app.go) converts them by `filepath.ToSlash`.
[lint-links](scripts/lint-links/internal/app.go) converts them by `filepath.ToSlash`.

## Add Go scripts to your project

Expand All @@ -351,7 +384,7 @@ This repository is such a project. Copy:
and [scripts/.mockery/](scripts/.mockery/);
- `scripts/go.mod` with your module path, e.g. `github.com/you/project/scripts`;
- from [mise.toml](mise.toml): `[task_config]`, tools of Go scripts,
`vars.scripts_cover`, tasks `bump` and `cachedir`,
`env.SCRIPTS_BIN`, `vars.scripts_cover`, tasks `bump` and `cachedir`,
and `scripts:*` tasks in root tasks `generate`, `fmt`, `lint` and `test`.

### A project in Go
Expand All @@ -371,8 +404,8 @@ The same, with a few differences:

**What exactly is the approach here?**
Only the task layer:
root mise tasks which build Go CLIs on demand into `.cache/bin`
and complete their flags and arguments.
root mise tasks which build Go CLIs on demand into `$SCRIPTS_BIN`,
run them directly and complete their flags and arguments.
Everything about the Go code itself is outside the approach:
its structure, [libraries](#libraries), code generators and linter config.
This example shows one recommended way to write and test it.
Expand Down
19 changes: 10 additions & 9 deletions mise.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ gh = 'latest'


[env]
SCRIPTS_BIN = '{{config_root}}/.cache/scripts' # Go scripts built by scripts:build.
PNPM_CONFIG_VIRTUAL_STORE_TYPE = 'project' # Fixed in 2026.11.1, remove after updating min_version.


Expand Down Expand Up @@ -137,13 +138,13 @@ run = "git ls-files -z -co --exclude-standard '*.yml' '*.yaml' | xargs -0 -r yam
description = 'Lint GitHub Action workflows'
run = 'actionlint'

[tasks.'lint:links']
description = 'Check local links in markdown files'
[tasks.'lint:docs']
description = 'Check tables of contents and local links in markdown files'
depends = ['scripts:build']
usage = 'mount run="mise run -q lint:links --usage-spec"'
usage = 'mount run="mise run -q lint:docs --usage-spec"'
raw_args = true
run = '.cache/bin/lint-links'
run_windows = '.cache\bin\lint-links'
run = '"${SCRIPTS_BIN:?}/lint-docs"'
run_windows = '"%SCRIPTS_BIN%\lint-docs"'

[tasks.'lint:markdown']
description = 'Lint markdown files'
Expand Down Expand Up @@ -171,17 +172,17 @@ description = 'Update tables of contents in markdown files'
depends = ['scripts:build']
usage = 'mount run="mise run -q toc --usage-spec"'
raw_args = true
run = '.cache/bin/toc'
run_windows = '.cache\bin\toc'
run = '"${SCRIPTS_BIN:?}/toc"'
run_windows = '"%SCRIPTS_BIN%\toc"'

# Not test:qemu: it's slow and needs docker, so `test:*` of a project must not run it.
[tasks.test-qemu]
description = 'Test Go scripts on other platforms emulated by QEMU in docker'
depends = ['scripts:build']
usage = 'mount run="mise run -q test-qemu --usage-spec"'
raw_args = true
run = '.cache/bin/test-qemu'
run_windows = '.cache\bin\test-qemu'
run = '"${SCRIPTS_BIN:?}/test-qemu"'
run_windows = '"%SCRIPTS_BIN%\test-qemu"'

[tasks.check-drift]
hide = true
Expand Down
4 changes: 4 additions & 0 deletions scripts/.mockery.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ _anchors:
filename: 'fake.{{.InterfaceFile | base | trimSuffix ".go"}}_test.go'
structname: 'Fake{{.InterfaceName | firstUpper}}'
packages:
github.com/powerman/go-scripts-example/scripts/lint-docs/internal:
interfaces:
World:
config: *fake
github.com/powerman/go-scripts-example/scripts/lint-links/internal:
interfaces:
World:
Expand Down
31 changes: 31 additions & 0 deletions scripts/lint-docs/internal/app.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package internal

import (
"errors"
"fmt"
)

type app struct {
world World
}

// NewApp creates an App which uses world.
func NewApp(world World) *app {
return &app{world: world}
}

// Check implements [App].
func (a *app) Check() error {
checks := [][]string{ // Binaries of Go scripts with their arguments.
{"toc", "--check"},
{"lint-links"},
}
var errs []error
for _, c := range checks {
err := a.world.RunScript(c[0], c[1:]...)
if err != nil {
errs = append(errs, fmt.Errorf("%s: %w", c[0], err))
}
}
return errors.Join(errs...)
}
74 changes: 74 additions & 0 deletions scripts/lint-docs/internal/app_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
package internal_test

import (
"errors"
"strings"
"testing"

"github.com/powerman/check"

"github.com/powerman/go-scripts-example/scripts/lint-docs/internal"
)

var errRun = errors.New("stub run error")

// newWorld returns a fake of binaries which all succeed.
func newWorld() *FakeWorld {
w := NewFakeWorld()
w.RunScriptFunc.SetDefaultReturn(nil)
return w
}

// runs returns command lines of binaries run by w.
func runs(w *FakeWorld) []string {
history := w.RunScriptFunc.History()
lines := make([]string, 0, len(history))
for _, c := range history {
lines = append(lines, strings.Join(append([]string{c.Name}, c.ArgArgs...), " "))
}
return lines
}

var allRuns = []string{"toc --check", "lint-links"}

func TestApp_Check(tt *testing.T) {
tt.Parallel()

tests := []struct {
name string
prepare func(*FakeWorld)
want string
}{
{
name: "OK",
prepare: func(*FakeWorld) {},
},
{
name: "FirstFails",
prepare: func(w *FakeWorld) { w.RunScriptFunc.PushReturn(errRun) },
want: "^toc: stub run error$",
},
{
name: "AllFail",
prepare: func(w *FakeWorld) { w.RunScriptFunc.SetDefaultReturn(errRun) },
want: "^toc: stub run error\nlint-links: stub run error$",
},
}
for _, tc := range tests {
tt.Run(tc.name, func(tt *testing.T) {
tt.Parallel()
t := check.T(tt).MustAll()
w := newWorld()
tc.prepare(w)

err := internal.NewApp(w).Check()
if tc.want == "" {
t.Nil(err)
} else {
t.Err(err, errRun)
t.Match(err, tc.want)
}
t.DeepEqual(runs(w), allRuns)
})
}
}
13 changes: 13 additions & 0 deletions scripts/lint-docs/internal/cmd.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package internal

import "github.com/powerman/kongusage"

// CLI defines the command-line interface of the program.
type CLI struct {
UsageSpec kongusage.Flag
}

// Run executes the CLI.
func (CLI) Run(a App) error {
return a.Check()
}
Loading
Loading