go-sio

Small Go helpers for line-based streaming I/O. The package has no third-party dependencies and is designed for newline-delimited streams such as logs, command output, and JSONL-like data.

Install

go-sio uses Go modules and requires Go 1.26 or newer.

go get github.com/maxwu/go-sio@latest

import "github.com/maxwu/go-sio"

// Optional explicit alias:
// import go_sio "github.com/maxwu/go-sio"

What It Provides

• StreamReader: reads an io.Reader line by line and applies a StringLineFilter.
• NewJSONFilterReadCloser: wraps an io.ReadCloser and keeps only newline-delimited tokens that are valid JSON.
• NewTeeReaderCloser: wraps an io.ReadCloser with io.TeeReader while preserving Close.
• NewReadCloser: combines an io.Reader and io.Closer into one io.ReadCloser.

Examples

Filter Lines

package main

import (
    "fmt"
    "io"
    "log"
    "strings"

    "github.com/maxwu/go-sio"
)

func main() {
    input := strings.NewReader("one\n\ntwo\nthree")

    filter := func(line string) (string, error) {
        if line == "\n" {
            return "", nil // drop blank lines
        }
        return strings.ToUpper(line), nil
    }

    sr := go_sio.NewStreamReader(input, filter)
    if sr == nil {
        log.Fatal("nil reader")
    }

    out, err := io.ReadAll(sr)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Print(string(out))
}

Keep Valid JSON Lines

package main

import (
    "fmt"
    "io"
    "log"
    "os"

    "github.com/maxwu/go-sio"
)

func main() {
    file, err := os.Open("stream.log")
    if err != nil {
        log.Fatal(err)
    }

    rc := go_sio.NewJSONFilterReadCloser(file)
    body, readErr := io.ReadAll(rc)
    closeErr := rc.Close()

    if readErr != nil {
        log.Fatal(readErr)
    }
    if closeErr != nil {
        log.Fatal(closeErr)
    }

    fmt.Print(string(body))
}

Tee A Stream

package main

import (
    "bytes"
    "fmt"
    "io"
    "log"
    "os"

    "github.com/maxwu/go-sio"
)

func main() {
    file, err := os.Open("stream.log")
    if err != nil {
        log.Fatal(err)
    }

    var captured bytes.Buffer
    rc := go_sio.NewTeeReaderCloser(file, &captured)

    _, copyErr := io.Copy(os.Stdout, rc)
    closeErr := rc.Close()

    if copyErr != nil {
        log.Fatal(copyErr)
    }
    if closeErr != nil {
        log.Fatal(closeErr)
    }

    fmt.Println("Captured:", captured.String())
}

API Summary

• type StringLineFilter func(string) (string, error): transforms one line at a time. Return "" to drop the line. Return an error to stop reading.
• var ErrNilReader error: returned by (*StreamReader).Read on a nil receiver.
• var NopFilter StringLineFilter: pass-through filter used when NewStreamReader receives a nil filter.
• func NewStreamReader(r io.Reader, f StringLineFilter) *StreamReader: returns nil when r is nil.
• func NewJSONFilterReadCloser(r io.ReadCloser) io.ReadCloser: filters newline-delimited tokens with encoding/json.Valid.
• func NewTeeReaderCloser(r io.ReadCloser, w io.Writer) *TeeReaderCloser: tees reads into w and closes r.
• func NewReadCloser(r io.Reader, c io.Closer) *ReadCloser: stores r and c as a combined io.ReadCloser.

Behavior Contract

Use this section as the source of truth for humans and AI agents changing the package.

• Input is line-oriented. StreamReader uses bufio.Scanner with a custom split function that keeps \n when present.
• The filter receives each line exactly as scanned. The last line may not include \n.
• Returning "" from a filter drops that line. It does not emit an empty line.
• Scanner token limits still apply. A line larger than Go's default scanner token size, about 64 KiB, returns a scanner error.
• NewJSONFilterReadCloser validates each newline-delimited token independently. It does not parse multi-line JSON records and does not reformat JSON.
• Any complete JSON value accepted by encoding/json.Valid is kept, including objects, arrays, strings, numbers, booleans, and null.
• Closing wrappers returned by NewJSONFilterReadCloser or NewTeeReaderCloser closes the original reader. Callers should close only the wrapper.
• NewReadCloser and NewTeeReaderCloser do not validate nil components. Passing nil readers, writers, or closers may panic later when read or closed.

Development

Project rules:

• Use only the Go standard library.
• Keep production code at 100% unit-test coverage.
• Keep docs aligned with go.mod, exported APIs, tests, and CI.

Run the same checks used by CI before opening a pull request:

golangci-lint run ./...
go test -count=1 -race ./... -coverprofile=coverage.out
go tool cover -func=coverage.out | grep -qE '^total:.*100\.0%$'
go test -bench . -run '^$' ./...

See CONTRIBUTING.md for the contributor workflow and AGENTS.md for agent-specific constraints.

License

MIT. See LICENSE.