feat(engine): read_file returns images as real Blob content

AGENTS.md

@@ -134,6 +134,81 @@ never written to the session log — a resumed session rediscovers them. Config
 `skills_dirs` (array; a non-empty project value overrides the user value
 entirely) and the repeatable `-skills-dir` run/serve flag drive it.
 
+### read_file image support
+
+The built-in `read_file` tool (`engine/filetools.go`) can return an image
+file as real visual content, not mangled text. `readPathContent` opens the
+target path exactly once and classifies it by its magic bytes
+(`http.DetectContentType` over at most the first 512 bytes) — never by its
+extension: a `.txt` file that is actually a PNG is still recognized as an
+image, and a `.png` file that is actually text stays a text read. On a
+recognized image (`image/png`, `image/jpeg`, `image/gif`, `image/webp`),
+`read_file` returns a `message.ToolResult` whose Content is `[Text, Blob]`:
+a one-line Text summary (format, byte size, and pixel dimensions) followed
+by a `message.Blob` carrying the real file bytes. This is the same
+`Text`+`Blob` shape MCP's `mcpContentToParts` already produces
+(`engine/mcp.go`) — `read_file` is a second producer of it — so every
+transcoder's existing Blob handling and the imageclamp dimension/byte-size
+pass (`imageclamp.Clamp`, called from every transcoder's
+`transcodeRequest`) apply with no new wiring. `read_file` never bypasses
+that clamp: it does not resize, re-encode, or otherwise touch pixels
+itself. Because `imageclamp.Clamp` runs later, at transcode time, an image
+it downscales or re-encodes can end up described by dimensions or a byte
+size that no longer match the summary `read_file` reported when it read
+the file; this is a known, accepted mismatch, not a defect to fix in
+`read_file` itself.
+
+**Only the Anthropic route puts a tool-result image on the wire.**
+`imageclamp.Limits.RecurseToolResults` is true for `provider/anthropic`
+only; `provider/openai` and `provider/openaicompat` set it false and
+instead replace a tool-result Blob with a text note,
+`"[N image attachment(s) omitted]"` (`toolResultOutput`,
+`provider/openai/transcode.go` and `provider/openaicompat/transcode.go`).
+This is pre-existing wire-format behavior `read_file` inherits, not
+something this feature introduces, but it means a `read_file` image reaches
+the model as pixels only on the Anthropic route; on the other two the model
+sees only the one-line Text summary.
+
+`readPathContent` applies three guards on the image path, in order:
+
+1. The sniff read uses `io.ReadFull`, not a single `Read`, so a short
+   `read(2)` — realistic on a pipe or FUSE mount — never misclassifies a
+   real image as plain text.
+2. The read is bounded at `readFileMaxImageBytes` (20MB), checked
+   against an `io.LimitReader` over the same open handle, never against a
+   separately captured `os.Stat` size a concurrently growing file could
+   outrun. This cap is separate from and smaller than any provider's own
+   wire limit, which `imageclamp.Clamp` enforces at transcode time; it
+   exists only so `read_file` itself never loads an unbounded file into
+   memory. An over-cap image returns a plain text error and no Blob.
+3. The body must decode with `image.DecodeConfig` before `read_file`
+   commits to the image outcome. A corrupt or truncated file that merely
+   opens with a matching magic-byte prefix fails this check; `read_file`
+   then reads the true remainder of the file (unbounded, same handle) and
+   returns it as ordinary text instead of shipping a Blob the model cannot
+   use. This guard is not airtight for GIF: the `GIF87a`/`GIF89a` header
+   carries no checksum, so text that happens to start with those exact six
+   bytes still "decodes" with fabricated dimensions. A real file colliding
+   with that prefix is vanishingly unlikely; this is a documented, accepted
+   residual.
+
+A non-image binary file (sniffed as `application/octet-stream` or similar)
+keeps `read_file`'s existing (unbounded) text-read behavior; `readPathContent`
+still reads it exactly once, through the same handle its sniff already
+opened.
+
+**Known gap, filed as issue #129**: a transcode-time degrade of an image
+Blob to a text placeholder for a model with no vision capability is not
+implemented. No per-model vision-capability signal exists anywhere in the
+codebase to gate it on — the embedded models.dev catalog this file's own
+"Architecture" section describes as a design goal is not yet built, and
+`provider.Request` carries no capability flag comparable to `Effort` or
+`SessionKey` that a caller could set from one. Building this now would mean
+inventing an ad hoc, likely-wrong static model list, so it is deferred to
+issue #129. Until it lands, a model with no vision support receives the
+image Blob exactly as any vision-capable model does; how it handles that
+block is between the model and its provider.
+
 ### Base loop retry
 
 The base interactive `Prompt` loop retries a transient provider error itself,

engine/filetools.go

@@ -1,13 +1,22 @@
 package engine
 
 import (
+	"bytes"
 	"context"
 	"encoding/json"
 	"fmt"
+	"image"
+	_ "image/gif"  // register GIF decoder for image.DecodeConfig
+	_ "image/jpeg" // register JPEG decoder for image.DecodeConfig
+	_ "image/png"  // register PNG decoder for image.DecodeConfig
+	"io"
+	"net/http"
 	"os"
 	"path/filepath"
 	"strings"
 
+	_ "golang.org/x/image/webp" // register WebP decoder for image.DecodeConfig
+
 	"github.com/majorcontext/harness/message"
 	"github.com/majorcontext/harness/provider"
 )
@@ -21,6 +30,145 @@ const (
 	readFileMaxLineLen = 2000
 )
 
+// readFileMaxImageBytes bounds the total bytes read_file will read from a
+// detected image. The transcode-time imageclamp pass (imageclamp.Clamp)
+// enforces each provider's own wire size limit; this cap exists only to
+// stop read_file itself from loading an unbounded file into memory before
+// that clamp ever runs. readPathContent checks this bound against the
+// read itself (an io.LimitReader over the open file handle), never against
+// a separately captured os.Stat size, which a file that grows after the
+// stat could outrun. It is a var, not a const, so a test can shrink it
+// instead of writing a real oversized fixture.
+var readFileMaxImageBytes = 20 * 1024 * 1024 // 20MB
+
+// imageSniffLen is how many leading bytes classify a file by magic bytes —
+// the same bound http.DetectContentType itself considers.
+const imageSniffLen = 512
+
+// readFileImageMediaTypes lists the image formats read_file recognizes and
+// returns as a message.Blob for the model to see directly. Every other
+// sniffed content type — including any other image/* MIME type
+// http.DetectContentType might report, such as image/bmp — keeps
+// read_file's existing text-reading behavior unchanged.
+var readFileImageMediaTypes = map[string]bool{
+	"image/png":  true,
+	"image/jpeg": true,
+	"image/gif":  true,
+	"image/webp": true,
+}
+
+// sniffMediaType reads up to imageSniffLen bytes from r via io.ReadFull and
+// classifies them via http.DetectContentType, returning the classification
+// and the sniffed bytes themselves (so a caller reading the rest of the
+// stream does not re-read them). Taking an io.Reader, not a path, is what
+// lets TestSniffMediaTypeSurvivesShortReads drive it with
+// iotest.OneByteReader: a plain single Read against such a source returns
+// one byte per call, so a caller using Read directly would misclassify
+// almost every real image — io.ReadFull is what makes this deterministic
+// against a short-read source (a pipe, a FUSE/network mount, a signal).
+func sniffMediaType(r io.Reader) (mediaType string, sniffed []byte, err error) {
+	buf := make([]byte, imageSniffLen)
+	n, rerr := io.ReadFull(r, buf)
+	if rerr != nil && rerr != io.ErrUnexpectedEOF && rerr != io.EOF {
+		return "", nil, rerr
+	}
+	buf = buf[:n]
+	return http.DetectContentType(buf), buf, nil
+}
+
+// fileContent is the outcome of readPathContent: either a detected image's
+// Blob-ready bytes, or a non-image file's raw bytes for the ordinary text
+// read. Exactly one of ImageData or TextData is populated.
+type fileContent struct {
+	IsImage       bool
+	ImageData     []byte
+	MediaType     string
+	Width, Height int
+	TextData      []byte
+}
+
+// readPathContent opens path exactly once and, on every outcome, reads it
+// exactly once — no second os.Open, no re-read of bytes already
+// classified. Earlier revisions of read_file's image detection opened the
+// file twice (once to sniff, once via os.ReadFile) even on a plain text
+// read, the far more common case; this unifies both outcomes behind one
+// handle.
+//
+// Classification is by magic bytes only (http.DetectContentType via
+// sniffMediaType), never by the file's extension: a ".txt" that is really
+// a PNG is still recognized; a ".png" that is really text is not.
+//
+// A recognized image type is read under a bound: readFileMaxImageBytes+1
+// bytes via io.LimitReader over the SAME open handle the sniff used — a
+// cap that binds on bytes actually read, never a pre-read os.Stat size a
+// concurrently growing file could outrun. An over-cap image returns a
+// non-nil error (no partial TextData) so the caller reports a clear error
+// instead of falling through to an unbounded read of a file already known
+// to be an oversized image.
+//
+// The image body must also decode with image.DecodeConfig before this
+// function commits to the image outcome: a corrupt or truncated file that
+// merely opens with a matching magic-byte prefix (PNG's 8-byte signature,
+// JPEG's SOI marker, WebP's RIFF/WEBP container) fails PNG/JPEG/WebP's own
+// structural header checks. On that failure this reads the true remainder
+// of the file (unbounded, via the same handle) so the text fallback is
+// complete rather than silently cut off at the image cap — reachable only
+// when magic bytes matched an image signature yet the body failed
+// structural validation, so an oversized file in this branch is already
+// known not to be a real image. This gate is NOT airtight for GIF: the
+// GIF87a/GIF89a header has no checksum, so any bytes following a literal
+// "GIF87a"/"GIF89a" prefix still "decode" with fabricated width/height. A
+// real-world text file beginning with those exact six bytes is
+// vanishingly unlikely; this is a documented, accepted residual, not a
+// silent gap.
+func readPathContent(path string) (fileContent, error) {
+	f, err := os.Open(path)
+	if err != nil {
+		return fileContent{}, err
+	}
+	defer f.Close()
+
+	mediaType, sniff, err := sniffMediaType(f)
+	if err != nil {
+		return fileContent{}, err
+	}
+	if !readFileImageMediaTypes[mediaType] {
+		rest, err := io.ReadAll(f)
+		if err != nil {
+			return fileContent{}, err
+		}
+		return fileContent{TextData: append(sniff, rest...)}, nil
+	}
+
+	budget := int64(readFileMaxImageBytes) - int64(len(sniff))
+	if budget < 0 {
+		budget = 0
+	}
+	capped, err := io.ReadAll(io.LimitReader(f, budget+1))
+	if err != nil {
+		return fileContent{}, err
+	}
+	full := append(sniff, capped...)
+	if len(full) > readFileMaxImageBytes {
+		return fileContent{}, fmt.Errorf("image (%s) exceeds the %d-byte read_file image limit", mediaType, readFileMaxImageBytes)
+	}
+
+	cfg, _, derr := image.DecodeConfig(bytes.NewReader(full))
+	if derr != nil {
+		// Sniffed as an image by magic bytes, but the body does not decode
+		// as one (corrupt, truncated, or a false-positive magic-byte
+		// match). Read the true remainder so the text fallback is
+		// complete, not silently cut off at the image cap; see the doc
+		// comment above for why this is safe.
+		rest, err := io.ReadAll(f)
+		if err != nil {
+			return fileContent{}, err
+		}
+		return fileContent{TextData: append(full, rest...)}, nil
+	}
+	return fileContent{IsImage: true, ImageData: full, MediaType: mediaType, Width: cfg.Width, Height: cfg.Height}, nil
+}
+
 // resolvePath resolves a tool path argument against the session working
 // directory. Absolute paths pass through unchanged.
 func (s *Session) resolvePath(path string) string {
@@ -34,7 +182,7 @@ func readFileTool() Tool {
 	return Tool{
 		Def: provider.ToolDef{
 			Name:        "read_file",
-			Description: "Read a file and return its content with line numbers (N→ prefixes). Prefer this over shell commands like cat, head, or sed for reading files. Relative paths resolve against the session working directory.",
+			Description: "Read a file and return its content with line numbers (N→ prefixes). A recognized image file (PNG, JPEG, GIF, WebP) is returned as an image where the current provider supports tool-result images. Prefer this over shell commands like cat, head, or sed for reading files. Relative paths resolve against the session working directory.",
 			InputSchema: json.RawMessage(`{
 				"type": "object",
 				"properties": {
@@ -62,10 +210,19 @@ func readFileTool() Tool {
 			if info.IsDir() {
 				return nil, fmt.Errorf("read_file: %s is a directory", path)
 			}
-			data, err := os.ReadFile(path)
+
+			content, err := readPathContent(path)
 			if err != nil {
-				return nil, fmt.Errorf("read_file: %w", err)
+				return nil, fmt.Errorf("read_file: %s: %w", path, err)
+			}
+			if content.IsImage {
+				summary := fmt.Sprintf("image (%s), %d bytes, %dx%d pixels", content.MediaType, len(content.ImageData), content.Width, content.Height)
+				return message.Parts{
+					&message.Text{Text: summary},
+					&message.Blob{MediaType: content.MediaType, Data: content.ImageData},
+				}, nil
 			}
+			data := content.TextData
 
 			lines := strings.Split(string(data), "\n")
 			// A trailing newline produces one empty trailing element; drop it.