diff --git a/AGENTS.md b/AGENTS.md index 9905d19..94dea69 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -134,6 +134,81 @@ never written to the session log — a resumed session rediscovers them. Config `skills_dirs` (array; a non-empty project value overrides the user value entirely) and the repeatable `-skills-dir` run/serve flag drive it. +### read_file image support + +The built-in `read_file` tool (`engine/filetools.go`) can return an image +file as real visual content, not mangled text. `readPathContent` opens the +target path exactly once and classifies it by its magic bytes +(`http.DetectContentType` over at most the first 512 bytes) — never by its +extension: a `.txt` file that is actually a PNG is still recognized as an +image, and a `.png` file that is actually text stays a text read. On a +recognized image (`image/png`, `image/jpeg`, `image/gif`, `image/webp`), +`read_file` returns a `message.ToolResult` whose Content is `[Text, Blob]`: +a one-line Text summary (format, byte size, and pixel dimensions) followed +by a `message.Blob` carrying the real file bytes. This is the same +`Text`+`Blob` shape MCP's `mcpContentToParts` already produces +(`engine/mcp.go`) — `read_file` is a second producer of it — so every +transcoder's existing Blob handling and the imageclamp dimension/byte-size +pass (`imageclamp.Clamp`, called from every transcoder's +`transcodeRequest`) apply with no new wiring. `read_file` never bypasses +that clamp: it does not resize, re-encode, or otherwise touch pixels +itself. Because `imageclamp.Clamp` runs later, at transcode time, an image +it downscales or re-encodes can end up described by dimensions or a byte +size that no longer match the summary `read_file` reported when it read +the file; this is a known, accepted mismatch, not a defect to fix in +`read_file` itself. + +**Only the Anthropic route puts a tool-result image on the wire.** +`imageclamp.Limits.RecurseToolResults` is true for `provider/anthropic` +only; `provider/openai` and `provider/openaicompat` set it false and +instead replace a tool-result Blob with a text note, +`"[N image attachment(s) omitted]"` (`toolResultOutput`, +`provider/openai/transcode.go` and `provider/openaicompat/transcode.go`). +This is pre-existing wire-format behavior `read_file` inherits, not +something this feature introduces, but it means a `read_file` image reaches +the model as pixels only on the Anthropic route; on the other two the model +sees only the one-line Text summary. + +`readPathContent` applies three guards on the image path, in order: + +1. The sniff read uses `io.ReadFull`, not a single `Read`, so a short + `read(2)` — realistic on a pipe or FUSE mount — never misclassifies a + real image as plain text. +2. The read is bounded at `readFileMaxImageBytes` (20MB), checked + against an `io.LimitReader` over the same open handle, never against a + separately captured `os.Stat` size a concurrently growing file could + outrun. This cap is separate from and smaller than any provider's own + wire limit, which `imageclamp.Clamp` enforces at transcode time; it + exists only so `read_file` itself never loads an unbounded file into + memory. An over-cap image returns a plain text error and no Blob. +3. The body must decode with `image.DecodeConfig` before `read_file` + commits to the image outcome. A corrupt or truncated file that merely + opens with a matching magic-byte prefix fails this check; `read_file` + then reads the true remainder of the file (unbounded, same handle) and + returns it as ordinary text instead of shipping a Blob the model cannot + use. This guard is not airtight for GIF: the `GIF87a`/`GIF89a` header + carries no checksum, so text that happens to start with those exact six + bytes still "decodes" with fabricated dimensions. A real file colliding + with that prefix is vanishingly unlikely; this is a documented, accepted + residual. + +A non-image binary file (sniffed as `application/octet-stream` or similar) +keeps `read_file`'s existing (unbounded) text-read behavior; `readPathContent` +still reads it exactly once, through the same handle its sniff already +opened. + +**Known gap, filed as issue #129**: a transcode-time degrade of an image +Blob to a text placeholder for a model with no vision capability is not +implemented. No per-model vision-capability signal exists anywhere in the +codebase to gate it on — the embedded models.dev catalog this file's own +"Architecture" section describes as a design goal is not yet built, and +`provider.Request` carries no capability flag comparable to `Effort` or +`SessionKey` that a caller could set from one. Building this now would mean +inventing an ad hoc, likely-wrong static model list, so it is deferred to +issue #129. Until it lands, a model with no vision support receives the +image Blob exactly as any vision-capable model does; how it handles that +block is between the model and its provider. + ### Base loop retry The base interactive `Prompt` loop retries a transient provider error itself, diff --git a/engine/filetools.go b/engine/filetools.go index 28ddf85..55b0b25 100644 --- a/engine/filetools.go +++ b/engine/filetools.go @@ -1,13 +1,22 @@ package engine import ( + "bytes" "context" "encoding/json" "fmt" + "image" + _ "image/gif" // register GIF decoder for image.DecodeConfig + _ "image/jpeg" // register JPEG decoder for image.DecodeConfig + _ "image/png" // register PNG decoder for image.DecodeConfig + "io" + "net/http" "os" "path/filepath" "strings" + _ "golang.org/x/image/webp" // register WebP decoder for image.DecodeConfig + "github.com/majorcontext/harness/message" "github.com/majorcontext/harness/provider" ) @@ -21,6 +30,145 @@ const ( readFileMaxLineLen = 2000 ) +// readFileMaxImageBytes bounds the total bytes read_file will read from a +// detected image. The transcode-time imageclamp pass (imageclamp.Clamp) +// enforces each provider's own wire size limit; this cap exists only to +// stop read_file itself from loading an unbounded file into memory before +// that clamp ever runs. readPathContent checks this bound against the +// read itself (an io.LimitReader over the open file handle), never against +// a separately captured os.Stat size, which a file that grows after the +// stat could outrun. It is a var, not a const, so a test can shrink it +// instead of writing a real oversized fixture. +var readFileMaxImageBytes = 20 * 1024 * 1024 // 20MB + +// imageSniffLen is how many leading bytes classify a file by magic bytes — +// the same bound http.DetectContentType itself considers. +const imageSniffLen = 512 + +// readFileImageMediaTypes lists the image formats read_file recognizes and +// returns as a message.Blob for the model to see directly. Every other +// sniffed content type — including any other image/* MIME type +// http.DetectContentType might report, such as image/bmp — keeps +// read_file's existing text-reading behavior unchanged. +var readFileImageMediaTypes = map[string]bool{ + "image/png": true, + "image/jpeg": true, + "image/gif": true, + "image/webp": true, +} + +// sniffMediaType reads up to imageSniffLen bytes from r via io.ReadFull and +// classifies them via http.DetectContentType, returning the classification +// and the sniffed bytes themselves (so a caller reading the rest of the +// stream does not re-read them). Taking an io.Reader, not a path, is what +// lets TestSniffMediaTypeSurvivesShortReads drive it with +// iotest.OneByteReader: a plain single Read against such a source returns +// one byte per call, so a caller using Read directly would misclassify +// almost every real image — io.ReadFull is what makes this deterministic +// against a short-read source (a pipe, a FUSE/network mount, a signal). +func sniffMediaType(r io.Reader) (mediaType string, sniffed []byte, err error) { + buf := make([]byte, imageSniffLen) + n, rerr := io.ReadFull(r, buf) + if rerr != nil && rerr != io.ErrUnexpectedEOF && rerr != io.EOF { + return "", nil, rerr + } + buf = buf[:n] + return http.DetectContentType(buf), buf, nil +} + +// fileContent is the outcome of readPathContent: either a detected image's +// Blob-ready bytes, or a non-image file's raw bytes for the ordinary text +// read. Exactly one of ImageData or TextData is populated. +type fileContent struct { + IsImage bool + ImageData []byte + MediaType string + Width, Height int + TextData []byte +} + +// readPathContent opens path exactly once and, on every outcome, reads it +// exactly once — no second os.Open, no re-read of bytes already +// classified. Earlier revisions of read_file's image detection opened the +// file twice (once to sniff, once via os.ReadFile) even on a plain text +// read, the far more common case; this unifies both outcomes behind one +// handle. +// +// Classification is by magic bytes only (http.DetectContentType via +// sniffMediaType), never by the file's extension: a ".txt" that is really +// a PNG is still recognized; a ".png" that is really text is not. +// +// A recognized image type is read under a bound: readFileMaxImageBytes+1 +// bytes via io.LimitReader over the SAME open handle the sniff used — a +// cap that binds on bytes actually read, never a pre-read os.Stat size a +// concurrently growing file could outrun. An over-cap image returns a +// non-nil error (no partial TextData) so the caller reports a clear error +// instead of falling through to an unbounded read of a file already known +// to be an oversized image. +// +// The image body must also decode with image.DecodeConfig before this +// function commits to the image outcome: a corrupt or truncated file that +// merely opens with a matching magic-byte prefix (PNG's 8-byte signature, +// JPEG's SOI marker, WebP's RIFF/WEBP container) fails PNG/JPEG/WebP's own +// structural header checks. On that failure this reads the true remainder +// of the file (unbounded, via the same handle) so the text fallback is +// complete rather than silently cut off at the image cap — reachable only +// when magic bytes matched an image signature yet the body failed +// structural validation, so an oversized file in this branch is already +// known not to be a real image. This gate is NOT airtight for GIF: the +// GIF87a/GIF89a header has no checksum, so any bytes following a literal +// "GIF87a"/"GIF89a" prefix still "decode" with fabricated width/height. A +// real-world text file beginning with those exact six bytes is +// vanishingly unlikely; this is a documented, accepted residual, not a +// silent gap. +func readPathContent(path string) (fileContent, error) { + f, err := os.Open(path) + if err != nil { + return fileContent{}, err + } + defer f.Close() + + mediaType, sniff, err := sniffMediaType(f) + if err != nil { + return fileContent{}, err + } + if !readFileImageMediaTypes[mediaType] { + rest, err := io.ReadAll(f) + if err != nil { + return fileContent{}, err + } + return fileContent{TextData: append(sniff, rest...)}, nil + } + + budget := int64(readFileMaxImageBytes) - int64(len(sniff)) + if budget < 0 { + budget = 0 + } + capped, err := io.ReadAll(io.LimitReader(f, budget+1)) + if err != nil { + return fileContent{}, err + } + full := append(sniff, capped...) + if len(full) > readFileMaxImageBytes { + return fileContent{}, fmt.Errorf("image (%s) exceeds the %d-byte read_file image limit", mediaType, readFileMaxImageBytes) + } + + cfg, _, derr := image.DecodeConfig(bytes.NewReader(full)) + if derr != nil { + // Sniffed as an image by magic bytes, but the body does not decode + // as one (corrupt, truncated, or a false-positive magic-byte + // match). Read the true remainder so the text fallback is + // complete, not silently cut off at the image cap; see the doc + // comment above for why this is safe. + rest, err := io.ReadAll(f) + if err != nil { + return fileContent{}, err + } + return fileContent{TextData: append(full, rest...)}, nil + } + return fileContent{IsImage: true, ImageData: full, MediaType: mediaType, Width: cfg.Width, Height: cfg.Height}, nil +} + // resolvePath resolves a tool path argument against the session working // directory. Absolute paths pass through unchanged. func (s *Session) resolvePath(path string) string { @@ -34,7 +182,7 @@ func readFileTool() Tool { return Tool{ Def: provider.ToolDef{ Name: "read_file", - Description: "Read a file and return its content with line numbers (N→ prefixes). Prefer this over shell commands like cat, head, or sed for reading files. Relative paths resolve against the session working directory.", + Description: "Read a file and return its content with line numbers (N→ prefixes). A recognized image file (PNG, JPEG, GIF, WebP) is returned as an image where the current provider supports tool-result images. Prefer this over shell commands like cat, head, or sed for reading files. Relative paths resolve against the session working directory.", InputSchema: json.RawMessage(`{ "type": "object", "properties": { @@ -62,10 +210,19 @@ func readFileTool() Tool { if info.IsDir() { return nil, fmt.Errorf("read_file: %s is a directory", path) } - data, err := os.ReadFile(path) + + content, err := readPathContent(path) if err != nil { - return nil, fmt.Errorf("read_file: %w", err) + return nil, fmt.Errorf("read_file: %s: %w", path, err) + } + if content.IsImage { + summary := fmt.Sprintf("image (%s), %d bytes, %dx%d pixels", content.MediaType, len(content.ImageData), content.Width, content.Height) + return message.Parts{ + &message.Text{Text: summary}, + &message.Blob{MediaType: content.MediaType, Data: content.ImageData}, + }, nil } + data := content.TextData lines := strings.Split(string(data), "\n") // A trailing newline produces one empty trailing element; drop it. diff --git a/engine/filetools_test.go b/engine/filetools_test.go index 1a91210..278f165 100644 --- a/engine/filetools_test.go +++ b/engine/filetools_test.go @@ -1,13 +1,17 @@ package engine import ( + "bytes" "context" "encoding/json" "fmt" + "image" + "image/png" "os" "path/filepath" "strings" "testing" + "testing/iotest" "github.com/majorcontext/harness/message" "github.com/majorcontext/harness/plugin" @@ -209,6 +213,230 @@ func TestReadFileAbsolutePath(t *testing.T) { } } +// runToolParts invokes a built-in tool on a throwaway session rooted at +// workDir and returns the raw message.Parts, not just the Text() +// concatenation runTool returns — needed to inspect a returned Blob part. +func runToolParts(t *testing.T, tool Tool, workDir, args string) (message.Parts, error) { + t.Helper() + s := NewSession(Config{WorkDir: workDir}) + return tool.Run(context.Background(), s, json.RawMessage(args)) +} + +// tinyPNG builds a real, compliant, tiny PNG — a 2x2 solid image — so image +// tests exercise genuine image bytes without a committed binary fixture +// (AGENTS.md's fixture-size lesson from #101: keep test images tiny). +func tinyPNG(t *testing.T) []byte { + t.Helper() + img := image.NewRGBA(image.Rect(0, 0, 2, 2)) + var buf bytes.Buffer + if err := png.Encode(&buf, img); err != nil { + t.Fatalf("png.Encode: %v", err) + } + return buf.Bytes() +} + +// TestSniffMediaTypeSurvivesShortReads red-verifies the io.ReadFull sniff +// in sniffMediaType: iotest.OneByteReader wraps the source so every Read +// call returns exactly one byte, the shape a pipe, a FUSE/network mount, or +// a signal-interrupted read(2) can produce. A single plain Read against +// such a source would see only the first byte and misclassify almost every +// real image; io.ReadFull is what makes classification correct regardless +// of how many underlying reads it takes. +func TestSniffMediaTypeSurvivesShortReads(t *testing.T) { + data := tinyPNG(t) + mediaType, sniff, err := sniffMediaType(iotest.OneByteReader(bytes.NewReader(data))) + if err != nil { + t.Fatal(err) + } + if mediaType != "image/png" { + t.Errorf("mediaType = %q, want image/png", mediaType) + } + if !bytes.Equal(sniff, data) { + t.Errorf("sniffed %d bytes, want all %d source bytes (file is under imageSniffLen)", len(sniff), len(data)) + } +} + +func TestReadFileImagePNGReturnsTextAndBlob(t *testing.T) { + dir := t.TempDir() + data := tinyPNG(t) + if err := os.WriteFile(filepath.Join(dir, "shot.png"), data, 0o644); err != nil { + t.Fatal(err) + } + + parts, err := runToolParts(t, readFileTool(), dir, `{"path":"shot.png"}`) + if err != nil { + t.Fatal(err) + } + if len(parts) != 2 { + t.Fatalf("parts = %d, want 2 (Text, Blob): %+v", len(parts), parts) + } + text, ok := parts[0].(*message.Text) + if !ok { + t.Fatalf("parts[0] = %T, want *message.Text", parts[0]) + } + for _, want := range []string{"image/png", fmt.Sprintf("%d bytes", len(data)), "2x2"} { + if !strings.Contains(text.Text, want) { + t.Errorf("summary %q missing %q", text.Text, want) + } + } + blob, ok := parts[1].(*message.Blob) + if !ok { + t.Fatalf("parts[1] = %T, want *message.Blob", parts[1]) + } + if blob.MediaType != "image/png" { + t.Errorf("MediaType = %q, want image/png", blob.MediaType) + } + if !bytes.Equal(blob.Data, data) { + t.Errorf("Blob.Data does not round-trip the source file bytes") + } +} + +// TestReadFileImageExtensionLieTextNamedPNGStaysText is the surplus-direction +// half of the extension-lies pair: a file NAMED .png that actually holds +// plain text bytes must NOT be sniffed as an image. Trusting the extension +// alone would wrongly wrap plain source text in an image Blob. +func TestReadFileImageExtensionLieTextNamedPNGStaysText(t *testing.T) { + dir := t.TempDir() + writeTestFile(t, filepath.Join(dir, "notes.png"), "just some text\nsecond line\n") + + parts, err := runToolParts(t, readFileTool(), dir, `{"path":"notes.png"}`) + if err != nil { + t.Fatal(err) + } + if len(parts) != 1 { + t.Fatalf("parts = %d, want 1 (Text only): %+v", len(parts), parts) + } + text, ok := parts[0].(*message.Text) + if !ok { + t.Fatalf("parts[0] = %T, want *message.Text", parts[0]) + } + if !strings.Contains(text.Text, "1→just some text") { + t.Errorf("out = %q, want ordinary line-numbered text", text.Text) + } +} + +// TestReadFileImageExtensionLiePNGNamedTxtIsSniffedAsImage is the missing- +// direction half: a file named .txt that actually holds PNG magic bytes must +// still be recognized as an image. Extension is a hint only; magic bytes are +// authoritative (AGENTS.md: read_file classifies "never by its extension"). +func TestReadFileImageExtensionLiePNGNamedTxtIsSniffedAsImage(t *testing.T) { + dir := t.TempDir() + data := tinyPNG(t) + if err := os.WriteFile(filepath.Join(dir, "disguised.txt"), data, 0o644); err != nil { + t.Fatal(err) + } + + parts, err := runToolParts(t, readFileTool(), dir, `{"path":"disguised.txt"}`) + if err != nil { + t.Fatal(err) + } + if len(parts) != 2 { + t.Fatalf("parts = %d, want 2 (Text, Blob): %+v", len(parts), parts) + } + blob, ok := parts[1].(*message.Blob) + if !ok { + t.Fatalf("parts[1] = %T, want *message.Blob", parts[1]) + } + if blob.MediaType != "image/png" { + t.Errorf("MediaType = %q, want image/png", blob.MediaType) + } +} + +func TestReadFileImageOverCapReturnsTextErrorNoBlob(t *testing.T) { + dir := t.TempDir() + data := tinyPNG(t) + if err := os.WriteFile(filepath.Join(dir, "shot.png"), data, 0o644); err != nil { + t.Fatal(err) + } + + orig := readFileMaxImageBytes + wantCap := len(data) - 1 + readFileMaxImageBytes = wantCap // force this exact file over cap + t.Cleanup(func() { readFileMaxImageBytes = orig }) + + parts, err := runToolParts(t, readFileTool(), dir, `{"path":"shot.png"}`) + if err == nil { + t.Fatal("want error for over-cap image") + } + if parts != nil { + t.Errorf("parts = %+v, want nil (no Blob) on an over-cap image error", parts) + } + wantSubstr := fmt.Sprintf("%d-byte read_file image limit", wantCap) + if !strings.Contains(err.Error(), wantSubstr) { + t.Errorf("error = %q, want it to contain %q", err, wantSubstr) + } +} + +// TestReadFileImageTruncatedPNGFallsBackToText proves the DecodeConfig gate +// in readPathContent: a file whose first 8 bytes are a genuine PNG +// signature, but whose body is not a real PNG (a truncated download, a +// corrupt write), fails image.DecodeConfig and is read as ordinary text +// instead of shipping a Blob the model cannot use. +func TestReadFileImageTruncatedPNGFallsBackToText(t *testing.T) { + dir := t.TempDir() + pngSignature := []byte("\x89PNG\r\n\x1a\n") + body := append(pngSignature, []byte("not a real IHDR chunk")...) + if err := os.WriteFile(filepath.Join(dir, "broken.png"), body, 0o644); err != nil { + t.Fatal(err) + } + + parts, err := runToolParts(t, readFileTool(), dir, `{"path":"broken.png"}`) + if err != nil { + t.Fatal(err) + } + if len(parts) != 1 { + t.Fatalf("parts = %d, want 1 (Text only, no Blob for an undecodable image): %+v", len(parts), parts) + } + if _, ok := parts[0].(*message.Text); !ok { + t.Fatalf("parts[0] = %T, want *message.Text", parts[0]) + } +} + +// TestReadFileImageToolCallProducesBlobToolResult drives read_file through +// the SAME production dispatch path a real turn uses — an assistant +// ToolCall executed by Session.runToolCalls, not a direct Tool.Run call — +// closing the gap between the engine-level Tool.Run tests above and the +// transcode-level golden test in provider/anthropic/transcode_test.go, +// which hand-builds a ToolResult shaped like read_file's output rather than +// obtaining one from read_file itself (AGENTS.md's "verification drives +// the production entry point" rule). +func TestReadFileImageToolCallProducesBlobToolResult(t *testing.T) { + dir := t.TempDir() + data := tinyPNG(t) + if err := os.WriteFile(filepath.Join(dir, "shot.png"), data, 0o644); err != nil { + t.Fatal(err) + } + s := NewSession(Config{WorkDir: dir}) + asst := &message.Message{ + Role: message.RoleAssistant, + Parts: message.Parts{ + &message.ToolCall{CallID: "call1", Name: "read_file", Arguments: json.RawMessage(`{"path":"shot.png"}`)}, + }, + } + + results := s.runToolCalls(context.Background(), asst) + if len(results) != 1 { + t.Fatalf("results = %d, want 1", len(results)) + } + tr, ok := results[0].(*message.ToolResult) + if !ok { + t.Fatalf("results[0] = %T, want *message.ToolResult", results[0]) + } + if tr.IsError { + t.Fatalf("ToolResult.IsError = true, content: %+v", tr.Content) + } + if len(tr.Content) != 2 { + t.Fatalf("ToolResult.Content = %d parts, want 2 (Text, Blob): %+v", len(tr.Content), tr.Content) + } + blob, ok := tr.Content[1].(*message.Blob) + if !ok { + t.Fatalf("ToolResult.Content[1] = %T, want *message.Blob", tr.Content[1]) + } + if blob.MediaType != "image/png" || !bytes.Equal(blob.Data, data) { + t.Errorf("Blob = %+v, want MediaType image/png and Data matching the source file", blob) + } +} + func TestWriteFileCreatesNestedDirs(t *testing.T) { dir := t.TempDir() out, err := runTool(t, writeFileTool(), dir, `{"path":"a/b/c.txt","content":"hello"}`) diff --git a/provider/anthropic/transcode_test.go b/provider/anthropic/transcode_test.go index 00915be..3e5623e 100644 --- a/provider/anthropic/transcode_test.go +++ b/provider/anthropic/transcode_test.go @@ -3,6 +3,7 @@ package anthropic import ( "encoding/base64" "encoding/json" + "fmt" "strings" "testing" @@ -245,6 +246,63 @@ func TestTranscodeToolCallAndResult(t *testing.T) { } } +// TestTranscodeReadFileImageArrivesAsRealWireImageBlock is the read_file +// counterpart of TestTranscodeToolCallAndResult above: engine/filetools.go's +// read_file tool now returns exactly this shape for an image file — a Text +// summary part ("image (image/png), N bytes, WxH pixels") followed by a Blob +// part carrying the real file bytes (engine/filetools_test.go's +// TestReadFileImagePNGReturnsTextAndBlob proves read_file itself builds this +// shape from its own production entry point, Tool.Run). This test proves the +// OTHER half: that shape, once it reaches a tool_result, transcodes to a +// real wire "image" content block on the Anthropic route — the only route +// that recurses into tool-result Blobs at all (Limits.RecurseToolResults; +// see imageclamp.Limits' doc comment) — with its bytes intact, not a text +// placeholder or an omission note. openai and openaicompat instead replace a +// tool-result Blob with a "[N image attachment(s) omitted]" note +// (toolResultOutput, provider/openai/transcode.go); that is pre-existing, +// unrelated wire-format behavior this PR does not change. +func TestTranscodeReadFileImageArrivesAsRealWireImageBlock(t *testing.T) { + png := tinyPNG(t) + summary := fmt.Sprintf("image (image/png), %d bytes, 2x2 pixels", len(png)) + out := mustTranscode(t, baseRequest( + message.Message{Role: message.RoleUser, Parts: message.Parts{&message.Text{Text: "read shot.png"}}}, + message.Message{Role: message.RoleAssistant, Parts: message.Parts{ + &message.ToolCall{CallID: "toolu_rf", Name: "read_file", Arguments: json.RawMessage(`{"path":"shot.png"}`)}, + }}, + message.Message{Role: message.RoleTool, Parts: message.Parts{ + &message.ToolResult{CallID: "toolu_rf", Content: message.Parts{ + &message.Text{Text: summary}, + &message.Blob{MediaType: "image/png", Data: png}, + }}, + }}, + )) + + res := out.Messages[2] + if res.Role != "user" { + t.Fatalf("tool result role = %s", res.Role) + } + tr := res.Content[0] + if tr.Type != "tool_result" || tr.ToolUseID != "toolu_rf" { + t.Fatalf("tool_result = %+v", tr) + } + if len(tr.Content) != 2 { + t.Fatalf("tool_result content = %d blocks, want 2 (text, image): %+v", len(tr.Content), tr.Content) + } + if tr.Content[0].Type != "text" || tr.Content[0].Text != summary { + t.Errorf("text block = %+v, want text %q", tr.Content[0], summary) + } + img := tr.Content[1] + if img.Type != "image" { + t.Fatalf("second block Type = %q, want %q (a real image block, not a placeholder)", img.Type, "image") + } + if img.Source == nil || img.Source.Type != "base64" || img.Source.MediaType != "image/png" { + t.Fatalf("image Source = %+v", img.Source) + } + if want := base64.StdEncoding.EncodeToString(png); img.Source.Data != want { + t.Errorf("image Source.Data does not round-trip read_file's original bytes") + } +} + // TestTranscodeEmptyToolResultContentNeverOmitsWireField is the red-first // regression test for NEP-5272's B1 finding: a ToolResult with empty // Content used to transcode to a tool_result block whose Content ended up