Skip to content
filip-papic-bsPublic

About

A pixel-art canvas a person and an AI agent draw on together. Six WebMCP site tools. Built for OpenAI's WebMCP Challenge.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

MCPixels

A wide pixel-art banner drawn in MCPixels: neon vignettes, arcade sprites, framed mini-scenes and a green mascot

A 1024×1024 pixel-art canvas that a person and an AI agent edit together, live, on the same page.

▶ Watch the two-minute demo — the six tools, one shared undo stack, and a ten-frame walk cycle exported as a spritesheet.

The editor is an ordinary pixel editor in any browser — draw, shapes, fill, selections, mirroring, import, PNG export, undo. In a WebMCP browser it also hands an agent six site tools covering everything but image import, so the agent is a real participant rather than a pixel pipe. No backend: the tools are registered by the page itself via document.modelContext.registerTool.

Built for OpenAI's WebMCP Challenge. Live at mcpixels.app.

Run locally

Requires Node 24+ — npm test runs .ts files directly, which needs unflagged native type stripping (Node 23.6 or later).

npm install
npm run dev      # vite dev server
npm test         # pure logic, node:test, no test dependency installed
npm run build    # tsc --noEmit && vite build
npm run lint     # Biome lint checks
npm run format   # format source and config files
npm run check    # format/lint gate, tests, typecheck and production build

npm test covers the canvas primitives (src/pixels.ts), the agent wire format (src/agent/encode.ts), and the six tools driven against a stand-in editor (src/tools.test.ts).

To exercise the agent side:

  • ChatGPT — open the site in the ChatGPT desktop app's built-in browser, then check Site tools in the address bar.
  • Chrome 149+ — enable chrome://flags/#enable-webmcp-testing, then from the console: navigator.modelContext.getTools() and executeTool("draw_pixel_art", { … }).

The six tools

The whole surface costs 1,949 tokens of context, because every tool's description and schema sits in the model's context on every turn. That budget is the reason for most of the design below.

# tool what it does
1 draw_pixel_art The only tool you send pixel data to. Takes a picture (rows of palette characters), a command string (ops — line, rect, ellipse, flood fill, recolour, pixels), or both, plus optional mirror. One call is one undo step, unless it changed nothing.
2 read_canvas Reads any region back in exactly the format draw_pixel_art accepts, with an exact flag saying whether the rows are faithful. A read costs at most ~2,600 tokens whatever is on the canvas.
3 selection Rearranges art already on the canvas: duplicate (with times, for a row in one call), move, erase, flip-left-right, flip-top-bottom, rotate, copy/cut/paste, set/dismiss. Takes an explicit region, falling back to the person's selection when you omit one. No pixel data travels through it.
4 edit undo, redo (up to 20 steps at once), or clear-canvas. The agent and the person share one edit timeline.
5 view Frames a region on screen. Draws nothing — edits bring themselves into view on their own, so this is only for navigation the person asked for.
6 export_pixel_art Saves a region as a PNG to the person's downloads at a chosen scale.

Why pictures instead of coordinates

Sending a full 32×32 sprite as [{x, y, color}, …] costs about 18,000 tokens. As rows of palette characters it costs about 690 — twenty six times less — and the agent can see what it is writing:

draw_pixel_art({
  origin: [-4, -3],
  palette: { k: "#161616", r: "#ff5c35", w: "#f5f1e8" },
  rows: ["..kkkk..", ".krrrrk.", "kr.ww.rk", "krrrrrrk", ".kkkkkk."],
})

. leaves a cell alone, - erases it — stamping never scrubs the art underneath. Larger geometry goes through ops instead — c #2d7ff9;rect -20 -20 20 20 f;bucket 0 0 — and both can be combined in one call. Because read_canvas returns the same shape, read → edit the rows → draw back is a closed loop with no reformatting.

The tools also nudge the agent toward the editor's own capabilities rather than spelling out every pixel: draw_pixel_art may return a single hint when a call had a substantially cheaper form (a mirrored half, or one filled rect). It is advice only — every call always does exactly what was asked.

Watching the agent work

Agent actions appear in a notice feed under the header; the person's own edits stay out of it. When an agent draws off screen the view eases across to frame it and the area flashes, so its work is not invisible — skipped while you are mid-gesture or have the import or export panel open, and switched off entirely with Follow agent edits in the dock's ••• menu.

Layout

src/
  pixels.ts        pure canvas logic: history, geometry, fill, RLE persistence
  agent/           encode.ts (wire format) · tools.ts (the six) · controller.ts
  editor/          state provider, hooks — pointer machine, keyboard, panels
  render/          canvas painter and PNG rasterizer, no React
  components/      presentational UI · icons.tsx (generated)

Agent discovery

The site advertises itself to agents and registries through static documents in public/ plus Link response headers in public/_headers. Everything here is a file on the CDN — no Functions, nothing to run.

path what it is
/.well-known/api-catalog RFC 9727 linkset, application/linkset+json
/.well-known/mcp/server-card.json MCP server card (SEP-1649) for the in-page WebMCP server
/.well-known/ai-catalog.json ARD capability manifest, CORS-open
/.well-known/agent-skills/index.json Agent Skills Discovery index v0.2.0, generated
/.well-known/agent-skills/drawing-on-mcpixels/SKILL.md the skill itself
/llms.txt short orientation for agents
/index.md the homepage as markdown
/auth.md states that there is no authentication, and why
/robots.txt Content-Signal preferences and an Agentmap pointer

Link: headers on / and /index.html carry api-catalog, service-desc, service-doc, alternate (the markdown) and describedby; <head> repeats the same set as <link rel> elements for clients that only parse the body.

Two more entrypoints live in DNS rather than in this repo:

; ARD — points registries at the capability manifest
_catalog._agents.mcpixels.app. 300 IN TXT "url=https://mcpixels.app/.well-known/ai-catalog.json"

; DNS-AID — ServiceMode HTTPS record for the agent entrypoint
_index._agents.mcpixels.app.   300 IN HTTPS 1 mcpixels.app. alpn="h2,http/1.1" port=443

The skills index is regenerated by the first step of npm run build, so editing a SKILL.md updates its sha256 digest without a separate step.

Deploy

Static Cloudflare Pages. No Pages Functions, no usage-billed services.

npm run deploy

Typechecks, builds, and uploads dist to the mcpixels Pages project on main.

License

GNU AGPL-3.0. Free to use, study, modify and share — but if you run a modified version as a network service, section 13 requires you to offer its source to the people using it.

Material Symbols is Apache-2.0 by Google; the glyphs inlined in src/components/icons.tsx keep that license.

About

A pixel-art canvas a person and an AI agent draw on together. Six WebMCP site tools. Built for OpenAI's WebMCP Challenge.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages