Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/LEGIBILITY-AUDIT-2026-07-01.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ follow from it.
~2.8×4.4 px. We were paying full price for pixels destroyed in transit.
- **Shipped:** page geometry clamped to **1568×728 = 1.14 MP** (WYSIWYG: billed/actual
pixel ratio 3.25 → 1.04). No token-cost change per image; 614/614 tests green.
- **Update (2026-07-04):** the `px/750` figure above is a ~4–5% continuous
approximation. Anthropic's current docs bill the exact **28-px patch count**
`⌈w/28⌉×⌈h/28⌉` (a 1568×728 page = 56×26 = **1456** tokens, ≈ the px/750 slope
measured here). The gate and export now use that exact formula via
`src/core/anthropic-vision.ts` (tiers: standard 1568/1568, high-res 2576/4784).
- Reading exact strings off even **crisp** 5×8 tops out at **63%** (24-token blind
test). Every miss is one of two classes the glyph-confusability matrix predicts:
**case-normalization** (camelCase) and **single-glyph substitution** (hex/num).
Expand Down
6 changes: 3 additions & 3 deletions docs/RENDER_SIZING.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@ The profitability gate resolves the same profile as the renderer and derives:
- per-image token cost

Changing cell spacing or font therefore cannot leave cost prediction on a
hardcoded 5×8 canvas. Claude uses the Anthropic pixel formula with a conservative
margin, GPT uses model-specific tile/patch pricing, and Grok uses the measured
tokens-per-megapixel rate.
hardcoded 5×8 canvas. Claude uses the Anthropic 28-px patch formula (with a
conservative margin at the gate), GPT uses model-specific tile/patch pricing, and
Grok uses the measured tokens-per-megapixel rate.

## Why not square pages

Expand Down
5 changes: 3 additions & 2 deletions docs/TRANSFORM_INFO.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,9 @@ to re-read that text in token form — Anthropic prompt-caches it, and image
blocks OCR cleanly at small font sizes. So pxpipe pulls the static prefix
out of the JSON body, renders it as one or more grayscale PNG image blocks,
and pins a single `cache_control` breakpoint on the last image. Anthropic
charges roughly `ceil(W*H/750)` tokens per image; the current Anthropic profile
caps pages at 1568×728 so rasterized pixels survive the provider's resize. A
charges `⌈W/28⌉×⌈H/28⌉` visual tokens per image (28-px patches); the current
Anthropic profile caps pages at 1568×728 (so `56×26 = 1456` tokens) and keeps
rasterized pixels under the provider's resize. A
large static slab becomes a small set of image pages on the first turn and a
cache-read (billed at 0.10×) on subsequent turns. The trade is real text tokens
for image tokens cached under the same stable prefix.
Expand Down
119 changes: 119 additions & 0 deletions src/core/anthropic-vision.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
/**
* Anthropic image / vision INPUT-TOKEN cost model.
*
* Anthropic bills images by 28×28-pixel PATCHES, not by a pixel ratio: an image
* costs `⌈width/28⌉ × ⌈height/28⌉` visual tokens, computed AFTER the image is
* downscaled to fit the model tier's long-edge and visual-token limits.
* (The older `(width×height)/750` figure was a ~4–5% continuous approximation of
* this same 28²=784 px²/patch grid; it is no longer the documented formula.)
* https://platform.claude.com/docs/en/build-with-claude/vision
* https://platform.claude.com/docs/en/build-with-claude/vision-coordinates
*
* This module is the single source of truth for that math. It is the *documented
* provider formula* only — any gate conservatism (safety margin) lives at the
* gate, not here, so this stays honest about what Anthropic actually charges.
*/

/** One visual token per 28×28-pixel patch. Also the Qwen2-VL grid; NOT OpenAI's
* 32-px patch / 512-px tile model — keep those on the OpenAI path only. */
export const ANTHROPIC_PATCH_PX = 28;

export interface AnthropicVisionProfile {
readonly tier: 'high-res' | 'standard';
/** Neither side may exceed this after downscale (px). */
readonly maxLongEdge: number;
/** ⌈w/28⌉×⌈h/28⌉ may not exceed this after downscale (visual tokens). */
readonly maxVisualTokens: number;
}

/** Model bases on Anthropic's high-resolution tier (max long edge 2576 px, max
* 4784 visual tokens). Everything else is standard (1568 px / 1568 tokens).
* Source: the Vision docs resolution-tier table. */
const HIGH_RES_BASES = [
'claude-fable-5',
'claude-mythos-5',
'claude-opus-4-8',
'claude-opus-4-7',
'claude-sonnet-5',
] as const;

const HIGH_RES: AnthropicVisionProfile = { tier: 'high-res', maxLongEdge: 2576, maxVisualTokens: 4784 };
const STANDARD: AnthropicVisionProfile = { tier: 'standard', maxLongEdge: 1568, maxVisualTokens: 1568 };

/** Resolve a model's vision tier. Unknown/blank models fall back to the
* conservative (smaller) standard tier. Matches exact base or `<base>-suffix` /
* `<base>[variant]` so aliases (e.g. `claude-fable-5-high`, `...[1m]`) tier alike. */
export function anthropicVisionProfile(model: string | null | undefined): AnthropicVisionProfile {
const m = (model ?? '').toLowerCase();
const isHighRes = HIGH_RES_BASES.some((b) => m === b || m.startsWith(`${b}-`) || m.startsWith(`${b}[`));
return isHighRes ? HIGH_RES : STANDARD;
}

/** Raw 28-px patch count for a `w×h` image, i.e. the visual-token cost when the
* image already fits the tier limits (no downscale). `⌈w/28⌉` inherently pads
* the right/bottom edge up to the next 28-px multiple, as Anthropic documents. */
export function patchTokens(width: number, height: number): number {
const w = Math.max(1, Math.floor(width));
const h = Math.max(1, Math.floor(height));
return Math.ceil(w / ANTHROPIC_PATCH_PX) * Math.ceil(h / ANTHROPIC_PATCH_PX);
}

/** True when a `w×h` image fits a tier with no resize: both padded patch edges
* are within the long-edge limit AND the patch count is within the token
* budget. Mirrors the `fits` predicate in Anthropic's reference resize. */
function fitsTier(w: number, h: number, maxLongEdge: number, maxVisualTokens: number): boolean {
return (
Math.ceil(w / ANTHROPIC_PATCH_PX) * ANTHROPIC_PATCH_PX <= maxLongEdge &&
Math.ceil(h / ANTHROPIC_PATCH_PX) * ANTHROPIC_PATCH_PX <= maxLongEdge &&
patchTokens(w, h) <= maxVisualTokens
);
}

/** Largest aspect-preserving size that fits the tier, exactly as Anthropic's
* reference does it: binary-search the long edge (short edge = round(long ×
* aspect)), recursing with a swap for portrait images. Original if it fits. */
function resizedSize(w: number, h: number, maxLongEdge: number, maxVisualTokens: number): [number, number] {
if (fitsTier(w, h, maxLongEdge, maxVisualTokens)) return [w, h];
if (h > w) {
const [rh, rw] = resizedSize(h, w, maxLongEdge, maxVisualTokens);
return [rw, rh];
}
// Landscape (w ≥ h): binary-search the largest long edge that still fits.
const aspect = h / w;
let lo = 1;
let hi = w;
let best = 1;
while (lo <= hi) {
const mid = (lo + hi) >> 1;
if (fitsTier(mid, Math.max(1, Math.round(mid * aspect)), maxLongEdge, maxVisualTokens)) {
best = mid;
lo = mid + 1;
} else {
hi = mid - 1;
}
}
return [best, Math.max(1, Math.round(best * aspect))];
}

/**
* Anthropic visual-token cost of an image at `width×height` for `model`'s tier.
* Applies the documented resize — the largest aspect-preserving size that fits
* BOTH the long-edge limit and the visual-token budget, found by binary search
* exactly as Anthropic's reference `count_image_tokens` — then the 28-px patch
* count.
*
* Note: pxpipe's own pages are always ≤ 1568×728, so the resize never fires for
* proxy output (both tiers charge the raw patch count). The resize path is for
* correctness as a general-purpose estimator (e.g. arbitrary export input).
*/
export function anthropicVisionTokens(
model: string | null | undefined,
width: number,
height: number,
): number {
const { maxLongEdge, maxVisualTokens } = anthropicVisionProfile(model);
const w = Math.max(1, Math.floor(width));
const h = Math.max(1, Math.floor(height));
const [rw, rh] = resizedSize(w, h, maxLongEdge, maxVisualTokens);
return patchTokens(rw, rh);
}
9 changes: 5 additions & 4 deletions src/core/export.ts
Original file line number Diff line number Diff line change
Expand Up @@ -248,8 +248,9 @@ export function parseExportArgv(
* Per-image vision-token cost for a rendered PNG at the given pixel dimensions.
*
* - **Claude / Anthropic models** (`model.startsWith('claude')` or
* `model.includes('anthropic')`): uses Anthropic's billing formula
* `ceil(width × height / 750 × 1.10)`, matching the live transform.
* `model.includes('anthropic')`): Anthropic's documented 28-px patch model
* `⌈w/28⌉×⌈h/28⌉` (tier-aware downscale), via `visionTokensForModel`. This is
* the exact provider cost — no gate margin — for reporting.
* - **OpenAI-shaped models**: delegates to the same exact-model router as the
* proxy (Grok measured pixel rate; GPT tile/patch rate).
*/
Expand Down Expand Up @@ -280,7 +281,7 @@ export interface ExportTokenReport {
* imageTokens = estimateImageCount(text, cols) × exportImageTokens(model, stripW, MAX_HEIGHT_PX)
* textTokens = sourceText.length / REPORT_CHARS_PER_TOKEN
*
* `exportImageTokens` routes to the Anthropic billing formula (width×height/750×1.10)
* `exportImageTokens` routes to the Anthropic 28-px patch model (⌈w/28⌉×⌈h/28⌉)
* for Claude models, and to the GPT tile-pricing formula for GPT/o-series models.
*
* `factsheetItemCount` is the number of unique precision-critical identifier strings
Expand Down Expand Up @@ -432,7 +433,7 @@ export async function runExportCore(
});

// Compute token costs using actual rendered image dimensions (more accurate than estimate).
// exportImageTokens routes to the Anthropic billing formula for claude-* models and to
// exportImageTokens routes to the Anthropic 28-px patch model for claude-* models and to
// the GPT tile-pricing formula for GPT/o-series models.
const textTokens = Math.round(sourceText.length / REPORT_CHARS_PER_TOKEN);
let imageTokens = 0;
Expand Down
7 changes: 4 additions & 3 deletions src/core/openai.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,10 @@ import {
countVisualRows,
estimateImageCount,
sha8,
ANTHROPIC_PIXELS_PER_TOKEN,
IMAGE_COST_SAFETY_MARGIN,
type TransformInfo,
type TransformOptions,
} from './transform.js';
import { anthropicVisionTokens } from './anthropic-vision.js';
import { stripSchemaDescriptions } from './schema-strip.js';
import {
planGptCollapse,
Expand Down Expand Up @@ -132,7 +131,9 @@ export const GROK_TOKENS_PER_MEGAPIXEL = 1000;
* OpenAI tile/patch formula. Model-based, not endpoint-based. */
export function visionTokensForModel(model: string, w: number, h: number): number {
if (isClaudeModel(model)) {
return Math.ceil((w * h / ANTHROPIC_PIXELS_PER_TOKEN) * IMAGE_COST_SAFETY_MARGIN);
// Anthropic's documented 28-px patch model (tier-aware downscale). This is the
// exact provider cost; the gate applies its own margin separately.
return anthropicVisionTokens(model, w, h);
}
if (isGrokModel(model)) {
const pixels = Math.max(0, w) * Math.max(0, h);
Expand Down
9 changes: 5 additions & 4 deletions src/core/render.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
/**
* Text → PNG renderer. Blits atlas glyphs into a grayscale framebuffer, then PNG-encodes.
* Iterates by codepoint so East Asian Wide chars (2-cell advance) and surrogate pairs handled correctly.
* Pages capped at ~1932×1932 px: Fable/Opus 4.8 >20-image requests are held to ≤2000 px/side
* (REJECTED if exceeded, not silently downscaled); ≤4784 token limit binds first at 1932 px.
* Pages capped at 1568×728 px (1.14 MP): under both Anthropic tiers' limits, so every
* page bills at its raw 28-px patch count with no server-side downscale (WYSIWYG).
*/

import {
Expand Down Expand Up @@ -141,7 +141,8 @@ function grayGlyph(codepoint: number, font: RenderFont | undefined): { atlas: Gr

/** Page-height ceiling. Measured (2026-07-01, count_tokens sweep, claude-sonnet-4-5 — see
* /tmp/pxexp/LEVER1-findings.md): the API downscales any image to fit BOTH long-edge ≤1568
* AND ~1.15 MP (≈1,143,750 px), then bills ≈ px/750 (≈1525 tok cap, applied per-image).
* AND ~1.15 MP (≈1,143,750 px), then bills the exact 28-px patch count ⌈w/28⌉×⌈h/28⌉
* (1568×728 = 56×26 = 1456 tokens/image; the old ≈px/750 slope was the 28²=784 approximation).
* The old 1932×1932 page was billed at cap but resampled 0.555× → 5×8 glyphs reached the
* encoder at ~2.8×4.4 px. New page shape 1568×728 = 1,141,504 px fits both bounds →
* WYSIWYG for the vision encoder (also satisfies ≤2000 px/side for >20-image requests). */
Expand Down Expand Up @@ -1103,7 +1104,7 @@ export async function renderTextToPngs(

const GUTTER_CELLS = 4;
// Width hard-capped at the API's long-edge bound: anything wider is resampled server-side
// (measured 2026-07-01: fit-within 1568-edge AND ~1.15 MP, then ≈px/750 billing).
// (measured 2026-07-01: fit-within 1568-edge AND ~1.15 MP, then 28-px patch billing ⌈w/28⌉×⌈h/28⌉).
const MAX_WIDTH_PX = 1568;

const GUTTER_DIVIDER_INK = 64; // pre-invert → 191 post-invert: light gray column separator
Expand Down
36 changes: 18 additions & 18 deletions src/core/transform.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ import { bytesToBase64 } from './png.js';
import { collapseHistory, HISTORY_SYNTHETIC_INTRO } from './history.js';
import type { GptHistoryOptions } from './openai-history.js';
import { CACHE_CREATE_RATE, CACHE_READ_RATE } from './baseline.js';
import { patchTokens } from './anthropic-vision.js';

/** Per-block descriptor passed to `TransformOptions.keepSharp`. */
export interface KeepSharpBlock {
Expand Down Expand Up @@ -142,7 +143,7 @@ const DEFAULTS: Required<TransformOptions> = {
historyAmortizationHorizon: 1,
priorWarmTokens: 0,
priorWarmImageTokens: 0,
// Multi-col off: single-col slab already holds ~50k chars; extra OCR risk not worth it.
// Multi-col off: single-col slab already holds ~28k chars/page at the 1568×728 cap; extra OCR risk not worth it.
multiCol: 1,
reflow: true,
keepSharp: () => false,
Expand All @@ -162,11 +163,11 @@ export const CLAUDE_CODE_OAUTH_IDENTITY =

// --- per-block break-even check ---
//
// Image token cost is computed from pixel area (Anthropic formula: w×h/750,
// empirically accurate to ~5% on dense PNGs). Constants bias CONSERVATIVE:
// CHARS_PER_TOKEN=4 under-estimates text savings; multi-col cost is linearly
// scaled from single-col + 10% margin. Mispredictions leave money on the
// table; they never generate net-loss images.
// Image token cost uses Anthropic's documented 28-px patch model (see
// src/core/anthropic-vision.ts). Constants bias CONSERVATIVE: CHARS_PER_TOKEN=4
// under-estimates text savings; the gate multiplies the patch count by
// ANTHROPIC_GATE_MARGIN on top. Mispredictions leave money on the table; they
// never generate net-loss images.

/** English ~4 chars per token average (conservative for code/JSON content). */
const CHARS_PER_TOKEN = 4;
Expand All @@ -189,16 +190,11 @@ export const HISTORY_CHARS_PER_TOKEN = 2.0;
* of truth — src/core/export.ts imports this rather than redefining it. */
export const REPORT_CHARS_PER_TOKEN = 3.7;

/** Anthropic image-billing formula: `tokens ≈ width × height / 750`.
* https://docs.anthropic.com/en/docs/build-with-claude/vision#image-tokens
* Accurate to ~5% on dense glyph PNGs (N=14 empirical calibration). The renderer
* sizes height to content, so per-block images cost far less than full-canvas.
* Exported so the export pipeline can reuse the same constant rather than hardcoding. */
export const ANTHROPIC_PIXELS_PER_TOKEN = 750;
/** Conservative 10% upward bias on Anthropic image token estimates — keeps the gate
* on the safe (pass-through) side when the true cost is near the break-even point.
* Exported so the export pipeline reuses the same value. */
export const IMAGE_COST_SAFETY_MARGIN = 1.10;
/** Gate-only conservatism: a 10% upward bias on the patch-count image estimate,
* keeping the gate on the safe (pass-through) side near break-even. This is NOT
* part of Anthropic's documented cost — the documented per-image formula lives in
* `anthropicVisionTokens` (anthropic-vision.ts); this margin only tunes the gate. */
export const ANTHROPIC_GATE_MARGIN = 1.10;

/** Width in px of a single-col PNG. Must stay in sync with `renderChunkToPng` (render.ts). */
function singleColWidthPx(cols: number): number {
Expand Down Expand Up @@ -241,8 +237,12 @@ function imageTokensForRows(
const rowsInLast = Math.min(Math.max(1, linesInLast), rowsPerImage);
const fullImageHeight = 2 * PAD_Y + rowsPerImage * CELL_H;
const lastImageHeight = 2 * PAD_Y + rowsInLast * CELL_H;
const totalPixels = fullImages * widthPx * fullImageHeight + widthPx * lastImageHeight;
return Math.ceil((totalPixels / ANTHROPIC_PIXELS_PER_TOKEN) * IMAGE_COST_SAFETY_MARGIN);
// Anthropic bills per-image by 28-px patches (not by summed pixel area), so
// count each image separately. pxpipe pages are ≤ 1568×728, so no tier
// downscale applies and the raw patch count is tier-agnostic.
const patchSum =
fullImages * patchTokens(widthPx, fullImageHeight) + patchTokens(widthPx, lastImageHeight);
return Math.ceil(patchSum * ANTHROPIC_GATE_MARGIN);
}

/** Exact image-token cost for `text`. Uses `countVisualRows` and optionally
Expand Down
Loading