fix(vision): patch-based Anthropic image-token model + doc sync#27
Merged
teamchong merged 5 commits intoJul 18, 2026
Merged
Conversation
Contributor
Author
|
Pushed a follow-up making |
…cols Anthropic no longer bills images by (width*height)/750; the current docs document a 28-px PATCH model: ceil(w/28)*ceil(h/28) visual tokens, after downscaling to the model tier's long-edge / token-budget limits (standard 1568/1568, high-res 2576/4784 for Fable 5 / Mythos 5 / Opus 4.8 / Opus 4.7 / Sonnet 5). /750 was a ~4-5% continuous approximation of that same 28^2 grid and ignored the downscale, so it grossly overcharged large images. - new src/core/anthropic-vision.ts: patchTokens(), anthropicVisionProfile(model), anthropicVisionTokens(model,w,h) with the exact documented downscale (matches Anthropic's reference count_image_tokens and its worked cost table) - gate (imageTokensForRows) now counts per-image 28-px patches; the 10% safety bias is renamed ANTHROPIC_GATE_MARGIN and kept ONLY at the gate (the documented formula in anthropic-vision.ts carries no margin). pxpipe pages are <=1568x728 so no tier downscale applies; net gate shift ~4% - export (exportImageTokens) routes claude models through anthropicVisionTokens (accurate, un-margined reporting) - remove ANTHROPIC_PIXELS_PER_TOKEN / IMAGE_COST_SAFETY_MARGIN (internal only) - transform DEFAULTS.cols 313 -> 312 so the slab is 1568 px (== render.ts MAX_WIDTH_PX; 313 -> 1573 px overshot the cap) - fix shrinkColsToContent's stale 'no-op / returns cols unchanged' docstring (it delegates to measureContentCols) - tests/anthropic-vision.test.ts covers the patch math, tiers, and downscale; export tests updated from the /750 numbers to the patch model Full suite green (638); typecheck + build green.
The docs described a superseded render/billing regime. Bring README, RENDER_SIZING, TRANSFORM_INFO, and the legibility audit in line with the code (render.ts constants) and Anthropic's current 28-px patch cost model: - geometry: 1928×1928 / 384 cols / 1932-px height / ~92k chars-per-page → 1568×728 / 312 cols / 728-px height / ~28k chars-per-page (render.ts today) - shrinkColsToContent: 'no-op, returns cols unchanged' → it delegates to measureContentCols (shrink-to-content is active; RENDER_SIZING's core thesis and the 'why full width' section were inverted) - thresholds: minReminderChars=1000 / minToolResultChars=2000 → 6000 / 6000 - billing: (w·h)/750 → ⌈w/28⌉×⌈h/28⌉ patch model + tiers; a 1568×728 page is 56×26 = 1456 tokens; RENDER_SIZING's 'unresolved billing gap' section is resolved and removed - legibility audit gets a dated note pointing px/750 → the exact patch count Docs-only; no code touched in this commit. Relative links still resolve.
The render.ts header still claimed pages cap at ~1932×1932 px (the pre-clamp regime). They cap at 1568×728 (1.14 MP) today — under both Anthropic tiers' limits, billed at the raw 28-px patch count with no server-side downscale.
…ize reference The downscale used a continuous sqrt seed + decrement loop, which matched Anthropic's worked cost table but drifted on extreme aspect ratios (it scaled both edges by one factor and floored, instead of pinning the long edge and rounding the short one). E.g. standard-tier 2510×7800 billed 1064, not 1008. - replace the approximation with a direct translation of the documented algorithm: fitsTier() (padded patch edges ≤ maxLongEdge AND patch count ≤ budget) + resizedSize() (binary-search the long edge, short edge = round(long × aspect), swap-recurse for portrait). patchTokens() stays the raw no-resize counter. - tests: the full official cost table on both tiers, the three extreme-aspect counterexamples (2510×7800→1064, 1194×4171→952, 2384×1625→1551), and a 400-case deterministic property test vs an independent reference translation. - fix stale comments: transform.ts slab '~50k chars' → '~28k chars/page at the 1568×728 cap'; render.ts MAX_HEIGHT_PX + MAX_WIDTH_PX comments now describe the exact 28-px patch billing (px/750 kept only as the historical measured slope). Only affects the general-purpose estimator (export/reporting); pxpipe pages are ≤1568×728 so the resize never fires for proxy output. Full suite green (639).
… (rebase integration) Rebase-integration step: main refactored all vision-cost math into the unified visionTokensForModel router (openai.ts) after this branch was opened, so the patch model is wired in there instead of the old export.ts path. The Claude branch of visionTokensForModel now calls anthropicVisionTokens; export reporting and the GPT-history estimate inherit it. Drop the now-unused ANTHROPIC_PIXELS_PER_TOKEN / IMAGE_COST_SAFETY_MARGIN imports, refresh the stale w*h/750 doc comments, and update the two tests that hardcoded the old formula (a full 1568x728 page is now 56x26 = 1456 tokens; claude-opus-4-8 768x1932 high-res = 28x69 = 1932). Also drop the obsolete "Claude >5x GPT" export test — with the accurate patch model a full page is ~equal to gpt-4o, which the adjacent not.toBe test already covers.
teamchong
force-pushed
the
fix/anthropic-vision-patch-model
branch
from
July 18, 2026 23:50
3a99a1a to
0cf790d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #26.
Grounds the Anthropic image-token cost on the current documented 28-px patch model and syncs the docs. Verified against Anthropic's primary docs (patch formula + tier table + reference
count_image_tokens, reproduced exactly).src/core/anthropic-vision.ts(new):patchTokens,anthropicVisionProfile(model),anthropicVisionTokens(model,w,h)with the documented long-edge + token-budget downscale (standard 1568/1568, high-res 2576/4784). Matches Anthropic's worked cost table (e.g. 1928² high-res → 4761; 1928² standard → 1521; 1568×728 → 1456).imageTokensForRows) now counts per-image 28-px patches; the 10% conservatism is renamedANTHROPIC_GATE_MARGINand kept ONLY at the gate — the documented formula carries no margin. pxpipe pages are ≤ 1568×728, so no tier downscale ever applies to proxy output and the net gate shift is ~4%.exportImageTokens) routes claude models throughanthropicVisionTokens(accurate, un-margined reporting).DEFAULTS.cols313 → 312 (1568 px ==MAX_WIDTH_PX); fixshrinkColsToContent's stale "no-op" docstring andrender.ts's stale page-cap header.tests/anthropic-vision.test.tscovers the patch math, tiers, and downscale; export tests updated off the/750numbers.Full suite green (638 on this branch); typecheck + build green. (I also verified the whole PR set merges and stays green together — see the integration note.)