Skip to content

[PR] fix: preserve image recovery for escaped JSON markers - #6701

Merged
ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-14_escaped-image-markers
Nov 14, 2026
Merged

ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-14_escaped-image-markers

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Problem

The raw-body image precheck only recognizes literal spellings of its JSON markers. A valid request can encode ASCII string characters with Unicode escapes; after JSON parsing it contains the same image, but the precheck returns false. The server consequently skips both the initial image index and its post-prepare refresh, leaving no archived pixels available for image recovery.

This affects OpenAI and Responses data URLs, Anthropic base64 sources, and Google inlineData. A real HTTP regression sends the same PNG through the public proxy with literal and escaped markers: the literal request has an image index, while the escaped request has an empty one on unchanged base.

Change

Compile the small marker patterns once, accepting each marker character as literal text or its JSON Unicode escape. Preserve case-insensitive data-URL tokens and case-sensitive JSON field names. The precheck still scans the raw body directly; it neither parses a second copy nor decodes/copies the full request. This implements the string spelling equivalence in RFC 8259 section 7.

Verification

  • Fifteen new tests: all four protocols use real kernel converters and the actual image-index builder, assert decoded PNG bytes, and compare guarded indexing with unconditional indexing. Other cases cover fully escaped keys/tokens/separators, uppercase hex digits, existing case-insensitive data-URL recognition, and escaped image-free text.
  • The public HTTP test starts the actual proxy and an upstream server, sends both wire spellings, and checks the resulting session's image index and archived PNG bytes.
  • On unchanged base 9495c764d36909eec263f404d4802d68b608f23f, the new suite has 7 failing regressions and 8 passing controls. After the fix, 34/34 related tests pass: all 15 new tests, the 15 existing image-recovery tests, and the four existing [性能回归] 0.1.193 长会话本地耗时从 1.3s 升至 3.2s:无用 SimHash 计算与重复历史转换 #2824 guard/index parity tests.
  • Full npm run typecheck, npm run build, and git diff --check pass. Full repository tests were not run locally; hosted CI remains pending.
  • A separate synthetic 16 MiB scan comparison was used to reject an earlier full-body-normalization approach. The submitted change compiles marker regexes once and retains the image-free fast path. This is a correctness fix, not a claim of faster scans for every text distribution.

Prior work checked

The complete Afloat16 production-patch index has only #6089 (incomplete image spill repair) and #6147 (Gemini tool-response image parity) touching this file; neither changes the raw-body guard. The original #2828 guard and the actual head of related #5901 retain the literal-only patterns. #6646 has no file patch. All-state escaped-image/index searches found no matching fix.

Closes #6700

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Escaped JSON image markers bypass recovery indexing

2 participants