Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag

### Features

- **Image billing modes (`imageBilling` / `BILI_IMAGE_BILLING`): pixel-tile upstreams no longer get permanently 502'd by byte-based image estimates (#767)**: follow-up to #496/#500 — on official Codex/OpenAI/Anthropic upstreams (pixel-tile billing), once a session's historical `lastInputTokens` baseline sits above the window, the default `base64 length / 4` image cost closes BOTH safe-forward paths at once (#496 forward-once requires a sub-window baseline; #300 stale-baseline fit requires `payloadEstimate < limit`), so preflight fails 502 `preflight_compress_failed` ("Images alone account for ~N tokens") forever even though the images bill only a few thousand tokens upstream (repro: two screenshots, byte estimate ~300K vs real billing ~4.4K). New global + per-provider `imageBilling` (`"auto" | "pixels" | "bytes"`; env `BILI_IMAGE_BILLING` overrides both, live-read per request): `pixels` parses the image header (PNG/JPEG/WebP/GIF/BMP, bounded decode — never the full body) and charges first-party high-detail tile billing (512px tiles, short side scaled up to 768px, long side capped at 2048px → 765–2805 tokens per image; unparsable formats fall back to a flat 16384); `bytes` keeps the conservative default so byte-counting relays retain their #488/#496 protection unchanged. `auto` classifies by upstream host (`openai.com` / `openai.azure.com` / `chatgpt.com` / `api.anthropic.com` suffixes → pixels; everything else → bytes). Remote URLs stay flat 4096 in both modes; `BILI_IMAGE_TOKEN_CAP` still clamps on top of both. A stale baseline now self-heals within one turn of pixel-billed traffic (usage reports replace the baseline absolutely), while #496's one-rejection-then-block guarantee for genuine byte-relay overflow is preserved — pinned by regression tests: recovery e2e (stale 696789 baseline + two large screenshots forwards and recovers under `pixels`), bytes-mode control (same payload still fails fast with zero images forwarded), learned-limit-only variant, and env-precedence check.
- **Session-file encryption at rest behind `BILI_ENCRYPTION_KEY` (#708, PR #711)**: for deployments on untrusted nodes, persisted session files (block summaries plus up to ~16k tokens of folded conversation per session — effectively full conversation content) can now be stored as `BILIENC1` AES-256-GCM over zstd-compressed JSON (zstd via built-in node:zlib on Node ≥ 22.15, raw mode byte on older runtimes; compression also shrinks files ~5–10×). The key is env-only — deliberately NO key-file support, since a key file would sit on the same untrusted filesystem as the data — exactly 32 bytes hex or base64, never written to disk or logs; an invalid value aborts startup (fail fast instead of silently running unencrypted). When enabled, boot takes over legacy plaintext files once: each is re-encoded and atomically replaced in place (the rename IS the old-file deletion; a crash mid-migration self-heals on the next boot), covering `.fb.json` spill files too; unreadable files are left in place with a warning. Wrong key at boot → GCM auth failure → affected sessions skip as corrupt (logged, no crash). Mechanism is the kernel's optional `StateStore` codec hook (acp-kernel#249 → 0.0.64); unset key = zero behavior change.
- **Per-route `passthrough` option for providers (#661, PR #662)**: `"providers": { "mitm://zcode.z.ai": { "passthrough": true } }` forwards matching requests byte-for-byte — no kernel round-trip, no render tags, no `prompt_cache_key` removal, response piped verbatim, no session state — the scoped sibling of the global `passthrough` switch. `mitm://` keys target only MITM (login-client) traffic; plain `https://` host keys cover both MITM and `/bili/` traffic.

Expand Down
27 changes: 25 additions & 2 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,18 @@ Top-level keys that control how the proxy listens and behaves globally.
- **Status:** ACTIVE
- **Description:** Upstream HTTP proxy (`http://host:port`) used for the proxy's **own** outbound connections to model providers. SOCKS5 is not supported. A per-URL `proxy` set inside a `providers` entry overrides this for that provider. An empty string means "explicitly direct" — it disables any environment/system proxy fallback for all providers.

### `imageBilling`

- **Type:** `"auto" | "pixels" | "bytes"`
- **Default:** `"auto"`
- **Status:** ACTIVE
- **Description:** How inline (base64) images are charged by the preflight size gate and output clamp (#488/#496/#767). `"bytes"` charges each image at `base64 length / 4` tokens — conservative, and correct for byte-billing relays. `"pixels"` parses the image header (PNG/JPEG/WebP/GIF/BMP) without decoding the body and charges first-party pixel-tile billing (OpenAI high-detail model: 512px tiles, short side scaled up to 768px, long side capped at 2048px → 765–2805 tokens per image; unparsable formats fall back to a flat 16384). Remote (`https://`) images always charge a flat 4096 in either mode. A per-provider `providers.<url>.imageBilling` wins over this global, and the `BILI_IMAGE_BILLING` env var wins over both (live-read, no restart).

---

## Providers

The `providers` block maps **upstream URLs** to per-provider configuration. Each key is a URL prefix; each value can declare model context windows, a per-provider proxy, a compression protocol, compression overrides, and a per-route passthrough.
The `providers` block maps **upstream URLs** to per-provider configuration. Each key is a URL prefix; each value can declare model context windows, a per-provider proxy, a compression protocol, compression overrides, an image billing mode, and a per-route passthrough.

```jsonc
{
Expand Down Expand Up @@ -201,6 +208,21 @@ A shallow key (`https://open.bigmodel.cn`) matches every path on that host. A de
}
```

### `imageBilling`

- **Type:** `"auto" | "pixels" | "bytes"`
- **Default:** *(global `imageBilling`, then `"auto"`)*
- **Status:** ACTIVE
- **Description:** Per-route override of how inline images are charged by the size gate (#767). Set `"pixels"` for official Codex/OpenAI/Anthropic endpoints (pixel-tile billing) and keep `"bytes"` for byte-counting relays — under byte billing, a stale over-window baseline plus large base64 screenshots fails preflight with 502 forever even though the images bill only a few thousand tokens upstream. When unset at both levels, billing is auto-selected from the upstream host: hosts ending in `openai.com`, `openai.azure.com`, `chatgpt.com`, or `api.anthropic.com` → `pixels`; everything else → `bytes`. The `BILI_IMAGE_BILLING` env var overrides both config levels:

```jsonc
{
"providers": {
"https://chatgpt.com/backend-api/codex": { "imageBilling": "pixels" }
}
}
```

---

## Compression Tuning
Expand Down Expand Up @@ -438,7 +460,8 @@ Environment variables take precedence over the config file. They are useful for
| `ACP_COMPRESS_TOOL` | Set to `0` to disable tool injection (same as `"compress.injectTool": false`). |
| `ACP_COMPRESS_NUDGE` | Set to `0` to disable nudge injection (same as `"compress.injectNudge": false`). |
| `ACP_MODEL_CONTEXT_LIMIT` | Override the context limit globally (absolute token count). |
| `BILI_IMAGE_TOKEN_CAP` | Cap the per-image token estimate used by the preflight size gate and output clamp (#488/#496). By default an inline `data:` image counts as `base64 length / 4` tokens with **no cap** — correct for byte-billing relays, but a large over-estimate for pixel-tile upstreams (official Anthropic/OpenAI), which bill each image at roughly 1.1K–1.6K tokens regardless of byte size. Set this to your upstream's per-image tile cost so the gate reflects real billing; unset = no cap (current default). |
| `BILI_IMAGE_TOKEN_CAP` | Cap the per-image token estimate used by the preflight size gate and output clamp (#488/#496). By default an inline `data:` image counts as `base64 length / 4` tokens with **no cap** — correct for byte-billing relays, but a large over-estimate for pixel-tile upstreams (official Anthropic/OpenAI). For pixel-tile upstreams prefer [`imageBilling`](#imagebilling) (`"pixels"`, or `BILI_IMAGE_BILLING=pixels`) which charges real tile billing instead of capping the byte estimate; the cap still applies on top of both billing modes as a blanket ceiling. Unset = no cap (default). |
| `BILI_IMAGE_BILLING` | Override the image billing mode used by the preflight size gate and output clamp (#767): `pixels` or `bytes`. Live-read per request (no restart); beats the global `imageBilling` and every per-provider `providers.<url>.imageBilling`. Use `bytes` to force conservative billing on a route configured `"pixels"` (e.g. a byte-counting relay behind an OpenAI lookalike host), or `pixels` to enable tile billing process-wide without editing config. See [`imageBilling`](#imagebilling). |
| `BILI_PREFLIGHT_HOLD_MS` | Grace period (ms) before a long preflight compression starts holding the client with keep-alive bytes (default `30000`; see #568 / README "Preflight hold"). |
| `BILI_CONFIG_FILE` | Override the config file path (point at any JSON file). |
| `ACP_PORT` / `PORT` | Override the listen port. |
Expand Down
27 changes: 25 additions & 2 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,18 @@
- **状态:** ACTIVE
- **说明:** 用于代理**自身**到模型 provider 的出站连接的上游 HTTP 代理(`http://host:port`)。不支持 SOCKS5。在 `providers` 条目内设置的按 URL 的 `proxy` 会针对该 provider 覆盖此项。空字符串表示"显式直连" —— 为所有 provider 禁用任何环境/系统代理回退。

### `imageBilling`

- **类型:** `"auto" | "pixels" | "bytes"`
- **默认值:** `"auto"`
- **状态:** ACTIVE
- **说明:** 预检尺寸门与输出钳制对内联(base64)图片的计费方式(#488/#496/#767)。`"bytes"` 按 `base64 长度 / 4` 计 token —— 保守,且对字节计费 relay 正确。`"pixels"` 只解析图片头(PNG/JPEG/WebP/GIF/BMP)、不解码完整图像,按第一方像素 tile 计费(OpenAI high-detail 模型:512px tile、短边放大到 768px、长边封顶 2048px → 每图 765–2805 token;无法解析的格式回退为固定 16384)。远程(`https://`)图片在两种模式下都固定计 4096。按 provider 的 `providers.<url>.imageBilling` 优先于本全局项,而 `BILI_IMAGE_BILLING` 环境变量优先于两者(实时读取,无需重启)。

---

## Providers

`providers` 块将**上游 URL** 映射到按 provider 的配置。每个键是一个 URL 前缀;每个值可以声明模型上下文窗口、按 provider 的代理、压缩协议、压缩覆盖项,以及按路由的透传开关。
`providers` 块将**上游 URL** 映射到按 provider 的配置。每个键是一个 URL 前缀;每个值可以声明模型上下文窗口、按 provider 的代理、压缩协议、压缩覆盖项、图片计费模式,以及按路由的透传开关。

```jsonc
{
Expand Down Expand Up @@ -199,6 +206,21 @@
}
```

### `imageBilling`

- **类型:** `"auto" | "pixels" | "bytes"`
- **默认值:** *(全局 `imageBilling`,再回退 `"auto"`)*
- **状态:** ACTIVE
- **说明:** 按路由覆盖尺寸门的图片计费方式(#767)。官方 Codex/OpenAI/Anthropic 端点(像素 tile 计费)设 `"pixels"`,字节计数 relay 保持 `"bytes"` —— 字节计费下,历史 baseline 超窗加上大 base64 截图会让 preflight 永远 502,而上游实际每图只收几千 token。两级都未显式设置时,按上游 host 自动选择:以 `openai.com`、`openai.azure.com`、`chatgpt.com`、`api.anthropic.com` 结尾的 host → `pixels`,其余 → `bytes`。`BILI_IMAGE_BILLING` 环境变量覆盖两级配置:

```jsonc
{
"providers": {
"https://chatgpt.com/backend-api/codex": { "imageBilling": "pixels" }
}
}
```

---

## 压缩调优
Expand Down Expand Up @@ -436,7 +458,8 @@
| `ACP_COMPRESS_TOOL` | 设为 `0` 禁用工具注入(等同 `"compress.injectTool": false`)。 |
| `ACP_COMPRESS_NUDGE` | 设为 `0` 禁用 nudge 注入(等同 `"compress.injectNudge": false`)。 |
| `ACP_MODEL_CONTEXT_LIMIT` | 全局覆盖上下文上限(绝对 token 数)。 |
| `BILI_IMAGE_TOKEN_CAP` | 预检尺寸门与输出钳制用的单图 token 估算上限(#488/#496)。默认内联 `data:` 图片按 `base64 长度 / 4` 计 token、**无上限** —— 对字节计费 relay 正确,但对像素 tile 计费的官方上游(Anthropic/OpenAI)会严重高估(后者无论字节多少,每图约计 1.1K–1.6K token)。设为你上游的单图 tile 成本,可让尺寸门反映真实计费;不设置 = 无上限(当前默认)。 |
| `BILI_IMAGE_TOKEN_CAP` | 预检尺寸门与输出钳制用的单图 token 估算上限(#488/#496)。默认内联 `data:` 图片按 `base64 长度 / 4` 计 token、**无上限** —— 对字节计费 relay 正确,但对像素 tile 计费的官方上游(Anthropic/OpenAI)会严重高估(后者无论字节多少,每图约计 1.1K–1.6K token)。像素 tile 上游建议改用 [`imageBilling`](#imagebilling)(`"pixels"`,或 `BILI_IMAGE_BILLING=pixels`),按真实 tile 计费而非截断字节估算;该上限仍在两种计费模式之上作为统一天花板生效。不设置 = 无上限(默认)。 |
| `BILI_IMAGE_BILLING` | 覆盖预检尺寸门与输出钳制的图片计费模式(#767):`pixels` 或 `bytes`。每次请求实时读取(无需重启);优先于全局 `imageBilling` 与所有按 provider 的 `providers.<url>.imageBilling`。在配置为 `"pixels"` 的路由上强制保守计费用 `bytes`(例如 OpenAI 同形 host 后面的字节计数 relay),或不想改配置文件就全进程启用 tile 计费用 `pixels`。详见 [`imageBilling`](#imagebilling)。 |
| `BILI_PREFLIGHT_HOLD_MS` | 预压缩超过该宽限期(毫秒)后,代理提前提交响应并用保活字节挂住客户端(默认 `30000`;见 #568 / README「预压缩挂起」)。 |
| `BILI_CONFIG_FILE` | 覆盖配置文件路径(指向任意 JSON 文件)。 |
| `ACP_PORT` / `PORT` | 覆盖监听端口。 |
Expand Down
23 changes: 22 additions & 1 deletion src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { log as loggerLog } from "./logger.js";
import { validateHttpProxy, type ProxyFallbackOptions } from "./upstream-proxy.js";

import { parseCompatRoles } from "./compat-roles.js";
import type { ImageBillingMode } from "./image-tokens.js";
import type { ReasoningGuardConfig } from "./reasoning-guard.js";

export function safeReadJson(path: string): unknown {
Expand Down Expand Up @@ -57,6 +58,12 @@ export type ProviderRoute = {
* verbatim, no session state. For upstreams whose anti-cheat fingerprints
* the request body (e.g. ZCode 405/3012). */
passthrough?: boolean;
/** Per-provider image billing mode (#767): "bytes" = ceil(base64/4)
* (conservative, matches byte-counting relays); "pixels" = dimension-
* based tile estimate (matches first-party pixel-tile upstreams);
* "auto" (default) classifies known first-party pixel hosts. Wins over
* the global `imageBilling`; env BILI_IMAGE_BILLING wins over both. */
imageBilling?: ImageBillingMode;
};
export type ProviderRoutes = Record<string, ProviderRoute>; // key = upstream URL prefix (the /bili/<this> string)

Expand Down Expand Up @@ -322,6 +329,9 @@ export type ProxyOptions = {
* forwarded body for upstreams without the developer role (#552). Empty =
* byte-for-byte transparent. */
compat: { roles: Record<string, string> };
/** Global-level image billing mode (#767); per-provider route entries
* override it, env BILI_IMAGE_BILLING overrides both. undefined = auto. */
imageBilling?: ImageBillingMode;
sessionHeader: string;
log: boolean;
debug: boolean;
Expand Down Expand Up @@ -464,6 +474,7 @@ export function loadOptions(env: NodeJS.ProcessEnv = process.env): ProxyOptions
routing: parsePromptCacheRouting(env.ACP_PROMPT_CACHE_ROUTING ?? fileConfig.promptCache?.routing),
},
compat: { roles: parseCompatRoles(fileConfig.compat?.roles) ?? {} },
imageBilling: parseImageBilling(fileConfig.imageBilling),
sessionHeader: env.ACP_SESSION_HEADER ?? fileConfig.sessionHeader ?? "x-acp-session",
log: env.ACP_LOG !== "0" && fileConfig.log !== false,
debug: (env.ACP_DEBUG ?? (fileConfig.debug ? "1" : "0")) === "1",
Expand Down Expand Up @@ -519,6 +530,10 @@ type FileConfig = {
* upstreams accept (e.g. `{"developer":"system"}`) — applied to the
* final forwarded body for openai/responses requests (#552). */
compat?: { roles?: Record<string, string> };
/** Global image billing mode (#767): "auto" | "pixels" | "bytes".
* Per-provider `imageBilling` overrides it; env BILI_IMAGE_BILLING wins
* over both. See ProviderRoute.imageBilling. */
imageBilling?: string;
};

function nonEmpty(value: string | undefined): string | undefined {
Expand Down Expand Up @@ -593,21 +608,27 @@ export function parseRouteEntry(v: unknown): ProviderRoute | undefined {
// is the KEY in the providers map (identical to the /bili/<url> string),
// so it is NOT repeated inside the value.
if (v && typeof v === "object" && !Array.isArray(v)) {
const obj = v as { models?: Record<string, ModelEntry>; proxy?: string; compressProtocol?: string; compress?: CompressSettings; compat?: { roles?: unknown }; passthrough?: boolean };
const obj = v as { models?: Record<string, ModelEntry>; proxy?: string; compressProtocol?: string; compress?: CompressSettings; compat?: { roles?: unknown }; passthrough?: boolean; imageBilling?: unknown };
const route: ProviderRoute = { models: obj.models };
if (typeof obj.proxy === "string") route.proxy = obj.proxy;
if (obj.compressProtocol === "marker" || obj.compressProtocol === "tools") route.compressProtocol = obj.compressProtocol;
if (obj.compress) route.compress = obj.compress;
const compatRoles = parseCompatRoles(obj.compat?.roles);
if (compatRoles) route.compat = { roles: compatRoles };
if (typeof obj.passthrough === "boolean") route.passthrough = obj.passthrough;
const imageBilling = parseImageBilling(obj.imageBilling);
if (imageBilling) route.imageBilling = imageBilling;
return route;
}
// A bare value (e.g. null) means "this upstream exists, no overrides".
if (v === null) return {};
return undefined;
}

export function parseImageBilling(value: unknown): ImageBillingMode | undefined {
return value === "auto" || value === "pixels" || value === "bytes" ? value : undefined;
}

export function parsePromptCacheRouting(value: string | undefined): PromptCacheRouting {
return value === "enabled" || value === "disabled" ? value : "auto";
}
Expand Down
Loading
Loading