Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ dist/
*.tgz
.bili-coop-build-stamp
tmp/
tests/.pp-stamp-state-*/
27 changes: 23 additions & 4 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,12 +432,31 @@ For each request, the proxy resolves the settings by longest-URL-prefix match (t
- `markerText: string` — nudge text appended as a commentary message each continued round (default `"Continue thinking..."`).
- `base: number` / `offset: number` — the lattice signature `tokens == base*n + offset` (defaults `518` / `-2`). Override if another model family truncates on a different lattice.
- `debugLog: boolean` — verbose per-round logging (default `false`).
```jsonc
// enable globally
{ "compress": { "reasoningGuard": { "enabled": true } } }
// tune per provider (placement scopes it to that provider's traffic)
{ "providers": { "https://your-relay.example": { "compress": { "reasoningGuard": { "enabled": true, "maxContinue": 2 } } } } }
```

#### `priceProfile`

- **Type:** `object` (`{ w?, r?, q? }`, all non-negative numbers)
- **Default:** *(unset — reports then price folds from the request model's models.dev price row in absolute $/Mtok; only models the registry cannot resolve fall back to the kernel's built-in relative ratios `{ w: 1, r: 0.1, q: 4 }`)*
- **Status:** ACTIVE
- **Description:** Price profile for the **cache-economics verdicts** in the session cache report (`acp_cache` tool / `/acp-cache` command / `GET /__bili/cache-report`, #800/#1279). The per-fold P&L fields (`oneTimeCostUnits`, `perTurnSavingUnits`, `breakevenTurns`, `paidBack`) are computed from three multipliers over the input-token unit: `w` (cache-write cost), `r` (cache-read cost), `q` (output cost). Two unit conventions coexist, both printed verbatim in the report header (`FOLD ECONOMICS (N folds @ w=.. r=.. q=..)`):
- **User config** uses **ratios normalized to the input price (p_in = 1)**: `w` = cacheWrite ÷ input, `r` = cacheRead ÷ input, `q` = output ÷ input. Sub-fields merge deepest-wins across the three levels like every other CompressSettings field (set `q` at provider level, refine one field at model level); fields left unset within a partial profile fall back to the kernel ratios `w: 1`, `r: 0.1`, `q: 4`.
- **Registry default** (no level sets the key): derived from the request model's models.dev price row — **absolute $/Mtok**, `w = cost.input`, `r = cost.cache_read ?? 0.1 × input`, `q = cost.output ?? 1.5 × input` (convention fallbacks for rows without those fields). Direct-to-provider traffic gets that host's own listing; unknown relays get the first matching listing across hosts (with a one-time warning when listings conflict). Live registry wins when reachable, bundled snapshot is the offline floor (#282).
User config wins wholesale — a profile set at any level is never mixed field-by-field with the registry row. The last request's effective value is stamped onto the session, so every report face prices folds with the profile that governed that session's most recent turn. **Report-only**: the profile never affects compression triggers, cadence, or any wire behavior. User-config examples (override the registry row, e.g. for relays with custom markup):
```jsonc
// enable globally
{ "compress": { "reasoningGuard": { "enabled": true } } }
// tune per provider (placement scopes it to that provider's traffic)
{ "providers": { "https://your-relay.example": { "compress": { "reasoningGuard": { "enabled": true, "maxContinue": 2 } } } } }
// DeepSeek-V3 ≈ low output multiple
{ "providers": { "https://api.deepseek.com": { "compress": { "priceProfile": { "w": 1, "r": 0.1, "q": 1.5 } } } } }
// OpenAI GPT-4o/o-series: 50% cached-read discount, flat writes, 4× output
{ "providers": { "https://api.openai.com": { "compress": { "priceProfile": { "w": 1, "r": 0.5, "q": 4 } } } } }
// Self-hosted / free tier: everything costs zero tokens of your budget
{ "compress": { "priceProfile": { "w": 0, "r": 0, "q": 0 } } }
```
Use list prices relative to the same model's normal input price; relays with custom markup should use their effective rates.

#### `outputSteering`

Expand Down
27 changes: 23 additions & 4 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,12 +432,31 @@
- `markerText: string` — 每个续写回合追加的 commentary 提示文本(默认 `"Continue thinking..."`)。
- `base: number` / `offset: number` — 晶格签名 `tokens == base*n + offset`(默认 `518` / `-2`)。若其他模型家族在不同晶格上截断则覆盖。
- `debugLog: boolean` — 逐回合详细日志(默认 `false`)。
```jsonc
// 全局开启
{ "compress": { "reasoningGuard": { "enabled": true } } }
// 按 provider 调参(放在哪一层就作用于哪一层的流量)
{ "providers": { "https://your-relay.example": { "compress": { "reasoningGuard": { "enabled": true, "maxContinue": 2 } } } } }
```

#### `priceProfile`

- **类型:** `object`(`{ w?, r?, q? }`,均为非负数)
- **默认值:** *(未设置 —— 报告改用请求模型在 models.dev 的价格行计价(绝对 $/Mtok);只有注册表解析不到的模型才回落到内核内置相对比例 `{ w: 1, r: 0.1, q: 4 }`)*
- **状态:** ACTIVE
- **说明:** 会话缓存报告(`acp_cache` 工具 / `/acp-cache` 命令 / `GET /__bili/cache-report`,#800/#1279)中**压缩经济学判定**所用的价格档。每个 fold 的损益字段(`oneTimeCostUnits`、`perTurnSavingUnits`、`breakevenTurns`、`paidBack`)由三个基于输入 token 单位的乘数计算得出:`w`(cache 写入成本)、`r`(cache 读取成本)、`q`(output 成本)。两种单位约定并存,且都会原样打印在报告头部(`FOLD ECONOMICS (N folds @ w=.. r=.. q=..)`):
- **用户配置**采用**相对输入价归一化(p_in = 1)的比例**:`w` = cacheWrite ÷ input,`r` = cacheRead ÷ input,`q` = output ÷ input。子字段与其他 CompressSettings 字段一样按“深层覆盖”三级合并(provider 层设 `q`、model 层精调单个字段均可);部分配置中未设置的字段回落到内核比例 `w: 1`、`r: 0.1`、`q: 4`。
- **注册表默认**(任何层级都未设置该键时):由请求模型在 models.dev 的价格行推导——**绝对 $/Mtok**,`w = cost.input`,`r = cost.cache_read ?? 0.1 × input`,`q = cost.output ?? 1.5 × input`(缺这些字段的行用惯例回落值)。直连供应商流量取该 host 自己的挂牌行;未知中转站取跨 host 第一个匹配行(挂牌冲突时一次性告警)。可达时实时注册表优先,随包快照为离线兜底(#282)。
用户配置整体胜出——任何层级设置了 profile 都不会与注册表行逐字段混用。最近一次请求生效的值会被戳记到会话上,因此所有报告出口都用该会话最近一轮所适用的价格档计价。**纯报表面**:价格档绝不影响压缩触发、频率或任何 wire 行为。用户配置示例(覆盖注册表行,例如中转站有自定义加成时):
```jsonc
// 全局开启
{ "compress": { "reasoningGuard": { "enabled": true } } }
// 按 provider 调参(放在哪一层就作用于哪一层的流量)
{ "providers": { "https://your-relay.example": { "compress": { "reasoningGuard": { "enabled": true, "maxContinue": 2 } } } } }
// DeepSeek-V3 ≈ output 倍数低
{ "providers": { "https://api.deepseek.com": { "compress": { "priceProfile": { "w": 1, "r": 0.1, "q": 1.5 } } } } }
// OpenAI GPT-4o/o 系列:缓存读取五折、写入平价、output 4×
{ "providers": { "https://api.openai.com": { "compress": { "priceProfile": { "w": 1, "r": 0.5, "q": 4 } } } } }
// 自托管 / 免费额度:一切不消耗你的 token 预算
{ "compress": { "priceProfile": { "w": 0, "r": 0, "q": 0 } } }
```
请用同一模型正常输入价的相对挂牌价;有自定义加成的中转站应填实际生效费率。

#### `outputSteering`

Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -883,6 +883,12 @@ API: `bili plugin install claude` writes a model-mediated
`commands/acp-cache.md` markdown command whose prompt drives the `acp_cache`
MCP tool and pastes the report back verbatim. codex/kimi/hermes expose no
user-typable command seam — ask the model to call its `acp_cache` tool directly.
Per-fold P&L verdicts are priced by the optional `compress.priceProfile`
(normalized multipliers over the input-token unit); when no level sets it, the
request model's models.dev price row applies in absolute $/Mtok (kernel ratio
defaults only for unresolvable models) — breakeven/PAID BACK therefore reflect
your upstream's actual economics out of the box; override per provider for
relays with custom markup (CONFIGURATION.md, #1279).

The same seam carries `/acp-rule` (#1251) — the human entry point to the
persistent-rules feature (identical output to the `acp_rule` tool): pi/omp
Expand Down
48 changes: 40 additions & 8 deletions scripts/update-registry-snapshot.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,20 @@ The FULL registry is committed to the repo and bundled into dist at build
models.dev is unreachable (and no upstream proxy is configured) still has
the entire dataset — every field models.dev ships (name, description,
family, reasoning, tool_call, modalities, limits, benchmarks, …), not just
context windows. Today only limit.context is consumed; future features
(pricing, provider metadata, modality checks) get the offline floor for
free.
context windows. limit.context and per-model pricing (cost rows) are
consumed today; everything else rides along for future features.

Source endpoint: https://models.dev/catalog.json — NOT models.json.
models.json carries NO pricing fields at all (verified 0 cost keys across
all 427 models); the per-model $/Mtok cost rows (input/output/cache_read/…)
live nested under providers.<host>.models.<id>.cost in catalog.json, whose
top-level `models` object is identical to models.json (verified entry-for-
entry), so windows data is unchanged while the offline floor gains prices
(#1279 follow-up). The snapshot stores both: `models` (flat, as before) and
`costs` flattened to "<host>/<model-id>" keys, keeping host-specific
pricing distinct (the same model listed by several hosts at different
prices). Only rows with a usable numeric input price are stored — without
an input anchor there is nothing to normalize against.

Run manually or before a release:
npm run registry:snapshot
Expand All @@ -21,7 +32,7 @@ import { writeFile } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";

const REGISTRY_URL = "https://models.dev/models.json";
const REGISTRY_URL = "https://models.dev/catalog.json";
const OUT_FILE = path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "src", "registry-snapshot.json");
const proxyUrl = process.env.https_proxy || process.env.HTTPS_PROXY || process.env.http_proxy || process.env.HTTP_PROXY;

Expand Down Expand Up @@ -51,16 +62,37 @@ async function fetchFull() {
return full;
}

let full;
let catalog;
try {
full = await fetchFull();
catalog = await fetchFull();
} catch (e) {
console.error(`could not fetch ${REGISTRY_URL}: ${e.message}`);
console.error(`keeping the existing ${path.basename(OUT_FILE)} untouched`);
process.exit(1);
}

const slim = { fetchedAt: new Date().toISOString(), count: Object.keys(full).length, models: full };
if (!catalog || typeof catalog !== "object" || !catalog.models || typeof catalog.models !== "object" || Array.isArray(catalog.models)) {
console.error(`unexpected catalog.json shape (missing top-level "models") — refusing to overwrite the snapshot`);
process.exit(1);
}

function flattenCosts(providers) {
const costs = {};
if (!providers || typeof providers !== "object" || Array.isArray(providers)) return costs;
for (const [pid, p] of Object.entries(providers)) {
const models = p && typeof p === "object" ? p.models : null;
if (!models || typeof models !== "object" || Array.isArray(models)) continue;
for (const [mid, m] of Object.entries(models)) {
const c = m && typeof m === "object" ? m.cost : null;
if (!c || typeof c !== "object" || Array.isArray(c)) continue;
if (typeof c.input !== "number" || !Number.isFinite(c.input) || c.input <= 0) continue;
costs[`${pid}/${mid}`] = c;
}
}
return costs;
}

const slim = { fetchedAt: new Date().toISOString(), count: Object.keys(catalog.models).length, models: catalog.models, costs: flattenCosts(catalog.providers) };
const body = JSON.stringify(slim) + "\n";
await writeFile(OUT_FILE, body, "utf8");
console.log(`wrote ${OUT_FILE} (${slim.count} models, FULL registry, ${(body.length / 1024).toFixed(1)} KB)`);
console.log(`wrote ${OUT_FILE} (${slim.count} models, ${Object.keys(slim.costs).length} cost rows, ${(body.length / 1024).toFixed(1)} KB)`);
26 changes: 24 additions & 2 deletions src/cache-ledger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
type CacheTotals,
type CompressionBlock,
type FoldEvent,
type PriceProfile,
} from "acp-kernel";
import { log as loggerLog } from "./logger.js";
import type { Session } from "./session.js";
Expand Down Expand Up @@ -220,8 +221,29 @@ export function recordCacheSample(session: Session, s: { at: number; input: numb
agg.tr += dec.ttlRepay;
}

/** [#1279] Price profile stamped by the last request (server.ts runPrepare).
* Metadata is persisted user-editable JSON, so re-validate on read: only
* finite non-negative numbers survive — a corrupt stamp degrades to the
* kernel defaults instead of poisoning the report. */
function stampedPriceProfile(session: Session): PriceProfile | undefined {
const v = session.metadata?.cachePriceProfile;
if (!v || typeof v !== "object" || Array.isArray(v)) return undefined;
const o = v as Record<string, unknown>;
const out: PriceProfile = {};
for (const key of ["w", "r", "q"] as const) {
const n = o[key];
if (typeof n === "number" && Number.isFinite(n) && n >= 0) out[key] = n;
}
return Object.keys(out).length > 0 ? out : undefined;
}

export function buildSessionCacheReport(session: Session): CacheReport {
const led = getCacheLedger(session);
// Effective profile = stamped value over kernel defaults (w=1, r=0.1, q=4),
// mirroring the per-field fallback inside computeFoldEconomics. Unstamped
// sessions keep the pre-#1279 Anthropic-ratio output byte-for-byte.
const price = stampedPriceProfile(session);
const effective: Required<PriceProfile> = { w: price?.w ?? 1, r: price?.r ?? 0.1, q: price?.q ?? 4 };
const a = led.agg;
const totals: CacheTotals = {
requests: a.requests,
Expand All @@ -247,11 +269,11 @@ export function buildSessionCacheReport(session: Session): CacheReport {
T: f.T,
requestsAfter: f.requestsAfter,
turnsToNextFold: f.k,
}),
}, effective),
);
return {
generatedAt: Date.now(),
profile: { w: 1.0, r: 0.1, q: 4.0 },
profile: effective,
totals,
economics: summarizeFoldEconomics(folds),
folds,
Expand Down
2 changes: 2 additions & 0 deletions src/compress-settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ export function mergeCompress(
const reasoningLevels = [global?.reasoning, provider?.reasoning, model?.reasoning].filter(Boolean) as NonNullable<CompressSettings["reasoning"]>[];
const reasoningGuardLevels = [global?.reasoningGuard, provider?.reasoningGuard, model?.reasoningGuard].filter(Boolean) as NonNullable<CompressSettings["reasoningGuard"]>[];
const outputSteeringLevels = [global?.outputSteering, provider?.outputSteering, model?.outputSteering].filter(Boolean) as NonNullable<CompressSettings["outputSteering"]>[];
const priceProfileLevels = [global?.priceProfile, provider?.priceProfile, model?.priceProfile].filter(Boolean) as NonNullable<CompressSettings["priceProfile"]>[];
return {
modelContextLimit: pick("modelContextLimit"),
outputHeadroomMaxPct: pick("outputHeadroomMaxPct"),
Expand Down Expand Up @@ -94,6 +95,7 @@ stripImages: pick("stripImages"),
reasoning: reasoningLevels.length > 0 ? Object.assign({}, ...reasoningLevels) : undefined,
reasoningGuard: reasoningGuardLevels.length > 0 ? Object.assign({}, ...reasoningGuardLevels) : undefined,
outputSteering: outputSteeringLevels.length > 0 ? Object.assign({}, ...outputSteeringLevels) : undefined,
priceProfile: priceProfileLevels.length > 0 ? Object.assign({}, ...priceProfileLevels) : undefined,
promptPack: pick("promptPack"),
};
}
Expand Down
43 changes: 36 additions & 7 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -355,14 +355,27 @@ export type CompressSettings = {
* across the three levels like `absorb`/`reasoning`; off unless enabled at some
* level. See src/reasoning-guard.ts. */
reasoningGuard?: ReasoningGuardConfig;
/** [#1093] Output-side compression levers — verbosity steering (a conciseness
* directive appended to the system-prompt tail) and effort routing (clamp an
* already-sent effort field down on mechanical continuation turns). Resolved
* through this same three-level cascade; sub-fields are validated by the
* kernel's resolveOutputSteeringConfig at resolution time (an out-of-range value
* falls back to its default with a warning rather than rejecting the whole block).
* Off unless enabled at some level. See src/output-steering.ts. */
/** [#1093] Output-side compression levers — verbosity steering (a conciseness
* directive appended to the system-prompt tail) and effort routing (clamp an
* already-sent effort field down on mechanical continuation turns). Resolved
* through this same three-level cascade; sub-fields are validated by the
* kernel's resolveOutputSteeringConfig at resolution time (an out-of-range value
* falls back to its default with a warning rather than rejecting the whole block).
* Off unless enabled at some level. See src/output-steering.ts. */
outputSteering?: Partial<OutputSteeringConfig>;
/** [#1279] Price profile for the session cache-economics report
* (`acp_cache` / `/acp-cache` / `GET /__bili/cache-report`). Normalized
* multipliers over the input-token unit (p_in = 1): `w` = cacheWrite/input,
* `r` = cacheRead/input, `q` = output/input — e.g. Anthropic ≈ {w:1, r:0.1,
* q:4}, DeepSeek-V3 ≈ {w:1, r:0.1, q:1.5}. Drives ONLY the per-fold
* one-time-cost / breakeven-turns / paid-back verdicts in the report — it
* never affects compression triggers, cadence, or any wire behavior. Each
* unset field falls back to the kernel default (w=1, r=0.1, q=4); a fully
* unset profile yields byte-identical reports to pre-#1279 behavior. The
* last request's effective value is stamped onto the session and used by
* every report face. Merged sub-field-wise across the three levels like
* `absorb`. */
priceProfile?: { w?: number; r?: number; q?: number };
};
export type PromptCacheRouting = "auto" | "enabled" | "disabled";
export type UpstreamProxyMode = "auto" | "manual" | "direct";
Expand Down Expand Up @@ -1388,6 +1401,22 @@ export function parseCompressSettings(v: unknown): (CompressSettings & { injectT
out.outputSteering = os as Partial<OutputSteeringConfig>;
}
}
if ("priceProfile" in obj && obj.priceProfile !== undefined) {
const pp = obj.priceProfile;
if (!pp || typeof pp !== "object" || Array.isArray(pp)) {
ok = false;
} else {
const ppo = pp as Record<string, unknown>;
const cleaned: NonNullable<CompressSettings["priceProfile"]> = {};
for (const key of ["w", "r", "q"] as const) {
if (!(key in ppo)) continue;
const v = ppo[key];
if (typeof v !== "number" || !Number.isFinite(v) || v < 0) { ok = false; continue; }
cleaned[key] = v;
}
if (ok) out.priceProfile = cleaned;
}
}
if (!ok) return undefined;
return out;
}
Expand Down
2 changes: 1 addition & 1 deletion src/registry-snapshot.json

Large diffs are not rendered by default.

Loading
Loading