Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@ this package adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- Replace the `global.forceDefaultOpsPtc` switch with one explicit action: the row's detail page carries a **Make OPS (PTC) the default** button that writes `selectedDefault: ops-ptc` once and answers `applied`/`no-update`/`unavailable`/`failed`. Apply, remount and unload no longer write the default, so a preset chosen on the settings page is not overridden on the next apply, and a saved `global.forceDefaultOpsPtc` key still loads with no effect.
- 用一次显式操作取代 `global.forceDefaultOpsPtc` 开关:行详情页有一个「设为 OPS (PTC) 默认」按钮,点一次写 `selectedDefault: ops-ptc`,并回报 `applied`/`no-update`/`unavailable`/`failed`。apply、重挂载与卸载都不再改写默认,设置页选定的预设不会在下一次 apply 被覆盖;已保存的 `global.forceDefaultOpsPtc` 键仍可加载但不再生效。

- `ops_edit_file` now accepts text copied from the reversibly escaped Native reader: raw `oldText` matches keep precedence, and only a raw miss tries the decoded rendering. The fallback also decodes `newText`, reports `renderedMatches`, and preserves ambiguity/CAS failures; raw edits and `write_file` remain byte-verbatim (#939).
- `ops_edit_file` 现在可接受从可逆转义的 Native 读取输出中复制的文本:优先匹配原始 `oldText`,只有原始匹配失败才尝试解码后的渲染形式。回退同样解码 `newText` 并返回 `renderedMatches`,歧义与 CAS 拒绝保持不变;原始编辑与 `write_file` 仍逐字节处理(#939)。
- The bounded classic-xref PDF reader resolves indirect `/Length` values in either object order while keeping cycle, depth, type, size and mismatch refusals. Direct lengths and the existing stream/xref budgets are unchanged (#788).
- 有界的经典 xref PDF 读取器现在可按任意对象顺序解析间接 `/Length`,同时保留循环、深度、类型、大小与不一致拒绝。直接长度及现有流/xref 预算不变(#788)。

## [0.5.5] — 2026-11-13 — Patch release

- A loader-less PTC scope no longer registers the configured `allowedTools` pool: with dynamic loading off nothing could ever select those APIs, and a host assembly without a step tool set declared all sixteen in `tools:sdk`. Resident publication, the default inspect + loader set and selection behavior are unchanged (#887).
Expand Down
6 changes: 4 additions & 2 deletions docs/file-batching.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,10 @@ structured.

`ops_edit_file` applies 1-100 edits to **one** UTF-8 file, atomically. Supply `path`, the exact
SHA-256 `expectedRevision` from a read/stat, and `edits: [{oldText,newText}]`. Each nonempty
`oldText` must occur exactly once in the same original bytes; overlaps, ambiguity or stale
revision reject all edits before publication. Empty `newText` deletes the match. Preserve
`oldText` must occur exactly once in the same original bytes. If that raw match fails, one
fallback decodes the reversible Native rendering of both `oldText` and `newText`; a raw match
never decodes either field, and a fallback receipt counts `renderedMatches`. Overlaps, ambiguity
or stale revision reject all edits before publication. Empty `newText` deletes the match. Preserve
CRLF/BOM explicitly. `dryRun: true` previews the same receipt without writing; read-only policy
still rejects mutation tools. This is not a cross-file transaction.

Expand Down
2 changes: 1 addition & 1 deletion docs/native-kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ reference pages retains the declared main-body scope independently of stored-row

The internal PDF container parser accepts a restricted PDF 1.0-1.7 subset: one classic
xref table, an exact final startxref/EOF footer, unique dictionary keys and object
identities, and directly declared stream lengths. Every active xref offset must point
identities, and stream lengths declared either directly or through an indirect reference resolved by the classic xref table. Every active xref offset must point
exactly at its object header. Objects cannot overlap; gaps must contain only whitespace
or comments. This deliberately rejects incremental updates, encryption, object/xref
streams, recovery scans and ambiguous duplicate data. `/Encrypt`, `/ObjStm` and `/XRef`
Expand Down
2 changes: 1 addition & 1 deletion docs/native-ranges.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ A fully returned `ops_read_files` range uses one path header, exact numbered tex
[revision=cfd1bc81432fd6b88748b827f46bb4242db17cf8655cf4ea90f3ba0e62065355; more lines; nextOffset=3]
```

The characters after `N|` are the source line, including leading spaces. Line bodies use reversible JSON escapes: backslashes are doubled, controls and Unicode line separators are escaped, and quotes remain literal. For example, an actual tab appears as `\t`, while the two source characters backslash-plus-t appear as `\\t`. This shared encoding applies to compact ranges and ordinary numbered source output, and prevents source text from forging extra rows or headers. LF/CRLF accounting remains the worker's physical-line accounting.
The characters after `N|` are the source line, including leading spaces. Line bodies use reversible JSON escapes: backslashes are doubled, controls and Unicode line separators are escaped, and quotes remain literal. For example, an actual tab appears as `\t`, while the two source characters backslash-plus-t appear as `\\t`. This shared encoding applies to compact ranges and ordinary numbered source output, and prevents source text from forging extra rows or headers. LF/CRLF accounting remains the worker's physical-line accounting. `ops_edit_file` first matches literal bytes; only after a raw miss may it decode this rendered form for `oldText` and `newText`, with `renderedMatches` in the receipt. It never decodes a raw match or `write_file` content.

“More lines” means the file has not been exhausted; use the stated `nextOffset`. “End of file; complete=true” means this range reached the end. The aggregate “requested ranges returned” receipt means every requested range body was returned, not that every whole file was read. Full revisions, total line counts and per-file continuation remain visible. Byte sizes and duplicate counters remain available in canonical/PTC data. Per-file revision receipts omit the repeated `Notice: ` prefix and terminal period; the aggregate `Notice` and every revision/continuation/completeness fact remain intact.

Expand Down
2 changes: 1 addition & 1 deletion docs/原生内核.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ FILE_CHANGED 始终使所请求页不完整。引用页生成的 source 类型
## PDF 解析基础(15c/16)

内部 PDF 容器解析器仅接受受限的 PDF 1.0–1.7 子集:单个传统 xref 表、
准确的末尾 startxref/EOF、唯一字典键及对象标识,以及直接整数形式的流长度。
准确的末尾 startxref/EOF、唯一字典键及对象标识,以及直接整数或经经典 xref 表解析的间接引用形式的流长度。
每个有效 xref 偏移必须准确指向对象头;对象不可重叠,间隙只能包含空白或注释。
增量更新、加密、对象流/xref 流、恢复扫描及重复歧义数据均被拒绝。
`/Encrypt`、`/ObjStm` 与 `/XRef` 无论出现在哪里都会被拒绝,名称转义后也不例外;
Expand Down
6 changes: 4 additions & 2 deletions docs/文件批量操作.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,10 @@ Native 输出文件标题、物理行号、逐文件错误与续读提示;程

`ops_edit_file` 对**单个** UTF-8 文件原子应用 1–100 段修改。参数包括 `path`、读取/stat 得到的准确
SHA-256 `expectedRevision` 与 `edits: [{oldText,newText}]`。每个非空 `oldText` 必须在同一原始字节中
唯一出现;重叠、歧义或版本过期会在发布前拒绝全部修改。空 `newText` 表示删除。CRLF/BOM 需显式
保留。`dryRun: true` 仅预览相同回执;只读策略仍拒绝写入工具。这不是跨文件事务。
唯一出现。若原始匹配失败,会尝试一次可逆 Native 渲染解码,同时解码 `oldText` 与 `newText`;
原始匹配不会解码任何字段,回退回执以 `renderedMatches` 计数。重叠、歧义或版本过期会在发布前拒绝全部修改。
空 `newText` 表示删除。CRLF/BOM 需显式保留。`dryRun: true` 仅预览相同回执;只读策略仍拒绝写入工具。
这不是跨文件事务。

它共享 `ops_edit_file` 的 no-follow、CAS 与平台元数据保护,返回包含版本、修改计数、字节/行数及
警告的短回执,不回显替换内容。重试前必须检查 `error.effectState`;该工具不会标记为并发安全。
2 changes: 1 addition & 1 deletion docs/紧凑文件范围.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ English: [Compact Native file ranges](native-ranges.md).
[revision=cfd1bc81432fd6b88748b827f46bb4242db17cf8655cf4ea90f3ba0e62065355; more lines; nextOffset=3]
```

`N|` 后紧跟原始行内容,包含原有缩进。正文使用可逆 JSON 转义:反斜杠写成双反斜杠,控制字符及 Unicode 换行分隔符被转义,引号保持原样。例如真实制表符显示为 `\t`,而反斜杠加字母 t 的两个源字符显示为 `\\t`。紧凑范围和普通行号正文共用此编码,防止伪造额外行号或文件标题;LF/CRLF 物理行规则与 worker 保持一致。
`N|` 后紧跟原始行内容,包含原有缩进。正文使用可逆 JSON 转义:反斜杠写成双反斜杠,控制字符及 Unicode 换行分隔符被转义,引号保持原样。例如真实制表符显示为 `\t`,而反斜杠加字母 t 的两个源字符显示为 `\\t`。紧凑范围和普通行号正文共用此编码,防止伪造额外行号或文件标题;LF/CRLF 物理行规则与 worker 保持一致。`ops_edit_file` 始终先匹配字面字节;只有原始匹配失败后才可能解码此渲染形式的 `oldText` 与 `newText`,并在回执中返回 `renderedMatches`。原始匹配与 `write_file` 内容不会被解码。

`more lines` 表示文件尚未读到末尾,按 `nextOffset` 继续;`end of file; complete=true` 表示本范围已经到达文件末尾。汇总的 `requested ranges returned` 仅证明所请求的范围正文全部返回,不代表读取了所有完整文件。完整版本散列、总行数和每个文件的续读位置继续保留;字节大小及重复计数仍在规范数据和 PTC 返回值中。逐文件版本回执省去重复的 `Notice: ` 前缀与末尾句点;汇总 `Notice` 以及版本、续读和完整性事实均保留。

Expand Down
74 changes: 63 additions & 11 deletions native/crates/ops-documents/src/pdf_container.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
//! A deliberately restricted PDF container: one classic xref table, direct stream lengths, no recovery.
//! A deliberately restricted PDF container: one classic xref table, lengths resolved through that xref, no recovery.
use crate::pdf_syntax::{Dict, Lexer, Value, bad, limit, unsupported, white};
use ops_protocol::Result;
use std::collections::{BTreeMap, BTreeSet};
pub const MAX_FILE: usize = 16 * 1024 * 1024;
pub const MAX_OBJECTS: usize = 10_000;
pub const MAX_STREAM: usize = 1024 * 1024;
pub type ObjectId = (u32, u16);
/// The classic xref table: object number to (byte offset, generation, active).
type XrefEntries = BTreeMap<usize, (usize, u16, bool)>;
#[derive(Debug)]
pub struct Object<'a> {
pub value: Value,
Expand Down Expand Up @@ -82,11 +84,62 @@ fn footer(data: &[u8]) -> Result<(usize, usize)> {
}
Ok((xref, footer_start))
}
/// Resolve one stream Length (#788): a direct bounded integer, or one indirect
/// reference whose xref entry names a plain integer object. The probe re-lexes
/// the provider's exact header, so absent, free, chained, non-integer and
/// identity-mismatched targets fail closed; the caller's node budget pays for
/// the probe, keeping the whole parse bounded. No recovery, no endstream scan.
fn stream_length(
data: &[u8],
entries: &XrefEntries,
dict: &Dict,
nodes: &mut usize,
) -> Result<usize> {
let length = match get(dict, b"Length")? {
Value::Ref(number, generation) => {
let (offset, entry_generation, active) =
*entries.get(&(*number as usize)).ok_or_else(|| {
bad("PDF stream Length does not name an xref entry")
})?;
if !active || entry_generation != *generation {
return Err(bad(
"PDF stream Length must name an active xref entry with a matching generation",
));
}
if !data.get(offset).is_some_and(u8::is_ascii_digit) {
return Err(bad("PDF stream Length xref must point exactly to an object header"));
}
let mut lexer = Lexer {
data,
at: offset,
nodes: *nodes,
};
if lexer.unsigned()? != *number as usize || lexer.unsigned()? != *generation as usize {
return Err(bad("PDF stream Length object identity mismatch"));
}
lexer.keyword(b"obj")?;
let resolved = lexer
.value(0)?
.integer()
.map_err(|_| bad("PDF stream Length must resolve to a bounded integer object"))?;
lexer.keyword(b"endobj")?;
*nodes = lexer.nodes;
resolved
}
value => value.integer()?,
};
if length > MAX_STREAM {
return Err(limit("PDF encoded stream exceeds 1 MiB"));
}
Ok(length)
}

fn object<'a>(
data: &'a [u8],
offset: usize,
id: ObjectId,
nodes: usize,
entries: &XrefEntries,
) -> Result<(Object<'a>, usize, usize)> {
if !data.get(offset).is_some_and(u8::is_ascii_digit) {
return Err(bad("PDF xref must point exactly to an object header"));
Expand All @@ -104,11 +157,10 @@ fn object<'a>(
let mut stream = None;
let token = lexer.word()?;
if token == b"stream" {
let length = get(value.dict()?, b"Length")?.integer()?;
if length > MAX_STREAM {
return Err(limit("PDF encoded stream exceeds 1 MiB"));
}
// Indirect Length is declined before reading stream bytes; no searching for endstream.
let mut nodes = lexer.nodes;
let length = stream_length(lexer.data, entries, value.dict()?, &mut nodes)?;
lexer.nodes = nodes;
// The Length is fully resolved before stream bytes are read; no searching for endstream.
if lexer.data.get(lexer.at..lexer.at + 2) == Some(b"\r\n") {
lexer.at += 2;
} else if lexer.data.get(lexer.at) == Some(&b'\n') {
Expand Down Expand Up @@ -228,16 +280,16 @@ pub fn parse(data: &[u8]) -> Result<Document<'_>> {
return Err(bad("Unexpected PDF data between trailer and final footer"));
}
let mut ordered = Vec::new();
for (number, (offset, generation, active)) in entries {
if !active {
for (number, (offset, generation, active)) in &entries {
if !*active {
continue;
}
if offset < 9 || offset >= xref {
if *offset < 9 || *offset >= xref {
return Err(bad(
"Active PDF object offset is outside the object section",
));
}
ordered.push((offset, (number as u32, generation)));
ordered.push((*offset, (*number as u32, *generation)));
}
ordered.sort_unstable();
let mut previous = 0;
Expand All @@ -258,7 +310,7 @@ pub fn parse(data: &[u8]) -> Result<Document<'_>> {
"Unindexed data or overlapping comments in PDF object section",
));
}
let (item, end, count) = object(&data[..xref], offset, id, nodes)?;
let (item, end, count) = object(&data[..xref], offset, id, nodes, &entries)?;
previous = end;
nodes = count;
objects.insert(id, item);
Expand Down
81 changes: 81 additions & 0 deletions native/crates/ops-documents/src/pdf_container_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -337,3 +337,84 @@ fn stream_roots_reserved_generations_and_direct_oversize_decode_are_refused() {
};
assert_eq!(decode(&raw).unwrap_err().code, "OUTPUT_LIMIT");
}

#[test]
fn indirect_stream_lengths_resolve_through_the_xref_without_recovery() {
// #788: Word and LibreOffice emit "/Length N G R". The integer provider may
// sit at any xref offset; one bounded probe re-lexes its exact header.
let pdf = fixture(&[
b"<<>>",
b"9",
b"<< /Length 2 0 R >>\nstream\n(Hi) Tj\n\nendstream",
]);
let document = parse(&pdf).unwrap();
assert_eq!(
document.object((3, 0)).unwrap().stream,
Some(b"(Hi) Tj\n\n".as_slice())
);
// The provider may also follow the stream object in file order.
let reversed = fixture(&[
b"<<>>",
b"<< /Length 3 0 R >>\nstream\nxy\nendstream",
b"2",
]);
assert_eq!(
parse(&reversed).unwrap().object((2, 0)).unwrap().stream,
Some(b"xy".as_slice())
);
// Only a plain integer object resolves: absent, dictionary, chained,
// generation-mismatched and oversized providers all fail closed.
rejected(
&fixture(&[b"<<>>", b"9", b"<< /Length 4 0 R >>\nstream\nxy\nendstream"]),
"DOCUMENT_PARSE_ERROR",
);
rejected(
&fixture(&[b"<<>>", b"<<>>", b"<< /Length 2 0 R >>\nstream\nxy\nendstream"]),
"DOCUMENT_PARSE_ERROR",
);
rejected(
&fixture(&[
b"<<>>",
b"3 0 R",
b"<< /Length 2 0 R >>\nstream\nxy\nendstream",
b"9",
]),
"DOCUMENT_PARSE_ERROR",
);
rejected(
&fixture(&[b"<<>>", b"9", b"<< /Length 2 1 R >>\nstream\nxy\nendstream"]),
"DOCUMENT_PARSE_ERROR",
);
rejected(
&fixture(&[
b"<<>>",
b"1048577",
b"<< /Length 2 0 R >>\nstream\nxy\nendstream",
]),
"OUTPUT_LIMIT",
);
// A free xref entry is never a Length provider.
let mut free = fixture(&[b"<<>>", b"9", b"<< /Length 2 0 R >>\nstream\nxy\nendstream"]);
let at = free.windows(7).position(|w| w == b"2 0 obj").unwrap();
free = replace(
&free,
format!("{at:010} 00000 n ").as_bytes(),
format!("{at:010} 65535 f ").as_bytes(),
);
rejected(&free, "DOCUMENT_PARSE_ERROR");
// The probe runs before the ordered scan reaches a duplicated offset and
// refuses a header that disagrees with the referenced identity.
let mut aliased = fixture(&[
b"<<>>",
b"<< /Length 3 0 R >>\nstream\nxy\nendstream",
b"2",
]);
let stream_at = aliased.windows(7).position(|w| w == b"2 0 obj").unwrap();
let provider_at = aliased.windows(7).position(|w| w == b"3 0 obj").unwrap();
aliased = replace(
&aliased,
format!("{provider_at:010} 00000 n ").as_bytes(),
format!("{stream_at:010} 00000 n ").as_bytes(),
);
rejected(&aliased, "DOCUMENT_PARSE_ERROR");
}
Loading
Loading