diff --git a/CHANGELOG.md b/CHANGELOG.md index abd414d..e4b08c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,26 @@ # Changelog +## 0.5.3 + +### Features + +- **`--filter-title-block-id` 精确过滤**:新增 CLI 选项与 `ConvertOptions.filterTitleBlockId`,按 heading 块 id 严格匹配目标章节,彻底规避同名标题歧义;与 `--filter-title` 互斥 +- **`get-titles` 子命令**:一键列出 docx/wiki 文档全部标题(1~9 级),支持 `yaml` / `yaml-tree` / `json` / `tree` / `text` 五种输出格式与 `--max-level` 限级;不支持 sheets +- **`--filter-title` / `--filter-title-block-id` 注入父级标题**:命中深层标题时,自动把所有更高层级的祖先标题(仅 heading 块本身)按文档顺序一并输出,保留章节层级上下文;顶级命中不注入,跳级不伪造中间层,旁支兄弟自动排除 +- **标题元信息结构化**:导出 `HeadingInfo`(`blockId` / `level` / `text`),`--filter-title` 未命中时错误信息改为与 `get-titles` 同形的 yaml 清单,AI/脚本可直接据此重选 blockId,形成错误恢复闭环 + +### Fixed + +- **表格 `row_span` / `col_span` 容错**:飞书接口可能返回 `undefined`,解析时兜底为 1,避免崩溃 + +### Changed + +- **docx 表格输出改为 Markdown 管道格式**:原 `` HTML 输出(包含 `rowspan` / `colspan`)改为标准 GFM 管道表格;合并单元格按「复制顶格值」策略展开,单元格内容自动转义 `|` 与换行;`Table` 与电子表格 `Sheet` 现使用同一 `renderMarkdownTable` 实现 +- **`--agent` 取值规范化**:CLI `--agent` 与环境变量 `LARK_DOCX2MD_AGENT` 显式接受 `stdout` 或 `local`;不带值的 `--agent` 仍等价于 `stdout`,但环境变量原 `true` 形式已不再支持,需改写为 `LARK_DOCX2MD_AGENT=stdout` +- **请求限速放宽**:`getDocxBlocks` 分页间隔 100 → 50ms、图片下载间隔 600 → 300ms,大文档转换更快 +- **代码组织拆分**:将 `getTitles` / `buildTitleTree` 从 `converter.ts` 抽离至独立模块 `src/get-titles.ts`,URL 解析逻辑迁移至 `src/url.ts` +- **SKILL 工作流升级**:`skills/lark-docx2md/SKILL.md` 指定标题场景改为「`get-titles` → 匹配 blockId → `dl --filter-title-block-id`」三步式,支持单标题、多级路径、同名同路径等多种消歧场景 + ## 0.5.2 ### Fixed diff --git a/README.md b/README.md index afc49d6..f92dd82 100644 --- a/README.md +++ b/README.md @@ -42,9 +42,10 @@ npx -y lark-docx2md@latest download | `--app-id ` | 飞书应用 App ID | `LARK_DOCX2MD_APP_ID` | — | | `--app-secret ` | 飞书应用 App Secret | `LARK_DOCX2MD_APP_SECRET` | — | | `-o, --output ` | 输出目录 | `LARK_DOCX2MD_OUTPUT` | `./larkDocx2mdOutput` | -| `--agent [mode]` | Agent 模式:日志 ERROR。不传值(或 `=true`)为在线模式,Markdown 输出到 stdout;传 `local` 则落盘后输出引导 AI 读取的提示词 | `LARK_DOCX2MD_AGENT=true\|local` | `false` | +| `--agent [mode]` | Agent 模式:日志 ERROR。不传值(或 `=stdout`)为在线模式,Markdown 输出到 stdout;传 `local` 则落盘后输出引导 AI 读取的提示词 | `LARK_DOCX2MD_AGENT=stdout\|local` | `false` | | `--image-mode ` | 图片处理模式:`local`(下载到本地)或 `online`(24h 临时链接) | `LARK_DOCX2MD_IMAGE_MODE` | `local` | | `--filter-title ` | 按标题过滤:仅转换匹配标题及其下级内容(匹配到同级或更高级标题时截止) | — | — | +| `--filter-title-block-id <id>` | 按 heading 块 id 精确过滤(无同名歧义),通常配合 `get-titles` 子命令获取;与 `--filter-title` 互斥 | — | — | | `--wb-format <format>` | 画板输出格式:`base64`、`inline-svg`、`svg`、`yaml` | `LARK_DOCX2MD_WB_FORMAT` | `svg`(agent 下默认 `yaml`) | | `--wb-bg <style>` | 画板 SVG 背景:`none`、`dot` 或颜色值如 `#fff` | `LARK_DOCX2MD_WB_BG` | `none` | | `--wb-image-mode <mode>` | 画板图片模式:`online`、`base64` 或 `local` | `LARK_DOCX2MD_WB_IMAGE_MODE` | `local` | @@ -54,7 +55,26 @@ npx -y lark-docx2md@latest download <url> > - `--agent`(在线):强制 `--image-mode=online`、`--wb-image-mode=online`;`--wb-format` 默认 `yaml`,仅允许 `inline-svg` / `yaml`;转换完成后 Markdown 直接通过 stdout 输出。 > - `--agent local`:强制 `--image-mode=local`、`--wb-image-mode=local`(Markdown、图片、画板中的图片均落盘);`--wb-format` 默认 `yaml`,仅允许 `inline-svg` / `yaml`;stdout 输出引导 AI 读取文件的提示词(包含绝对路径)。 > - 非 agent 模式下 `--wb-format yaml` 时:`--wb-image-mode` 强制为 `online`。 -> - `--filter-title`:按标题文本精确匹配(忽略前后空格),收集该标题及其所有子级块,遇到同级或更高级标题时停止。未匹配到则报错提示。 +> - `--filter-title`:按标题文本精确匹配(忽略前后空格),收集该标题及其所有子级块,遇到同级或更高级标题时停止。同名标题取首个;未匹配时错误信息附全文标题 yaml 清单。 +> - `--filter-title-block-id`:按 heading 块 id 严格相等匹配,适用于同名标题或脚本化场景;通常先用 `get-titles` 查出目标 `blockId` 再传入。与 `--filter-title` 互斥。 +> - **命中深层标题时自动注入父级标题(仅 heading 块本身)**:两个过滤参数均会按文档顺序补齐包含路径上的顶层→该标题的所有祖先标题,以保留章节层级上下文;不会引入旁支兄弟或伪造跳级。 + +## 子命令:`get-titles` + +列出 docx/wiki 文档全部标题(不支持 sheets),配合 `--filter-title-block-id` 使用可避开同名歧义。 + +```bash +npx -y lark-docx2md@latest get-titles --agent <url> +``` + +| 参数 | 说明 | 默认值 | +|-----------------------|------------------------------------------------------------------------------------------|--------| +| `<url>` | 飞书 wiki/docx URL | — | +| `--max-level <n>` | 仅输出 `level <= n` 的标题(1~9) | `9` | +| `--format <format>` | 输出格式:`yaml`(扁平) \| `yaml-tree`(嵌套) \| `json` \| `tree`(json 嵌套) \| `text`(缩进的 markdown 标题) | `yaml` | +| `--agent [mode]` | 同 `dl`,降低日志级别 | — | + +输出包含 `blockId` / `level` / `text`,可被 AI 直接消费用于选择目标章节(嵌套关系由 `yaml-tree` / `tree` 格式天然表达)。 ## 功能 @@ -81,7 +101,7 @@ npx -y lark-docx2md@latest download <url> | Callout | 高亮块 | `>[!TIP]` + 子块 | | Divider | 分割线 | `---` | | Image | 图片 | `![图片](url)` | -| Table / TableCell | 表格 | `<table>` HTML(支持合并单元格) | +| Table / TableCell | 表格 | GFM 管道表格(合并单元格按值展开) | | QuoteContainer | 引用容器 | `> 子块内容` | | Grid / GridColumn | 分栏布局 | 展平为子块内容 | | Sheet | 电子表格 | GFM 表格(合并单元格自动展开) | diff --git a/package.json b/package.json index af0e4db..c93a087 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "lark-docx2md", - "version": "0.5.2", + "version": "0.5.3", "description": "Convert Lark/Feishu documents to Markdown", "type": "module", "main": "./dist/converter.js", diff --git a/skills/lark-docx2md/SKILL.md b/skills/lark-docx2md/SKILL.md index 81f66f2..55558d3 100644 --- a/skills/lark-docx2md/SKILL.md +++ b/skills/lark-docx2md/SKILL.md @@ -5,69 +5,53 @@ description: 读取飞书 Wiki 文档或电子表格(`https://*.feishu.cn/wiki # lark-docx2md -将飞书文档或电子表格 URL 转换为本地 Markdown 文件,命令标准输出包含文件绝对路径的提示词。 +将飞书文档或电子表格 URL 转换为本地 Markdown 文件。 -## 必要输入 +## 输入 -- 飞书文档 URL,格式为 `https://*.feishu.cn/wiki/...` 或 `https://*.feishu.cn/sheets/...`(可带 `?sheet=xxx` 参数) +- **URL**(必填):`https://*.feishu.cn/{wiki,docx,docs,sheets}/...`,可带 `?sheet=xxx`。 +- **标题**(可选,仅 `wiki/docx/docs` 适用;`sheets` 跳过):用户想要的某一章节。按以下规则识别,命中即视为指定: + 1. 显式关键词「标题/章节/小节/部分/节选/只要/其中的」+ 名词短语。 + 2. 成对引号 `""` `''` `“”` `‘’` `「」` `《》` 包裹的短文本。 + 3. 介词结构:`X 中的 Y`、`X 里的 Y`、`X 那一段`、`关于 X 的内容`。 + 4. 多级路径:`A 下/中/里的 B`、`A > B`、`A/B`、`A → B` → 取为有序路径 `[A, B, ...]`。 + 5. 用户要求「完整/全部/整篇」或无法稳定抽取 → **不指定**。 -## 可选输入:标题(用于 `--filter-title`) - -当用户只想要文档中的某一节内容时,可通过 `--filter-title "<标题>"` 仅转换匹配该标题及其子级的部分(标题文本需与文档中的标题**完全一致**,仅做 trim 比较)。 - -### 识别「用户是否指定了标题」的启发式规则 (重要) - -按如下顺序判断,命中任一条即视为用户指定了标题,并将命中的文本作为 `<title>`: - -1. **显式指令**:用户消息中出现「标题」「章节」「小节」「部分」「节选」「只要/只读/仅要/仅读」「其中的」等关键词,并紧跟一个可识别的名词短语或引号文本。 - - 例:`只要「接入指南」这一节` → title = `接入指南` - - 例:`把其中“常见问题”部分转成 md` → title = `常见问题` -2. **引号包裹**:用户消息中出现成对的引号(`""` / `''` / `“”` / `‘’` / `「」` / `《》`)包裹的短文本,且语义上指代文档中的某一节。 - - 例:`读一下文档的「部署流程」` → title = `部署流程` -3. **介词结构**:形如 `URL 中的 X`、`文档里的 X`、`X 那一段`、`关于 X 的内容` 等结构,X 为短名词短语(一般 ≤ 30 字、不含句号/换行)。 -4. **未命中即视为未指定**:若用户仅给出 URL、或要求「完整」「全部」「整篇」内容,或无法稳定抽取标题文本,则**不要**添加 `--filter-title`。 - -### 标题文本的规范化 - -- 去除首尾空白与包裹用的引号。 -- **保留**原文大小写、标点、空格与中英文字符(过滤器按精确匹配比较)。 -- 不要自行翻译、改写或补全标题。 -- 命令行中用双引号包裹;若标题内含英文双引号,用 `\"` 转义。 + 抽到的标题文本:去首尾空白与包裹引号;保留原文大小写、标点、空格、中英文;不翻译/改写/补全。 ## 工作流程 -1. 验证 URL 包含 `/wiki/` 或 `/sheets/`。 -2. 按上文规则判断用户是否指定了标题,并据此构造命令: +### sheets 或未指定标题 - - **未指定标题**(默认): +URL 含 `/sheets/`,或 docx/wiki 下未识别出标题,直接: - ```bash - npx -y lark-docx2md@latest dl --agent local "<url>" - ``` +```bash +npx -y lark-docx2md@latest dl --agent local "<url>" +``` - - **指定了标题**:在上述命令基础上追加 `--filter-title "<title>"`: +### docx/wiki 且指定标题(三步) - ```bash - npx -y lark-docx2md@latest dl --agent local --filter-title "<title>" "<url>" - ``` +1. 取标题表: - > **注意**:`--agent local` 参数在任何情况下都不可省略,缺少它会导致输出格式不适合 AI 处理。 + ```bash + npx -y lark-docx2md@latest get-titles --agent local "<url>" + ``` -3. 捕获标准输出,从提示词中解析出 Markdown 文件的绝对路径。 -4. 读取该绝对路径对应的 Markdown 文件,获得文档完整内容。 +2. 匹配标题并获取 `blockId`: + - 单标题:找 `text === <title>`。唯一直接用;多候选用上下文(父节点关键词)消歧。 + - 同父级同名兄弟:按同父节点下 children 的出现顺序列出全部候选,附上下文提示回问用户选择。 + - 无候选:报告并展示可用标题。 -## 关联图片读取要求 +3. 用 `blockId` 下载: -- Markdown 文件中的图片以本地相对路径引用(如 `static/xxx.png`),画板内的图片同样如此。 -- 读取 Markdown 后,**必须**一并读取其中引用的所有图片文件内容,以确保对文档的理解包含图片信息。 + ```bash + npx -y lark-docx2md@latest dl --agent local --filter-title-block-id "<blockId>" "<url>" + ``` -## 异常处理 +### 读取结果 -- 如果 URL 格式无效,要求提供有效的 `https://*.feishu.cn/wiki/` 或 `https://*.feishu.cn/sheets/` 链接。 -- 如果输出为空,报告所使用的命令并附上 stderr 关键信息。 +从 `dl` 的 stdout 解析 Markdown 绝对路径并读取该文件;同时**必须**读取其中引用的所有图片(含画板内图片,路径形如 `static/xxx.png`)。 -## 响应风格 +## 约束 -- 保持简洁、以执行为导向。 -- 转换成功时附上准确的文件路径。 -- 当用户输入不完整时,明确指出所做的假设。 +- `dl` 必须带 `--agent local`;`get-titles` 必须带 `--agent`。 diff --git a/src/cli.ts b/src/cli.ts index 2fc85c6..a252a79 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -1,13 +1,39 @@ #!/usr/bin/env node +import * as fs from 'node:fs'; +import * as path from 'node:path'; import { Command } from 'commander'; import { LoggerLevel } from '@larksuiteoapi/node-sdk'; import { convert } from './converter.js'; +import { buildTitleTree, getTitles } from './get-titles.js'; import { setLogLevel } from './logger.js'; -import type { SvgBackground, WbFormat, WbImageMode } from './types.js'; +import { serializeYaml } from './whiteboard/yaml/serialize.js'; +import type { SvgBackground, WbFormat, WbImageMode, AgentMode } from './types.js'; const program = new Command(); program.name('larkDocx2md').description('Download Lark/Feishu documents to markdown'); +/** + * 把 commander 解析出的 --agent 原值(undefined | true | string)与 + * LARK_DOCX2MD_AGENT 环境变量统一收敛为精确的 AgentMode | false。 + * + * 优先级:显式 --agent > 环境变量 > 默认 false。 + * 非法取值返回错误信息字符串,由调用方决定如何上报。 + */ +function resolveAgentMode (raw: boolean | string | undefined): { value: AgentMode | false; error?: string } { + if (raw === undefined) { + const env = process.env.LARK_DOCX2MD_AGENT; + if (env === 'stdout' || env === 'local') return { value: env }; + if (env !== undefined && env !== '') { + return { value: false, error: `Invalid LARK_DOCX2MD_AGENT="${env}", must be "stdout" or "local"` }; + } + return { value: false }; + } + // --agent 不带值:默认 stdout + if (raw === true) return { value: 'stdout' }; + if (raw === 'stdout' || raw === 'local') return { value: raw }; + return { value: false, error: `Invalid --agent value "${raw}", must be "stdout" or "local"` }; +} + program .command('download') .alias('dl') @@ -15,38 +41,40 @@ program .option('--app-id <id>', 'Feishu app ID (or read from LARK_DOCX2MD_APP_ID)') .option('--app-secret <secret>', 'Feishu app secret (or read from LARK_DOCX2MD_APP_SECRET)') .option('-o, --output <dir>', 'Output directory (or LARK_DOCX2MD_OUTPUT)') - .option('--agent [mode]', 'Enable agent mode: ERROR log level, and AI-oriented stdout. Pass "local" to save markdown/images/whiteboards to disk and print a read-file prompt (or LARK_DOCX2MD_AGENT=true|local)') + .option('--agent [mode]', 'Enable agent mode: ERROR log level, and AI-oriented stdout. Modes: "stdout" (default, print markdown to stdout) or "local" (save markdown/images/whiteboards to disk and print a read-file prompt). Or LARK_DOCX2MD_AGENT=stdout|local') .option('--wb-format <format>', 'Whiteboard output format: "base64", "inline-svg", "svg", or "yaml" (or LARK_DOCX2MD_WB_FORMAT)') .option('--wb-bg <style>', 'Whiteboard SVG background: "none", "dot", or a color like "#fff" (or LARK_DOCX2MD_WB_BG)') .option('--wb-image-mode <mode>', 'Whiteboard image mode: "online", "base64", or "local" (or LARK_DOCX2MD_WB_IMAGE_MODE)') .option('--image-mode <mode>', 'Image handling mode: "local" or "online" (or LARK_DOCX2MD_IMAGE_MODE)') - .option('--filter-title <title>', 'Only convert the section matching this heading title') + .option('--filter-title <title>', 'Only convert the section matching this heading title (single title, first match wins on duplicates)') + .option('--filter-title-block-id <id>', 'Only convert the section whose heading block id matches (most precise; obtain from get-titles)') .argument('<url>', 'Feishu wiki document URL: https://*.feishu.cn/wiki/*') - .action(async (url: string, opts: { appId?: string; appSecret?: string; output?: string; agent?: boolean | string; imageMode?: string; wbImageMode?: string; wbBg?: SvgBackground; wbFormat?: string; filterTitle?: string }) => { + .action(async (url: string, opts: { appId?: string; appSecret?: string; output?: string; agent?: boolean | string; imageMode?: string; wbImageMode?: string; wbBg?: SvgBackground; wbFormat?: string; filterTitle?: string; filterTitleBlockId?: string }) => { // ─── 环境变量默认值(直接指定 > 环境变量 > 内置默认值)──────────────── opts.appId = opts.appId ?? process.env.LARK_DOCX2MD_APP_ID; opts.appSecret = opts.appSecret ?? process.env.LARK_DOCX2MD_APP_SECRET; opts.output = opts.output ?? process.env.LARK_DOCX2MD_OUTPUT ?? './larkDocx2mdOutput'; - // 解析 --agent:可能为 undefined | true | 'local' | 其他字符串 - if (opts.agent === undefined) { - const envAgent = process.env.LARK_DOCX2MD_AGENT; - if (envAgent === 'true') opts.agent = true; - else if (envAgent === 'local') opts.agent = 'local'; - else opts.agent = false; - } else if (typeof opts.agent === 'string' && opts.agent !== 'local') { - program.error(`Invalid --agent value "${opts.agent}", only "local" is supported (or omit the value)`); + // 解析 --agent:可能为 undefined | true | 'stdout' | 'local' | 其他字符串 + const agentResolved = resolveAgentMode(opts.agent); + if (agentResolved.error) { + program.error(agentResolved.error); + return; } - const agentEnabled = opts.agent === true || opts.agent === 'local'; + opts.agent = agentResolved.value; + const agentEnabled = opts.agent === 'stdout' || opts.agent === 'local'; const agentLocal = opts.agent === 'local'; + const agentStdout = opts.agent === 'stdout'; opts.imageMode = opts.imageMode ?? process.env.LARK_DOCX2MD_IMAGE_MODE ?? 'local'; opts.wbFormat = opts.wbFormat ?? process.env.LARK_DOCX2MD_WB_FORMAT; opts.wbBg = opts.wbBg ?? process.env.LARK_DOCX2MD_WB_BG ?? 'none'; opts.wbImageMode = opts.wbImageMode ?? process.env.LARK_DOCX2MD_WB_IMAGE_MODE ?? 'local'; - // 设置 wb-format 默认值:--agent local 默认 inline-svg(兼容本地画板图片),--agent(在线)默认 yaml,其余 svg + // 设置 wb-format 默认值:--agent local 默认 inline-svg(兼容本地画板图片),--agent stdout 默认 yaml,其余 svg if (!opts.wbFormat) { - opts.wbFormat = agentEnabled ? 'yaml' : 'svg'; + if (agentLocal) opts.wbFormat = 'inline-svg'; + else if (agentStdout) opts.wbFormat = 'yaml'; + else opts.wbFormat = 'svg'; } if (agentEnabled) { @@ -56,7 +84,7 @@ program opts.imageMode = 'local'; opts.wbImageMode = 'local'; } else { - // --agent(在线):一律在线,且画板仅支持内嵌形式 + // --agent stdout:一律在线,且画板仅支持内嵌形式 opts.imageMode = 'online'; opts.wbImageMode = 'online'; } @@ -79,6 +107,9 @@ program if (!['online', 'base64', 'local'].includes(opts.wbImageMode)) { program.error(`Invalid --wb-image-mode "${opts.wbImageMode}", must be "online", "base64", or "local"`); } + if (opts.filterTitle && opts.filterTitleBlockId) { + program.error('--filter-title and --filter-title-block-id are mutually exclusive; choose one'); + } const appId = opts.appId!; const appSecret = opts.appSecret!; @@ -95,8 +126,9 @@ program wbImageMode: opts.wbImageMode as WbImageMode, wbBg: opts.wbBg, wbFormat: opts.wbFormat as WbFormat, - agent: agentLocal ? 'local' : (opts.agent === true), + agent: opts.agent as AgentMode | false, filterTitle: opts.filterTitle?.trim(), + filterTitleBlockId: opts.filterTitleBlockId?.trim(), }); if (agentLocal) { @@ -106,9 +138,68 @@ program `\`${result.filePath}\`\n\n` + `**Read this file to access the full markdown content.**\n`, ); - } else if (opts.agent === true) { + } else if (agentStdout) { process.stdout.write(result.markdown); } }); +program + .command('get-titles') + .description('Print all headings (level 1~9) of a wiki/docx document as a nested yaml tree. Useful before --filter-title-block-id.') + .option('--app-id <id>', 'Feishu app ID (or read from LARK_DOCX2MD_APP_ID)') + .option('--app-secret <secret>', 'Feishu app secret (or read from LARK_DOCX2MD_APP_SECRET)') + .option('-o, --output <dir>', 'Output directory used by --agent local (or LARK_DOCX2MD_OUTPUT)') + .option('--max-level <n>', 'Only output headings whose level <= n (1~9)', '9') + .option('--agent [mode]', 'Enable agent mode: ERROR log level, AI-oriented stdout. Modes: "stdout" (default, print titles to stdout) or "local" (save titles to disk and print a read-file prompt). Or LARK_DOCX2MD_AGENT=stdout|local') + .argument('<url>', 'Feishu wiki/docx URL: https://*.feishu.cn/{wiki,docx,docs}/*') + .action(async (url: string, opts: { appId?: string; appSecret?: string; output?: string; maxLevel?: string; agent?: boolean | string }) => { + opts.appId = opts.appId ?? process.env.LARK_DOCX2MD_APP_ID; + opts.appSecret = opts.appSecret ?? process.env.LARK_DOCX2MD_APP_SECRET; + opts.output = opts.output ?? process.env.LARK_DOCX2MD_OUTPUT ?? './larkDocx2mdOutput'; + const agentResolved = resolveAgentMode(opts.agent); + if (agentResolved.error) { + program.error(agentResolved.error); + return; + } + opts.agent = agentResolved.value; + const agentEnabled = opts.agent === 'stdout' || opts.agent === 'local'; + const agentLocal = opts.agent === 'local'; + if (agentEnabled) setLogLevel(LoggerLevel.error); + + const maxLevel = parseInt(opts.maxLevel ?? '9', 10); + if (!Number.isInteger(maxLevel) || maxLevel < 1 || maxLevel > 9) { + program.error(`Invalid --max-level "${opts.maxLevel}", must be an integer in [1, 9]`); + } + + const appId = opts.appId!; + const appSecret = opts.appSecret!; + if (!appId || !appSecret) { + program.error('Missing credentials: pass --app-id/--app-secret or set LARK_DOCX2MD_APP_ID/LARK_DOCX2MD_APP_SECRET'); + } + + const result = await getTitles({ + appId, + appSecret, + url, + agent: opts.agent as AgentMode | false, + }); + const filtered = result.titles.filter(t => t.level <= maxLevel); + const tree = buildTitleTree(filtered); + const content = serializeYaml({ url: result.url, docToken: result.docToken, titles: tree }); + + if (agentLocal) { + // 本地模式:落盘 + 输出引导 AI 读取文件的提示词(绝对路径) + fs.mkdirSync(opts.output, { recursive: true }); + const filePath = path.resolve(opts.output, `${result.docToken}.titles.yaml`); + fs.writeFileSync(filePath, content); + process.stdout.write( + `**The Feishu document titles have been downloaded to the following absolute path:**\n\n` + + `\`${filePath}\`\n\n` + + `**Read this file to access the full titles list.**\n`, + ); + } else { + process.stdout.write(content); + } + }); + program.parse(); diff --git a/src/client.ts b/src/client.ts index 4c05563..542251d 100644 --- a/src/client.ts +++ b/src/client.ts @@ -64,7 +64,7 @@ export function createClient (appId: string, appSecret: string, loggerLevel: Log for (let i = 0; true ; i++) { if (i > 0) { // 单个应用调用频率上限为每秒 5 次 - await sleep(100); + await sleep(50); } const data = await call('getDocxBlocks', () => client.docx.v1.documentBlock.list({ diff --git a/src/converter.ts b/src/converter.ts index 5d8e5f5..edc1765 100644 --- a/src/converter.ts +++ b/src/converter.ts @@ -8,19 +8,19 @@ import { MdSerializer, registerBuiltinSerializers } from './md-ast/serializer.js import { MdTransformer } from './md-ast/transformer.js'; import { createLogger } from './logger.js'; import type { ConvertOptions, ConvertResult, DocxBlock } from './types.js'; -import { createTitleFilter } from './title-filter.js'; +import { + createTitleFilter, + createTitleBlockIdFilter, + type TitleFilterResult, +} from './title-filter.js'; +import { serializeYaml } from './whiteboard/yaml/serialize.js'; +import { buildTitleTree } from './get-titles.js'; +import { parseWikiUrl } from './url.js'; -const logger = createLogger('converter'); - -// ─── URL Parsing ───────────────────────────────────────────────────────────── +// 保留 “parseWikiUrl from converter” 公开 API(供外部代码 / 测试向后兼容) +export { parseWikiUrl }; -export function parseWikiUrl (url: string): { docType: string; docToken: string; sheetId?: string } { - const m = url.match(/^https:\/\/[\w.-]+\/(docs|docx|wiki|sheets)\/([a-zA-Z0-9]+)/); - if (!m) throw new Error('Invalid feishu document URL'); - // 解析 ?sheet=XXX 查询参数(仅对 sheets 类型有意义) - const sheetId = new URL(url).searchParams.get('sheet') ?? undefined; - return { docType: m[1]!, docToken: m[2]!, sheetId }; -} +const logger = createLogger('converter'); // ─── Core Convert ──────────────────────────────────────────────────────────── @@ -59,19 +59,12 @@ export async function convert (opts: ConvertOptions): Promise<ConvertResult> { // 原 docx 流程 const doc = await client.getDocxDocument(docToken); let blocks: DocxBlock[]; - if (opts.filterTitle) { - const filter = createTitleFilter({ title: opts.filterTitle }); + const filter = createDocxFilter(opts); + if (filter) { await client.getDocxBlocks(docToken, filter.pageHandler); const result = filter.getResult(); if (!result.matched) { - let msg = `No heading matched "${opts.filterTitle}". Please verify the heading text.`; - if (result.availableHeadings.length > 0) { - const list = result.availableHeadings - .map(h => `${'#'.repeat(h.level)} ${h.text}`) - .join('\n'); - msg += `\n\nAvailable headings in the document:\n\n${list}`; - } - throw new Error(msg); + throw new Error(buildFilterErrorMessage(opts, result, opts.url, docToken)); } blocks = result.blocks; } else { @@ -104,3 +97,28 @@ export async function convert (opts: ConvertOptions): Promise<ConvertResult> { return { markdown, docToken, filePath }; } + +// ─── Filter Selection ─────────────────────────────────────────────────────── + +/** 优先级:filterTitleBlockId > filterTitle。返回 null 表示不过滤。 */ +function createDocxFilter (opts: ConvertOptions): { + pageHandler: (blocks: DocxBlock[]) => boolean; + getResult: () => TitleFilterResult; +} | null { + if (opts.filterTitleBlockId) return createTitleBlockIdFilter({ blockId: opts.filterTitleBlockId }); + if (opts.filterTitle) return createTitleFilter({ title: opts.filterTitle }); + return null; +} + +function buildFilterErrorMessage (opts: ConvertOptions, result: TitleFilterResult, url: string, docToken: string): string { + let target: string; + if (opts.filterTitleBlockId) target = `block id "${opts.filterTitleBlockId}"`; + else target = `"${opts.filterTitle}"`; + let msg = `No heading matched ${target}. Please verify the heading text/id.`; + if (result.availableHeadings.length > 0) { + const tree = buildTitleTree(result.availableHeadings); + const yaml = serializeYaml({ url, docToken, titles: tree }); + msg += `\n\nFull title list of the document:\n\n${yaml}`; + } + return msg; +} diff --git a/src/get-titles.ts b/src/get-titles.ts new file mode 100644 index 0000000..5d26003 --- /dev/null +++ b/src/get-titles.ts @@ -0,0 +1,78 @@ +import { LoggerLevel } from '@larksuiteoapi/node-sdk'; +import { createClient } from './client.js'; +import { createLogger } from './logger.js'; +import { parseWikiUrl } from './url.js'; +import { createHeadingCollector, type HeadingInfo } from './title-filter.js'; +import type { AgentMode } from './types.js'; + +const logger = createLogger('get-titles'); + +// ─── Options & Result ─────────────────────────────────────────────────────── + +export interface GetTitlesOptions { + appId: string; + appSecret: string; + url: string; + /** 仅影响 SDK 日志级别(与 dl 保持一致) */ + agent?: AgentMode | false; +} + +export interface GetTitlesResult { + url: string; + docToken: string; + titles: HeadingInfo[]; +} + +// ─── Core ─────────────────────────────────────────────────────────────────── + +/** 拉取 docx/wiki 文档中所有标题信息(扁平列表,不下载图片)。不支持电子表格。 */ +export async function getTitles (opts: GetTitlesOptions): Promise<GetTitlesResult> { + const { docType, docToken: rawToken } = parseWikiUrl(opts.url); + if (docType === 'sheets') { + throw new Error('get-titles does not support spreadsheets, only docx/wiki documents are supported'); + } + + const sdkLoggerLevel = opts.agent ? LoggerLevel.error : LoggerLevel.warn; + const client = createClient(opts.appId, opts.appSecret, sdkLoggerLevel); + + let docToken = rawToken; + if (docType === 'wiki') { + const node = await client.getWikiNodeInfo(docToken); + docToken = node.obj_token!; + logger.info('Resolved wiki node:', node.obj_type, docToken); + if (node.obj_type === 'sheet') { + throw new Error('get-titles does not support spreadsheets (wiki node points to a spreadsheet), only docx/wiki documents are supported'); + } + } + + const collector = createHeadingCollector(); + await client.getDocxBlocks(docToken, collector.pageHandler); + const titles = collector.getHeadings(); + logger.info(`Collected ${titles.length} headings`); + return { url: opts.url, docToken, titles }; +} + +// ─── Tree & Output Helpers ────────────────────────────────────────────────── + +/** 标题树节点(buildTitleTree 输出)。 */ +export interface TitleTreeNode extends HeadingInfo { + children?: TitleTreeNode[]; +} + +/** 按 level 栈式回溯将扁平标题列表转为树(容忍跳级标题)。 */ +export function buildTitleTree (titles: HeadingInfo[]): TitleTreeNode[] { + const roots: TitleTreeNode[] = []; + const stack: TitleTreeNode[] = []; + for (const t of titles) { + const node: TitleTreeNode = { ...t }; + while (stack.length > 0 && stack[stack.length - 1]!.level >= node.level) stack.pop(); + if (stack.length === 0) { + roots.push(node); + } else { + const parent = stack[stack.length - 1]!; + (parent.children ??= []).push(node); + } + stack.push(node); + } + return roots; +} diff --git a/src/md-ast/parsers/table.ts b/src/md-ast/parsers/table.ts index 714201b..ff8ce9c 100644 --- a/src/md-ast/parsers/table.ts +++ b/src/md-ast/parsers/table.ts @@ -41,8 +41,10 @@ export const tableParser: BlockParser = { const row = Math.floor(i / cols); const col = i % cols; if (rows[row]?.cells[col]) { - rows[row]!.cells[col]!.rowSpan = m.row_span > 1 ? m.row_span : undefined; - rows[row]!.cells[col]!.colSpan = m.col_span > 1 ? m.col_span : undefined; + const rowSpan = m.row_span ?? 1; + const colSpan = m.col_span ?? 1; + rows[row]!.cells[col]!.rowSpan = rowSpan > 1 ? rowSpan : undefined; + rows[row]!.cells[col]!.colSpan = colSpan > 1 ? colSpan : undefined; } } diff --git a/src/md-ast/serializer.ts b/src/md-ast/serializer.ts index fd10353..6e49a2b 100644 --- a/src/md-ast/serializer.ts +++ b/src/md-ast/serializer.ts @@ -1,6 +1,7 @@ import { Registry } from '../core/registry.js'; +import { escapeCell, renderMarkdownTable } from '../sheet/index.js'; import type { DocSourceType } from './transformer.js'; -import type { MdBlockNode, MdInlineNode } from './types.js'; +import type { MdBlockNode, MdInlineNode, MdTableRow } from './types.js'; export interface NodeSerializer { type: string; @@ -221,23 +222,47 @@ const tableSerializer: NodeSerializer = { type: 'table', serialize (node, ctx) { if (node.type !== 'table') return ''; - let buf = '<table>\n'; - for (const row of node.rows) { - buf += '<tr>\n'; - for (const cell of row.cells) { - let attrs = ''; - if (cell.rowSpan && cell.rowSpan > 1) attrs += ` rowspan="${cell.rowSpan}"`; - if (cell.colSpan && cell.colSpan > 1) attrs += ` colspan="${cell.colSpan}"`; - const content = ctx.serializeInline(cell.content); - buf += `<td${attrs}>${content}</td>`; - } - buf += '</tr>\n'; - } - buf += '</table>\n'; - return buf; + const grid = rebuildGrid(node.rows, ctx); + return renderMarkdownTable(grid); }, }; +/** + * 基于 parser 产出的稀疏 rows(被合并覆盖的格已被过滤,顶格包含 rowSpan/colSpan) + * 还原为稠密二维字符串网格:Markdown 不支持合并,按"复制左上角值"展开。 + */ +function rebuildGrid (rows: MdTableRow[], ctx: SerializeContext): string[][] { + const occupied = new Set<string>(); + const grid: string[][] = []; + for (let r = 0; r < rows.length; r++) { + if (!grid[r]) grid[r] = []; + let c = 0; + for (const cell of rows[r]!.cells) { + while (occupied.has(`${r}-${c}`)) c++; + const raw = ctx.serializeInline(cell.content); + const value = escapeCell(raw); + const rs = cell.rowSpan ?? 1; + const cs = cell.colSpan ?? 1; + for (let rr = r; rr < r + rs; rr++) { + if (!grid[rr]) grid[rr] = []; + for (let cc = c; cc < c + cs; cc++) { + grid[rr]![cc] = value; + if (rr !== r || cc !== c) occupied.add(`${rr}-${cc}`); + } + } + c += cs; + } + } + // 补齐列宽:避免稀疏数组导致 join 出 undefined + const maxCols = grid.reduce((m, row) => Math.max(m, row.length), 0); + for (const row of grid) { + for (let i = 0; i < maxCols; i++) { + if (row[i] === undefined) row[i] = ''; + } + } + return grid; +} + const gridSerializer: NodeSerializer = { type: 'grid', serialize (node, ctx) { @@ -264,13 +289,7 @@ const sheetResolvedSerializer: NodeSerializer = { out += `## ${node.title}-${s.title}\n\n`; } if (s.error) { out += `> ${s.error}\n\n`; continue; } - if (!s.rows.length) { out += '_(空表)_\n\n'; continue; } - const [head, ...body] = s.rows; - if (!head) { out += '_(空表)_\n\n'; continue; } - out += `| ${head.join(' | ')} |\n`; - out += `| ${head.map(() => '---').join(' | ')} |\n`; - for (const r of body) out += `| ${r.join(' | ')} |\n`; - out += '\n'; + out += renderMarkdownTable(s.rows); } return out; }, diff --git a/src/md-ast/transformer.ts b/src/md-ast/transformer.ts index 14ce1a7..53c1aa0 100644 --- a/src/md-ast/transformer.ts +++ b/src/md-ast/transformer.ts @@ -47,7 +47,7 @@ export class MdTransformer { const uniqueTokens = [...new Set(tokens)]; if (uniqueTokens.length === 0) return map; - if (this.opts.imageMode === 'online' || this.opts.agent === true) { + if (this.opts.imageMode === 'online' || this.opts.agent === 'stdout') { for (let i = 0; i < uniqueTokens.length; i += 5) { const batch = uniqueTokens.slice(i, i + 5); const urlMap = await this.client.batchGetTmpDownloadUrl(batch); @@ -280,7 +280,7 @@ export class MdTransformer { error: `读取失败:${e.message}`, }); } finally { - await sleep(600); + await sleep(300); } } map.set(raw, { diff --git a/src/sheet/index.ts b/src/sheet/index.ts index f4eb09d..155f056 100644 --- a/src/sheet/index.ts +++ b/src/sheet/index.ts @@ -21,6 +21,8 @@ export interface ResolvedSheet { const escapeCell = (s: string) => s.replace(/\|/g, '\\|').replace(/\n/g, '<br>'); +export { escapeCell }; + export function cellToMd (cell: unknown): string { if (cell == null) return ''; if (typeof cell !== 'object') return escapeCell(String(cell)); @@ -104,3 +106,19 @@ export function columnIndexToLetter (n: number): string { } return result; } + +// ─── renderMarkdownTable: 二维字符串 → Markdown 管道表格 ────────────────── +// +// 首行作为表头,其余为表体;空表输出占位文案。调用方负责对单元格内容做 +// `escapeCell` 转义与合并展开。 + +export function renderMarkdownTable (rows: string[][]): string { + if (!rows.length) return '_(空表)_\n\n'; + const [head, ...body] = rows; + if (!head || head.length === 0) return '_(空表)_\n\n'; + let out = `| ${head.join(' | ')} |\n`; + out += `| ${head.map(() => '---').join(' | ')} |\n`; + for (const r of body) out += `| ${r.join(' | ')} |\n`; + out += '\n'; + return out; +} diff --git a/src/title-filter.ts b/src/title-filter.ts index 0fdd785..4963e10 100644 --- a/src/title-filter.ts +++ b/src/title-filter.ts @@ -15,8 +15,19 @@ export interface TitleFilterResult { blocks: DocxBlock[]; /** 是否已找到匹配 */ matched: boolean; - /** 扫描期间遇到的所有 heading(按出现顺序,不去重;仅 scanning 阶段收集) */ - availableHeadings: Array<{ level: number; text: string }>; + /** 扫描期间遇到的所有 heading(按出现顺序,不去重;仅 scanning 阶段收集)。 + * 未匹配时扫描会走完整份文档,此列表与 get-titles 输出同形。 */ + availableHeadings: HeadingInfo[]; +} + +/** 单条标题信息(扁平形式)。多级路径/同名兄弟由上层 buildTitleTree 生成的嵌套结构隐含。 */ +export interface HeadingInfo { + /** 飞书块 id(最稳定锚点) */ + blockId: string; + /** 标题级别 1~9 */ + level: number; + /** 标题文本(trim 后) */ + text: string; } /** @@ -47,20 +58,46 @@ export function extractHeadingText (block: DocxBlock): string | null { } /** - * 创建标题过滤器,返回一个 pageHandler 兼容的回调和结果获取器。 - * 纯函数工厂,无副作用,易于测试。 + * 将 heading block 转为 HeadingInfo;非 heading 返回 null。纯函数,无状态。 */ -export function createTitleFilter (options: TitleFilterOptions): { - /** 传入 getDocxBlocks 的 pageHandler */ - pageHandler: (blocks: DocxBlock[]) => boolean; - /** 获取最终结果 */ - getResult: () => TitleFilterResult; -} { - const targetTitle = options.title.trim(); +function toHeadingInfo (block: DocxBlock): HeadingInfo | null { + const level = getHeadingLevel(block); + if (level === null) return null; + return { + blockId: block.block_id ?? '', + level, + text: extractHeadingText(block) ?? '', + }; +} + +// ─── 公共骨架 ────────────────────────────────────────────────────────────── + +/** 祖先栈条目:扫描期维护的「严格递增 level」标题链。 */ +interface HeadingStackEntry { + info: HeadingInfo; + block: DocxBlock; +} + +/** + * 通用 heading 过滤器骨架:扫描 → 命中 → 收集 → 终止 状态机。 + * 调用方仅需提供 `match(block, info)` 谓词决定何时进入 collecting。 + * + * 复用要点: + * - page 节点(block_type=1)始终保留 + * - scanning 阶段把所有 heading 推入 availableHeadings,并以栈式回溯维护祖先链 + * - 命中时把祖先 heading(level < 命中 level)按文档顺序注入 collected, + * 作为「仅标题」上下文(heading 的非命中兄弟内容不在 blockMap 中,Parser 会自动跳过) + * - collecting 阶段遇到同级或更高级标题终止 + */ +function createHeadingMatchFilter ( + match: (block: DocxBlock, info: HeadingInfo) => boolean, +): { pageHandler: (blocks: DocxBlock[]) => boolean; getResult: () => TitleFilterResult } { let state: FilterState = 'scanning'; let matchedLevel = 0; const collected: DocxBlock[] = []; - const seenHeadings: Array<{ level: number; text: string }> = []; + const seen: HeadingInfo[] = []; + // 扫描期祖先栈:始终保持栈内 level 严格递增(与 buildTitleTree 的栈式回溯一致) + const ancestorStack: HeadingStackEntry[] = []; function pageHandler (blocks: DocxBlock[]): boolean { for (const block of blocks) { @@ -71,15 +108,26 @@ export function createTitleFilter (options: TitleFilterOptions): { } switch (state) { case 'scanning': { - const level = getHeadingLevel(block); - if (level !== null) { - const text = extractHeadingText(block) ?? ''; - // 记录扫描到的所有标题(不去重、按出现顺序),用于未匹配时给出可用标题列表 - seenHeadings.push({ level, text }); - if (text === targetTitle) { + const info = toHeadingInfo(block); + if (info) { + // 栈式回溯:弹出所有 level >= 当前的旧条目,保证栈内即为「祖先链」 + while ( + ancestorStack.length > 0 + && ancestorStack[ancestorStack.length - 1]!.info.level >= info.level + ) { + ancestorStack.pop(); + } + seen.push(info); + if (match(block, info)) { + // 命中:按文档顺序注入祖先 heading(仅 heading block 本身) + for (const entry of ancestorStack) { + collected.push(entry.block); + } state = 'collecting'; - matchedLevel = level; + matchedLevel = info.level; collected.push(block); + } else { + ancestorStack.push({ info, block }); } } break; @@ -104,10 +152,55 @@ export function createTitleFilter (options: TitleFilterOptions): { function getResult (): TitleFilterResult { return { blocks: [...collected], - matched: state === 'collecting' || state === 'done', - availableHeadings: [...seenHeadings], + matched: state !== 'scanning', + availableHeadings: [...seen], }; } return { pageHandler, getResult }; } + +/** + * 按标题文本过滤(首个匹配生效;若有同名标题,请改用 createTitleBlockIdFilter)。 + */ +export function createTitleFilter (options: TitleFilterOptions): { + pageHandler: (blocks: DocxBlock[]) => boolean; + getResult: () => TitleFilterResult; +} { + const target = options.title.trim(); + return createHeadingMatchFilter((_block, info) => info.text === target); +} + +/** + * 按 heading 块 id 过滤(最精确,不会受同名标题干扰)。 + * 仅匹配 block_type 为 heading(1~9)且 block_id 严格相等的块。 + */ +export function createTitleBlockIdFilter (options: { blockId: string }): { + pageHandler: (blocks: DocxBlock[]) => boolean; + getResult: () => TitleFilterResult; +} { + const target = options.blockId.trim(); + // info 非 null 已在骨架中保证(仅 heading 才会进入 match),block.block_id 比较即可 + return createHeadingMatchFilter(block => block.block_id === target); +} + +/** + * 流式收集文档中所有标题,用于 get-titles 命令。 + * 返回的 pageHandler 始终返回 true,以遵循分页拉取全量文档。 + */ +export function createHeadingCollector (): { + pageHandler: (blocks: DocxBlock[]) => boolean; + getHeadings: () => HeadingInfo[]; +} { + const headings: HeadingInfo[] = []; + + function pageHandler (blocks: DocxBlock[]): boolean { + for (const block of blocks) { + const info = toHeadingInfo(block); + if (info) headings.push(info); + } + return true; + } + + return { pageHandler, getHeadings: () => [...headings] }; +} diff --git a/src/types.ts b/src/types.ts index 2645ee9..5ac9b73 100644 --- a/src/types.ts +++ b/src/types.ts @@ -31,6 +31,14 @@ export type WbFormat = 'base64' | 'inline-svg' | 'svg' | 'yaml'; /** 画板 SVG 背景样式:'none' | '#xxx' 颜色值 | 'dot' 点阵 */ export type SvgBackground = 'none' | 'dot' | (string & {}); +/** + * Agent 模式: + * - `'stdout'`:图片/画板全部走在线 URL,markdown 输出到 stdout + * - `'local'`:图片/画板中的图片/markdown 均落盘,stdout 输出引导 AI 读取的提示词 + * - `false` / 省略:禁用 agent 模式 + */ +export type AgentMode = 'stdout' | 'local'; + export interface ConvertOptions { appId: string; appSecret: string; @@ -40,14 +48,12 @@ export interface ConvertOptions { wbImageMode: WbImageMode; wbBg: SvgBackground; wbFormat: WbFormat; - /** - * Agent 模式: - * - `true`:图片/画板全部走在线 URL,markdown 输出到 stdout - * - `'local'`:图片/画板中的图片/markdown 均落盘,stdout 输出引导 AI 读取的提示词 - */ - agent?: boolean | 'local'; - /** 按标题过滤:仅转换匹配标题及其下级内容 */ + /** Agent 模式,详见 {@link AgentMode}。 */ + agent?: AgentMode | false; + /** 按标题过滤:仅转换匹配标题及其下级内容(单一标题,同名时取首个) */ filterTitle?: string; + /** 按 heading 块 id 过滤:最精确,避开一切同名歧义 */ + filterTitleBlockId?: string; } export interface ConvertResult { diff --git a/src/url.ts b/src/url.ts new file mode 100644 index 0000000..7e8f4fc --- /dev/null +++ b/src/url.ts @@ -0,0 +1,20 @@ +/** + * 飞书文档 URL 解析工具。独立模块,避免 converter / get-titles 之间的循环依赖。 + */ + +export interface ParsedFeishuUrl { + /** 'docs' | 'docx' | 'wiki' | 'sheets' */ + docType: string; + /** 文档/wiki 节点 token */ + docToken: string; + /** ?sheet=XXX 查询参数(仅对 sheets 类型有意义) */ + sheetId?: string; +} + +export function parseWikiUrl (url: string): ParsedFeishuUrl { + const m = url.match(/^https:\/\/[\w.-]+\/(docs|docx|wiki|sheets)\/([a-zA-Z0-9]+)/); + if (!m) throw new Error('Invalid feishu document URL'); + // 解析 ?sheet=XXX 查询参数(仅对 sheets 类型有意义) + const sheetId = new URL(url).searchParams.get('sheet') ?? undefined; + return { docType: m[1]!, docToken: m[2]!, sheetId }; +} diff --git a/tests/converter.test.ts b/tests/converter.test.ts index 08eb523..c961138 100644 --- a/tests/converter.test.ts +++ b/tests/converter.test.ts @@ -1,5 +1,7 @@ import { describe, it, expect } from 'vitest'; import { parseWikiUrl } from '../src/converter.js'; +import { buildTitleTree } from '../src/get-titles.js'; +import type { HeadingInfo } from '../src/title-filter.js'; describe('parseWikiUrl', () => { it('parses docx URL', () => { @@ -42,3 +44,51 @@ describe('parseWikiUrl', () => { expect(result.docToken).toBe('abc123'); }); }); + +// ─── buildTitleTree ────────────────────────────────────────────────────────────────────────── + +function h (blockId: string, level: number, text: string): HeadingInfo { + return { blockId, level, text }; +} + +describe('buildTitleTree', () => { + it('builds tree from flat headings using level', () => { + const flat: HeadingInfo[] = [ + h('h1', 1, 'A'), + h('h2', 2, 'A.1'), + h('h3', 2, 'A.2'), + h('h4', 1, 'B'), + ]; + const tree = buildTitleTree(flat); + expect(tree).toHaveLength(2); + expect(tree[0]!.text).toBe('A'); + expect(tree[0]!.children).toHaveLength(2); + expect(tree[0]!.children![0]!.text).toBe('A.1'); + expect(tree[1]!.text).toBe('B'); + expect(tree[1]!.children).toBeUndefined(); + }); + + it('handles skipped levels by attaching to nearest higher-level ancestor', () => { + const flat: HeadingInfo[] = [ + h('h1', 1, 'A'), + h('h3', 3, 'C'), // 跳过 H2,C 仍作为 A 的子节点 + ]; + const tree = buildTitleTree(flat); + expect(tree).toHaveLength(1); + expect(tree[0]!.children).toHaveLength(1); + expect(tree[0]!.children![0]!.text).toBe('C'); + }); + + it('returns empty array for empty input', () => { + expect(buildTitleTree([])).toEqual([]); + }); + + it('multiple roots when first heading is not the lowest level', () => { + const flat: HeadingInfo[] = [ + h('h2', 2, 'X'), + h('h2b', 2, 'Y'), + ]; + const tree = buildTitleTree(flat); + expect(tree).toHaveLength(2); + }); +}); diff --git a/tests/md-ast/serializer.test.ts b/tests/md-ast/serializer.test.ts index b71ab93..61bf025 100644 --- a/tests/md-ast/serializer.test.ts +++ b/tests/md-ast/serializer.test.ts @@ -124,7 +124,7 @@ describe('MdSerializer', () => { }); describe('table', () => { - it('serializes as HTML table', () => { + it('serializes as Markdown pipe table', () => { const node: MdBlockNode = { type: 'table', rows: [ @@ -133,19 +133,40 @@ describe('MdSerializer', () => { ], }; const result = s.serialize(node); - expect(result).toContain('<table>'); - expect(result).toContain('<td>A</td>'); - expect(result).toContain('</table>'); + expect(result).toContain('| A | B |'); + expect(result).toContain('| --- | --- |'); + expect(result).toContain('| C | D |'); + expect(result).not.toContain('<table>'); + expect(result).not.toContain('<td>'); }); - it('serializes colspan/rowspan', () => { + it('expands colspan/rowspan by duplicating the top-left value', () => { + // 3x3 grid with top-left cell merged as 2x2 const node: MdBlockNode = { type: 'table', - rows: [{ cells: [{ content: [text('X')], colSpan: 2, rowSpan: 3 }] }], + rows: [ + { cells: [{ content: [text('X')], rowSpan: 2, colSpan: 2 }, { content: [text('c')] }] }, + { cells: [{ content: [text('f')] }] }, + { cells: [{ content: [text('g')] }, { content: [text('h')] }, { content: [text('i')] }] }, + ], + }; + const result = s.serialize(node); + expect(result).toContain('| X | X | c |'); + expect(result).toContain('| X | X | f |'); + expect(result).toContain('| g | h | i |'); + }); + + it('escapes pipe and converts inline marks', () => { + const node: MdBlockNode = { + type: 'table', + rows: [ + { cells: [{ content: [text('a|b')] }, { content: [bold('B')] }] }, + { cells: [{ content: [text('c')] }, { content: [link('d', 'http://x')] }] }, + ], }; const result = s.serialize(node); - expect(result).toContain('rowspan="3"'); - expect(result).toContain('colspan="2"'); + expect(result).toContain('| a\\|b | **B** |'); + expect(result).toContain('| c | [d](http://x) |'); }); }); diff --git a/tests/md-ast/transformer.test.ts b/tests/md-ast/transformer.test.ts index e04d869..d1b4aec 100644 --- a/tests/md-ast/transformer.test.ts +++ b/tests/md-ast/transformer.test.ts @@ -84,10 +84,10 @@ describe('MdTransformer - image resolution (online)', () => { expect(batchFn).toHaveBeenCalledTimes(3); // 12 tokens / 5 = 3 batches (5,5,2) }); - it('agent=true forces online mode', async () => { + it('agent=stdout forces online mode', async () => { const batchFn = vi.fn().mockResolvedValue({ T: 'https://cdn/t' }); const client = mockClient({ batchGetTmpDownloadUrl: batchFn } as any); - const tr = new MdTransformer(client, makeOpts({ imageMode: 'local', agent: true })); + const tr = new MdTransformer(client, makeOpts({ imageMode: 'local', agent: 'stdout' })); const ast: MdBlockNode = { type: 'page', title: [], children: [{ type: 'image', alt: '', src: 'T' }], diff --git a/tests/sheet/index.test.ts b/tests/sheet/index.test.ts index 383284c..cb1e225 100644 --- a/tests/sheet/index.test.ts +++ b/tests/sheet/index.test.ts @@ -1,5 +1,5 @@ import { describe, it, expect } from 'vitest'; -import { cellToMd, expandMerges, trimTrailingEmpty, columnIndexToLetter } from '../../src/sheet/index.js'; +import { cellToMd, expandMerges, trimTrailingEmpty, columnIndexToLetter, renderMarkdownTable, escapeCell } from '../../src/sheet/index.js'; // ─── cellToMd ─────────────────────────────────────────────────────────────── @@ -135,3 +135,47 @@ describe('columnIndexToLetter', () => { expect(columnIndexToLetter(0)).toBe(''); }); }); + +// ─── escapeCell ─────────────────────────────────────────────────── + +describe('escapeCell', () => { + it('escapes pipe characters', () => { + expect(escapeCell('a|b|c')).toBe('a\\|b\\|c'); + }); + + it('converts newline to <br>', () => { + expect(escapeCell('a\nb')).toBe('a<br>b'); + }); + + it('leaves plain text unchanged', () => { + expect(escapeCell('hello world')).toBe('hello world'); + }); +}); + +// ─── renderMarkdownTable ──────────────────────────────────────────── + +describe('renderMarkdownTable', () => { + it('outputs placeholder for empty grid', () => { + expect(renderMarkdownTable([])).toBe('_(空表)_\n\n'); + expect(renderMarkdownTable([[]])).toBe('_(空表)_\n\n'); + }); + + it('renders header + body', () => { + const result = renderMarkdownTable([ + ['h1', 'h2'], + ['a', 'b'], + ['c', 'd'], + ]); + expect(result).toContain('| h1 | h2 |'); + expect(result).toContain('| --- | --- |'); + expect(result).toContain('| a | b |'); + expect(result).toContain('| c | d |'); + expect(result.endsWith('\n\n')).toBe(true); + }); + + it('renders header-only table', () => { + const result = renderMarkdownTable([['h1', 'h2']]); + expect(result).toContain('| h1 | h2 |'); + expect(result).toContain('| --- | --- |'); + }); +}); diff --git a/tests/title-filter.test.ts b/tests/title-filter.test.ts index 9fae007..96dc3eb 100644 --- a/tests/title-filter.test.ts +++ b/tests/title-filter.test.ts @@ -1,5 +1,11 @@ import { describe, it, expect } from 'vitest'; -import { createTitleFilter, getHeadingLevel, extractHeadingText } from '../src/title-filter.js'; +import { + createHeadingCollector, + createTitleBlockIdFilter, + createTitleFilter, + extractHeadingText, + getHeadingLevel, +} from '../src/title-filter.js'; import type { DocxBlock } from '../src/types.js'; // ─── Helper: 创建模拟 DocxBlock ───────────────────────────────────────────── @@ -167,11 +173,11 @@ describe('createTitleFilter', () => { const result = getResult(); expect(result.matched).toBe(false); expect(result.availableHeadings).toEqual([ - { level: 1, text: 'Intro' }, - { level: 2, text: 'Section A' }, - { level: 3, text: 'Section A' }, - { level: 2, text: 'Section A' }, - { level: 1, text: 'Outro' }, + { blockId: 'blk', level: 1, text: 'Intro' }, + { blockId: 'blk', level: 2, text: 'Section A' }, + { blockId: 'blk', level: 3, text: 'Section A' }, + { blockId: 'blk', level: 2, text: 'Section A' }, + { blockId: 'blk', level: 1, text: 'Outro' }, ]); }); @@ -188,8 +194,8 @@ describe('createTitleFilter', () => { const result = getResult(); expect(result.matched).toBe(true); expect(result.availableHeadings).toEqual([ - { level: 1, text: 'Before' }, - { level: 2, text: 'Target' }, + { blockId: 'blk', level: 1, text: 'Before' }, + { blockId: 'blk', level: 2, text: 'Target' }, ]); }); @@ -229,3 +235,358 @@ describe('createTitleFilter', () => { expect(result.matched).toBe(true); }); }); + +// ─── createTitleFilter - ancestor injection ──────────────────────────────── + +describe('createTitleFilter - ancestor injection', () => { + it('injects direct parent heading (heading-only) when matching a deeper heading', () => { + const { pageHandler, getResult } = createTitleFilter({ title: 'X' }); + + const blocks = [ + makeHeadingBlock(1, 'A', 'h-a'), + makeTextBlock('a-text'), // 父级 A 的兄弟内容(不应注入) + makeHeadingBlock(2, 'B', 'h-b'), + makeTextBlock('b-text'), // 父级 B 的兄弟内容(不应注入) + makeHeadingBlock(3, 'X', 'h-x'), + makeTextBlock('x-content', ), + ]; + pageHandler(blocks); + const result = getResult(); + + expect(result.matched).toBe(true); + // 祖先 A、B + 命中 X + X 的内容(注意 a-text、b-text 因不在 blockMap 中会被 Parser 跳过, + // 但本层 collected 不包含它们就足够保证「heading-only」语义) + expect(result.blocks.map(b => b.block_id)).toEqual([ + 'h-a', 'h-b', 'h-x', 'x-content', + ]); + }); + + it('injects multi-level ancestors in document order', () => { + const { pageHandler, getResult } = createTitleFilter({ title: 'Deep' }); + + pageHandler([ + makeHeadingBlock(1, 'L1', 'h1'), + makeHeadingBlock(2, 'L2', 'h2'), + makeHeadingBlock(3, 'L3', 'h3'), + makeHeadingBlock(4, 'Deep', 'h-deep'), + ]); + + const result = getResult(); + expect(result.matched).toBe(true); + expect(result.blocks.map(b => b.block_id)).toEqual(['h1', 'h2', 'h3', 'h-deep']); + }); + + it('does not inject when match is top-level', () => { + const { pageHandler, getResult } = createTitleFilter({ title: 'Top' }); + + pageHandler([ + makeHeadingBlock(1, 'Top', 'h-top'), + makeTextBlock('c1'), + ]); + + const result = getResult(); + expect(result.blocks.map(b => b.block_id)).toEqual(['h-top', 'c1']); + }); + + it('does not inject sibling headings that are not ancestors', () => { + const { pageHandler, getResult } = createTitleFilter({ title: 'X' }); + + // 旁支:## B 在遇到下一个 ## B2 时已被弹出,不应作为 ### X 的祖先 + pageHandler([ + makeHeadingBlock(1, 'A', 'h-a'), + makeHeadingBlock(2, 'B', 'h-b'), + makeTextBlock('b-text'), + makeHeadingBlock(2, 'B2', 'h-b2'), + makeHeadingBlock(3, 'X', 'h-x'), + makeTextBlock('x-content'), + ]); + + const result = getResult(); + expect(result.matched).toBe(true); + expect(result.blocks.map(b => b.block_id)).toEqual([ + 'h-a', 'h-b2', 'h-x', 'x-content', + ]); + // 关键负断言:不能含有 h-b + expect(result.blocks.some(b => b.block_id === 'h-b')).toBe(false); + }); + + it('handles skipped levels (H1 → H3) without fabricating H2', () => { + const { pageHandler, getResult } = createTitleFilter({ title: 'Skip' }); + + pageHandler([ + makeHeadingBlock(1, 'A', 'h1'), + makeHeadingBlock(3, 'Skip', 'h3'), + makeTextBlock('t'), + ]); + + const result = getResult(); + expect(result.blocks.map(b => b.block_id)).toEqual(['h1', 'h3', 't']); + }); + + it('keeps ancestor stack across multiple pageHandler calls (cross-page)', () => { + const { pageHandler, getResult } = createTitleFilter({ title: 'X' }); + + // Page 1:仅祖先 + pageHandler([ + makeHeadingBlock(1, 'A', 'h-a'), + makeHeadingBlock(2, 'B', 'h-b'), + ]); + // Page 2:命中 + pageHandler([ + makeHeadingBlock(3, 'X', 'h-x'), + makeTextBlock('x-content'), + ]); + + const result = getResult(); + expect(result.matched).toBe(true); + expect(result.blocks.map(b => b.block_id)).toEqual([ + 'h-a', 'h-b', 'h-x', 'x-content', + ]); + }); + + it('availableHeadings is unaffected by ancestor injection', () => { + const { pageHandler, getResult } = createTitleFilter({ title: 'X' }); + + pageHandler([ + makeHeadingBlock(1, 'A', 'h-a'), + makeHeadingBlock(2, 'B', 'h-b'), + makeHeadingBlock(3, 'X', 'h-x'), + makeHeadingBlock(4, 'Child', 'h-child'), // collecting 阶段,不进 availableHeadings + ]); + + const result = getResult(); + expect(result.availableHeadings).toEqual([ + { blockId: 'h-a', level: 1, text: 'A' }, + { blockId: 'h-b', level: 2, text: 'B' }, + { blockId: 'h-x', level: 3, text: 'X' }, + ]); + }); + + it('keeps page block before injected ancestors', () => { + const { pageHandler, getResult } = createTitleFilter({ title: 'X' }); + + pageHandler([ + makePageBlock('page'), + makeHeadingBlock(1, 'A', 'h-a'), + makeHeadingBlock(2, 'X', 'h-x'), + makeTextBlock('c'), + ]); + + const result = getResult(); + expect(result.blocks.map(b => b.block_id)).toEqual([ + 'page', 'h-a', 'h-x', 'c', + ]); + }); +}); + +// ─── createTitleBlockIdFilter - ancestor injection ───────────────────────── + +describe('createTitleBlockIdFilter - ancestor injection', () => { + it('injects multi-level ancestors when matching by block id', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: 'h-x' }); + + pageHandler([ + makeHeadingBlock(1, 'A', 'h-a'), + makeHeadingBlock(2, 'B', 'h-b'), + makeHeadingBlock(3, 'Same', 'h-x'), + makeTextBlock('c'), + ]); + + const result = getResult(); + expect(result.matched).toBe(true); + expect(result.blocks.map(b => b.block_id)).toEqual(['h-a', 'h-b', 'h-x', 'c']); + }); + + it('does not inject sibling headings that are not ancestors', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: 'h-x' }); + + pageHandler([ + makeHeadingBlock(1, 'A', 'h-a'), + makeHeadingBlock(2, 'B', 'h-b'), + makeHeadingBlock(2, 'B2', 'h-b2'), + makeHeadingBlock(3, 'X', 'h-x'), + makeTextBlock('c'), + ]); + + const result = getResult(); + expect(result.blocks.map(b => b.block_id)).toEqual(['h-a', 'h-b2', 'h-x', 'c']); + expect(result.blocks.some(b => b.block_id === 'h-b')).toBe(false); + }); +}); + +// ─── createTitleBlockIdFilter ──────────────────────────────────────────────── + +describe('createTitleBlockIdFilter', () => { + it('matches by block id rather than text (disambiguates same-name headings)', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: 'h-target' }); + + pageHandler([ + makeHeadingBlock(2, 'Same Name', 'h-other'), + makeTextBlock('a'), + makeHeadingBlock(2, 'Same Name', 'h-target'), + makeTextBlock('b'), + makeHeadingBlock(2, 'Same Name', 'h-third'), + ]); + + const result = getResult(); + expect(result.matched).toBe(true); + // 命中的 heading + b,遇到同级标题停止 + expect(result.blocks).toHaveLength(2); + expect(result.blocks[0]!.block_id).toBe('h-target'); + expect(result.blocks[1]!.block_id).toBe('b'); + }); + + it('non-heading blocks with matching id never match', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: 'txt' }); + + pageHandler([makeHeadingBlock(1, 'A', 'h1'), makeTextBlock('txt')]); + + const result = getResult(); + expect(result.matched).toBe(false); + }); + + it('trims whitespace from blockId option', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: ' h1 ' }); + + pageHandler([makeHeadingBlock(1, 'A', 'h1'), makeTextBlock('c')]); + + const result = getResult(); + expect(result.matched).toBe(true); + }); + + it('always keeps page blocks', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: 'h1' }); + + pageHandler([makePageBlock(), makeHeadingBlock(1, 'X', 'h1'), makeTextBlock()]); + + expect(getResult().blocks[0]!.block_type).toBe(1); + }); + + it('returns false from pageHandler after done', () => { + const { pageHandler } = createTitleBlockIdFilter({ blockId: 'h1' }); + + pageHandler([ + makeHeadingBlock(1, 'A', 'h1'), + makeHeadingBlock(1, 'B', 'h2'), // 同级 → done + ]); + expect(pageHandler([makeTextBlock()])).toBe(false); + }); + + it('availableHeadings on miss carries blockId/level/text (yaml-shape)', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: 'never' }); + + pageHandler([ + makeHeadingBlock(1, 'A', 'h1'), + makeHeadingBlock(2, 'B', 'h2'), + makeHeadingBlock(2, 'B', 'h3'), // 同名同父级 + ]); + + const result = getResult(); + expect(result.matched).toBe(false); + expect(result.availableHeadings).toEqual([ + { blockId: 'h1', level: 1, text: 'A' }, + { blockId: 'h2', level: 2, text: 'B' }, + { blockId: 'h3', level: 2, text: 'B' }, + ]); + }); + + it('stops collecting at higher-level heading (level < matchedLevel)', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: 'h-sub' }); + + pageHandler([ + makeHeadingBlock(2, 'Sub', 'h-sub'), + makeTextBlock('c1'), + makeHeadingBlock(1, 'Top', 'h-top'), // 更高级 → 停止 + makeTextBlock('after'), + ]); + + const result = getResult(); + expect(result.blocks).toHaveLength(2); // Sub + c1 + }); + + it('handles multi-page scanning before match', () => { + const { pageHandler, getResult } = createTitleBlockIdFilter({ blockId: 'h-found' }); + + expect(pageHandler([makeHeadingBlock(1, 'A', 'h1')])).toBe(true); + expect(pageHandler([makeHeadingBlock(1, 'B', 'h-found'), makeTextBlock('c')])).toBe(true); + + const result = getResult(); + expect(result.matched).toBe(true); + expect(result.blocks).toHaveLength(2); + }); +}); + +// ─── createHeadingCollector ────────────────────────────────────────────────── + +describe('createHeadingCollector', () => { + it('collects flat HeadingInfo (blockId/level/text)', () => { + const { pageHandler, getHeadings } = createHeadingCollector(); + + pageHandler([ + makePageBlock(), + makeHeadingBlock(1, 'A', 'h1'), + makeTextBlock('t'), + makeHeadingBlock(2, 'A.1', 'h2'), + makeHeadingBlock(2, 'A.2', 'h3'), + makeHeadingBlock(1, 'B', 'h4'), + ]); + + expect(getHeadings()).toEqual([ + { blockId: 'h1', level: 1, text: 'A' }, + { blockId: 'h2', level: 2, text: 'A.1' }, + { blockId: 'h3', level: 2, text: 'A.2' }, + { blockId: 'h4', level: 1, text: 'B' }, + ]); + }); + + it('handles skipped levels (e.g. H1 then H3)', () => { + const { pageHandler, getHeadings } = createHeadingCollector(); + + pageHandler([ + makeHeadingBlock(1, 'A', 'h1'), + makeHeadingBlock(3, 'C', 'h3'), + makeHeadingBlock(2, 'B', 'h2'), + ]); + + expect(getHeadings()).toEqual([ + { blockId: 'h1', level: 1, text: 'A' }, + { blockId: 'h3', level: 3, text: 'C' }, + { blockId: 'h2', level: 2, text: 'B' }, + ]); + }); + + it('returns empty array when no headings present', () => { + const { pageHandler, getHeadings } = createHeadingCollector(); + pageHandler([makePageBlock(), makeTextBlock('t1'), makeTextBlock('t2')]); + expect(getHeadings()).toEqual([]); + }); + + it('pageHandler always returns true to drain document', () => { + const { pageHandler } = createHeadingCollector(); + expect(pageHandler([makeHeadingBlock(1, 'A', 'h1')])).toBe(true); + expect(pageHandler([])).toBe(true); + }); + + it('accumulates across multiple page calls', () => { + const { pageHandler, getHeadings } = createHeadingCollector(); + + pageHandler([makeHeadingBlock(1, 'A', 'h1'), makeHeadingBlock(2, 'A.1', 'h2')]); + pageHandler([makeHeadingBlock(1, 'B', 'h3')]); + + expect(getHeadings()).toHaveLength(3); + expect(getHeadings()[2]).toEqual({ + blockId: 'h3', level: 1, text: 'B', + }); + }); + + it('falls back to empty blockId when block_id is missing', () => { + const { pageHandler, getHeadings } = createHeadingCollector(); + + pageHandler([{ + block_type: 3, + heading1: { elements: [{ text_run: { content: 'NoId' } }] }, + } as unknown as DocxBlock]); + + expect(getHeadings()[0]!.blockId).toBe(''); + }); +});