diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9ae0963..2eb0edd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -29,5 +29,5 @@ jobs: python -m pip install --upgrade pip python -m pip install -e ".[dev,llm]" - - name: Run base test suite - run: python -m pytest -k "not integration" + - name: Run test suite + run: python -m pytest diff --git a/CONFIG.zh-CN.md b/CONFIG.zh-CN.md index 3c27203..ca766d7 100644 --- a/CONFIG.zh-CN.md +++ b/CONFIG.zh-CN.md @@ -1,10 +1,10 @@ # SlideNote 配置指南 -SlideNote 现在把普通用户入口收敛到两个 preset：`lecture` 和 `local`。底层仍然保留 OCR、Vision、图文锚定、Lecture-Weave、缓存和质量报告等能力，但这些不再作为 `slidenote build` 的日常参数暴露。 +SlideNote 的普通用户入口是两个 preset：`lecture` 和 `local`。`build` 只公开少量常用选项；更细的图文锚定、Lecture-Weave、缓存与质量检查由流水线内部处理。 ## 我该怎么跑？ -高质量讲义，默认推荐： +正式生成讲义，使用默认 `lecture`： ```powershell $env:DEEPSEEK_API_KEY="..." @@ -18,11 +18,11 @@ python -m slidenote build lecture.pdf --out outputs\lecture --provider deepseek python -m slidenote build lecture.pdf --out outputs\local --preset local --export markdown-zip ``` -关闭视觉理解，只用文本模型写讲义： +关闭视觉模型来写讲义（课件图片仍可能进入笔记，OCR 也可单独运行）： ```powershell $env:DEEPSEEK_API_KEY="..." -python -m slidenote build lecture.pdf --out outputs\text-only --provider deepseek --vision off +python -m slidenote build lecture.pdf --out outputs\no-vision --provider deepseek --vision off ``` 从已有笔记生成复习包： @@ -31,7 +31,7 @@ python -m slidenote build lecture.pdf --out outputs\text-only --provider deepsee python -m slidenote study-pack outputs\lecture --question-count 12 ``` -把 PDF 教材构建成 RAG-ready 文档库： +把 PDF 教材解析成供后续检索使用的分块语料（当前尚无向量索引，也未接入笔记生成）： ```powershell python -m slidenote textbook-index textbook.pdf --out outputs\textbook --ocr auto @@ -43,7 +43,7 @@ python -m slidenote textbook-index textbook.pdf --out outputs\textbook --ocr aut python -m slidenote build lecture.pdf --out outputs\paper --export docx,pdf ``` -分享 Markdown 给别人时，优先使用 `--export markdown-zip`。`notes.zip` 里包含 `notes.md` 和 `notes.assets/`，对方解压后打开 `notes.md` 才能看到图片。 +分享 Markdown 给别人时，优先使用 `--export markdown-zip`。`notes.zip` 包含 `notes.md`；有图片资源时还包含 `notes.assets/`。对方解压后打开 `notes.md` 即可查看。 ## Build 参数 @@ -51,11 +51,12 @@ python -m slidenote build lecture.pdf --out outputs\paper --export docx,pdf | 参数 | 默认值 | 说明 | | --- | --- | --- | -| `input` | 必填 | 输入 `.pptx` / `.ppt` / `.pdf`。 | +| `input` | 必填 | 输入 `.pptx` / `.ppt` / `.pdf`；其他格式取决于可选外部 parser。 | | `--out` | `outputs/slidenote` | 输出目录。 | | `--preset` | `lecture` | `lecture` 走强质量 AI 讲义流程；`local` 不调用 API。 | | `--provider` | `deepseek` | 文本模型 provider。支持 `deepseek`、`openai`、`qwen`、`doubao`、`glm`、`gemini`、`claude`。 | -| `--vision` | `auto` | `auto` 启用视觉理解；`off` 跳过视觉 API。`local` preset 会强制关闭。 | +| `--vision` | `auto` | `auto` 按需调用视觉模型；`off` 跳过视觉 API，但不删除课件图片。`local` preset 会强制关闭。 | +| `--ocr` | `auto` | `auto` 只处理低文本/扫描页；`off` 跳过 OCR；`all` 处理所有页。`local` preset 会强制关闭。 | | `--export` | 无 | 额外导出：`markdown-zip`、`markdown-toc`、`docx`、`pdf`、`latex`、`all`。 | | `--parser` | `auto` | 可选解析器入口，普通用户不用改。 | | `--progress-json` | `/progress.json` | GUI/自动化使用的进度文件。 | @@ -76,8 +77,10 @@ python -m slidenote build lecture.pdf --out outputs\paper --export docx,pdf | Preset | 适合场景 | 行为 | | --- | --- | --- | -| `lecture` | 正式学习、长期保存、希望笔记像讲义。 | 默认启用 LLM、OCR auto、Vision auto、图文锚定、Deck Brief、Content Guard、Lecture-Weave、teaching enrichment 和本地缓存。需要 provider API key。 | -| `local` | 没有 API key、离线预览、检查解析是否正常。 | 不调用文本模型、视觉模型或 OCR API，只用本地规则生成基础 Markdown 和质量报告。 | +| `lecture` | 正式学习、长期保存、需要图文理解；像老师重新讲一遍。 | 默认启用 LLM、OCR auto、Vision auto、图文锚定、Deck Brief、Content Guard、Lecture-Weave、teaching enrichment 和本地缓存。需要相应 API key。 | +| `local` | 首次预览、没有 API key、检查课件能否被读出。 | 不调用文本模型、视觉模型或 OCR API，只用本地规则生成基础 Markdown 和质量报告。 | + +首次处理新课件可先用 `local` 检查解析；正式学习或分享笔记再用 `lecture`。图表较多时保留 `--vision auto`；只想关闭视觉模型时使用 `--vision off`。复习题由独立的 `study-pack` 命令生成。 ## 环境变量 diff --git a/README.md b/README.md index e3a191b..bf93de8 100644 --- a/README.md +++ b/README.md @@ -37,17 +37,19 @@ ## Contents - [Quick Start](#quick-start) +- [Modes and Pipeline](#modes-and-pipeline) +- [Outputs and Review](#outputs-and-review) - [Optional GUI](#optional-gui) -- [Pipeline And Presets](#slidenote-pipeline) +- [Textbook Chunks](#textbook-chunks) - [Origin](#origin) -- [Setup](#setup) -- [Common Workflows](#common-workflows) -- [Technical Docs](#technical-docs) -- [Future Outlook](#future-outlook) -- [License And Acknowledgements](#license) +- [Setup and Docs](#setup-and-docs) +- [License](#license) +- [Acknowledgements](#acknowledgements) ## Quick Start +On Windows / PowerShell: + ```powershell git clone https://github.com/Cat-blizzard/SlideNote.git cd SlideNote @@ -55,208 +57,88 @@ cd SlideNote .\run_gui.ps1 ``` -The setup script creates `.venv`, installs SlideNote with GUI/LLM extras, and runs `slidenote doctor`. The GUI lets you paste API keys in the page for a single run, so you do not have to set terminal environment variables first. - -Manual setup is still available: - -```powershell -python -m venv .venv -.\.venv\Scripts\Activate.ps1 -python -m pip install -e ".[dev,llm]" -python -m slidenote doctor -``` - -Manual setup does not include the GUI extra; `.\run_gui.ps1` automatically installs `".[dev,llm,gui]"` when it is missing. - -For a local preview without API calls: +The installer creates `.venv`, installs GUI and model dependencies, and checks the environment. You can enter API keys in the GUI for a single run. Start with a local preview to check extraction and note generation: ```powershell python -m slidenote build path\to\lecture.pdf --out outputs\local --preset local --export markdown-zip ``` -After the first install, run this Local preview command first. Confirm that `notes.md` and the shareable `notes.zip` are generated before switching to the `lecture` quality workflow. - -For higher-quality notes with visual understanding: +For model-assisted writing and visual understanding, configure the relevant API keys. For example, with DeepSeek for text and the default vision provider: ```powershell -$env:DASHSCOPE_API_KEY="..." $env:DEEPSEEK_API_KEY="..." +$env:DASHSCOPE_API_KEY="..." python -m slidenote build path\to\lecture.pdf --out outputs\lecture --provider deepseek --export markdown-zip ``` -Open `outputs\lecture\notes.md` after generation. Images are copied into `outputs\lecture\notes.assets\` by default. - -## Optional GUI - -SlideNote Studio is a Streamlit interface around the same CLI pipeline. It supports uploading PPT/PDF files, entering API keys in the page, selecting presets, watching progress and ETA, reviewing token/cost reports, checking page-level sources, and downloading generated results. - -```powershell -.\run_gui.ps1 -``` +The main output is `outputs\lecture\notes.md`. Check generated explanations against the slides. -See [gui/README_GUI.md](gui/README_GUI.md) and [gui/README_GUI.zh-CN.md](gui/README_GUI.zh-CN.md) for GUI details. +For manual setup, use `python -m pip install -e "."` for local mode or `python -m pip install -e ".[llm]"` for model mode. `.\run_gui.ps1` installs the GUI extra when needed. The `dev` extra is mainly for project tests. -## Textbook Library +## Modes and Pipeline -SlideNote can also build a RAG-ready corpus from a PDF textbook. This entrypoint only parses the textbook, detects the table of contents, maps sections to pages, and writes retrievable chunks; it is not connected to note generation yet. - -```powershell -python -m slidenote textbook-index path\to\textbook.pdf --out outputs\textbook --ocr auto -``` - -`--ocr auto` extracts native PDF text first and only calls OCR for scanned or low-text pages. Use `--ocr off` for copyable digital textbooks. - -## SlideNote Pipeline +| Mode | Use | Behavior | +| --- | --- | --- | +| Default `lecture` | Model-assisted detailed study notes | Uses a text model and, as configured, OCR, visual understanding, and Lecture-Weave writing. Quality depends on the source and model output. | +| `local` | Offline preview and extraction checks | Makes no text, vision, or OCR API calls; local rules produce basic notes. | -SlideNote is organized as a five-stage product pipeline. Low-level modules can stay fine-grained for caching, debugging, and partial refresh; the user-facing workflow should remain simple. +`--vision off` disables visual model calls; it does not prevent images from appearing in notes or exports. Use `--ocr off|auto|all` to adjust OCR. See [configuration](CONFIG.zh-CN.md) for options and presets. ```text Ingest -> Understand -> Write -> Guard -> Export ``` -| Stage | Purpose | Main artifacts | +| Stage | Main work | Main artifacts | | --- | --- | --- | -| **1. Ingest** | Parse PPT/PDF into stable, traceable structure. | `content.json`, `element_ir.json`, `source_map.json`, screenshots, assets, parser adapters | -| **2. Understand** | Decide what the courseware is teaching. | `deck_understanding.json`, `page_understanding.json`, `sections.json`, `deck_brief.json`, figure/table understanding | -| **3. Write** | Turn structured material into readable study notes. | `notes.md`, Lecture-Weave page notes, teaching enrichment | -| **4. Guard** | Check faithfulness, coverage, and study quality. | `coverage.json`, `coverage.md`, `content_guard.json`, `quality_report.json` | -| **5. Export** | Publish notes and reports. | `notes.zip`, `notes.toc.md`, `notes.docx`, `notes.pdf`, `notes.tex`; review/exam packs are generated separately by `study-pack` | - -More detail: [SlideNote Pipeline](docs/pipeline.zh-CN.md). +| **Ingest** | Parse PPT/PDF and extract pages, screenshots, and image assets. | Pages and assets for later stages | +| **Understand** | Run OCR, visual and structural understanding, and assemble structured content. | `content.json`, `deck_understanding.json`, `page_understanding.json`; `content_guard.json` when configured | +| **Write** | Generate readable study notes. | `notes.md` | +| **Guard** | Build source mappings, coverage reports, and quality diagnostics. | Final `element_ir.json`, `source_map.json`, `coverage.json`, `coverage.md`, `quality_report.json` | +| **Export** | Produce requested sharing or reading formats. | `notes.zip`, `notes.docx`, `notes.pdf`, and others | -## User Presets +See the [pipeline guide](docs/pipeline.zh-CN.md) for implementation details. -Use top-level `--preset` for product workflows. Everyday users now only need two modes: the default `lecture` mode and the no-API `local` mode. +## Outputs and Review -| Preset | Best for | Behavior | -| --- | --- | --- | -| `lecture` | Teacher-style detailed lecture notes. | Enables LLM, OCR auto, Vision auto, Lecture-Weave, deck brief, content guard, and teaching enrichment. | -| `local` | No API key, offline preview, parser checks. | Uses local rules only and does not call text, vision, or OCR APIs. | +`notes.md` is the main output. With `--export markdown-zip`, SlideNote writes `notes.zip` containing the notes; it includes files from `notes.assets/` when images are referenced. Word and PDF export require external tools; see [configuration](CONFIG.zh-CN.md). Generate review and exam materials separately after a build: ```powershell -python -m slidenote build lecture.pdf --out outputs\lecture --provider deepseek -python -m slidenote build lecture.pdf --out outputs\local --preset local +python -m slidenote study-pack outputs\lecture --question-count 20 ``` -More detail: [User Presets](docs/presets.zh-CN.md). - -## Origin - -SlideNote started from a very personal learning problem. - -I have never been the kind of student who learns best by simply listening to lectures. Sometimes I cannot fully follow a teacher's explanation in real time, and I usually learn more efficiently by reading. Reading lets me slow down, go back, skip ahead, and control the pace of understanding by myself. - -But lecture slides are not the same as readable notes. After class, reading the PPT directly often feels incomplete: the bullets are fragmented, the logic is implicit, and many important details live in diagrams, screenshots, formulas, or the teacher's spoken explanation. Manually rewriting everything into notes is possible, but it is time-consuming, hard to keep complete, and not always pleasant to revisit later. - -So I wanted to build a tool that could turn course slides into structured, readable, traceable notes: not just a summary, but a faithful learning document that preserves images, keeps page references, checks coverage, and helps convert lecture materials into something I can actually study from. - -That idea became SlideNote. +Coverage uses element IDs and text markers to flag potentially missing source items. Quality scores are mostly heuristic diagnostics. Neither proves factual accuracy, completeness, or question validity. Before sharing notes, compare important claims, figure placement, equations, tables, and exported layout with the source slides. -## Setup - -SlideNote does not require a local GPU. The local parser can run with only Python dependencies; LLM rewriting, OCR, and visual understanding require API keys for the providers you choose. - -Minimum setup: - -- Python `3.10` or newer. -- A virtual environment is recommended. -- New users can run `.\install.ps1` and then `.\run_gui.ps1`. -- `python -m pip install -e ".[dev]"` for local parsing. -- `python -m pip install -e ".[dev,llm]"` for LLM providers. - -Optional software: - -| Software | Purpose | -| --- | --- | -| LibreOffice | Converts `.ppt` / `.pptx` to PDF and enables full-slide screenshots when PowerPoint is unavailable. | -| Microsoft PowerPoint + `pywin32` | Windows-only PPTX screenshot export route. | -| Pandoc | Word and LaTeX export. | -| LibreOffice + Pandoc | PDF export from `notes.docx`, usually more stable for CJK layout. | - -Configuration details live in [CONFIG.zh-CN.md](CONFIG.zh-CN.md). The `build` entrypoint is intentionally small; provider, OCR, Vision, and cache details are handled mostly through strong defaults and environment variables. - -> The setup scripts target Windows PowerShell; the `python -m slidenote ...` commands themselves are cross-platform (Linux/macOS users can skip the `.ps1` scripts and call the commands directly). - -## Common Workflows - -Local rule-based draft: - -```powershell -python -m slidenote build path\to\lecture.pptx --out outputs\local --preset local --export markdown-zip -``` +## Optional GUI -Teacher-style lecture notes: +SlideNote Studio lets you upload PPT/PDF, enter temporary API keys, select a mode, view progress and reports, inspect page screenshots and notes, and download outputs: ```powershell -python -m slidenote build path\to\lecture.pdf ` - --out outputs\lecture-notes ` - --provider deepseek ` - --export markdown-zip +.\run_gui.ps1 ``` -Review and exam pack: +See the [GUI guide](gui/README_GUI.md). -```powershell -python -m slidenote build path\to\lecture.pdf ` - --out outputs\lecture-review ` - --provider deepseek -python -m slidenote study-pack outputs\lecture-review --question-count 20 -``` +## Textbook Chunks -Text-only lecture notes: +`textbook-index` turns a PDF textbook into a chunked corpus with table-of-contents and section mapping for future retrieval features. It currently creates no vector index, provides no search, and does not feed note generation. ```powershell -python -m slidenote build path\to\lecture.pdf ` - --out outputs\text-only ` - --provider deepseek ` - --vision off +python -m slidenote textbook-index path\to\textbook.pdf --out outputs\textbook --ocr auto ``` -## Technical Docs +For a digital PDF with selectable text, try `--ocr off`. The `auto` setting uses OCR on scanned or low-text pages. -README is intentionally kept as a landing page. Detailed behavior lives in the docs (currently Chinese-first; the tables and code examples remain readable for English users): - -| Topic | Link | -| --- | --- | -| Documentation index | [docs/index.zh-CN.md](docs/index.zh-CN.md) | -| Pipeline stages | [docs/pipeline.zh-CN.md](docs/pipeline.zh-CN.md) | -| Presets | [docs/presets.zh-CN.md](docs/presets.zh-CN.md) | -| Coverage, content guard, quality report, review/exam packs | [docs/quality-and-guard.zh-CN.md](docs/quality-and-guard.zh-CN.md) | -| Element IR, source map, assets | [docs/ir-and-source-map.zh-CN.md](docs/ir-and-source-map.zh-CN.md) | -| LLM providers, OCR, vision, cache, cost | [docs/providers-and-cost.zh-CN.md](docs/providers-and-cost.zh-CN.md) | -| Roadmap design notes | [docs/roadmap/extension-notes.zh-CN.md](docs/roadmap/extension-notes.zh-CN.md) | - -The main output is `notes.md`. To share Markdown notes with images, export `notes.zip`; it contains `notes.md` and the `notes.assets/` image folder. Depending on options, SlideNote can also write `content.json`, `deck_understanding.json`, `page_understanding.json`, `element_ir.json`, `source_map.json`, `coverage.md`, `quality_report.json`, `review.md`, `exam.md`, `exam.json`, `exam.html`, `notes.docx`, `notes.pdf`, and other reports. - -## Future Outlook - -SlideNote is built with a hopeful assumption: future AI systems will become stronger, faster, cheaper, and easier to orchestrate through mature open-source agent frameworks. If that happens, this project should not merely run the same prompts for less money. Its ceiling should rise. - -Models and providers such as DeepSeek are one example of the direction that makes this exciting: better price/performance, broader access, and a more open ecosystem can make high-quality multi-pass workflows practical for ordinary study materials. When API latency drops and agent frameworks become more reliable, SlideNote can afford to run richer stages by default: deeper deck understanding, page-level visual reasoning, teacher-style section writing, teaching enrichment, coverage repair, exam generation, wrong-answer review, and source verification. - -The reason this matters is that SlideNote's bottleneck is not only "can the model summarize a slide?" The harder problem is coordinating parsing, vision, writing, grounding, quality checks, and revision without losing traceability. That is why the project invests in `element_ir.json`, `source_map.json`, coverage reports, artifact registries, presets, cache keys, and review/exam packs. Those structures let SlideNote absorb future model gains without being tied to one model, provider, or agent runtime. - -The long-term vision is: - -> SlideNote should grow from a courseware converter into a course learning operating system. - -In that version, slides, readings, personal notes, figures, formulas, quizzes, mistakes, and revisions all live in one traceable learning workflow. - -## Design Principle +## Origin -SlideNote deliberately avoids this shortcut: +I learn more comfortably by reading and revisiting material at my own pace. Lecture slides, however, are often prompts for a live explanation: the logic is scattered, and important details may be in figures, tables, formulas, or what the teacher says. Rewriting them into notes after class takes time and can miss details. -```text -PPT -> LLM -> Summary -``` +SlideNote grew from the idea of turning slides into structured study notes that preserve images and links to their source pages, with coverage reports that point to material worth checking. The aim is to make courseware easier to read and review while keeping the original material available for verification. -Instead, it follows: +## Setup and Docs -```text -PPT/PDF -> structured extraction -> source inventory -> note generation -> coverage check -> export -``` +SlideNote needs Python 3.10 or newer. Local mode needs no GPU. LibreOffice or PowerPoint may be needed for slide conversion and full-page screenshots; Pandoc and LibreOffice may be needed for Word, PDF, or LaTeX exports. The setup scripts above target Windows; Linux/macOS users can call the same `python -m slidenote ...` commands. -The local rule-based draft is only a baseline for debugging extraction and coverage. Production notes should use the default `lecture` preset, while coverage checks still rely on element IDs so the model cannot silently summarize away details. +See the [documentation index](docs/index.zh-CN.md), [configuration](CONFIG.zh-CN.md), and [roadmap](ROADMAP.zh-CN.md). The detailed docs are currently Chinese-first. The longer-term aim is a traceable workflow across slides, textbooks, review questions, and personal notes; the roadmap tracks actual priorities. ## License @@ -280,19 +162,3 @@ The SlideNote name, logo, and other brand assets are not licensed for standalone - SlideNote's future retrieval, source tracing, and post-generation QA direction is informed by systems such as [RAGFlow](https://github.com/infiniflow/ragflow). These projects are references and inspirations, not bundled dependencies unless explicitly listed elsewhere. - Thanks to [LEO690201](https://github.com/LEO690201) for contributing bug fixes that improved SlideNote's reliability. - SlideNote's development has also benefited from code analysis, implementation, and debugging assistance provided by Codex, Claude Code, and DeepSeek Harness. All AI-assisted changes remain subject to maintainer review and project testing. - -## References - -- [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat/create) -- [OpenAI Images and vision](https://developers.openai.com/api/docs/guides/images-vision) -- [DeepSeek API](https://api-docs.deepseek.com/) -- [Alibaba Cloud Model Studio OpenAI-compatible API](https://help.aliyun.com/zh/model-studio/compatibility-of-openai-with-dashscope) -- [Volcengine Ark OpenAI SDK compatibility](https://www.volcengine.com/docs/82379/1330626) -- [Zhipu GLM OpenAI compatibility](https://docs.bigmodel.cn/cn/guide/develop/openai/introduction) -- [Baidu OCR API](https://ai.baidu.com/ai-doc/REFERENCE/4kru2vqdg) -- [Mathpix OCR API](https://docs.mathpix.com/reference/post-v3-text) -- [Google Cloud Vision OCR](https://cloud.google.com/vision/docs/ocr) -- [Gemini generateContent API](https://ai.google.dev/gemini-api/docs/text-generation) -- [Gemini image understanding](https://ai.google.dev/gemini-api/docs/image-understanding) -- [Claude Messages API](https://docs.anthropic.com/en/api/messages) -- [Claude Vision](https://platform.claude.com/docs/en/build-with-claude/vision) diff --git a/README.zh-CN.md b/README.zh-CN.md index 14b5870..74fe8d7 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -37,17 +37,19 @@ ## 目录 - [快速开始](#快速开始) +- [模式与工作流程](#模式与工作流程) +- [结果与复核](#结果与复核) - [可选 GUI](#可选-gui) -- [Pipeline 与 Preset](#slidenote-pipeline) +- [教材分块](#教材分块) - [起源](#起源) -- [环境与安装](#环境与安装) -- [常用工作流](#常用工作流) -- [技术文档](#技术文档) -- [未来展望](#未来展望) -- [许可证与致谢](#许可证) +- [配置与文档](#配置与文档) +- [许可证](#许可证) +- [致谢](#致谢) ## 快速开始 +Windows / PowerShell 用户可以运行： + ```powershell git clone https://github.com/Cat-blizzard/SlideNote.git cd SlideNote @@ -55,204 +57,88 @@ cd SlideNote .\run_gui.ps1 ``` -安装脚本会创建 `.venv`、安装带 GUI/LLM 的依赖，并运行 `slidenote doctor`。GUI 可以在页面里临时填写 API key，所以新手不需要先理解终端环境变量。 - -也可以手动安装： - -```powershell -python -m venv .venv -.\.venv\Scripts\Activate.ps1 -python -m pip install -e ".[dev,llm]" -python -m slidenote doctor -``` - -手动安装默认不含 GUI 依赖；运行 `.\run_gui.ps1` 时脚本会按需自动补装 `".[dev,llm,gui]"`。 - -如果只想先本地预览，确认课件能正常解析： +安装脚本会创建 `.venv`、安装 GUI 和模型相关依赖，并运行环境检查。GUI 支持在页面中临时填写 API key。先用本地模式检查课件能否解析、笔记能否生成： ```powershell python -m slidenote build path\to\lecture.pdf --out outputs\local --preset local --export markdown-zip ``` -第一次安装后建议先跑这条 Local preview 命令，确认 `notes.md` 和可分享的 `notes.zip` 都能生成，再切换到 `lecture` 质量模式。 - -如果想生成带图片理解的高质量讲义式笔记： +需要模型辅助讲解和视觉理解时，再配置对应的 API key。例如，使用 DeepSeek 文本模型和默认视觉模型： ```powershell -$env:DASHSCOPE_API_KEY="..." $env:DEEPSEEK_API_KEY="..." +$env:DASHSCOPE_API_KEY="..." python -m slidenote build path\to\lecture.pdf --out outputs\lecture --provider deepseek --export markdown-zip ``` -生成后打开 `outputs\lecture\notes.md`。图片默认会复制到 `outputs\lecture\notes.assets\`。 - -## 可选 GUI - -SlideNote Studio 是套在同一条 CLI pipeline 外面的 Streamlit 图形界面。它支持上传 PPT/PDF、在页面内配置 API key、选择运行 preset、查看进度和 ETA、查看 token / 成本报告、逐页检查来源，并下载生成结果。 - -```powershell -.\run_gui.ps1 -``` - -GUI 详情见 [gui/README_GUI.zh-CN.md](gui/README_GUI.zh-CN.md) 和 [gui/README_GUI.md](gui/README_GUI.md)。 - -## 教材库 - -SlideNote 也可以先把 PDF 教材构建成 RAG-ready 文档库。这个入口只做教材解析、目录识别、章节映射和 chunk 切片；当前不会自动参与笔记生成。 +主输出在 `outputs\lecture\notes.md`。模型生成的内容仍应对照课件核查。 -```powershell -python -m slidenote textbook-index path\to\textbook.pdf --out outputs\textbook --ocr auto -``` +手动安装可用 `python -m pip install -e "."`（本地模式）或 `python -m pip install -e ".[llm]"`（模型模式）；运行 GUI 时，`.\run_gui.ps1` 会按需补装 GUI 依赖。`dev` extra 主要用于项目测试，不是普通使用的前提。 -`--ocr auto` 会先抽取 PDF 原生文本，只对扫描页或低文本页调用 OCR。可复制文字的电子教材可以使用 `--ocr off`。 +## 模式与工作流程 -## SlideNote Pipeline +| 模式 | 用途 | 行为 | +| --- | --- | --- | +| 默认 `lecture` | 生成模型辅助的详细讲义 | 使用文本模型，并按配置执行 OCR、视觉理解和 Lecture-Weave 写作；质量取决于课件与模型输出。 | +| `local` | 离线预览与解析检查 | 不调用文本、视觉或 OCR API；用本地规则生成基础笔记。 | -SlideNote 按五个产品阶段组织。底层模块可以继续保持细粒度，方便缓存、调试和局部刷新；但用户侧应该先看到一条清楚的流水线，而不是一长串彼此独立的开关。 +`--vision off` 只关闭视觉模型调用，并不禁止笔记引用或导出图片；需要调整 OCR 时可用 `--ocr off|auto|all`。参数及预设说明见 [配置参考](CONFIG.zh-CN.md)。 ```text Ingest -> Understand -> Write -> Guard -> Export ``` -| 阶段 | 作用 | 主要产物 | +| 阶段 | 主要工作 | 主要产物 | | --- | --- | --- | -| **1. Ingest** | 稳定解析 PPT/PDF，并保留可追溯来源。 | `content.json`、`element_ir.json`、`source_map.json`、截图、图片资产、parser adapter | -| **2. Understand** | 理解课件在讲什么，每页/每图/每表起什么作用。 | `deck_understanding.json`、`page_understanding.json`、`sections.json`、`deck_brief.json`、图表理解 | -| **3. Write** | 把结构化材料写成可读学习笔记。 | `notes.md`、Lecture-Weave 逐页讲解、teaching enrichment | -| **4. Guard** | 检查保真、覆盖率和学习质量。 | `coverage.json`、`coverage.md`、`content_guard.json`、`quality_report.json` | -| **5. Export** | 发布最终结果和运行报告。 | `notes.zip`、`notes.toc.md`、`notes.docx`、`notes.pdf`、`notes.tex`；复习/考试包由 `study-pack` 另行生成 | +| **Ingest** | 解析 PPT/PDF，提取页面、截图和图片资产。 | 页面与素材，供后续阶段使用 | +| **Understand** | OCR、视觉与结构理解，并整理结构化内容。 | `content.json`、`deck_understanding.json`、`page_understanding.json`；按配置生成 `content_guard.json` 等 | +| **Write** | 生成可阅读的学习笔记。 | `notes.md` | +| **Guard** | 生成来源映射、覆盖率和质量诊断。 | 最终 `element_ir.json`、`source_map.json`、`coverage.json`、`coverage.md`、`quality_report.json` | +| **Export** | 按需导出分享或阅读格式。 | `notes.zip`、`notes.docx`、`notes.pdf` 等 | -详细说明见 [SlideNote Pipeline](docs/pipeline.zh-CN.md)。 +实现细节见 [Pipeline 文档](docs/pipeline.zh-CN.md)。 -## 用户侧 Preset +## 结果与复核 -顶层 `--preset` 是用户侧工作流入口。现在普通用户只需要理解两个模式：默认 `lecture` 和无 API 的 `local`。 - -| Preset | 适合场景 | 背后行为 | -| --- | --- | --- | -| `lecture` | 想要“像老师重新讲一遍”的详细讲义。 | 默认启用 LLM、OCR auto、Vision auto、Lecture-Weave、Deck Brief、Content Guard 和 teaching enrichment。 | -| `local` | 没有 API key、离线预览、检查解析是否正常。 | 不调用文本模型、视觉模型或 OCR API，只用本地规则生成基础 Markdown。 | +`notes.md` 是主输出。选择 `--export markdown-zip` 后会生成 `notes.zip`，其中包含笔记；笔记引用了图片时，还会包含 `notes.assets/` 中的相应文件。Word、PDF 等格式需要相应的外部工具，见 [配置参考](CONFIG.zh-CN.md)。复习和考试材料可在构建后单独生成： ```powershell -python -m slidenote build lecture.pdf --out outputs\lecture --provider deepseek -python -m slidenote build lecture.pdf --out outputs\local --preset local +python -m slidenote study-pack outputs\lecture --question-count 20 ``` -详细说明见 [用户侧 Preset](docs/presets.zh-CN.md)。 - -## 起源 - -SlideNote 来自一个个人的学习困境。 - -我一直不是那种特别适合“只靠听课”学习的人。有时候老师讲得很快，或者表达方式不太适合我，我在课堂上并不能完全跟上。相比听课，我更喜欢阅读：文字可以反复看，可以停下来想，也可以按照自己的节奏跳转、回看和整理。 - -但课下直接读 PPT，我又总觉得差点意思。PPT 本质上更像是老师讲课时的提示板，而不是一份真正适合阅读和复习的笔记。很多内容都是零散的，逻辑藏在老师的讲解里，关键知识还经常出现在图、表、流程图、公式截图和页面布局中。 - -当然，我也试过自己整理笔记，但这件事既耗时间，也很难保证不遗漏。而且手写笔记的字迹和排版有时会让我自己都不太想回头看。 +覆盖率通过元素 ID 和正文标记提示可能漏写的来源；质量分数主要是启发式诊断。它们不能证明解释准确、内容完整或题目有效。分享笔记前，建议对照课件检查关键事实、图文位置、公式表格及导出版式。 -所以我想做一个工具，把课程 PPT/PDF 转换成结构清晰、内容完整、保留图片、可追溯到原页码、并且经过覆盖率校验的课程笔记。它不只是总结课件，而是尽量把展示材料变成真正适合学习的文字材料。 - -于是就有了 SlideNote。 - -## 环境与安装 - -SlideNote 不需要本机 GPU。基础解析只需要 Python 依赖；LLM 改写、OCR 和视觉理解按需配置对应 provider 的 API key。 - -最低环境： - -- Python `3.10` 或更高版本。 -- 推荐使用虚拟环境。 -- 新手可以直接运行 `.\install.ps1`，然后运行 `.\run_gui.ps1`。 -- 本地解析：`python -m pip install -e ".[dev]"`。 -- LLM provider：`python -m pip install -e ".[dev,llm]"`。 - -可选软件： - -| 软件 | 用途 | -| --- | --- | -| LibreOffice | 将 `.ppt` / `.pptx` 转 PDF，并在没有 PowerPoint 时生成整页截图。 | -| Microsoft PowerPoint + `pywin32` | Windows 上的 PPTX 整页截图导出路线。 | -| Pandoc | Word 和 LaTeX 导出。 | -| LibreOffice + Pandoc | PDF 导出会优先从 `notes.docx` 转换，中文/CJK 排版更稳。 | - -配置指南见 [CONFIG.zh-CN.md](CONFIG.zh-CN.md)。现在 `build` 入口已经简化，provider、OCR、Vision 和缓存细节主要通过强默认和环境变量处理。 - -> 安装脚本面向 Windows / PowerShell；`python -m slidenote ...` 命令本身跨平台可用（Linux/macOS 用户直接调用命令即可，跳过 `.ps1` 脚本）。 - -## 常用工作流 +## 可选 GUI -本地规则草稿： +SlideNote Studio 提供上传 PPT/PDF、临时填写 API key、选择模式、查看进度和报告、逐页查看截图与笔记、下载结果等操作： ```powershell -python -m slidenote build path\to\lecture.pptx --out outputs\local --preset local --export markdown-zip +.\run_gui.ps1 ``` -教师讲义式笔记： +详情见 [GUI 使用说明](gui/README_GUI.zh-CN.md)。 -```powershell -python -m slidenote build path\to\lecture.pdf ` - --out outputs\lecture-notes ` - --provider deepseek ` - --export markdown-zip -``` +## 教材分块 -复习 / 考试包： +`textbook-index` 可把 PDF 教材解析为带目录与章节映射的分块语料，供后续检索功能使用。目前它不创建向量索引、不提供检索，也不会自动参与笔记生成。 ```powershell -python -m slidenote build path\to\lecture.pdf ` - --out outputs\lecture-review ` - --provider deepseek -python -m slidenote study-pack outputs\lecture-review --question-count 20 -``` - -纯文本讲义： - -```powershell -python -m slidenote build path\to\lecture.pdf ` - --out outputs\text-only ` - --provider deepseek ` - --vision off +python -m slidenote textbook-index path\to\textbook.pdf --out outputs\textbook --ocr auto ``` -## 技术文档 - -README 现在只作为项目首页。细节放到文档中心： - -| 主题 | 链接 | -| --- | --- | -| 文档导航 | [docs/index.zh-CN.md](docs/index.zh-CN.md) | -| 五阶段 Pipeline | [docs/pipeline.zh-CN.md](docs/pipeline.zh-CN.md) | -| 用户侧 Preset | [docs/presets.zh-CN.md](docs/presets.zh-CN.md) | -| Coverage、Content Guard、Quality Report、复习/考试包 | [docs/quality-and-guard.zh-CN.md](docs/quality-and-guard.zh-CN.md) | -| Element IR、Source Map、图片资产 | [docs/ir-and-source-map.zh-CN.md](docs/ir-and-source-map.zh-CN.md) | -| LLM Provider、OCR、Vision、缓存与成本 | [docs/providers-and-cost.zh-CN.md](docs/providers-and-cost.zh-CN.md) | -| 路线图设计笔记 | [docs/roadmap/extension-notes.zh-CN.md](docs/roadmap/extension-notes.zh-CN.md) | - -主输出是 `notes.md`。如果要把 Markdown 笔记发给别人，推荐导出 `notes.zip`，里面包含 `notes.md` 和 `notes.assets/` 图片资源。根据选项不同，SlideNote 还会写出 `content.json`、`deck_understanding.json`、`page_understanding.json`、`element_ir.json`、`source_map.json`、`coverage.md`、`quality_report.json`、`review.md`、`exam.md`、`exam.json`、`exam.html`、`notes.docx`、`notes.pdf` 等报告和导出文件。 - -## 未来展望 - -SlideNote 带着一个乐观前提在建设：未来 AI 会更强、更快、更便宜，也会更容易通过成熟的开源智能体框架来组织复杂工作流。如果这件事发生，SlideNote 不应该只是“用更低成本跑同一套 prompt”，而应该让项目上限被真正抬高。 +电子版 PDF 可尝试 `--ocr off`；`auto` 会对扫描页或低文本页使用 OCR。 -以 DeepSeek 这类强调性价比、可获得性和开放生态的模型 / 服务为例，它让人看到一种很值得期待的方向：当高质量 API 的价格、速度和可用性继续改善，多 pass 的高质量流程就不再只是少数重型场景才能负担的奢侈品。SlideNote 可以把更深的课件理解、逐页视觉推理、教师讲义式写作、teaching enrichment、coverage repair、考试题生成、错题复盘和来源校验变成更自然的默认能力。 - -这件事之所以重要，是因为 SlideNote 的难点不只是“模型能不能总结一页 PPT”。真正难的是在解析、视觉理解、写作、图文锚定、质量检查和局部修订之间保持协调，同时不丢失可追溯性。所以项目会持续投入 `element_ir.json`、`source_map.json`、coverage、artifact registry、preset、cache key 和 review/exam 学习包这些工程结构。它们让 SlideNote 能吃到未来模型进步的红利，而不是被某一个模型、某一家 provider 或某一种 agent runtime 绑死。 +## 起源 -长期愿景是： +我更习惯按自己的节奏阅读和复习，但课堂 PPT 往往只是讲课提示：逻辑分散，关键内容还可能藏在图、表、公式和老师的讲解里。课后从头整理笔记很耗时，也容易漏掉细节。 -> SlideNote 从课件转换器，成长为课程学习操作系统。 +SlideNote 因此尝试把课件整理为有结构、保留图片、能回看来源的学习笔记，并用覆盖报告提示需要复核的地方。目标是让笔记更适合阅读和复习，而不是替代对原始课件的判断。 -在这个愿景里，课件、教材、个人笔记、图表、公式、测验、错题和局部修订都处在同一条可检查、可追溯、可复习的学习工作流里。 +## 配置与文档 -## 设计原则 +SlideNote 需要 Python 3.10 或更高版本，本地模式不需要 GPU。PPT 转换和整页截图可能需要 LibreOffice 或 PowerPoint；Word、PDF、LaTeX 导出可能需要 Pandoc 和 LibreOffice。Windows 安装脚本见上文，Linux/macOS 可使用相同的 `python -m slidenote ...` 命令。 -SlideNote 不走 `PPT -> LLM -> 总结` 的捷径，而是： - -```text -PPT/PDF -> 结构化解析 -> 内容清单 -> 笔记生成 -> 覆盖率校验 -> 导出 -``` - -本地规则草稿只负责把结构化内容“保底写出来”，方便调试解析和覆盖率。正式笔记默认使用 `lecture` preset，但覆盖率检查仍然依靠元素 ID 做硬校验，避免模型把细节悄悄总结掉。 +更多内容见[文档中心](docs/index.zh-CN.md)、[配置参考](CONFIG.zh-CN.md)和[路线图](ROADMAP.zh-CN.md)。长期希望把课件、教材、复习题和个人笔记连成可追溯的学习流程；具体开发优先级以路线图为准。 ## 许可证 @@ -276,19 +162,3 @@ SlideNote 名称、logo 和其它品牌素材不授权作独立复用。具体 - SlideNote 后续的检索、来源追踪和生成后质检方向也参考了 [RAGFlow](https://github.com/infiniflow/ragflow) 这类深度文档理解 / RAG 系统。这些项目是思路参考，不代表已作为依赖打包进 SlideNote。 - 感谢 [LEO690201](https://github.com/LEO690201) 为 SlideNote 修复 bug、提升项目稳定性所作的贡献。 - SlideNote 的开发也得到了 Codex、Claude Code 和 DeepSeek Harness 在代码分析、实现与调试方面的辅助。所有 AI 辅助改动仍须经过维护者审核和项目测试。 - -## 参考文档 - -- [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat/create) -- [OpenAI Images and vision](https://developers.openai.com/api/docs/guides/images-vision) -- [DeepSeek API](https://api-docs.deepseek.com/) -- [阿里云百炼 OpenAI 兼容接口](https://help.aliyun.com/zh/model-studio/compatibility-of-openai-with-dashscope) -- [火山方舟 OpenAI SDK 兼容](https://www.volcengine.com/docs/82379/1330626) -- [智谱 GLM OpenAI 兼容](https://docs.bigmodel.cn/cn/guide/develop/openai/introduction) -- [百度 OCR API](https://ai.baidu.com/ai-doc/REFERENCE/4kru2vqdg) -- [Mathpix OCR API](https://docs.mathpix.com/reference/post-v3-text) -- [Google Cloud Vision OCR](https://cloud.google.com/vision/docs/ocr) -- [Gemini generateContent API](https://ai.google.dev/gemini-api/docs/text-generation) -- [Gemini image understanding](https://ai.google.dev/gemini-api/docs/image-understanding) -- [Claude Messages API](https://docs.anthropic.com/en/api/messages) -- [Claude Vision](https://platform.claude.com/docs/en/build-with-claude/vision) diff --git a/ROADMAP.zh-CN.md b/ROADMAP.zh-CN.md index 8ff449d..5156961 100644 --- a/ROADMAP.zh-CN.md +++ b/ROADMAP.zh-CN.md @@ -1,132 +1,80 @@ # SlideNote 后续扩展路线图 -这个文档只保留方向、优先级和关键决策。较长的模块设计说明已经迁移到 [docs/roadmap/extension-notes.zh-CN.md](docs/roadmap/extension-notes.zh-CN.md)。 +本文只记录当前能力、下一步优先级和关键原则。较长的设计想法见 [路线图设计笔记](docs/roadmap/extension-notes.zh-CN.md)，具体参数见 [CONFIG.zh-CN.md](CONFIG.zh-CN.md)。 -SlideNote 当前的核心定位是： - -> 保真型课程笔记生成器：不是简单总结 PPT/PDF，而是先解析、再理解、再写作、再检查覆盖率。 - -长期愿景是： - -> 从课件转换器，成长为课程学习操作系统。 - -## 目录 - -- [五阶段产品流水线](#五阶段产品流水线) -- [当前基础](#当前基础) -- [建议优先级](#建议优先级) -- [长期愿景](#长期愿景吃到未来-ai-红利) -- [关键原则](#关键原则) +SlideNote 的核心定位是：把 PPT/PDF 转成可阅读、可追溯、可复习的课程笔记，并报告可能遗漏的内容。长期愿景是把课件、教材、个人笔记和练习连接成一条学习工作流。 ## 五阶段产品流水线 -SlideNote 的底层能力很多，但用户侧和 LLM 工作流应该按五个阶段收束： - ```text Ingest -> Understand -> Write -> Guard -> Export ``` -| 阶段 | 当前角色 | 下一步方向 | +| 阶段 | 当前职责与主要产物 | 后续重点 | | --- | --- | --- | -| **Ingest** | 解析 PPT/PDF，生成 `content.json`、`element_ir.json`、`source_map.json`、截图和图片资产。 | 已引入 parser adapter 架构；内置解析器和 Docling / Marker / MinerU 外部 CLI adapter 通过统一 `Deck` 契约接入。 | -| **Understand** | 生成章节、页面类型、语义版面、图表理解、Deck Brief。 | 已收束为 `deck_understanding.json` 和 `page_understanding.json`，作为 GUI、Agent 和局部 revise 的稳定认知入口。 | -| **Write** | 生成 `notes.md`，支持 Lecture-Weave、section context、lecture-notes profile、teaching enrichment。 | 继续把默认高质量路线从“总结”推向“教学重构”。 | -| **Guard** | coverage、content guard、quality report、review/exam 题目质量检查。 | 加强来源校验、幻觉风险检测和错题复盘闭环。 | -| **Export** | Markdown、Word、PDF、LaTeX、review/exam pack、GUI 下载。 | 模板系统、课程级导出、多 PPT 整合。 | - -更多细节见 [Pipeline 文档](docs/pipeline.zh-CN.md)。 - -## 当前基础 +| **Ingest** | 解析课件并提取页面、截图和图片资产；内置解析器与可选外部 parser adapter 统一输出 `Deck`。 | 提高复杂课件的解析保真度。 | +| **Understand** | 执行 OCR、视觉与版面增强、章节识别、Content Guard 分类；生成 `content.json`、`deck_understanding.json` 和 `page_understanding.json`。 | 核对理解结果与原课件的对应关系。 | +| **Write** | 生成 `notes.md`；Lecture-Weave 先生成逐页内容，再按章节等上下文组织正文。 | 提高解释的准确性和阅读连贯性。 | +| **Guard** | 检查覆盖情况，生成 `coverage.json`、最终 `element_ir.json`、`source_map.json` 和 `quality_report.json`。 | 区分结构性漏项提示与需要人工判断的语义质量。 | +| **Export** | 按需导出 Word、PDF、LaTeX 等格式，并生成运行摘要。 | 验收实际渲染后的阅读排版。 | -已经具备的产品基础： +学习包由单独的 `study-pack` 命令生成，不属于 `build` 的 Guard 阶段。更多细节见 [Pipeline 文档](docs/pipeline.zh-CN.md)。 -- 支持 `.pptx` / `.pdf` 解析，`.ppt` 可尝试借助 LibreOffice 转 PDF；解析入口已抽象为 parser adapter，外部 Docling / Marker / MinerU 可选接入。 -- 生成 `content.json`、`element_ir.json`、`source_map.json`、页面截图和图片资产。 -- 支持 OCR、Vision、语义版面增强、表格理解、图片重要性排序、组合图识别、局部图裁剪、图文锚定。 -- 生成 `deck_understanding.json` 和 `page_understanding.json`，统一承载 Deck Brief、章节、页面角色、图表理解和图片排序。 -- 支持 `--preset lecture|local`，把底层参数收束成默认高质量讲义和无 API 本地预览两条用户侧工作流。 -- 支持 Lecture-Weave、Deck Brief、Content Guard、teaching enrichment 和质量报告。 -- 支持 coverage 报告、review/exam 学习包、题目质量指标和错题复盘 prompt。 -- 支持 LLM/OCR/Vision 缓存、并发、用量报告、`progress.json`、`run_summary.json` 和 GUI。 -- 支持 Markdown、带目录 Markdown、Word、PDF、LaTeX 导出。 +## 已实现的基础 -详细配置见 [CONFIG.zh-CN.md](CONFIG.zh-CN.md)，详细机制见 [docs/index.zh-CN.md](docs/index.zh-CN.md)。 +- 内置 `.pptx` / `.pdf` 解析；`.ppt` 可尝试经 LibreOffice 转换。parser adapter 已接入统一 `Deck` 契约，Docling、Marker、MinerU 是需要相应工具的可选外部 CLI adapter。 +- `build` 已按五阶段组织，支持 `lecture|local` preset，以及 OCR、Vision、图表与版面相关的可选增强。 +- 已生成 `deck_understanding.json` 和 `page_understanding.json`，汇集章节、页面角色和相关理解结果。它们目前是构建产物，GUI 和局部 revise 尚未将其作为稳定的交互入口。 +- 已生成 `notes.md`、coverage、source map 和启发式质量报告。Coverage 能提示部分漏项和来源标记问题，不能证明解释准确或学习效果。 +- 独立的 `study-pack` 命令可生成 review、exam、交互式 `exam.html`、结构化学习包与错题复盘提示词；答题记录尚未形成持久化闭环。 +- 已有 GUI 查看与下载、进度报告，以及 Markdown、Word、PDF、LaTeX 等输出能力；导出排版仍需用真实课件系统验收。 ## 建议优先级 -### P0：近期最值得做 - -1. **错题复盘闭环继续产品化** - - 让 `exam.html` 的答题结果更自然地进入 `wrong_answer_review_prompt.md`。 - - 把错题关联到 section、source page、concept 和 review pack。 - - 在 GUI 中展示“错在哪里、该回看哪里”。 - -2. **题目质量约束继续增强** - - 扩展 `quality_report.json` 的题目质量指标。 - - 检查答案唯一性、干扰项质量、解析充分性和来源覆盖。 - - 图表题保持图文就地嵌入。 - -3. **统一理解产物（已落地，2026-06-03）** - - 已生成 `deck_understanding.json`：聚合 Deck Brief、章节、页面角色、关键概念、跨页关联、重要图表。 - - 已生成 `page_understanding.json`：聚合每页 section、role、modality、key points、表格、图示、语义组和 required items。 - - Deck Brief、section detection、page role、figure/table understanding、image ranking 已收束为稳定认知包，底层调试产物仍保留。 - -4. **Parser Adapter 架构（已落地，2026-06-03）** - - 已定义内置解析器 adapter 接口，默认 `auto` 仍优先走内置 PPT/PDF 解析。 - - 已注册 Docling / Marker / MinerU 外部 CLI adapter，可通过 `--parser docling|marker|mineru` 或命令模板环境变量接入。 - - 核心 pipeline 只依赖统一 `Deck` 数据模型，不直接绑定某个外部解析库。 - -### P1：中期能力 - -1. **课程级学习包** - - 支持多 PPT / 多讲次整合。 - - 生成课程级概念图、术语表、章节导航和复习计划。 - -2. **GUI 局部编辑与 revise** - - 在 GUI 中按页、按 section 或按 source element 发起局部重写。 - - 依赖 `element_ir.json`、`source_map.json` 和 artifact registry。 - -3. **质量审阅升级** - - 引入可选 LLM 质量评审。 - - 检查机械逐页复述、解释深度、例子密度、图表整合和幻觉风险。 - - 审阅模型和写作模型尽量分离。 +### P0：先让单份课件的笔记可靠、好读 -4. **教材 / 个人笔记接入** - - 教材作为可追溯背景知识，不替代课件来源。 - - 个人笔记作为补充上下文，明确区分来源。 +1. **真实课件评测与内容核对** + - 建立包含文字密集页、图表、表格、公式、扫描页和长课件的固定样本集。 + - 对照原课件人工抽查关键事实、漏项、图文对应、来源定位；同时记录运行时间和成本。 + - 每次改动在同一批样本上比较，并留下可复核的失败案例。不要仅用 coverage 分数代表内容质量。 -### P2：长期增强 +2. **笔记阅读与导出排版验收** + - 检查标题层级、段落密度、图表与解释的位置、长表格、公式和中文换行。 + - 把 Markdown、Word、PDF 的实际渲染纳入验收，发现断页、溢出、图文错位时能定位到样本与页面。 + - 先稳定默认版式和图文关系；封面、主题、页眉页脚等装饰性模板留待后续。 -1. **开放式 Agent 工作流** - - 用 artifact registry、IR 和 source map 作为共享状态。 - - 让 agent 调用明确阶段，而不是把整个 pipeline 交给自由聊天。 +3. **来源与质量提示说清边界** + - 在文档和 GUI 中明确区分结构性 coverage、启发式质量分数与人工内容复核。 + - 让缺失项、来源页和笔记位置更容易一起查看；对高风险页面给出复查提示。 -2. **模板与发布系统** - - 支持课程封面、页眉页脚、引用样式、主题和导出模板。 - - 支持更完整的 PDF / Word / LaTeX 发布路线。 +### P1：围绕可信笔记完善学习与修订流程 -3. **更多输入格式** - - 在 parser adapter 基础上支持 Word、HTML、图片集合、教材 PDF 和更多文档格式。 +1. **错题复盘闭环** + - 保存 `exam.html` 的作答与错题记录，再关联章节、来源页、概念和 review 内容。 + - 让用户从错题进入对应笔记及复习提示；当前仅有可复制的复盘提示词。 -4. **局部性能重写** - - 如果确定性模块成为瓶颈，再考虑 Rust 或其它语言重写局部组件。 - - 当前阶段优先产品结构和质量闭环，不急于语言迁移。 +2. **局部编辑与 revise** + - 在 GUI 中按页或章节发起修订，并明确受影响的缓存和产物。 + - 现有缺失项修补会重写其传入的整段上下文；真正只改局部笔记块还需实现并验证不会破坏其余内容。 -## 长期愿景：吃到未来 AI 红利 +3. **学习包与质量审阅** + - 在 `study-pack` 中核对题目来源、答案唯一性、解析和干扰项质量；题目指标与构建笔记的 `quality_report.json` 分开呈现。 + - 可选增加独立审阅流程，重点检查事实与解释边界，并用真实样本验证效果。 -SlideNote 带着一个乐观前提在建设：未来 AI 会更强、更快、更便宜，也会更容易通过成熟的开源智能体框架来组织复杂工作流。 +### P2：在单份课件体验稳定后扩展 -以 DeepSeek 这类强调性价比、可获得性和开放生态的模型 / 服务为例，当高质量 API 的价格、速度和可用性继续改善，多 pass 的高质量流程就会更适合普通课程材料。SlideNote 可以把更深的课件理解、逐页视觉推理、教师讲义式写作、teaching enrichment、coverage repair、考试题生成、错题复盘和来源校验变成更自然的默认能力。 +- **课程级整合：**多讲次导航、概念关联、术语表和复习计划。 +- **教材与个人笔记：**作为可追溯的补充来源，清楚区分课件、外部资料和 AI 解释。 +- **受控 Agent 工作流：**基于明确阶段、产物和来源映射探索，不预设多 Agent 必然优于现有流程。 +- **发布与输入扩展：**主题、封面、页眉页脚，以及更多输入格式，按用户需求和评测结果决定顺序。 -项目真正要抓住的不是某一个模型，而是一个可迁移的工程底座：`element_ir.json`、`source_map.json`、coverage、artifact registry、preset、cache key、review/exam 学习包和 GUI 审阅工作台。这些结构让未来模型能力提升时，SlideNote 的上限也能一起升高。 +## 长期愿景 -愿景是让课件、教材、个人笔记、图表、公式、测验、错题和局部修订都处在同一条可检查、可追溯、可复习的学习工作流里。 +模型能力和成本会变化；SlideNote 应把稳定的数据结构、来源追踪和可复核的质量流程留在自身工程中，再逐步利用更好的模型。课程级工作流是长期方向，不作为近期交付承诺。 ## 关键原则 -- **不要让 README 变成参数手册。** 首页负责让人快速理解项目，细节放到 `docs/`。 -- **不要把确定性工程交给 LLM。** 元素 ID、source map、缓存、导出、成本统计应保持稳定。 -- **让 LLM 做它擅长的事。** 章节理解、图表解释、讲义写作、易错点、自测题和语义修复适合模型参与。 -- **Coverage 是质检器，不是写作模板。** 正文应该像讲义，coverage 负责最后查漏和局部修补。 -- **Preset 是用户入口。** 普通用户应优先看到 `lecture|local`；底层质量、缓存、并发、OCR/Vision 细节尽量留在内部默认里。 -- **保留项目气质。** 起源、愿景、致谢和对未来 AI 红利的期待属于 README，不是可以随手迁走的噪音。 +- **Coverage 是质检线索，不是写作模板或语义正确性的证明。** 正文应像讲义，异常项提示复查。 +- **确定性工作交给程序。** 元素 ID、来源映射、缓存、导出和报告应可重复检查。 +- **模型处理适合语义判断的任务。** 章节理解、图表解释、讲义写作和修订可以使用模型，但结果需要样本验证。 +- **Preset 是普通用户入口。** 高级参数留在配置文档，README 保持简洁。 diff --git a/docs/benchmark.zh-CN.md b/docs/benchmark.zh-CN.md index 6016cfa..1e7d026 100644 --- a/docs/benchmark.zh-CN.md +++ b/docs/benchmark.zh-CN.md @@ -1,30 +1,31 @@ # SlideNote 基准测试指南（Benchmark Guide） -> 目的：用**真实课件**建立 SlideNote 笔记生成质量的量化基线，让测试人员可以在 -> 不同分支 / 不同配置之间做可复现的对比。基准结论是后续一切优化（prompt、 -> 模型、架构）的对照组——**先有数字，再谈改进**。 +> 目的：用**真实课件**建立 SlideNote 笔记生成的基线，让测试人员可以在 +> 不同分支 / 不同配置之间做可复现的对比。记录结构指标、人工核对结果、 +> 耗时和成本，再判断 prompt、模型或架构改动是否真的改善了笔记。 -## 适用分支 +## 适用范围 + +先用 `main` 的 `slidenote build` 建立单份课件的笔记基线。下文“实验分支附录”只适用于 `experiment/dsh-backend`，其 `agent-*` 命令不在 `main` 中。两条流程的自动指标可用于发现回归，质量结论还需要阅读原课件与笔记。 | 基准 | 命令 | 分支 | | --- | --- | --- | -| A. 确定性管线 | `slidenote build`（`lecture` vs `local`） | `main`（所有分支可用） | -| B. Agent 后端 | `slidenote agent-eval` / `agent-build --backend dsh` | `experiment/dsh-backend` | +| A. 主线构建 | `slidenote build`（`lecture` 与 `local`） | `main` | +| B. Agent 实验 | `slidenote agent-eval` / `agent-build --backend dsh` | `experiment/dsh-backend` | ## 准备 -1. 环境：`.\install.ps1`（或 `python -m pip install -e ".[dev,llm]"`），`python -m slidenote doctor` 确认无红叉。 -2. API key（基准 B 与 lecture preset 需要）：`DEEPSEEK_API_KEY`（文本模型）；课件含图时可再配 vision/OCR 的 key。 +1. 环境：`.\install.ps1`（或 `python -m pip install -e ".[llm]"`），运行 `python -m slidenote doctor` 检查当前流程所需依赖。 +2. API key（lecture 和基准 B 需要）：`DEEPSEEK_API_KEY`（文本模型）；课件含图时按实际配置补齐 Vision/OCR 的 key。 3. 课件建议（覆盖面比数量重要，至少准备 3-5 份）： - 类型：理论课（多文字）、图文课（多图/流程图）、表格多的课 - 规模：10-20 页的短课件 + 50 页以上的长课件各若干 - 格式：PDF（原生文本）和扫描/低文本 PDF（触发 OCR）各准备 - - 用途说明：同一份课件在**同一个输出目录**下重复跑时，LLM 缓存会命中， - 第二次结果用于验证缓存；**换输出目录**才算新的一次独立生成。 + - 同一份课件在同一个输出目录下重复跑时可能命中缓存；测量独立生成的耗时与用量时，使用新的输出目录并记录缓存状态。 --- -## 基准 A：确定性管线（main 分支） +## 基准 A：主线构建（main 分支） ### A1. 离线基线（无 API） @@ -45,13 +46,15 @@ python -m slidenote build path\to\lecture.pdf --out outputs\baseline-lecture --e | --- | --- | | `notes.md` | 直接阅读：结构、讲解深度、图片是否插入且解释 | | `coverage.md` / `coverage.json` | `missing`（未覆盖元素数）、`coverage_ratio`、`required_visible_coverage`（必讲内容漏没漏）、`figure_coverage`（图片缺失/未解释数） | -| `quality_report.json` | `coherence_score` / `explanation_depth_score` / `figure_integration_score` / `hallucination_risk` / `suggested_repairs` | +| `quality_report.json` | `coherence_score` / `explanation_depth_score` / `figure_integration_score` / `hallucination_risk` / `suggested_repairs`；均为启发式诊断，不能代替事实核查 | | `run_summary.json` | `run.preset` / `stage_timings`（各阶段耗时，找瓶颈） | | `progress.json` | 阶段进度与 ETA | --- -## 基准 B：Agent 后端对比（experiment/dsh-backend 分支） +## 实验分支附录：Agent 后端（experiment/dsh-backend） + +以下命令只在实验分支存在。`agent-eval` 内置的 `local` 对照适合检查实验流程能否运行及其结构指标；它与使用文本模型的 Agent 输入条件不同，不能据此断言 Agent 的讲义质量优于主线 `lecture`。质量对比应另跑同一课件的 `lecture`，记录所用模型、视觉/OCR 配置、缓存状态，并人工盲评笔记。 ```powershell git checkout experiment/dsh-backend @@ -60,8 +63,8 @@ python -m slidenote doctor ### B1. 一键对比（推荐起点） -`agent-eval` 自动跑两条线并出对比报告：基线（`slidenote build --preset local`） -vs agent 流程（`agent-pack` + `agent-run --backend dsh`）。 +`agent-eval` 自动跑两条线并出结构指标报告：基线（`slidenote build --preset local`） +与 Agent 流程（`agent-pack` + `agent-run --backend dsh`）。 ```powershell $env:DEEPSEEK_API_KEY="..." @@ -96,18 +99,20 @@ python -m slidenote agent-build path\to\lecture.pdf --out outputs\agent-build-ou --- -## 对比维度（A/B 通用记录表） +## 对比维度与人工验收 -对每一份课件，按下面模板记录（建议存成 `benchmark-YYYYMMDD.md`）： +对每一份课件，按下面模板记录（建议存成 `benchmark-YYYYMMDD.md`）。人工评分应对照原页检查事实、条件、数字、公式和图表解释；最好隐藏生成路线，让评审先读结果再揭晓配置。导出 DOCX/PDF 时还要打开实际文件，检查标题层级、图文相邻、中文换行、公式表格和分页。 | 维度 | 说明 | 怎么判 | | --- | --- | --- | -| coverage ratio | 元素覆盖率 | 越高越好；`missing` 应说明遗漏类型 | -| required visible missing | 必讲内容缺失 | 应为 0；>0 说明关键知识点被跳过 | +| coverage ratio | 元素结构覆盖率 | 记录数值与遗漏类型；高分不证明讲解正确 | +| required visible missing | 必讲项可见覆盖缺失 | 记录并回看原页；0 也不证明语义完整 | | figure missing / unexplained | 图片缺失/插了没解释 | 越低越好 | | 讲义结构 | 章节、小标题、连贯性 | 人工阅读评分（1-5） | | 讲解深度 | 是否解释了"为什么" | 人工阅读评分（1-5） | -| 图片解释质量 | 图旁是否有说明文字 | 人工阅读评分（1-5） | +| 图片解释质量 | 图片说明是否准确、是否支持相邻论述 | 对照原图评分（1-5） | +| 内容准确性 | 数字、条件、公式、图表解读是否与原页相符 | 对照课件抽查，记录错误页码 | +| 阅读排版 | 标题、图文位置、公式表格、换行与分页 | 检查 Markdown；导出时检查实际 DOCX/PDF | | 耗时 | 全程耗时 + 各阶段 | 记录，便于后续优化对比 | | token / 成本 | `llm_usage.json` / `agent_run.json` | 记录总量 | | 失败与警告 | 诊断、repair 失败、warnings | 如实记录 | @@ -129,6 +134,8 @@ python -m slidenote agent-build path\to\lecture.pdf --out outputs\agent-build-ou | 讲义结构（1-5） | | | | | | 讲解深度（1-5） | | | | | | 图片解释（1-5） | | | | | +| 内容准确性（1-5） | | | | | +| 阅读排版（1-5） | | | | | | 耗时 | | | | | | token 总量 | | | | | | 失败/警告 | | | | | @@ -141,13 +148,10 @@ python -m slidenote agent-build path\to\lecture.pdf --out outputs\agent-build-ou ## 注意事项 -1. **对比一致性**：同一课件对比时，尽量固定 API key、模型（`--provider`/`--dsh-model`）、 - 并发数；不同日期的重跑因模型版本变化可能有波动，记录时注明。 -2. **缓存**：LLM 输出有本地磁盘缓存（lecture 与 agent 流程均有）。"换输出目录 = 新生成"； - 想强制重跑用 `--cache refresh`（build）或 `--dsh-cache refresh`（agent-run）。 +1. **对比一致性**：同一课件对比时，固定或记录模型、provider、视觉/OCR 配置和运行日期；实验分支额外记录 `--dsh-model`。`local`、`lecture` 和 Agent 成本与能力不同，不宜只凭一个总分排序。 +2. **缓存**：LLM 输出有本地磁盘缓存。主线 `build` 没有公开的 `--cache refresh` 参数；测独立运行时使用新的输出目录并记录缓存状态。实验分支的 `agent-run` 可使用 `--dsh-cache refresh`。 3. **耗时测量**：`build` 看 `run_summary.json` 的 `stage_timings`；agent 流程可对比 `--dsh-concurrency 1` 与默认 3 的差异。 -4. **不要只看数字**：coverage 高不代表笔记好——必须人工读 1-2 份 `notes.md` 评 - 讲解质量。数字防回归，阅读定质量。 +4. **不要只看数字**：coverage 和本地质量分数是结构信号，不验证事实。每种课件类型都应人工核对原页与笔记，记录具体错误和导出排版问题。 5. **Windows**：命令均为 PowerShell 语法；Linux/macOS 去掉 `.ps1` 脚本与 `$env:`， 直接用 `python -m slidenote ...`。 diff --git a/docs/index.zh-CN.md b/docs/index.zh-CN.md index 6d1d445..8f6e896 100644 --- a/docs/index.zh-CN.md +++ b/docs/index.zh-CN.md @@ -6,25 +6,23 @@ README 只保留项目首页需要的信息：它是什么、怎么跑、为什 1. 新用户先看 [README.zh-CN.md](../README.zh-CN.md) 的快速开始和 preset。 2. 想理解整体结构，看 [Pipeline](pipeline.zh-CN.md)。 -3. 想选择运行方式，看 [Presets](presets.zh-CN.md)。 -4. 想快速查命令、preset 和环境变量，看 [CONFIG 配置指南](../CONFIG.zh-CN.md)。 -5. 想理解 provider、OCR、Vision 和成本背景，看 [Provider、OCR、Vision、缓存与成本](providers-and-cost.zh-CN.md)。 -6. 想理解可追溯结构，看 [Element IR 与 Source Map](ir-and-source-map.zh-CN.md)。 -7. 想看覆盖率、复习包和考试包，看 [Quality And Guard](quality-and-guard.zh-CN.md)。 -8. 想看后续方向，看 [ROADMAP.zh-CN.md](../ROADMAP.zh-CN.md) 和 [路线图设计笔记](roadmap/extension-notes.zh-CN.md)。 +3. 想选择 preset、查命令和环境变量，看 [CONFIG 配置指南](../CONFIG.zh-CN.md)。 +4. 想理解 provider、OCR、Vision 和成本背景，看 [Provider、OCR、Vision、缓存与成本](providers-and-cost.zh-CN.md)。 +5. 想理解可追溯结构，看 [Element IR 与 Source Map](ir-and-source-map.zh-CN.md)。 +6. 想看覆盖率、复习包和考试包，看 [Quality And Guard](quality-and-guard.zh-CN.md)。 +7. 想看后续方向，看 [ROADMAP.zh-CN.md](../ROADMAP.zh-CN.md) 和 [路线图想法档案](roadmap/extension-notes.zh-CN.md)。 ## 文档列表 | 文档 | 说明 | | --- | --- | | [Pipeline](pipeline.zh-CN.md) | 五阶段产品流水线：Ingest、Understand、Write、Guard、Export。 | -| [Presets](presets.zh-CN.md) | 用户侧 `--preset lecture|local` 的定位。 | | [Quality And Guard](quality-and-guard.zh-CN.md) | coverage、content guard、quality report、review/exam 学习包。 | | [Element IR And Source Map](ir-and-source-map.zh-CN.md) | `content.json`、`element_ir.json`、`source_map.json` 和图片资产。 | | [Providers And Cost](providers-and-cost.zh-CN.md) | LLM provider、OCR、Vision、缓存、并发和导出依赖。 | -| [基准测试指南](benchmark.zh-CN.md) | 用真实课件建立质量基线与对比流程（含实验分支 agent 基准）。 | -| [Roadmap Design Notes](roadmap/extension-notes.zh-CN.md) | 从旧 Roadmap 迁出的模块设计笔记。 | -| [CONFIG.zh-CN.md](../CONFIG.zh-CN.md) | 简化配置指南；先按场景选命令，再看环境变量和旧参数迁移。 | +| [基准测试指南](benchmark.zh-CN.md) | 用真实课件建立主线质量基线；实验分支另列。 | +| [路线图想法档案](roadmap/extension-notes.zh-CN.md) | 历史设计设想，现行优先级以 ROADMAP 为准。 | +| [CONFIG.zh-CN.md](../CONFIG.zh-CN.md) | preset、常用命令、公开参数、环境变量和旧参数迁移的统一说明。 | | [GUI 文档](../gui/README_GUI.zh-CN.md) | SlideNote Studio 的安装、运行和界面说明。 | ## 维护原则 diff --git a/docs/ir-and-source-map.zh-CN.md b/docs/ir-and-source-map.zh-CN.md index e9847da..bec2131 100644 --- a/docs/ir-and-source-map.zh-CN.md +++ b/docs/ir-and-source-map.zh-CN.md @@ -1,18 +1,21 @@ # Element IR 与 Source Map -SlideNote 的可追溯能力依赖几个稳定结构： +SlideNote 用以下产物保留课件内容与笔记的结构化关联： ```text -content.json -> element_ir.json -> source_map.json -> notes.md / GUI / coverage +Ingest: Deck / 截图 + -> Understand: content.json + -> Write: notes.md（含来源注释） + -> Guard: coverage.json + 最终 element_ir.json + source_map.json ``` -它们让正文可以读起来干净，同时保留页面、元素、图片和覆盖率信息。 +`source_map.json` 需要读取已生成的 `notes.md`，因此在 Guard 阶段建立，而不是笔记的输入。结构化来源有助于定位和复核，不证明解释在语义上完全准确。 ## content.json -`content.json` 是解析后的基础内容清单，记录每页的标题、文本块、表格、图片、截图路径、OCR 文本和视觉摘要。 +`content.json` 是 Understand 阶段写出的页面内容清单，记录每页的标题、文本块、表格、图片、截图路径，以及实际启用的 OCR 和视觉理解结果。 -它偏向“原始解析结果”，适合调试输入材料是否被正确读取。 +它保留解析结果及后续理解步骤写回的字段，适合检查输入材料是否被正确读取和补充。 常见字段包括： @@ -24,7 +27,7 @@ content.json -> element_ir.json -> source_map.json -> notes.md / GUI / coverage ## element_ir.json -`element_ir.json` 是统一 Element IR，面向 prompt、coverage、source map、GUI 和后续 Agent 工作流。 +`element_ir.json` 是 Guard 阶段写出的最终 Element IR，供元素检查和后续工具使用。构建中的 prompt 与 coverage 也会从当前 `Deck` 构造需要的元素视图。 每个元素尽量包含： @@ -46,12 +49,7 @@ content.json -> element_ir.json -> source_map.json -> notes.md / GUI / coverage ## 构建时机 -IR 会在两个阶段写入： - -1. `export_content` 阶段写基础 IR，供后续 prompt、source map 和 coverage 使用。 -2. coverage 阶段结束后刷新最终 IR，合入 `covered`、`missing`、`marker-only` 等实际状态。 - -这样最终 `element_ir.json` 不是只停留在前置状态，而是反映生成后的覆盖结果。 +构建过程中，prompt、coverage 和 source map 会直接从当前 `Deck` 构造需要的元素视图。coverage 阶段结束后写入一次最终 `element_ir.json`，合入 `covered`、`missing`、`marker-only` 等实际状态，避免重复生成中间文件。 ## source_map.json @@ -67,7 +65,7 @@ note block -> PPT/PDF page -> text/table/image element id ``` -阅读正文时不会被元素 ID 打断，但 GUI、coverage、导出和局部 revise 仍能找到来源。 +阅读正文时不会被元素 ID 打断；构建中的覆盖检查会使用这些标记，`source_map.json` 可供后续来源高亮和局部 revise 使用。目前 GUI 尚未直接读取该文件。 ## 图片资产 diff --git a/docs/pipeline.zh-CN.md b/docs/pipeline.zh-CN.md index 9385e17..89c2d08 100644 --- a/docs/pipeline.zh-CN.md +++ b/docs/pipeline.zh-CN.md @@ -8,15 +8,19 @@ Ingest -> Understand -> Write -> Guard -> Export 底层模块可以保持细粒度，方便缓存、调试和局部刷新；用户侧和 LLM 工作流应该看到清楚的阶段边界。 +实现中由 `BUILD_PHASES` 按这五个阶段组织步骤。构建开始时会根据 preset 和选项排除未启用的 OCR、Vision、图裁剪等步骤，并把实际计划写入 `progress.json` 的 `planned_stages`；`current_phase` 表示当前产品阶段。逐步耗时仍保留在 `run_summary.json`，方便定位慢点。 + ## 阶段总览 | 阶段 | 目标 | 典型产物 | | --- | --- | --- | -| Ingest | 把 PPT/PDF 变成稳定、可追溯、可复现的结构化材料。 | `content.json`、`element_ir.json`、`source_map.json`、截图、图片资产、parser adapter | -| Understand | 理解课件主题、章节结构、页面角色、图表含义和关键元素。 | `deck_understanding.json`、`page_understanding.json`、`sections.json`、`deck_brief.json`、`semantic_layout.json`、`table_understanding.json`、`figure_grounding.json` | +| Ingest | 解析 PPT/PDF，提取页面元素和资源。 | 内存中的 `Deck`、截图、图片资产 | +| Understand | 理解课件主题、章节结构、页面角色、图表含义和关键元素。 | `content.json`、`page_modalities.json`、`deck_understanding.json`、`page_understanding.json`、`sections.json`、`deck_brief.json`、`semantic_layout.json`、`table_understanding.json`、`figure_grounding.json`、`content_guard.json` | | Write | 生成可读学习笔记，而不是机械逐页搬运。 | `notes.md`、`page_notes.json`、`weave_report.json`、`teaching_enrichment.json` | -| Guard | 检查是否漏掉关键内容、是否有来源、是否像讲义。 | `coverage.json`、`coverage.md`、`content_guard.json`、`quality_report.json` | -| Export | 输出阅读和复习材料。 | `notes.toc.md`、`notes.docx`、`notes.pdf`、`notes.tex`、`review.md`、`exam.html` | +| Guard | 检查是否漏掉关键内容、是否有来源、是否像讲义。 | `coverage.json`、`coverage.md`、`element_ir.json`、`source_map.json`、`quality_report.json` | +| Export | 输出阅读材料和构建摘要。 | `notes.toc.md`、`notes.docx`、`notes.pdf`、`notes.tex`、`run_summary.json` | + +`lecture` 的教学补充采用按章节判断：整合后的章节稿已有足够正文，并包含例子、易错点和自测线索时，跳过额外模型调用；`force` 仍会执行补充。 ## 什么不交给 LLM @@ -38,7 +42,7 @@ Ingest -> Understand -> Write -> Guard -> Export - 图表、流程图、公式截图的含义解释。 - 讲义式正文生成。 - 背景直觉、例子、易错点、自测题。 -- 语义层面的遗漏修复和质量审阅。 +- 针对必讲内容生成补漏候选稿；候选稿仍需本地覆盖检查和人工复核。 SlideNote 的原则是：本地规则管边界和证据，模型管理解和表达。 @@ -68,15 +72,15 @@ figures/ screenshots/ ``` -不是每次运行都会生成所有文件。具体取决于 `--preset`、`--vision`、`--export`，以及是否单独运行 `slidenote study-pack`。 +不是每次运行都会生成所有文件，具体取决于 `--preset`、`--vision`、`--export` 等选项。`slidenote study-pack` 是构建后的独立命令，会另外生成复习和考试材料。 ## 稳定认知包 `deck_understanding.json` 聚合 Deck Brief、章节计划、页面角色、核心问题、关键术语、跨页关联、重要表格和高价值图示。它是全局导航入口，不替代底层 `sections.json` / `deck_brief.json` 调试产物。 -`page_understanding.json` 聚合每页的 section、role、modality、key points、文本摘要、表格结论、图示解释、semantic groups 和 content guard required items。它适合作为 GUI、Agent backend 和局部 revise 的逐页稳定入口。 +`page_understanding.json` 聚合每页的 section、role、modality、key points、文本摘要、表格结论、图示解释、semantic groups 和 content guard required items。当前构建会产出该文件；GUI 和局部 revise 尚未把它作为统一入口。 -`quality_report.json` 继续负责质量审阅：覆盖率、机械逐页复述风险、解释深度、图表整合和题目质量。 +`quality_report.json` 用本地启发式指标提示讲义结构、解释深度、图表整合和机械逐页复述等风险，结合 `coverage.json` 供人工复核。独立运行的 `study-pack` 不会回写构建报告的题目质量字段。 ## Parser Adapter diff --git a/docs/presets.zh-CN.md b/docs/presets.zh-CN.md deleted file mode 100644 index b67fb6d..0000000 --- a/docs/presets.zh-CN.md +++ /dev/null @@ -1,51 +0,0 @@ -# 用户侧 Preset - -`--preset` 是普通用户选择工作流的入口。现在只保留两个值：`lecture` 和 `local`。 - -## Preset 总览 - -| Preset | 适合场景 | 推荐心智模型 | -| --- | --- | --- | -| `lecture` | 正式学习、长期保存、需要图文理解。 | 像老师重新讲一遍。 | -| `local` | 没有 API key、离线预览、检查解析是否正常。 | 先确认课件能被读出来。 | - -## 建议命令 - -教师讲义： - -```powershell -$env:DEEPSEEK_API_KEY="..." -$env:DASHSCOPE_API_KEY="..." -python -m slidenote build lecture.pdf --out outputs\lecture --provider deepseek --export markdown-zip -``` - -本地预览： - -```powershell -python -m slidenote build lecture.pdf --out outputs\local --preset local --export markdown-zip -``` - -已有输出目录生成复习包： - -```powershell -python -m slidenote study-pack outputs\lecture --question-count 12 -``` - -## 背后行为 - -| Preset | 行为 | -| --- | --- | -| `lecture` | 默认启用 LLM、OCR auto、Vision auto、图文锚定、Deck Brief、Content Guard、Lecture-Weave、teaching enrichment 和本地缓存。 | -| `local` | 强制关闭外部 API 调用，只保留本地解析、本地语义布局、本地图片排序、coverage、source map 和基础 Markdown。 | - -`lecture` 是质量优先默认值，不再要求用户手动理解 `note-profile`、`note-depth`、`speed-mode`、缓存、并发、OCR/Vision targets 等底层参数。 - -## 选择建议 - -- 第一次跑新文件且没有 API key，用 `--preset local`。 -- 正式学习或要分享笔记，用默认 `lecture`，并加 `--export markdown-zip`。 -- 图很多、流程图多、截图多的课件，保持 `--vision auto`。 -- 只想用文本模型、不想调用视觉模型，用 `--vision off`。 -- 复习题和自测题不要放进 build 命令里，使用 `slidenote study-pack <输出目录>`。 - -旧的 `fast` / `faithful` preset 已经移除。对应迁移方式见 [CONFIG.zh-CN.md](../CONFIG.zh-CN.md)。 diff --git a/docs/providers-and-cost.zh-CN.md b/docs/providers-and-cost.zh-CN.md index e419d14..4599864 100644 --- a/docs/providers-and-cost.zh-CN.md +++ b/docs/providers-and-cost.zh-CN.md @@ -6,8 +6,8 @@ - Python `3.10+`。 - 简单入口：在仓库根目录运行 `.\install.ps1`，然后运行 `.\run_gui.ps1`。 -- 本地解析：`python -m pip install -e ".[dev]"`。 -- LLM / GUI：`python -m pip install -e ".[dev,llm,gui]"`。 +- 本地解析：`python -m pip install -e .`。 +- LLM / GUI：`python -m pip install -e ".[llm,gui]"`；开发测试另装 `.[dev]`。 可选外部软件： @@ -79,16 +79,17 @@ Vision 负责解释图、流程、趋势、布局和视觉关系。公开参数 $env:DASHSCOPE_API_KEY="..." ``` -相关输出只在实际运行 Vision 时出现： +下列输出与视觉调用或图裁剪有关，按实际运行的步骤生成： ```text visuals.json vision_usage.json figures.json figure_usage.json -figure_grounding.json ``` +`figure_grounding.json` 也可能由本地规则生成；例如 `local` preset 关闭视觉 API 后仍会执行本地图示定位。它不能单独作为 Vision API 是否调用的依据。 + ## 缓存、并发和成本 这些现在是内部强默认，不再作为普通 CLI 参数暴露： @@ -97,7 +98,7 @@ figure_grounding.json - API 并发使用保守内部默认值。 - 视觉/OCR/figure target 上限使用质量优先默认值。 -构建完成后仍会保留统计文件，供 GUI 和后续诊断使用： +`build` 会写出 `run_summary.json`；各类 API 使用量文件只在对应步骤实际运行时出现： ```text llm_usage.json @@ -105,9 +106,22 @@ vision_usage.json ocr_usage.json figure_usage.json run_summary.json +``` + +GUI 会根据这些文件生成成本报告，也可以在已有构建目录中手动运行： + +```powershell +python scripts/generate_cost_report.py outputs\lecture +``` + +该操作生成： + +```text cost_report.json cost_report.md cost_dashboard.html ``` +CLI `slidenote build` 本身不会自动生成成本报告。成本取决于所用模型与价格配置，报告中的估算应按实际账单复核。 + 如果你需要重新开放成本或并发调参，请先确认默认 `lecture` 路线的真实瓶颈，再把它作为开发者配置处理，而不是直接恢复到普通用户界面。 diff --git a/docs/quality-and-guard.zh-CN.md b/docs/quality-and-guard.zh-CN.md index 08ac7c8..60b0627 100644 --- a/docs/quality-and-guard.zh-CN.md +++ b/docs/quality-and-guard.zh-CN.md @@ -1,11 +1,11 @@ # Coverage、Content Guard 与学习质量 -SlideNote 的质量控制分两层： +SlideNote 的质量诊断分两层： -1. 硬检查：关键元素有没有进入笔记，来源是否可追溯。 -2. 软检查：笔记是否像讲义，是否有解释深度、图表整合、例子、自测和易错点。 +1. 结构检查：关键元素是否在可见正文中出现，来源标记能否关联到原始元素。 +2. 启发式检查：笔记的章节、解释、图表引用、例子、自测和易错点是否具备讲义特征。 -Coverage 不应该决定正文结构。它更适合作为最后的质检器，发现遗漏后只做局部修补，避免把正文改回逐页清单。 +Coverage 在 Write 之后运行，适合作为漏项提示和人工复核入口。覆盖率高只能说明结构上的来源关联较完整，不能证明讲解正确或没有编造事实。针对缺失项进行局部修补是后续目标；当前自动补漏仍需校验候选稿，不能把它视为独立的逐项语义验证。 ## Coverage @@ -26,7 +26,7 @@ coverage.md ## Content Guard -Content Guard 负责先找出“必须解释”的学习内容，再把它们交给写作和修复阶段。 +Content Guard 在 Understand 阶段先找出“必须解释”的学习内容，再把它们交给写作和修复阶段；Guard 阶段会把最终覆盖状态写回 `content_guard.json`。未启用时不会生成该文件。 它会优先关注： @@ -50,20 +50,21 @@ content_guard.json ## Quality Report -`quality_report.json` 是学习质量报告。第一版主要使用本地启发式指标，避免额外增加 LLM 成本。 +`quality_report.json` 是笔记质量诊断报告，当前主要使用本地启发式指标，避免额外增加 LLM 成本。分数来自段落长度、标题、关键词、图片引用和来源标记等信号，应结合课件与笔记人工判断。 重点指标包括： | 字段 | 含义 | | --- | --- | -| `coherence_score` | 章节是否连贯。 | -| `explanation_depth_score` | 是否解释“是什么、为什么、怎么运作”。 | -| `example_score` | 是否包含例子、类比或直观说明。 | -| `figure_integration_score` | 图表是否融入正文，而不是只附在页尾。 | -| `mechanical_page_listing_score` | 是否像“第 1 页讲 A，第 2 页讲 B”的机械复述。 | -| `self_check_coverage_score` | 是否包含自测题。 | -| `misconception_coverage_score` | 是否覆盖易错点或常见误解。 | -| `question_quality_score` | 复习/考试题是否有明确答案、来源和解析。 | +| `coherence_score` | 段落和标题结构的启发式分数，不验证逻辑连贯性。 | +| `explanation_depth_score` | 段落长度及“为什么”“如何”等词的启发式分数。 | +| `example_score` | 例子、类比等词的出现情况。 | +| `figure_integration_score` | 图片引用与原图数量等结构信号。 | +| `mechanical_page_listing_score` | 机械逐页复述的文本模式信号。 | +| `self_test_score` | 自测相关词的出现情况。 | +| `pitfall_score` | 易错点、误解等词的出现情况。 | +| `hallucination_risk` | 由来源标记密度和覆盖缺失推断的复核优先级；不做事实核查。 | +| `question_quality_score` | 当前 `build` 没有接入独立学习包，值为 `null`；`study-pack` 的题目质量另行计算。 | 未来可以增加轻量 LLM 审阅 pass，但不应该让同一个写作模型无约束地自己审自己。 @@ -87,7 +88,7 @@ content_guard.json ## Review / Exam 学习包 -`slidenote study-pack` 把最终 `notes.md` 延伸成复习材料： +`slidenote study-pack` 是构建后的独立命令，读取已有的 `notes.md` 和 `content.json`，生成复习材料： ```text review.md @@ -103,7 +104,7 @@ wrong_answer_review_prompt.md 设计目标： -- 让复习从“看一份笔记”变成“做题、批改、复盘、定位知识漏洞”的闭环。 +- 逐步让复习从“看一份笔记”延伸到做题、批改和复盘；目前错题复盘以提示词文件为主，尚无持久化答题历史。 - 题目要有来源页和解析，不只是随机问答。 - 涉及图表的题目应尽量把相关图文就地放在题目附近。 - 错题复盘 prompt 应帮助学生追问：到底漏掉了哪个知识点。 @@ -114,13 +115,13 @@ wrong_answer_review_prompt.md ```text parse content - -> deck/page understanding + -> deck/page understanding 与必讲项识别（Understand） -> section lecture writing -> teaching enrichment - -> content guard repair - -> coverage check - -> quality report - -> review/exam pack + -> 候选补漏与校验（Write，启用时） + -> coverage check、最终 element IR 与 source map（Guard） + -> quality report（Guard） + -> review/exam pack（单独运行 study-pack） ``` -核心思想是：Write 负责把内容讲清楚，Guard 负责不漏和不乱编。 +核心思想是：Write 负责生成可读正文，Guard 负责提示漏项和结构风险；准确性仍要对照课件复核。 diff --git a/docs/roadmap/extension-notes.zh-CN.md b/docs/roadmap/extension-notes.zh-CN.md index 82321f6..570a522 100644 --- a/docs/roadmap/extension-notes.zh-CN.md +++ b/docs/roadmap/extension-notes.zh-CN.md @@ -1,6 +1,6 @@ -# 路线图设计笔记 +# 路线图设计笔记（想法档案） -这份文档承接旧版 ROADMAP 中较长的模块设计说明。它不是优先级列表；真正的优先级以 [ROADMAP.zh-CN.md](../../ROADMAP.zh-CN.md) 为准。 +最后核对：2026-09-23。本文件保存设计想法和可选方向，不代表已排期或承诺交付；现行优先级以 [ROADMAP.zh-CN.md](../../ROADMAP.zh-CN.md) 为准。下面的“已实现”只表示当前代码中存在相应产物或入口，不代表已完成真实课件质量验收。 ## 上下文、覆盖率与章节切分 @@ -9,7 +9,7 @@ SlideNote 需要在 page / section / document 三种上下文之间取舍： - page：来源最清晰，适合调试和局部刷新，但容易有逐页拼接感。 -- section：默认更适合正式笔记，逻辑更连贯，请求数也比逐页更少。 +- section：适合组织正式笔记，逻辑更连贯。当前 Lecture-Weave 仍会先逐页生成内容，再做 section weave；不能把它理解成总请求数一定少于逐页模式。 - document：适合短文件和全局摘要，长文件容易超上下文。 推荐路线是：逐页理解负责证据和局部讲解，section weave 负责最终正文。 @@ -21,7 +21,7 @@ Coverage 的价值是“不漏”，不是决定正文形状。高质量笔记 后续增强方向： - coverage 报告按章节聚合。 -- missing item 只触发局部修复。 +- missing item 尽量只触发受影响笔记块的修复。当前 Content Guard 修补会重写传入的整段页或章节上下文，并检查 coverage 回退、图片丢失和正文截断；尚未实现笔记块级修改。 - source marker 不污染正文阅读。 - 图表 coverage 单独显示插入位置、解释状态和复查建议。 diff --git a/gui/README_GUI.md b/gui/README_GUI.md index 4caf012..cb207c2 100644 --- a/gui/README_GUI.md +++ b/gui/README_GUI.md @@ -13,6 +13,7 @@ SlideNote Studio is a Streamlit interface for `python -m slidenote build` and `p - Enter Text / Vision / OCR API keys on the page. Keys are passed only through the child-process environment, not command-line flags. - Select extra exports: Markdown ZIP, TOC Markdown, Word, PDF, or LaTeX. - Keep progress, ETA, Doctor readiness, usage, and cost details in compact diagnostics panels. +- Saved page modality corrections apply to the next build of the same source file; a different file does not inherit them. - Generate a study pack from the Notes workspace: `review.md`, `exam.md`, `exam.json`, `exam.html`, and related files. - Download `notes.zip`, `notes.md`, `coverage.md`, export files, or the complete output ZIP. - Switch to **Textbook library**, upload a PDF textbook, and build a RAG-ready corpus. The corpus is not connected to note generation yet. diff --git a/gui/README_GUI.zh-CN.md b/gui/README_GUI.zh-CN.md index f8d0987..5a2d10a 100644 --- a/gui/README_GUI.zh-CN.md +++ b/gui/README_GUI.zh-CN.md @@ -13,11 +13,12 @@ SlideNote Studio 是一个基于 Streamlit 的图形界面。它包装 `python - - 在页面里临时填写 Text / Vision / OCR API key；key 只通过本次子进程环境变量传入，不写进命令行。 - 选择是否导出 `notes.zip`、目录 Markdown、Word、PDF 或 LaTeX。 - 进度、ETA、Doctor、用量和成本信息收在紧凑的诊断区里。 +- 在页面里保存的模态修正会用于同一源文件的下一次构建；更换文件后不会沿用旧修正。 - 在 Notes workspace 基于已有输出目录生成复习包：`review.md`、`exam.md`、`exam.json`、`exam.html` 等。 - 下载 `notes.zip`、`notes.md`、`coverage.md`、导出文件或完整结果 ZIP。 - 切换到 **Textbook library**，上传 PDF 教材，构建 RAG-ready 教材库；该库当前不会自动参与笔记生成。 -分享 Markdown 时优先下载 `notes.zip`。压缩包里包含 `notes.md` 和 `notes.assets/`，别人解压后图片才能正常显示。 +分享 Markdown 时优先下载 `notes.zip`。压缩包包含 `notes.md`；笔记引用图片时还包含 `notes.assets/`，别人解压后即可查看图片。 ## 安装 @@ -75,7 +76,7 @@ python -m slidenote study-pack <输出目录> --question-count 12 左侧 **Run** 面板里的 **Exports** 可以勾选： -- `notes.zip`：Markdown 笔记包，包含图片资源，不需要 Pandoc。 +- `notes.zip`：Markdown 笔记包；有图片时包含相关资源，不需要 Pandoc。 - `notes.toc.md`：带目录 Markdown，不需要 Pandoc。 - `notes.docx`：Word 文档，需要 Pandoc。 - `notes.pdf`：PDF 讲义，需要 Pandoc + LibreOffice。 diff --git a/gui/app.py b/gui/app.py index 8cf6e04..a14c53e 100644 --- a/gui/app.py +++ b/gui/app.py @@ -1,16 +1,18 @@ from __future__ import annotations -import io +import hashlib import html import json import os import re import shutil import subprocess +import threading import time -import zipfile -from datetime import datetime +from collections import deque +from datetime import datetime, timezone from pathlib import Path +from queue import Empty, Queue from typing import Any import streamlit as st @@ -25,8 +27,13 @@ except Exception: # pragma: no cover - GUI fallback only run_doctor = None +from slidenote.costing import read_json +from slidenote.exporting import find_libreoffice + from gui.studio_core import ( + DEFAULT_TEXT_PROVIDER, PROVIDER_ENV_KEYS, + TEXT_PROVIDERS, StudioConfig, TextbookConfig, build_env, @@ -34,14 +41,20 @@ build_study_pack_command, build_textbook_command, command_for_display, + coverage_missing_items, + coverage_summary, discover_outputs, discover_textbook_outputs, + format_cost, + format_count, needs_text_api, needs_vision_api, + output_signature, performance_tips, progress_percent, provider_env_key, safe_run_name, + zip_output_dir, ) ROOT = Path(__file__).resolve().parents[1] @@ -58,6 +71,8 @@ MODALITY_OPTIONS = ["native_text", "mixed", "image_only", "shape_diagram", "decorative", "unknown"] +OCR_MODES = {"Auto scanned pages": "auto", "Off": "off", "All pages": "all"} + def _run_simplified_app() -> None: st.set_page_config(page_title="SlideNote Studio", page_icon="SN", layout="wide", initial_sidebar_state="collapsed") @@ -100,11 +115,19 @@ def _run_simplified_app() -> None: preset_name = st.selectbox("Workflow preset", list(PRESETS.keys()), index=0) preset = PRESETS[preset_name] preset_value = str(preset["preset"]) - provider = st.selectbox("Text provider", ["deepseek", "openai", "qwen", "doubao", "glm", "gemini", "claude"], index=0) + provider = st.selectbox("Text provider", TEXT_PROVIDERS, index=TEXT_PROVIDERS.index(DEFAULT_TEXT_PROVIDER)) vision = st.selectbox("Vision", ["auto", "off"], index=["auto", "off"].index(str(preset["vision"])), disabled=preset_value == "local") if preset_value == "local": vision = "off" vision_provider = "qwen" + ocr_label = st.selectbox( + "OCR", + ["Auto scanned pages", "Off", "All pages"], + index=0, + disabled=preset_value == "local", + help="auto only OCRs low-text/scanned pages; the Local preview preset always skips OCR.", + ) + ocr_mode = "off" if preset_value == "local" else OCR_MODES[ocr_label] with st.expander("API keys", expanded=preset_value == "lecture"): api_key = st.text_input("Text API key", type="password", placeholder="Used for lecture builds and study packs") @@ -150,7 +173,7 @@ def _run_simplified_app() -> None: vision=vision, vision_provider=vision_provider, vision_api_key=vision_api_key or (api_key if provider == vision_provider else None) or None, - ocr="auto" if preset_value == "lecture" else "off", + ocr=ocr_mode, ocr_api_key=ocr_api_key or None, ocr_secret_key=ocr_secret_key or None, export=",".join(export_options) if export_options else None, @@ -180,6 +203,12 @@ def _run_simplified_app() -> None: st.error(f"Could not prepare output folder: {exc}") return config = _clone_config_for_run(preview_config, input_path=input_path, output_dir=output_dir, progress_json=progress_json) + if _carry_modality_overrides( + Path(st.session_state["last_output_dir"]) if st.session_state.get("last_output_dir") else None, + input_path, + output_dir, + ): + st.caption("Saved page modality corrections will be used for this build.") _run_build(config) st.session_state["last_output_dir"] = str(output_dir) @@ -285,8 +314,8 @@ def _render_textbook_library() -> None: _render_source_file(uploaded) st.markdown("### Build library") - ocr_label = st.selectbox("OCR", ["Auto scanned pages", "Off", "All pages"], index=0) - ocr_mode = {"Auto scanned pages": "auto", "Off": "off", "All pages": "all"}[ocr_label] + ocr_label = st.selectbox("OCR", list(OCR_MODES), index=0) + ocr_mode = OCR_MODES[ocr_label] with st.expander("OCR API key", expanded=ocr_mode != "off"): ocr_api_key = st.text_input("OCR API key / app id", type="password", key="textbook_ocr_api_key") ocr_secret_key = st.text_input("OCR secret / app key", type="password", key="textbook_ocr_secret_key") @@ -464,7 +493,7 @@ def _render_workspace_downloads(output_dir: Path, outputs: dict[str, Path]) -> N c2.download_button("notes.md", data=notes.read_bytes(), file_name="notes.md", mime="text/markdown", use_container_width=True) else: c2.button("notes.md", disabled=True, use_container_width=True) - c3.download_button("all results", data=_zip_output_dir(output_dir), file_name=f"{output_dir.name}.zip", mime="application/zip", use_container_width=True) + c3.download_button("all results", data=_cached_output_zip(str(output_dir), output_signature(output_dir)), file_name=f"{output_dir.name}.zip", mime="application/zip", use_container_width=True) if notes_zip: st.caption("Share Markdown notes with notes.zip; it includes notes.md and notes.assets.") @@ -495,8 +524,8 @@ def _render_study_pack_compact(output_dir: Path, config: StudioConfig | None = N def _render_usage_snapshot(output_dir: Path) -> None: - run_summary = _read_json(output_dir / "run_summary.json") or {} - cost_report = _read_json(output_dir / "cost_report.json") or {} + run_summary = read_json(output_dir / "run_summary.json") or {} + cost_report = read_json(output_dir / "cost_report.json") or {} counts = run_summary.get("counts") if isinstance(run_summary.get("counts"), dict) else {} cost_summary = cost_report.get("summary") if isinstance(cost_report.get("summary"), dict) else {} rows = [ @@ -531,7 +560,7 @@ def _render_export_readiness(export_options: list[str]) -> None: needs_pandoc = any(fmt in export_options for fmt in ("docx", "pdf", "latex")) needs_libreoffice = "pdf" in export_options pandoc_path = shutil.which("pandoc") - libreoffice_path = _find_libreoffice() + libreoffice_path = find_libreoffice() if needs_pandoc and not pandoc_path: st.warning("Word/LaTeX/PDF exports need Pandoc. Install Pandoc, then rerun. Markdown TOC still works.") st.code("winget install JohnMacFarlane.Pandoc", language="powershell") @@ -546,14 +575,6 @@ def _render_export_readiness(export_options: list[str]) -> None: st.success(f"LibreOffice ready for PDF: {libreoffice_path}") -def _find_libreoffice() -> str | None: - for executable in ("soffice", "soffice.com", "libreoffice"): - found = shutil.which(executable) - if found: - return found - return None - - def _api_status(enabled: bool, typed_key: str | None, provider: str) -> tuple[str, str, str]: if not enabled: return "Off", "Not used", "muted" @@ -582,6 +603,8 @@ def _ocr_status(enabled: bool, api_key: str | None, secret_key: str | None, prov def _status_card(label: str, status: str, detail: str, tone: str, icon: str = "•") -> None: + # Doctor output includes paths and tool messages; escape everything rendered as HTML. + label, status, detail, tone, icon = (html.escape(str(value)) for value in (label, status, detail, tone, icon)) st.markdown( f"""
@@ -662,7 +685,8 @@ def _clone_config_for_run(config: StudioConfig, input_path: Path, output_dir: Pa def _prepare_run_paths(uploaded, output_base: Path, timestamped_subfolder: bool) -> tuple[Path, Path, Path]: - run_name = f"{safe_run_name(uploaded.name)}_{int(time.time())}" + # Keep each uploaded source immutable so corrections can be checked against its bytes. + run_name = f"{safe_run_name(uploaded.name)}_{time.time_ns()}" input_path = UPLOADS_DIR / f"{run_name}{Path(uploaded.name).suffix.lower()}" input_path.write_bytes(uploaded.getbuffer()) output_base.mkdir(parents=True, exist_ok=True) @@ -672,6 +696,65 @@ def _prepare_run_paths(uploaded, output_base: Path, timestamped_subfolder: bool) return input_path, output_dir, progress_json +def _output_source_path(output_dir: Path) -> Path | None: + content = read_json(output_dir / "content.json") or {} + if not isinstance(content, dict): + return None + source_name = content.get("source_path") + if not isinstance(source_name, str) or not source_name: + return None + source_path = Path(source_name) + if not source_path.is_absolute(): + source_path = ROOT / source_path + return source_path + + +def _output_source_matches(output_dir: Path, input_path: Path) -> bool: + manifest = read_json(output_dir / "page_modalities.overrides.json") or {} + source_hash = manifest.get("source_sha256") if isinstance(manifest, dict) else None + if source_hash is not None: + if not isinstance(source_hash, str) or len(source_hash) != 64 or any(char not in "0123456789abcdefABCDEF" for char in source_hash): + return False + try: + digest = hashlib.sha256() + with input_path.open("rb") as current_file: + while chunk := current_file.read(1024 * 1024): + digest.update(chunk) + return digest.hexdigest() == source_hash.lower() + except OSError: + return False + source_path = _output_source_path(output_dir) + if source_path is None: + return False + try: + if source_path.stat().st_size != input_path.stat().st_size: + return False + with source_path.open("rb") as old_file, input_path.open("rb") as new_file: + while old_chunk := old_file.read(1024 * 1024): + if old_chunk != new_file.read(len(old_chunk)): + return False + return not new_file.read(1) + except OSError: + return False + + +def _carry_modality_overrides(previous_output_dir: Path | None, input_path: Path, output_dir: Path) -> bool: + manifest_name = "page_modalities.overrides.json" + target = output_dir / manifest_name + if target.is_file(): + if _output_source_matches(output_dir, input_path): + return True + # Preserve corrections for the old source without applying them to a different upload. + target.replace(output_dir / f"page_modalities.overrides.stale-{time.time_ns()}.json") + if previous_output_dir is None or previous_output_dir == output_dir: + return False + source = previous_output_dir / manifest_name + if not source.is_file() or not _output_source_matches(previous_output_dir, input_path): + return False + shutil.copy2(source, target) + return True + + def _prepare_textbook_paths(uploaded) -> tuple[Path, Path]: if Path(uploaded.name).suffix.lower() != ".pdf": raise ValueError("Textbook library v1 only accepts PDF files.") @@ -691,7 +774,7 @@ def _run_build(config: StudioConfig) -> None: status_box = st.empty() stage_box = st.empty() log_box = st.empty() - logs: list[str] = [] + logs: deque[str] = deque(maxlen=120) process = subprocess.Popen( cmd, @@ -704,21 +787,38 @@ def _run_build(config: StudioConfig) -> None: errors="replace", bufsize=1, ) - while process.poll() is None: + output_queue: Queue[str] = Queue() + + def read_output() -> None: if process.stdout is not None: - line = process.stdout.readline() - if line: - logs.append(line.rstrip()) + with process.stdout: + for line in process.stdout: + output_queue.put(line.rstrip("\r\n")) + + reader = threading.Thread(target=read_output, daemon=True) + reader.start() + + while True: + for _ in range(200): + try: + logs.append(output_queue.get_nowait()) + except Empty: + break _update_progress_ui(config.progress_json, progress_bar, status_box, stage_box) - log_box.code("\n".join(logs[-80:]) or "Running...", language="text") + log_box.code("\n".join(list(logs)[-80:]) or "Running...", language="text") + if process.poll() is not None: + break time.sleep(0.25) - if process.stdout is not None: - rest = process.stdout.read() - if rest: - logs.extend(rest.splitlines()) + reader.join(timeout=2.0) + while True: + try: + logs.append(output_queue.get_nowait()) + except Empty: + break + process.wait() _update_progress_ui(config.progress_json, progress_bar, status_box, stage_box) - log_box.code("\n".join(logs[-120:]) or "No console output.", language="text") + log_box.code("\n".join(logs) or "No console output.", language="text") if process.returncode == 0: _generate_cost_report(config.output_dir) @@ -770,7 +870,7 @@ def _run_study_pack(config: StudioConfig, question_count: int) -> None: def _update_progress_ui(progress_path: Path, progress_bar, status_box, stage_box) -> None: - progress = _read_json(progress_path) + progress = read_json(progress_path) if not progress: status_box.info("Waiting for progress.json...") return @@ -850,7 +950,7 @@ def _render_textbook_workspace(output_dir: Path | None) -> None: return outputs = discover_textbook_outputs(output_dir) st.markdown(f"
Output saved to
{html.escape(str(output_dir))}
", unsafe_allow_html=True) - manifest = _read_json(outputs.get("manifest") or output_dir / "textbook_manifest.json") or {} + manifest = read_json(outputs.get("manifest") or output_dir / "textbook_manifest.json") or {} counts = manifest.get("counts") if isinstance(manifest.get("counts"), dict) else {} c1, c2, c3, c4 = st.columns(4) c1.metric("Pages", counts.get("pages", "—")) @@ -893,12 +993,12 @@ def _render_textbook_downloads(output_dir: Path, outputs: dict[str, Path]) -> No col.download_button(path.name, data=path.read_bytes(), file_name=path.name, mime=_mime_for_path(path), use_container_width=True) else: col.button(label, disabled=True, use_container_width=True) - st.download_button("all textbook files", data=_zip_output_dir(output_dir), file_name=f"{output_dir.name}.zip", mime="application/zip", use_container_width=True) + st.download_button("all textbook files", data=_cached_output_zip(str(output_dir), output_signature(output_dir)), file_name=f"{output_dir.name}.zip", mime="application/zip", use_container_width=True) def _render_exports_tab(output_dir: Path) -> None: outputs = discover_outputs(output_dir) - export_report = _read_json(output_dir / "export_report.json") + export_report = read_json(output_dir / "export_report.json") export_paths = [ ("Markdown ZIP", outputs.get("notes_zip"), "notes.zip"), ("Markdown TOC", outputs.get("notes_toc"), "notes.toc.md"), @@ -931,71 +1031,42 @@ def _render_exports_tab(output_dir: Path) -> None: st.caption("export_report.json will appear when extra exports are requested.") -def _zip_output_dir(output_dir: Path) -> bytes: - buffer = io.BytesIO() - with zipfile.ZipFile(buffer, "w", compression=zipfile.ZIP_DEFLATED) as archive: - for path in sorted(output_dir.rglob("*")): - if path.is_file(): - archive.write(path, path.relative_to(output_dir).as_posix()) - buffer.seek(0) - return buffer.getvalue() +@st.cache_data(show_spinner=False, max_entries=4) +def _cached_output_zip(output_dir: str, signature: tuple[tuple[str, int, int], ...]) -> bytes: + # `signature` only keys the cache: the zip is rebuilt when files change, not on + # every Streamlit rerun. zip_output_dir leaves out `.cache/`. + del signature + return zip_output_dir(Path(output_dir)) def _render_quality_panel(output_dir: Path) -> None: - coverage = _read_json(output_dir / "coverage.json") or {} - run_summary = _read_json(output_dir / "run_summary.json") or {} - total = int(coverage.get("total") or 0) - covered = int(coverage.get("covered") or 0) - missing = int(coverage.get("missing") or 0) - score = (covered / total * 100) if total else 100.0 + coverage = read_json(output_dir / "coverage.json") or {} + run_summary = read_json(output_dir / "run_summary.json") or {} + summary = coverage_summary(coverage) + score = summary["ratio"] * 100 + label = "Visible coverage" if summary["visible"] else "Coverage" q1, q2, q3, q4 = st.columns(4) - q1.metric("Coverage score", f"{score:.1f}%") - q2.metric("Covered elements", covered) - q3.metric("Missing elements", missing) + q1.metric(label, f"{score:.1f}%") + q2.metric("Explained elements", summary["covered"]) + q3.metric("Not explained", summary["missing"]) q4.metric("Pages", (run_summary.get("counts") or {}).get("pages", "—")) st.progress(min(max(score / 100, 0), 1)) - - if missing: - st.warning("Some elements are not visibly covered. Use the repair queue below to decide which pages need refresh.") - missing_items = _coverage_missing_items(coverage) - if missing_items: - st.dataframe(missing_items, use_container_width=True, hide_index=True) - missing_pages = sorted({str(item.get("slide_id")) for item in missing_items if item.get("slide_id")}) - if missing_pages: - pages_text = ",".join(missing_pages) - st.code(pages_text, language="text") - st.caption("Copy this into 'Refresh only these pages' to rerun only the affected pages.") + if summary["required_total"]: + st.caption(f"Required items not explained: {summary['required_missing']} / {summary['required_total']}") + + missing_items = coverage_missing_items(coverage) + if missing_items: + st.warning("Some slide elements are not explained in the note text. Review these pages against the slides.") + st.dataframe(missing_items, use_container_width=True, hide_index=True) + missing_pages = sorted({int(item["slide_id"]) for item in missing_items if item.get("slide_id")}) + if missing_pages: + st.caption("Pages to review: " + ", ".join(str(page) for page in missing_pages)) else: st.success("No missing coverage items reported.") _render_stage_timings(run_summary) -def _coverage_missing_items(coverage: dict[str, Any]) -> list[dict[str, Any]]: - explicit = coverage.get("missing_items") or coverage.get("marker_only_items") or [] - items = explicit if isinstance(explicit, list) else [] - if not items and isinstance(coverage.get("items"), list): - items = [item for item in coverage["items"] if not item.get("covered")] - normalized = [] - for item in items[:200]: - normalized.append( - { - "slide_id": item.get("slide_id") or _slide_id_from_element(item.get("id") or item.get("element_id")), - "element_id": item.get("id") or item.get("element_id"), - "kind": item.get("kind") or item.get("type") or "", - "reason": item.get("reason") or item.get("status") or "missing", - } - ) - return normalized - - -def _slide_id_from_element(element_id: str | None) -> int | None: - if not element_id: - return None - match = re.match(r"s(\d+)_", str(element_id)) - return int(match.group(1)) if match else None - - def _render_stage_timings(run_summary: dict[str, Any]) -> None: timings = run_summary.get("stage_timings") or {} rows: list[dict[str, Any]] = [] @@ -1020,8 +1091,8 @@ def _render_stage_timings(run_summary: dict[str, Any]) -> None: def _render_page_explorer(output_dir: Path) -> None: - content = _read_json(output_dir / "content.json") or {} - modalities = _read_json(output_dir / "page_modalities.json") or {} + content = read_json(output_dir / "content.json") or {} + modalities = read_json(output_dir / "page_modalities.json") or {} pages = content.get("pages") or [] if not pages: st.info("content.json not found or has no pages.") @@ -1119,9 +1190,23 @@ def _note_excerpt_for_page(notes_path: Path, slide_id: int) -> str: def _save_modality_override(output_dir: Path, slide_id: int, modality: str, note: str) -> None: path = output_dir / "page_modalities.overrides.json" - data = _read_json(path) or {"schema_version": 1, "pages": {}} + data = read_json(path) or {"schema_version": 1, "pages": {}} pages = data.setdefault("pages", {}) - pages[str(slide_id)] = {"modality": modality, "note": note, "updated_at": datetime.utcnow().isoformat(timespec="seconds") + "Z"} + pages[str(slide_id)] = { + "modality": modality, + "note": note, + "updated_at": datetime.now(timezone.utc).isoformat(timespec="seconds").replace("+00:00", "Z"), + } + source_path = _output_source_path(output_dir) + if source_path is not None: + try: + digest = hashlib.sha256() + with source_path.open("rb") as source_file: + while chunk := source_file.read(1024 * 1024): + digest.update(chunk) + data["source_sha256"] = digest.hexdigest() + except OSError: + pass path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8") @@ -1131,14 +1216,14 @@ def _shorten(value: str, limit: int) -> str: def _render_cost_tab(output_dir: Path) -> None: - cost = _read_json(output_dir / "cost_report.json") + cost = read_json(output_dir / "cost_report.json") if cost: summary = cost.get("summary", {}) c1, c2, c3, c4, c5 = st.columns(5) - c1.metric("Estimated cost", f"{summary.get('estimated_cost', 0):.6f} {cost.get('currency', 'USD')}") + c1.metric("Estimated cost", format_cost(summary.get("estimated_cost"), cost.get("currency"))) c2.metric("Calls", summary.get("calls", 0)) - c3.metric("Input tokens", f"{summary.get('input_tokens', 0):,}") - c4.metric("Output tokens", f"{summary.get('output_tokens', 0):,}") + c3.metric("Input tokens", format_count(summary.get("input_tokens", 0))) + c4.metric("Output tokens", format_count(summary.get("output_tokens", 0))) c5.metric("Cache hits", summary.get("local_cache_hits", 0)) stages = cost.get("stages", []) if stages: @@ -1158,7 +1243,7 @@ def _render_run_summary_tab(output_dir: Path) -> None: path = output_dir / filename if path.exists(): with st.expander(filename, expanded=filename == "run_summary.json"): - st.json(_read_json(path) or {}) + st.json(read_json(path) or {}) def _render_markdown_file(path: Path, label: str) -> None: @@ -1193,15 +1278,6 @@ def _mime_for_path(path: Path) -> str: return "application/octet-stream" -def _read_json(path: Path) -> dict[str, Any] | None: - if not path.exists(): - return None - try: - return json.loads(path.read_text(encoding="utf-8")) - except Exception: - return None - - def _read_jsonl(path: Path | None) -> list[dict[str, Any]]: if not path or not path.exists(): return [] @@ -1388,9 +1464,6 @@ def _style() -> None: .device-log code { color:#cbd5e1; background:transparent; } .device-log code.ok { color:#86efac; } - .topbar { - display: none; - } .chip-row { display: flex; align-items: center; flex-wrap: wrap; gap: .5rem; margin: .8rem 0 .6rem; } .chip { display: inline-flex; align-items: center; min-height: 28px; max-width: 100%; diff --git a/gui/studio_core.py b/gui/studio_core.py index fb90bf0..f3a9bae 100644 --- a/gui/studio_core.py +++ b/gui/studio_core.py @@ -1,88 +1,51 @@ from __future__ import annotations +import io import os import re import sys +import zipfile from dataclasses import dataclass from pathlib import Path from typing import Any from slidenote.llm import PROVIDERS as LLM_PROVIDERS +from slidenote.utils import as_float, as_int # Provider metadata derives from slidenote.llm.ProviderSpec (single source of truth). PROVIDER_ENV_KEYS: dict[str, tuple[str, ...]] = {name: spec.api_key_envs for name, spec in LLM_PROVIDERS.items()} -DEFAULT_MODELS: dict[str, str] = {name: (spec.default_model or "") for name, spec in LLM_PROVIDERS.items()} +DEFAULT_TEXT_PROVIDER = "deepseek" +# Selectable text providers, default first, then slidenote.llm registry order. +TEXT_PROVIDERS: list[str] = sorted(LLM_PROVIDERS, key=lambda name: name != DEFAULT_TEXT_PROVIDER) -VISION_DEFAULT_MODELS: dict[str, str] = {name: (spec.default_vision_model or "") for name, spec in LLM_PROVIDERS.items()} +# Build caches are large, machine-local and not useful to share. +ZIP_EXCLUDED_DIRS = frozenset({".cache"}) SAFE_OUTPUT_RE = re.compile(r"[^a-zA-Z0-9_.-]+") +# The build and textbook-index CLIs only use Baidu OCR (no --ocr-provider flag); +# credentials are passed through these environment variables. +OCR_API_KEY_ENV = "BAIDU_OCR_API_KEY" +OCR_SECRET_KEY_ENV = "BAIDU_OCR_SECRET_KEY" + @dataclass(slots=True) class StudioConfig: + """GUI build settings. Only options the `slidenote build` CLI accepts are kept.""" + input_path: Path output_dir: Path progress_json: Path preset: str = "lecture" - speed_mode: str = "quality" - concurrency: int = 1 - llm_concurrency: int | None = None - vision_concurrency: int | None = None - ocr_concurrency: int | None = None - figure_concurrency: int | None = None - global_cache_dir: Path | None = None - refresh_pages: str | None = None - use_llm: bool = True provider: str = "deepseek" - model: str | None = None api_key: str | None = None - base_url: str | None = None - max_output_tokens: int | None = None - temperature: float | None = None - content_guard: str = "auto" - note_context: str = "section" - note_style: str = "article" - note_language: str = "zh" - term_policy: str = "bilingual" - note_strategy: str = "lecture-weave" - note_depth: str = "very-detailed" - weave_dedup: str = "normal" - page_neighborhood: int = 1 - deck_brief: str = "auto" - section_detection: str = "auto" - section_cache: str = "on" - cache: str = "on" ocr: str = "auto" - ocr_provider: str = "baidu" ocr_api_key: str | None = None ocr_secret_key: str | None = None - ocr_language: str = "CHN_ENG" - ocr_cache: str = "on" - ocr_max_targets: int | None = None - ocr_max_edge: int | None = None vision: str = "auto" vision_provider: str = "qwen" - vision_model: str | None = None vision_api_key: str | None = None - vision_base_url: str | None = None - vision_cache: str = "on" - vision_max_targets: int | None = None - vision_max_edge: int | None = None - vision_detail: str | None = "low" - vision_max_output_tokens: int | None = None - figure_crop: str = "auto" - figure_max_targets: int | None = None - figure_grounding: str = "auto" - figure_audit: str = "local" - composite_figures: str = "auto" - image_ranking: str = "local" - screenshot_policy: str = "fallback" - source_display: str = "hidden" - asset_mode: str = "bundle" - review_mode: str = "off" - exam_mode: str = "off" - exam_question_count: int = 12 export: str | None = None quiet: bool = True @@ -92,7 +55,6 @@ class TextbookConfig: input_path: Path output_dir: Path ocr: str = "auto" - ocr_provider: str = "baidu" ocr_api_key: str | None = None ocr_secret_key: str | None = None quiet: bool = True @@ -125,24 +87,22 @@ def needs_text_api(config: StudioConfig) -> bool: def build_env(base_env: dict[str, str] | None, config: StudioConfig | TextbookConfig) -> dict[str, str]: - env = dict(base_env or os.environ) - if isinstance(config, StudioConfig) and needs_text_api(config) and config.api_key: - env[provider_env_key(config.provider)] = config.api_key - if isinstance(config, StudioConfig) and needs_vision_api(config) and config.vision_api_key: - env[provider_env_key(config.vision_provider)] = config.vision_api_key - if config.ocr != "off" and config.ocr_api_key: - env[f"{config.ocr_provider.upper()}_OCR_API_KEY"] = config.ocr_api_key - if config.ocr_provider == "baidu": - env["BAIDU_OCR_API_KEY"] = config.ocr_api_key - if config.ocr_provider == "mathpix": - env["MATHPIX_APP_ID"] = config.ocr_api_key - if config.ocr_provider == "google": - env["GOOGLE_VISION_API_KEY"] = config.ocr_api_key - if config.ocr != "off" and config.ocr_secret_key: - if config.ocr_provider == "baidu": - env["BAIDU_OCR_SECRET_KEY"] = config.ocr_secret_key - if config.ocr_provider == "mathpix": - env["MATHPIX_APP_KEY"] = config.ocr_secret_key + env = dict(os.environ if base_env is None else base_env) + if isinstance(config, StudioConfig): + text_env = None + if needs_text_api(config) and config.api_key: + text_env = provider_env_key(config.provider) + env[text_env] = config.api_key + if needs_vision_api(config) and config.vision_api_key: + vision_env = provider_env_key(config.vision_provider) + # Same provider for text and vision shares one env var: keep the text key. + if vision_env != text_env: + env[vision_env] = config.vision_api_key + if config.ocr != "off": + if config.ocr_api_key: + env[OCR_API_KEY_ENV] = config.ocr_api_key + if config.ocr_secret_key: + env[OCR_SECRET_KEY_ENV] = config.ocr_secret_key return env @@ -163,6 +123,8 @@ def build_slidenote_command(config: StudioConfig) -> list[str]: config.provider, "--vision", config.vision, + "--ocr", + config.ocr, ] if config.quiet: cmd.append("--quiet") @@ -221,19 +183,20 @@ def performance_tips(config: StudioConfig) -> list[str]: def progress_percent(progress: dict[str, Any]) -> float: + if progress.get("status") == "complete": + return 1.0 + planned = progress.get("planned_stages") + if not isinstance(planned, list) or not planned: + # No stage plan yet (build just started): progress is indeterminate. + return 0.02 current = progress.get("current_stage") or {} - stages = progress.get("stages") or [] - completed = len(stages) - total_known_stages = 13 - base = min(completed / total_known_stages, 0.95) + completed = len(progress.get("stages") or []) + total_stages = len(planned) + base = min(completed / total_stages, 0.95) stage_total = current.get("total") or 0 stage_current = current.get("current") or 0 if stage_total: - base = min((completed + min(stage_current / stage_total, 1.0)) / total_known_stages, 0.98) - if progress.get("status") == "complete": - return 1.0 - if progress.get("status") == "failed": - return max(base, 0.02) + base = min((completed + min(stage_current / stage_total, 1.0)) / total_stages, 0.98) return max(base, 0.02) @@ -276,3 +239,101 @@ def discover_textbook_outputs(output_dir: Path) -> dict[str, Path]: "ocr_usage": "ocr_usage.json", } return {key: output_dir / filename for key, filename in names.items() if (output_dir / filename).exists()} + + +def _zip_members(output_dir: Path) -> list[Path]: + return sorted( + path + for path in output_dir.rglob("*") + if path.is_file() and not ZIP_EXCLUDED_DIRS.intersection(path.relative_to(output_dir).parts) + ) + + +def output_signature(output_dir: Path) -> tuple[tuple[str, int, int], ...]: + """Cheap fingerprint of the files `zip_output_dir` would package.""" + signature = [] + for path in _zip_members(output_dir): + stat = path.stat() + signature.append((path.relative_to(output_dir).as_posix(), stat.st_size, stat.st_mtime_ns)) + return tuple(signature) + + +def zip_output_dir(output_dir: Path) -> bytes: + buffer = io.BytesIO() + with zipfile.ZipFile(buffer, "w", compression=zipfile.ZIP_DEFLATED) as archive: + for path in _zip_members(output_dir): + archive.write(path, path.relative_to(output_dir).as_posix()) + return buffer.getvalue() + + +def coverage_summary(coverage: dict[str, Any]) -> dict[str, Any]: + """Visible (prose) coverage metrics from coverage.json, falling back to trace totals for old reports.""" + visible = coverage.get("visible_coverage") + source = visible if isinstance(visible, dict) else coverage + required = coverage.get("required_visible_coverage") + required = required if isinstance(required, dict) else {} + return { + "total": as_int(source.get("total")), + "covered": as_int(source.get("covered")), + "missing": as_int(source.get("missing")), + "ratio": as_float(source.get("coverage_ratio"), 1.0), + "visible": isinstance(visible, dict), + "trace_ratio": as_float(coverage.get("coverage_ratio"), 1.0), + "required_total": as_int(required.get("total")), + "required_missing": as_int(required.get("missing")), + } + + +def slide_id_from_element(element_id: str | None) -> int | None: + if not element_id: + return None + match = re.match(r"s(\d+)_", str(element_id)) + return int(match.group(1)) if match else None + + +def coverage_missing_items(coverage: dict[str, Any], limit: int = 200) -> list[dict[str, Any]]: + """Elements not explained in visible prose (the coverage.json `items` schema), required ones first.""" + items = [item for item in coverage.get("items") or [] if isinstance(item, dict)] + has_visible = any("visible_covered" in item for item in items) + rows: list[dict[str, Any]] = [] + for item in items: + required = bool(item.get("required")) + if has_visible: + # Structural pages are exempt from prose coverage unless an item is required. + if item.get("visible_covered") or (item.get("structural") and not required): + continue + elif item.get("covered"): + continue + if required: + reason = "required, not explained in prose" + elif not item.get("trace_covered", item.get("covered")): + reason = "not referenced in notes" + else: + reason = "source marker only, no prose explanation" + element_id = item.get("id") or item.get("element_id") + rows.append( + { + "slide_id": item.get("slide_id") or slide_id_from_element(element_id), + "element_id": element_id, + "kind": item.get("kind") or item.get("type") or "", + "required": required, + "reason": reason, + } + ) + rows.sort(key=lambda row: (not row["required"], as_int(row["slide_id"]))) + return rows[:limit] + + +def format_cost(value: Any, currency: str | None = "USD") -> str: + try: + amount = float(value) + except (TypeError, ValueError): + return "not recorded" + return f"{amount:.6f} {currency or 'USD'}" + + +def format_count(value: Any) -> str: + try: + return f"{int(value):,}" + except (TypeError, ValueError): + return "—" diff --git a/install.ps1 b/install.ps1 index e7e532d..31c5d1b 100644 --- a/install.ps1 +++ b/install.ps1 @@ -16,7 +16,8 @@ function Find-Python { ) foreach ($candidate in $candidates) { try { - $null = & $candidate["Exe"] @($candidate["Args"]) --version 2>$null + # Accept the interpreter only if it is Python 3.10 or newer. + $null = & $candidate["Exe"] @($candidate["Args"]) -c "import sys; sys.exit(0 if sys.version_info >= (3, 10) else 1)" 2>$null if ($LASTEXITCODE -eq 0) { return $candidate } @@ -28,6 +29,15 @@ function Find-Python { throw "Python 3.10+ was not found. Install Python, then run this script again." } +# $ErrorActionPreference does not stop on failing native commands, so check exit codes. +function Invoke-Checked { + param([string]$Description, [scriptblock]$Command) + & $Command + if ($LASTEXITCODE -ne 0) { + throw "$Description failed with exit code $LASTEXITCODE." + } +} + $python = Find-Python $venvFullPath = Join-Path $Root $VenvPath $venvPython = Join-Path $venvFullPath "Scripts\python.exe" @@ -37,22 +47,25 @@ Write-Host "Project: $Root" if (-not (Test-Path $venvPython)) { Write-Host "Creating virtual environment: $VenvPath" - & $python["Exe"] @($python["Args"]) -m venv $venvFullPath + Invoke-Checked "Creating the virtual environment" { & $python["Exe"] @($python["Args"]) -m venv $venvFullPath } } else { Write-Host "Using existing virtual environment: $VenvPath" } Write-Host "Upgrading pip" -& $venvPython -m pip install --upgrade pip +Invoke-Checked "Upgrading pip" { & $venvPython -m pip install --upgrade pip } $extras = if ($NoGui) { ".[dev,llm]" } else { ".[dev,llm,gui]" } Write-Host "Installing SlideNote: $extras" -& $venvPython -m pip install -e $extras +Invoke-Checked "Installing SlideNote" { & $venvPython -m pip install -e $extras } Write-Host "" Write-Host "Running environment check" & $venvPython -m slidenote doctor +if ($LASTEXITCODE -ne 0) { + Write-Warning "The environment check reported problems; see the output above." +} Write-Host "" Write-Host "Setup complete." diff --git a/run_gui.ps1 b/run_gui.ps1 index bdf2726..b8df677 100644 --- a/run_gui.ps1 +++ b/run_gui.ps1 @@ -13,18 +13,24 @@ $venvPython = Join-Path $Root ".venv\Scripts\python.exe" if (-not (Test-Path $venvPython)) { Write-Host "Virtual environment not found. Running setup first." & (Join-Path $Root "install.ps1") + if (-not $? -or -not (Test-Path $venvPython)) { + throw "Setup failed; fix the errors above and run .\install.ps1 again." + } } & $venvPython -m streamlit --version *> $null if ($LASTEXITCODE -ne 0) { Write-Host "GUI dependency missing. Installing GUI extras." & $venvPython -m pip install -e ".[dev,llm,gui]" + if ($LASTEXITCODE -ne 0) { + throw "Installing the GUI extras failed with exit code $LASTEXITCODE." + } } -$args = @("-m", "streamlit", "run", "gui/app.py", "--server.port", "$Port") +$streamlitArgs = @("-m", "streamlit", "run", "gui/app.py", "--server.port", "$Port") if ($NoBrowser) { - $args += @("--server.headless", "true") + $streamlitArgs += @("--server.headless", "true") } Write-Host "Starting SlideNote Studio..." -& $venvPython @args +& $venvPython @streamlitArgs diff --git a/scripts/_smoke_common.py b/scripts/_smoke_common.py new file mode 100644 index 0000000..0d8761e --- /dev/null +++ b/scripts/_smoke_common.py @@ -0,0 +1,36 @@ +"""Helpers shared by the no-API smoke scripts in this directory.""" + +from __future__ import annotations + +import os +import subprocess + +from slidenote.llm import PROVIDERS + +OCR_ENV_KEYS = frozenset( + { + "BAIDU_OCR_API_KEY", + "BAIDU_OCR_SECRET_KEY", + "GOOGLE_API_KEY", + "GOOGLE_VISION_API_KEY", + "MATHPIX_APP_ID", + "MATHPIX_APP_KEY", + } +) +# Every credential a SlideNote run could pick up; derived from the provider registry. +API_ENV_KEYS = frozenset( + {"SLIDENOTE_API_KEY", *OCR_ENV_KEYS, *(key for spec in PROVIDERS.values() for key in spec.api_key_envs)} +) + + +def env_without(keys: frozenset[str]) -> dict[str, str]: + env = dict(os.environ) + for key in keys: + env.pop(key, None) + return env + + +def run(command: list[str], *, env: dict[str, str]) -> None: + completed = subprocess.run(command, text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=env, check=False) + if completed.returncode != 0: + raise RuntimeError(f"Command failed with exit code {completed.returncode}: {' '.join(command)}\n{completed.stdout}") diff --git a/scripts/audit_redundancy.py b/scripts/audit_redundancy.py index 763d912..94eb229 100644 --- a/scripts/audit_redundancy.py +++ b/scripts/audit_redundancy.py @@ -4,7 +4,8 @@ 1. unused imports per module (names bound by import but never referenced, unless exported via __all__ or used in type-comment/string contexts) 2. modules that are never imported anywhere in the repository - 3. duplicate function bodies (same AST, same module or cross-module) + 3. duplicate function bodies (same AST, same module or cross-module), + including the gui/ and scripts/ helpers Usage: python scripts/audit_redundancy.py """ @@ -19,6 +20,8 @@ ROOT = Path(__file__).resolve().parents[1] PACKAGE = ROOT / "slidenote" +# Directories whose functions are also checked for duplicated bodies. +DUPLICATE_SCAN_DIRS = (ROOT / "gui", ROOT / "scripts") # Modules that are legitimate entry points / never need importers. ENTRY_POINTS = { @@ -42,9 +45,18 @@ def module_name(path: Path, root: Path, package_prefix: str | None = None) -> st return ".".join(parts) -def analyze_file(path: Path) -> tuple[set[str], list[tuple[int, str]], list[ast.stmt]]: +def _annotations(node: ast.AST) -> list[ast.expr]: + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + args = [*node.args.posonlyargs, *node.args.args, *node.args.kwonlyargs] + args += [arg for arg in (node.args.vararg, node.args.kwarg) if arg is not None] + return [annotation for annotation in [node.returns, *(arg.annotation for arg in args)] if annotation is not None] + if isinstance(node, ast.AnnAssign): + return [node.annotation] + return [] + + +def analyze_tree(tree: ast.Module) -> tuple[set[str], list[tuple[int, str, str]], list[ast.stmt]]: """Return (used_names, unused_imports, module_functions).""" - tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path)) used: set[str] = set() imported: list[tuple[int, str, str]] = [] # (lineno, name, source) functions: list[ast.stmt] = [] @@ -54,12 +66,21 @@ def analyze_file(path: Path) -> tuple[set[str], list[tuple[int, str]], list[ast. for node in tree.body: if isinstance(node, ast.Assign): for target in node.targets: - if isinstance(target, ast.Name) and target.id == "__all__": - for elt in node.value.elts: # type: ignore[attr-defined] + # Only literal lists/tuples can be read statically; skip computed __all__. + if isinstance(target, ast.Name) and target.id == "__all__" and isinstance(node.value, (ast.List, ast.Tuple)): + for elt in node.value.elts: if isinstance(elt, ast.Constant) and isinstance(elt.value, str): exported.add(elt.value) for node in ast.walk(tree): + # Quoted annotations ("NoteOptions") reference TYPE_CHECKING-only imports. + for annotation in _annotations(node): + if isinstance(annotation, ast.Constant) and isinstance(annotation.value, str): + try: + parsed = ast.parse(annotation.value, mode="eval") + except SyntaxError: + continue + used.update(name.id for name in ast.walk(parsed) if isinstance(name, ast.Name)) if isinstance(node, ast.Name): used.add(node.id) elif isinstance(node, ast.Import): @@ -73,7 +94,8 @@ def analyze_file(path: Path) -> tuple[set[str], list[tuple[int, str]], list[ast. if alias.name == "*": continue local = alias.asname or alias.name - imported.append((node.lineno, local, f"{node.module}.{alias.name}")) + source = "." * node.level + (node.module or "") + imported.append((node.lineno, local, f"{source}.{alias.name}" if node.module else f"{source}{alias.name}")) elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): if node.name.startswith("test_") or node.name.startswith("_test"): continue @@ -113,10 +135,12 @@ def resolve_relative(importer: str, path: Path, level: int, module: str | None) for path in repository_files: mod = importer_name(path) tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path)) - if path in module_of: - _, unused, functions = analyze_file(path) - for lineno, name, src in unused: - all_unused.append((mod, lineno, name, src)) + in_package = path in module_of + if in_package or any(scan_dir in path.parents for scan_dir in DUPLICATE_SCAN_DIRS): + _, unused, functions = analyze_tree(tree) + if in_package: + for lineno, name, src in unused: + all_unused.append((mod, lineno, name, src)) for fn in functions: if len(fn.body) == 1 and ( isinstance(fn.body[0], ast.Pass) diff --git a/scripts/smoke_first_run.py b/scripts/smoke_first_run.py index cda0ff2..2863df4 100644 --- a/scripts/smoke_first_run.py +++ b/scripts/smoke_first_run.py @@ -1,40 +1,16 @@ from __future__ import annotations import argparse -import os import shutil -import subprocess import sys import tempfile import zipfile from pathlib import Path import fitz +from _smoke_common import API_ENV_KEYS, env_without, run -API_ENV_KEYS = { - "ANTHROPIC_API_KEY", - "ARK_API_KEY", - "BAIDU_OCR_API_KEY", - "BAIDU_OCR_SECRET_KEY", - "CLAUDE_API_KEY", - "DASHSCOPE_API_KEY", - "DEEPSEEK_API_KEY", - "DOUBAO_API_KEY", - "GEMINI_API_KEY", - "GLM_API_KEY", - "GOOGLE_API_KEY", - "GOOGLE_VISION_API_KEY", - "MATHPIX_APP_ID", - "MATHPIX_APP_KEY", - "OPENAI_API_KEY", - "QWEN_API_KEY", - "SLIDENOTE_API_KEY", - "VOLCENGINE_API_KEY", - "ZAI_API_KEY", - "ZHIPUAI_API_KEY", -} - REQUIRED_BUILD_FILES = { "notes.md", "notes.zip", @@ -68,9 +44,9 @@ def main() -> int: try: root.mkdir(parents=True, exist_ok=True) _write_fixture_pdf(source) - env = _no_api_env() + env = env_without(API_ENV_KEYS) - _run( + run( [ sys.executable, "-m", @@ -90,7 +66,7 @@ def main() -> int: _require_files(build_out, REQUIRED_BUILD_FILES) _require_zip_members(build_out / "notes.zip", {"notes.md", "README.txt"}) - _run( + run( [ sys.executable, "-m", @@ -122,26 +98,6 @@ def _write_fixture_pdf(path: Path) -> None: doc.close() -def _no_api_env() -> dict[str, str]: - env = dict(os.environ) - for key in API_ENV_KEYS: - env.pop(key, None) - return env - - -def _run(command: list[str], *, env: dict[str, str]) -> None: - completed = subprocess.run( - command, - text=True, - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, - env=env, - check=False, - ) - if completed.returncode != 0: - raise RuntimeError(f"Command failed with exit code {completed.returncode}: {' '.join(command)}\n{completed.stdout}") - - def _require_files(root: Path, names: set[str]) -> None: missing = sorted(name for name in names if not (root / name).exists()) if missing: diff --git a/scripts/smoke_textbook_index.py b/scripts/smoke_textbook_index.py index 28416ff..3f80513 100644 --- a/scripts/smoke_textbook_index.py +++ b/scripts/smoke_textbook_index.py @@ -2,25 +2,15 @@ import argparse import json -import os import shutil -import subprocess import sys import tempfile from pathlib import Path import fitz +from _smoke_common import OCR_ENV_KEYS, env_without, run -API_ENV_KEYS = { - "BAIDU_OCR_API_KEY", - "BAIDU_OCR_SECRET_KEY", - "GOOGLE_API_KEY", - "GOOGLE_VISION_API_KEY", - "MATHPIX_APP_ID", - "MATHPIX_APP_KEY", -} - REQUIRED_FILES = { "textbook_manifest.json", "textbook_pages.jsonl", @@ -46,8 +36,8 @@ def main() -> int: try: root.mkdir(parents=True, exist_ok=True) _write_fixture_pdf(source) - env = _no_ocr_env() - _run( + env = env_without(OCR_ENV_KEYS) + run( [ sys.executable, "-m", @@ -92,19 +82,6 @@ def _write_fixture_pdf(path: Path) -> None: doc.close() -def _no_ocr_env() -> dict[str, str]: - env = dict(os.environ) - for key in API_ENV_KEYS: - env.pop(key, None) - return env - - -def _run(command: list[str], *, env: dict[str, str]) -> None: - completed = subprocess.run(command, text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=env, check=False) - if completed.returncode != 0: - raise RuntimeError(f"Command failed with exit code {completed.returncode}: {' '.join(command)}\n{completed.stdout}") - - def _require_files(root: Path) -> None: missing = sorted(name for name in REQUIRED_FILES if not (root / name).exists()) if missing: diff --git a/slidenote/api_retry.py b/slidenote/api_retry.py index 4fccc55..e005d16 100644 --- a/slidenote/api_retry.py +++ b/slidenote/api_retry.py @@ -9,6 +9,12 @@ T = TypeVar("T") +DEFAULT_MAX_RETRIES = 2 +TRANSIENT_STATUS_CODES = frozenset({408, 409, 425, 429}) +# A status code only counts when it appears next to an HTTP/status marker, so +# numbers such as token counts or model names ("gpt-4o-2024-05-13") never match. +_STATUS_IN_MESSAGE = re.compile(r"\b(?:https?|status(?:[\s_]+code)?|error[\s_]+code|code)[\s:=]*([1-5]\d\d)\b") + @dataclass(slots=True) class RetryResult(Generic[T]): @@ -19,7 +25,7 @@ class RetryResult(Generic[T]): def with_api_retries( call: Callable[[], T], *, - max_retries: int = 2, + max_retries: int = DEFAULT_MAX_RETRIES, base_delay: float = 0.5, jitter: float = 0.15, ) -> RetryResult[T]: @@ -35,18 +41,31 @@ def with_api_retries( time.sleep(delay) -def is_transient_api_error(exc: Exception) -> bool: +def is_transient_status(status: int) -> bool: + return status in TRANSIENT_STATUS_CODES or 500 <= status <= 599 + + +def is_transient_api_error(exc: BaseException) -> bool: + # HTTPError subclasses URLError, so its status must be checked first: + # 400/401/403/404 are permanent and must not be retried. + if isinstance(exc, urllib.error.HTTPError): + return is_transient_status(exc.code) if isinstance(exc, (TimeoutError, ConnectionError, urllib.error.URLError)): return True + cause = exc.__cause__ + if isinstance(cause, (urllib.error.URLError, TimeoutError, ConnectionError)): + # Providers wrap urllib errors in RuntimeError; classify the original. + return is_transient_api_error(cause) status = _status_code(exc) if status is not None: - return status in {408, 409, 425, 429} or 500 <= status <= 599 + return is_transient_status(status) name = exc.__class__.__name__.lower() if any(marker in name for marker in ("ratelimit", "timeout", "connection", "serviceunavailable", "internalserver")): return True message = str(exc).lower() - if _http_status_in_message(message) in {408, 409, 425, 429, 500, 502, 503, 504}: - return True + message_status = _http_status_in_message(message) + if message_status is not None: + return is_transient_status(message_status) transient_markers = ( "rate limit", "ratelimit", @@ -62,10 +81,10 @@ def is_transient_api_error(exc: Exception) -> bool: return any(marker in message for marker in transient_markers) -def _status_code(exc: Exception) -> int | None: +def _status_code(exc: BaseException) -> int | None: for attr in ("status_code", "status", "code"): value = getattr(exc, attr, None) - if isinstance(value, int): + if isinstance(value, int) and not isinstance(value, bool): return value response = getattr(exc, "response", None) value = getattr(response, "status_code", None) @@ -73,10 +92,5 @@ def _status_code(exc: Exception) -> int | None: def _http_status_in_message(message: str) -> int | None: - match = re.search(r"\b(?:http\s*)?([45]\d\d)\b", message) - if not match: - return None - try: - return int(match.group(1)) - except ValueError: - return None + match = _STATUS_IN_MESSAGE.search(message) + return int(match.group(1)) if match else None diff --git a/slidenote/build/artifacts.py b/slidenote/build/artifacts.py index 3d3d58a..43cdfd5 100644 --- a/slidenote/build/artifacts.py +++ b/slidenote/build/artifacts.py @@ -1,14 +1,131 @@ from __future__ import annotations +import json +import shutil from pathlib import Path from typing import Any -from slidenote.pipeline import ArtifactRegistry, BuildContext, FunctionStage, StageResult, run_stage +from slidenote.pipeline import ArtifactRegistry + +# Files a build (or a follow-up study-pack/cost run) writes into the output +# directory. They are removed at build start so artifacts from an earlier run +# with different options (e.g. vision_usage.json after `--vision off`) cannot be +# mistaken for results of the current run. `.cache/`, user-written +# `page_modalities.overrides.json`, progress.json (rewritten immediately) and +# any unknown files are intentionally preserved. +GENERATED_ARTIFACT_FILES = ( + "content.json", + "element_ir.json", + "notes.md", + "coverage.md", + "coverage.json", + "quality_report.json", + "source_map.json", + "run_summary.json", + "page_modalities.json", + "table_understanding.json", + "semantic_layout.json", + "image_importance.json", + "composite_figures.json", + "sections.json", + "deck_brief.json", + "deck_brief.md", + "deck_understanding.json", + "page_understanding.json", + "content_guard.json", + "figures.json", + "figure_usage.json", + "figure_grounding.json", + "ocr.json", + "ocr_usage.json", + "visuals.json", + "vision_usage.json", + "llm_usage.json", + "page_notes.json", + "page_notes.md", + "weave_report.json", + "teaching_enrichment.json", + "export_report.json", + "notes.zip", + "notes.toc.md", + "notes.docx", + "notes.pdf", + "notes.tex", + "cost_report.json", + "cost_report.md", + "cost_dashboard.html", + "study_pack.json", + "review.md", + "exam.json", + "exam.md", + "exam.html", + "section_study_pack.json", + "exam_review_pack.json", + "final_exam.md", + "final_exam.answers.md", + "wrong_answer_review_prompt.md", +) +GENERATED_ARTIFACT_DIRS = ("notes.assets", "figures", "images", "screenshots") +# A progress.json file can also be created by a failed setup attempt, so it is +# not enough to identify a directory as a previous SlideNote build. + + +def _is_previous_build(output_root: Path) -> bool: + for name in ("run_summary.json", "content.json"): + try: + marker = json.loads((output_root / name).read_text(encoding="utf-8")) + except (OSError, UnicodeError, json.JSONDecodeError): + continue + if not isinstance(marker, dict): + continue + if name == "run_summary.json": + artifacts = marker.get("artifacts") + if ( + marker.get("schema_version") == 1 + and isinstance(marker.get("source_path"), str) + and isinstance(marker.get("source_type"), str) + and isinstance(artifacts, dict) + and artifacts.get("content") == "content.json" + ): + return True + elif ( + isinstance(marker.get("source_path"), str) + and marker.get("source_type") in {"pdf", "pptx"} + and isinstance(marker.get("pages"), list) + and all(isinstance(page, dict) and isinstance(page.get("slide_id"), int) for page in marker["pages"]) + ): + return True + return False + + +def remove_stale_build_artifacts(output_root: Path, keep: tuple[Path, ...] = ()) -> list[str]: + """Delete known SlideNote-generated artifacts from a previous run. + + Paths in ``keep`` (and anything containing them, such as the input file + living inside ``images/``) are never removed. Returns the removed names. + """ + if not _is_previous_build(output_root): + return [] + resolved_keep = [path.resolve() for path in keep] + removed: list[str] = [] + for name in (*GENERATED_ARTIFACT_FILES, *GENERATED_ARTIFACT_DIRS): + target = output_root / name + if not target.exists() and not target.is_symlink(): + continue + resolved = target.resolve() + if any(kept == resolved or resolved in kept.parents for kept in resolved_keep): + continue + if target.is_dir() and not target.is_symlink(): + shutil.rmtree(target) + else: + target.unlink() + removed.append(name) + return removed def _run_json_stage( deck, - context: BuildContext, + state, *, name: str, artifact_name: str, @@ -16,30 +133,22 @@ def _run_json_stage( message: str, complete_message: str, runner, - dependencies: list[str] | None = None, ) -> dict[str, Any]: - progress = context.progress + progress = state.progress progress.start_stage(name, message=message) - - def stage_runner(stage_deck, stage_context: BuildContext) -> StageResult: - report = runner(stage_deck) - artifacts: dict[str, str] = {} - if stage_context.artifacts is not None: - stage_context.artifacts.write_json(artifact_name, artifact_path, report) - registered = stage_context.artifacts.relative_path(artifact_name) - if registered: - artifacts[artifact_name] = registered - return StageResult(name=name, report=report, artifacts=artifacts) - - stage = FunctionStage( - name=name, - dependencies=dependencies or [], - artifacts=[artifact_name], - runner=stage_runner, - ) - result = run_stage(deck, context, stage) + report = runner(deck) + state.artifacts.write_json(artifact_name, artifact_path, report) progress.finish_stage(complete_message) - return result.report or {} + return report or {} + + +EXPORT_ARTIFACT_NAMES = { + "markdown-zip": "notes_zip", + "markdown-toc": "notes_toc", + "docx": "notes_docx", + "pdf": "notes_pdf", + "latex": "notes_latex", +} def _register_export_artifacts(artifacts: ArtifactRegistry, export_report: dict[str, Any]) -> None: @@ -50,13 +159,13 @@ def _register_export_artifacts(artifacts: ArtifactRegistry, export_report: dict[ if not isinstance(result, dict) or result.get("status") != "ok": continue path = result.get("path") - fmt = str(result.get("format") or "").replace("-", "_") + fmt = str(result.get("format") or "") if not path or not fmt: continue resolved = Path(path) if not resolved.is_absolute(): resolved = artifacts.output_root / resolved - artifacts.register(f"notes_{fmt}", resolved) + artifacts.register(EXPORT_ARTIFACT_NAMES.get(fmt, f"notes_{fmt.replace('-', '_')}"), resolved) def _build_ocr_export(deck, ocr_report): return { diff --git a/slidenote/build/config.py b/slidenote/build/config.py index 8a2b08a..a4050e4 100644 --- a/slidenote/build/config.py +++ b/slidenote/build/config.py @@ -6,6 +6,23 @@ from slidenote.llm import get_provider_spec +# Per-stage output-token caps and temperatures for auxiliary model passes. +SEMANTIC_LAYOUT_MAX_OUTPUT_TOKENS = 1400 +FIGURE_CROP_MAX_OUTPUT_TOKENS = 1200 +FIGURE_GROUNDING_MAX_OUTPUT_TOKENS = 1400 +VISION_AUX_DEFAULT_OUTPUT_TOKENS = 1000 +SECTIONS_MAX_OUTPUT_TOKENS = 2500 +SECTIONS_DEFAULT_OUTPUT_TOKENS = 1800 +DECK_BRIEF_MAX_OUTPUT_TOKENS = 5000 +DECK_BRIEF_DEFAULT_OUTPUT_TOKENS = 3000 +CONTENT_GUARD_MAX_OUTPUT_TOKENS = 2500 +CONTENT_GUARD_DEFAULT_OUTPUT_TOKENS = 1800 +AUXILIARY_TEXT_TEMPERATURE = 0.0 + + +def _capped_tokens(requested: int | None, default: int, cap: int) -> int: + return min(requested or default, cap) + def _friendly_build_error(exc: Exception, args: argparse.Namespace) -> str | None: message = str(exc) @@ -225,15 +242,39 @@ def _apply_note_profile_defaults(args: argparse.Namespace) -> None: } +# Presets whose values always win over explicit CLI flags (with a warning). +FORCED_BUILD_PRESETS = {"local"} + + def _apply_build_preset_defaults(args: argparse.Namespace) -> None: preset = getattr(args, "preset", "lecture") preset_defaults = BUILD_PRESET_DEFAULTS[preset] explicit_options = set(getattr(args, "_explicit_options", set()) or set()) + warnings: list[str] = list(getattr(args, "_config_warnings", None) or []) + forced = preset in FORCED_BUILD_PRESETS for name, value in preset_defaults.items(): - if preset == "local" or name not in explicit_options: - setattr(args, name, value) - if args.vision == "off" and args.semantic_layout == "auto": + if name in explicit_options and not forced: + continue + if forced and name in explicit_options and getattr(args, name, value) != value: + flag = "--" + name.replace("_", "-") + warnings.append( + f"`--preset {preset}` forces `{flag} {value}`; ignoring explicit `{flag} {getattr(args, name)}`." + ) + setattr(args, name, value) + _disable_vision_dependent_modes(args) + args._config_warnings = warnings + + +def _disable_vision_dependent_modes(args: argparse.Namespace) -> None: + """`--vision off` means no vision API calls anywhere, so downgrade dependents to local modes.""" + if args.vision != "off": + return + if args.semantic_layout in {"auto", "vision"}: args.semantic_layout = "local" + if args.figure_grounding == "vision": + args.figure_grounding = "auto" + if args.figure_crop == "vision": + args.figure_crop = "off" def _resolve_api_concurrency(args: argparse.Namespace) -> dict[str, int]: diff --git a/slidenote/build/progress.py b/slidenote/build/progress.py index 91b58cd..1610cc6 100644 --- a/slidenote/build/progress.py +++ b/slidenote/build/progress.py @@ -27,6 +27,9 @@ def callback(event: dict[str, Any]) -> None: def _llm_progress(progress: ProgressReporter): def callback(record: dict[str, Any]) -> None: + if record.get("event") == "total": + progress.set_total(record.get("total")) + return label = record.get("context_id") or record.get("slide_id") progress.advance( message=f"LLM context {label}", @@ -37,21 +40,18 @@ def callback(record: dict[str, Any]) -> None: return callback -def _stage_metrics(progress: ProgressReporter) -> dict[str, Any]: +def _stage_metrics(progress: ProgressReporter, limit: int = 3) -> dict[str, Any]: snapshot = progress.snapshot() - stages = snapshot.get("stages") if isinstance(snapshot, dict) else [] - stage_records = [stage for stage in stages if isinstance(stage, dict)] + stage_records = [stage for stage in snapshot.get("stages") or [] if isinstance(stage, dict)] return { - "elapsed_seconds": snapshot.get("elapsed_seconds") if isinstance(snapshot, dict) else None, + "elapsed_seconds": snapshot.get("elapsed_seconds"), "stages": stage_records, - "slowest_stages": _slowest_stage_records(stage_records, limit=3), + "slowest_stages": _slowest_stage_records(stage_records, limit=limit), } def _slowest_stages(progress: ProgressReporter, limit: int = 3) -> list[dict[str, Any]]: - snapshot = progress.snapshot() - stages = snapshot.get("stages") if isinstance(snapshot, dict) else [] - return _slowest_stage_records([stage for stage in stages if isinstance(stage, dict)], limit=limit) + return _stage_metrics(progress, limit=limit)["slowest_stages"] def _slowest_stage_records(stages: list[dict[str, Any]], limit: int = 3) -> list[dict[str, Any]]: diff --git a/slidenote/build/runner.py b/slidenote/build/runner.py index 227a1e9..54a41e6 100644 --- a/slidenote/build/runner.py +++ b/slidenote/build/runner.py @@ -1,6 +1,7 @@ from __future__ import annotations import argparse +import sys from slidenote.build.config import ( _apply_build_preset_defaults, @@ -8,23 +9,30 @@ _friendly_build_error, ) from slidenote.build.errors import UserFacingConfigError -from slidenote.build.stages import BUILD_STAGES, _print_build_outputs -from slidenote.build.state import create_build_state +from slidenote.build.stages import BUILD_PHASES, _print_build_outputs +from slidenote.build.state import create_build_state, resolve_progress_path from slidenote.exporting import parse_export_formats +from slidenote.pipeline import run_build_plan +from slidenote.progress import ProgressReporter def run_build(args: argparse.Namespace) -> int: _apply_build_preset_defaults(args) _apply_note_profile_defaults(args) + for warning in getattr(args, "_config_warnings", None) or []: + print(f"Warning: {warning}", file=sys.stderr) try: export_formats = parse_export_formats(args.export) + state = create_build_state(args, export_formats) except ValueError as exc: + _record_setup_failure(args, str(exc)) raise UserFacingConfigError(str(exc)) from exc + except Exception as exc: + _record_setup_failure(args, str(exc)) + raise - state = create_build_state(args, export_formats) try: - for stage in BUILD_STAGES: - stage(state) + run_build_plan(state, BUILD_PHASES) except Exception as exc: friendly_message = _friendly_build_error(exc, args) if friendly_message: @@ -35,3 +43,11 @@ def run_build(args: argparse.Namespace) -> int: _print_build_outputs(state) return state.export_exit_code + + +def _record_setup_failure(args: argparse.Namespace, message: str) -> None: + """Record setup errors in progress.json so GUI pollers see the failure.""" + try: + ProgressReporter(resolve_progress_path(args), quiet=True).fail(message) + except OSError: + pass diff --git a/slidenote/build/stages.py b/slidenote/build/stages.py index a41811a..eea6e10 100644 --- a/slidenote/build/stages.py +++ b/slidenote/build/stages.py @@ -9,7 +9,21 @@ _register_export_artifacts, _run_json_stage, ) -from slidenote.build.config import _should_build_deck_brief +from slidenote.build.config import ( + AUXILIARY_TEXT_TEMPERATURE, + CONTENT_GUARD_DEFAULT_OUTPUT_TOKENS, + CONTENT_GUARD_MAX_OUTPUT_TOKENS, + DECK_BRIEF_DEFAULT_OUTPUT_TOKENS, + DECK_BRIEF_MAX_OUTPUT_TOKENS, + FIGURE_CROP_MAX_OUTPUT_TOKENS, + FIGURE_GROUNDING_MAX_OUTPUT_TOKENS, + SECTIONS_DEFAULT_OUTPUT_TOKENS, + SECTIONS_MAX_OUTPUT_TOKENS, + SEMANTIC_LAYOUT_MAX_OUTPUT_TOKENS, + VISION_AUX_DEFAULT_OUTPUT_TOKENS, + _capped_tokens, + _should_build_deck_brief, +) from slidenote.build.progress import _llm_progress, _slowest_stages, _target_progress from slidenote.build.state import BuildState from slidenote.build.summary import _build_run_summary @@ -21,6 +35,7 @@ from slidenote.image_ranking import rank_deck_images from slidenote.ir import build_deck_ir from slidenote.models import Deck +from slidenote.pipeline import BuildPhase, BuildStep from slidenote.notes import NoteGenerationResult, NoteOptions, estimate_note_generation_steps, generate_notes_result from slidenote.notes.quality import build_note_quality_report from slidenote.ocr import enrich_deck_with_ocr @@ -29,9 +44,10 @@ from slidenote.table_understanding import enrich_deck_with_table_understanding from slidenote.understanding import build_understanding_reports from slidenote.composite_figures import enrich_deck_with_composite_figures +from slidenote.notes.contexts import _resolved_context_mode from slidenote.figure_grounding import enrich_deck_with_figure_grounding from slidenote.figures import enrich_deck_with_figures -from slidenote.modality import enrich_deck_with_modalities +from slidenote.modality import apply_modality_overrides, enrich_deck_with_modalities from slidenote.semantic_layout import enrich_deck_with_semantic_layout from slidenote.vision import enrich_deck_with_vision @@ -46,13 +62,17 @@ def _stage_modality(state: BuildState) -> None: deck = _require_deck(state) state.modality_report = _run_json_stage( deck, - state.build_context, + state, name="modality", artifact_name="page_modalities", artifact_path="page_modalities.json", message="Classifying page modalities", complete_message="Page modality classification complete", - runner=lambda stage_deck: enrich_deck_with_modalities(stage_deck), + runner=lambda stage_deck: apply_modality_overrides( + stage_deck, + enrich_deck_with_modalities(stage_deck), + state.output_root / "page_modalities.overrides.json", + ), ) @@ -60,9 +80,8 @@ def _stage_table_understanding(state: BuildState) -> None: deck = _require_deck(state) state.table_understanding_report = _run_json_stage( deck, - state.build_context, + state, name="table_understanding", - dependencies=["modality"], artifact_name="table_understanding", artifact_path="table_understanding.json", message="Summarizing table conclusions", @@ -76,9 +95,8 @@ def _stage_semantic_layout(state: BuildState) -> None: deck = _require_deck(state) state.semantic_layout_report = _run_json_stage( deck, - state.build_context, + state, name="semantic_layout", - dependencies=["table_understanding"], artifact_name="semantic_layout", artifact_path="semantic_layout.json", message="Building semantic page blocks", @@ -93,7 +111,9 @@ def _stage_semantic_layout(state: BuildState) -> None: base_url=args.vision_base_url, cache_mode=args.vision_cache, cache_dir=state.cache_dirs["vision"], - max_output_tokens=min(args.vision_max_output_tokens or 1000, 1400), + max_output_tokens=_capped_tokens( + args.vision_max_output_tokens, VISION_AUX_DEFAULT_OUTPUT_TOKENS, SEMANTIC_LAYOUT_MAX_OUTPUT_TOKENS + ), temperature=args.vision_temperature, detail=args.vision_detail or "low", max_edge=args.vision_max_edge, @@ -105,8 +125,6 @@ def _stage_semantic_layout(state: BuildState) -> None: def _stage_composite_figures(state: BuildState) -> None: args = state.args - if args.composite_figures == "off": - return deck = _require_deck(state) state.progress.start_stage("composite_figures", message="Detecting composite figures") state.composite_figure_report = enrich_deck_with_composite_figures( @@ -120,9 +138,6 @@ def _stage_composite_figures(state: BuildState) -> None: def _stage_figure_crop(state: BuildState) -> None: args = state.args - should_run_figure_crop = args.figure_crop == "vision" or (args.figure_crop == "auto" and args.vision != "off") - if not should_run_figure_crop: - return deck = _require_deck(state) state.progress.start_stage("figure_crop", message="Cropping local figures") state.figure_report = enrich_deck_with_figures( @@ -139,7 +154,7 @@ def _stage_figure_crop(state: BuildState) -> None: max_crops_per_page=args.figure_max_crops_per_page, min_confidence=args.figure_min_confidence, min_area=args.figure_min_area, - max_output_tokens=min(args.vision_max_output_tokens or 1000, 1200), + max_output_tokens=_capped_tokens(args.vision_max_output_tokens, VISION_AUX_DEFAULT_OUTPUT_TOKENS, FIGURE_CROP_MAX_OUTPUT_TOKENS), temperature=args.vision_temperature, detail=args.vision_detail, max_edge=args.vision_max_edge, @@ -152,8 +167,6 @@ def _stage_figure_crop(state: BuildState) -> None: def _stage_image_importance(state: BuildState) -> None: args = state.args - if args.image_ranking == "off": - return deck = _require_deck(state) state.progress.start_stage("image_importance", message="Ranking image importance") state.image_importance_report = rank_deck_images(deck, state.output_root, mode=args.image_ranking, stage="pre_vision") @@ -162,8 +175,6 @@ def _stage_image_importance(state: BuildState) -> None: def _stage_ocr(state: BuildState) -> None: args = state.args - if args.ocr == "off": - return deck = _require_deck(state) state.progress.start_stage("ocr", message="Running OCR") state.ocr_report = enrich_deck_with_ocr( @@ -190,16 +201,12 @@ def _stage_ocr(state: BuildState) -> None: def _stage_vision(state: BuildState) -> None: args = state.args - should_run_vision = args.vision != "off" or args.figure_grounding == "vision" - if not should_run_vision: - return deck = _require_deck(state) - vision_mode = args.vision if args.vision != "off" else "auto" state.progress.start_stage("vision", message="Running vision analysis") state.vision_report = enrich_deck_with_vision( deck, output_root=state.output_root, - mode=vision_mode, + mode=args.vision, provider=args.vision_provider, model=args.vision_model, api_key=args.vision_api_key, @@ -223,8 +230,6 @@ def _stage_vision(state: BuildState) -> None: def _stage_figure_grounding(state: BuildState) -> None: args = state.args - if args.figure_grounding == "off": - return deck = _require_deck(state) state.progress.start_stage("figure_grounding", message="Grounding figures to page text") state.figure_grounding_report = enrich_deck_with_figure_grounding( @@ -239,7 +244,9 @@ def _stage_figure_grounding(state: BuildState) -> None: base_url=args.vision_base_url, cache_mode=args.vision_cache, cache_dir=state.cache_dirs["vision"], - max_output_tokens=min(args.vision_max_output_tokens or 1000, 1400), + max_output_tokens=_capped_tokens( + args.vision_max_output_tokens, VISION_AUX_DEFAULT_OUTPUT_TOKENS, FIGURE_GROUNDING_MAX_OUTPUT_TOKENS + ), temperature=args.vision_temperature, detail=args.vision_detail or "low", max_edge=args.vision_max_edge, @@ -258,15 +265,15 @@ def _stage_sections(state: BuildState) -> None: deck, output_root=state.output_root, mode=args.section_detection, - use_llm=args.use_llm and (args.note_context == "section" or (args.note_context == "auto" and len(deck.pages) > 12)), + use_llm=args.use_llm and _resolved_context_mode(deck, args.note_context) == "section", provider=args.provider, model=args.model, api_key=args.api_key, base_url=args.base_url, cache_mode=args.section_cache, cache_dir=state.cache_dirs["sections"], - max_output_tokens=min(args.max_output_tokens or 1800, 2500), - temperature=0.0, + max_output_tokens=_capped_tokens(args.max_output_tokens, SECTIONS_DEFAULT_OUTPUT_TOKENS, SECTIONS_MAX_OUTPUT_TOKENS), + temperature=AUXILIARY_TEXT_TEMPERATURE, ) state.artifacts.write_json("sections", "sections.json", state.section_report) state.progress.finish_stage("Section detection complete") @@ -274,8 +281,6 @@ def _stage_sections(state: BuildState) -> None: def _stage_deck_brief(state: BuildState) -> None: args = state.args - if not _should_build_deck_brief(args): - return deck = _require_deck(state) state.progress.start_stage("deck_brief", message="Building deck brief") state.deck_brief_report = build_deck_brief( @@ -288,8 +293,8 @@ def _stage_deck_brief(state: BuildState) -> None: base_url=args.base_url, cache_mode=args.cache, cache_dir=state.cache_dirs["llm"], - max_output_tokens=min(args.max_output_tokens or 3000, 5000), - temperature=0.0, + max_output_tokens=_capped_tokens(args.max_output_tokens, DECK_BRIEF_DEFAULT_OUTPUT_TOKENS, DECK_BRIEF_MAX_OUTPUT_TOKENS), + temperature=AUXILIARY_TEXT_TEMPERATURE, ) state.artifacts.write_json("deck_brief", "deck_brief.json", state.deck_brief_report) state.artifacts.write_text("deck_brief_markdown", "deck_brief.md", render_deck_brief_markdown(state.deck_brief_report)) @@ -298,8 +303,6 @@ def _stage_deck_brief(state: BuildState) -> None: def _stage_content_guard(state: BuildState) -> None: args = state.args - if args.content_guard == "off": - return deck = _require_deck(state) state.progress.start_stage("content_guard", message="Classifying required learning content") state.content_guard_report = build_content_guard( @@ -313,8 +316,8 @@ def _stage_content_guard(state: BuildState) -> None: base_url=args.base_url, cache_mode=args.cache, cache_dir=state.cache_dirs["llm"], - max_output_tokens=min(args.max_output_tokens or 1800, 2500), - temperature=0.0, + max_output_tokens=_capped_tokens(args.max_output_tokens, CONTENT_GUARD_DEFAULT_OUTPUT_TOKENS, CONTENT_GUARD_MAX_OUTPUT_TOKENS), + temperature=AUXILIARY_TEXT_TEMPERATURE, ) if state.content_guard_report is not None: state.artifacts.write_json("content_guard", "content_guard.json", state.content_guard_report) @@ -341,7 +344,6 @@ def _stage_export_content(state: BuildState) -> None: deck = _require_deck(state) state.progress.start_stage("export_content", message="Writing structured content") state.artifacts.write_json("content", "content.json", deck.to_dict()) - state.artifacts.write_json("element_ir", "element_ir.json", build_deck_ir(deck, content_guard=state.content_guard_report)) if state.image_importance_report is not None: state.artifacts.write_json("image_importance", "image_importance.json", state.image_importance_report) if state.figure_report is not None: @@ -440,7 +442,13 @@ def _stage_coverage(state: BuildState) -> None: "element_ir.json", build_deck_ir(deck, content_guard=state.content_guard_report, coverage_report=state.coverage_report), ) - state.source_map = build_source_map(deck, state.notes_markdown, state.output_root) + state.source_map = build_source_map( + deck, + state.notes_markdown, + state.output_root, + content_guard=state.content_guard_report, + coverage_report=state.coverage_report, + ) state.artifacts.write_json("source_map", "source_map.json", state.source_map) state.progress.finish_stage("Coverage complete") @@ -457,7 +465,6 @@ def _stage_quality_report(state: BuildState) -> None: note_context=args.note_context, note_strategy=args.note_strategy, note_depth=args.note_depth, - study_pack_report=state.study_pack_report, ) state.artifacts.write_json("quality_report", "quality_report.json", state.quality_report) state.progress.finish_stage("Quality report complete") @@ -465,8 +472,6 @@ def _stage_quality_report(state: BuildState) -> None: def _stage_export(state: BuildState) -> None: args = state.args - if not state.export_formats: - return state.progress.start_stage("export", message="Exporting requested note formats") state.export_report = build_export_artifacts(state.notes_markdown, state.output_root, state.export_formats, export_toc=args.export_toc) if state.export_report is not None: @@ -478,102 +483,27 @@ def _stage_export(state: BuildState) -> None: def _stage_run_summary(state: BuildState) -> None: - deck = _require_deck(state) - notes_result = _require_notes_result(state) - coverage_report = _require_report(state.coverage_report, "coverage") - source_map = _require_report(state.source_map, "source_map") + _require_deck(state) + _require_notes_result(state) + _require_report(state.coverage_report, "coverage") + _require_report(state.source_map, "source_map") + state.progress.set_phase(None) state.progress.complete("Build complete") state.artifacts.register("run_summary", state.output_root / "run_summary.json") - run_summary = _build_run_summary( - args=state.args, - input_path=state.input_path, - output_root=state.output_root, - deck=deck, - modality_report=state.modality_report or {}, - table_understanding_report=state.table_understanding_report or {}, - semantic_layout_report=state.semantic_layout_report or {}, - image_importance_report=state.image_importance_report, - section_report=state.section_report or {}, - deck_brief_report=state.deck_brief_report, - deck_understanding_report=state.deck_understanding_report, - page_understanding_report=state.page_understanding_report, - composite_figure_report=state.composite_figure_report, - figure_report=state.figure_report, - figure_grounding_report=state.figure_grounding_report, - ocr_report=state.ocr_report, - vision_report=state.vision_report, - content_guard_report=state.content_guard_report, - llm_usage=notes_result.llm_usage, - coverage_report=coverage_report, - quality_report=state.quality_report, - source_map=source_map, - cache_dirs=state.cache_dirs, - refresh_slide_ids=state.refresh_slide_ids, - progress=state.progress, - note_asset_warnings=notes_result.asset_warnings or [], - export_report=state.export_report, - study_pack_report=state.study_pack_report, - artifact_registry=state.artifacts, - api_concurrency=state.api_concurrency, - ) - state.artifacts.write_json("run_summary", "run_summary.json", run_summary) + state.artifacts.write_json("run_summary", "run_summary.json", _build_run_summary(state)) def _print_build_outputs(state: BuildState) -> None: - args = state.args - if args.quiet: + if state.args.quiet: return - notes_result = _require_notes_result(state) output_root = state.output_root print(f"SlideNote build complete: {output_root}") - print(f"- content: {output_root / 'content.json'}") - print(f"- notes: {output_root / 'notes.md'}") - print(f"- coverage: {output_root / 'coverage.md'}") - print(f"- quality: {output_root / 'quality_report.json'}") - print(f"- sources: {output_root / 'source_map.json'}") - print(f"- element IR: {output_root / 'element_ir.json'}") - print(f"- modalities: {output_root / 'page_modalities.json'}") - print(f"- tables: {output_root / 'table_understanding.json'}") - print(f"- semantic: {output_root / 'semantic_layout.json'}") - print(f"- sections: {output_root / 'sections.json'}") - if state.deck_brief_report is not None: - print(f"- deck brief: {output_root / 'deck_brief.json'}") - if state.content_guard_report is not None: - print(f"- content guard: {output_root / 'content_guard.json'}") - if state.deck_understanding_report is not None: - print(f"- deck understanding: {output_root / 'deck_understanding.json'}") - if state.page_understanding_report is not None: - print(f"- page understanding: {output_root / 'page_understanding.json'}") - print(f"- progress: {state.progress.path}") - print(f"- summary: {output_root / 'run_summary.json'}") - if state.image_importance_report is not None: - print(f"- image rank: {output_root / 'image_importance.json'}") - if state.composite_figure_report is not None: - print(f"- composite figs: {output_root / 'composite_figures.json'}") - if notes_result.llm_usage is not None: - print(f"- llm use: {output_root / 'llm_usage.json'}") - if notes_result.page_notes is not None: - print(f"- page notes: {output_root / 'page_notes.md'}") - print(f"- page json: {output_root / 'page_notes.json'}") - if notes_result.weave_report is not None: - print(f"- weave: {output_root / 'weave_report.json'}") - if state.figure_report is not None: - print(f"- figures: {output_root / 'figures.json'}") - print(f"- fig use: {output_root / 'figure_usage.json'}") - if state.figure_grounding_report is not None: - print(f"- fig ground: {output_root / 'figure_grounding.json'}") - if state.ocr_report is not None: - print(f"- ocr: {output_root / 'ocr.json'}") - print(f"- ocr use: {output_root / 'ocr_usage.json'}") - if state.vision_report is not None: - print(f"- visuals: {output_root / 'visuals.json'}") - print(f"- vision: {output_root / 'vision_usage.json'}") - if state.export_report is not None: - print(f"- exports: {output_root / 'export_report.json'}") - notes_zip = output_root / "notes.zip" - if notes_zip.exists(): - print(f"- note package: {notes_zip}") - print(" Reminder: Markdown notes are inside notes.zip with image assets.") + artifacts = state.artifacts.as_summary() + width = max((len(name) for name in artifacts), default=0) + for name, path in artifacts.items(): + print(f"- {name.ljust(width)} {output_root / path}") + if "notes_zip" in artifacts: + print(" Reminder: Markdown notes are inside notes.zip with image assets.") for stage in _slowest_stages(state.progress, limit=3): print(f"- slow stage: {stage['name']} {stage['elapsed_seconds']:.1f}s") @@ -596,25 +526,35 @@ def _require_report(report: dict[str, Any] | None, name: str) -> dict[str, Any]: return report -BUILD_STAGES = ( - _stage_parse, - _stage_modality, - _stage_table_understanding, - _stage_semantic_layout, - _stage_composite_figures, - _stage_figure_crop, - _stage_image_importance, - _stage_ocr, - _stage_vision, - _stage_figure_grounding, - _stage_sections, - _stage_deck_brief, - _stage_content_guard, - _stage_understanding, - _stage_export_content, - _stage_notes, - _stage_coverage, - _stage_quality_report, - _stage_export, - _stage_run_summary, +BUILD_PHASES = ( + BuildPhase("ingest", ( + BuildStep("parse", _stage_parse), + )), + BuildPhase("understand", ( + BuildStep("modality", _stage_modality), + BuildStep("table_understanding", _stage_table_understanding), + BuildStep("semantic_layout", _stage_semantic_layout), + BuildStep("composite_figures", _stage_composite_figures, lambda state: state.args.composite_figures != "off"), + BuildStep("figure_crop", _stage_figure_crop, lambda state: state.args.figure_crop == "vision" or (state.args.figure_crop == "auto" and state.args.vision != "off")), + BuildStep("image_importance", _stage_image_importance, lambda state: state.args.image_ranking != "off"), + BuildStep("ocr", _stage_ocr, lambda state: state.args.ocr != "off"), + BuildStep("vision", _stage_vision, lambda state: state.args.vision != "off"), + BuildStep("figure_grounding", _stage_figure_grounding, lambda state: state.args.figure_grounding != "off"), + BuildStep("sections", _stage_sections), + BuildStep("deck_brief", _stage_deck_brief, lambda state: _should_build_deck_brief(state.args)), + BuildStep("content_guard", _stage_content_guard, lambda state: state.args.content_guard != "off"), + BuildStep("understanding", _stage_understanding), + BuildStep("export_content", _stage_export_content), + )), + BuildPhase("write", ( + BuildStep("notes", _stage_notes), + )), + BuildPhase("guard", ( + BuildStep("coverage", _stage_coverage), + BuildStep("quality_report", _stage_quality_report), + )), + BuildPhase("export", ( + BuildStep("export", _stage_export, lambda state: bool(state.export_formats)), + BuildStep("run_summary", _stage_run_summary, tracks_progress=False), + )), ) diff --git a/slidenote/build/state.py b/slidenote/build/state.py index 652b682..0788f63 100644 --- a/slidenote/build/state.py +++ b/slidenote/build/state.py @@ -5,10 +5,12 @@ from pathlib import Path from typing import Any +from slidenote.build.artifacts import remove_stale_build_artifacts +from slidenote.build.errors import UserFacingConfigError from slidenote.build.config import _parse_slide_ranges, _resolve_api_concurrency, _resolve_cache_dirs from slidenote.models import Deck from slidenote.notes import NoteGenerationResult -from slidenote.pipeline import ArtifactRegistry, BuildContext +from slidenote.pipeline import ArtifactRegistry from slidenote.progress import ProgressReporter from slidenote.utils import ensure_clean_dir @@ -24,7 +26,6 @@ class BuildState: api_concurrency: dict[str, int] cache_dirs: dict[str, Path | None] artifacts: ArtifactRegistry - build_context: BuildContext export_formats: list[str] deck: Deck | None = None modality_report: dict[str, Any] | None = None @@ -46,34 +47,32 @@ class BuildState: coverage_report: dict[str, Any] | None = None quality_report: dict[str, Any] | None = None source_map: dict[str, Any] | None = None - study_pack_report: dict[str, Any] | None = None export_report: dict[str, Any] | None = None export_exit_code: int = 0 +def resolve_progress_path(args: argparse.Namespace) -> Path: + return (args.progress_json or (args.out / "progress.json")).resolve() + + def create_build_state(args: argparse.Namespace, export_formats: list[str]) -> BuildState: input_path = args.input.resolve() output_root = args.out.resolve() if not input_path.exists(): - raise FileNotFoundError(input_path) + raise UserFacingConfigError(f"Input file not found: {input_path}") + try: + refresh_slide_ids = _parse_slide_ranges(args.refresh_pages) + except ValueError as exc: + raise UserFacingConfigError(f"Invalid refresh page range `{args.refresh_pages}`: {exc}") from exc ensure_clean_dir(output_root) - progress = ProgressReporter((args.progress_json or (output_root / "progress.json")).resolve(), quiet=args.quiet) - refresh_slide_ids = _parse_slide_ranges(args.refresh_pages) + progress_path = resolve_progress_path(args) + remove_stale_build_artifacts(output_root, keep=(input_path, progress_path)) + progress = ProgressReporter(progress_path, quiet=args.quiet) concurrency = max(1, args.concurrency) api_concurrency = _resolve_api_concurrency(args) cache_dirs = _resolve_cache_dirs(args, output_root) artifacts = ArtifactRegistry(output_root) artifacts.register("progress", progress.path) - build_context = BuildContext( - args=args, - input_path=input_path, - output_root=output_root, - progress=progress, - cache_dirs=cache_dirs, - refresh_slide_ids=refresh_slide_ids, - concurrency=concurrency, - artifacts=artifacts, - ) return BuildState( args=args, input_path=input_path, @@ -84,6 +83,5 @@ def create_build_state(args: argparse.Namespace, export_formats: list[str]) -> B api_concurrency=api_concurrency, cache_dirs=cache_dirs, artifacts=artifacts, - build_context=build_context, export_formats=export_formats, ) diff --git a/slidenote/build/summary.py b/slidenote/build/summary.py index c77b514..8345765 100644 --- a/slidenote/build/summary.py +++ b/slidenote/build/summary.py @@ -1,66 +1,89 @@ from __future__ import annotations -import argparse -from pathlib import Path from typing import Any -from slidenote.build.config import _resolve_api_concurrency from slidenote.build.progress import _stage_metrics from slidenote.content_guard import content_guard_warnings -from slidenote.exporting import export_warnings, parse_export_formats -from slidenote.pipeline import ArtifactRegistry -from slidenote.progress import ProgressReporter -from slidenote.utils import display_path +from slidenote.exporting import export_warnings +from slidenote.llm import resolve_provider_runtime +# Artifact keys always present in run_summary["artifacts"] (None when the +# stage did not run); actual paths come from the ArtifactRegistry. +RUN_SUMMARY_ARTIFACT_KEYS = ( + "content", + "element_ir", + "notes", + "note_assets", + "coverage", + "quality_report", + "source_map", + "progress", + "run_summary", + "page_modalities", + "table_understanding", + "semantic_layout", + "image_importance", + "composite_figures", + "sections", + "deck_brief", + "deck_brief_markdown", + "deck_understanding", + "page_understanding", + "content_guard", + "export_report", + "notes_zip", + "notes_toc", + "notes_docx", + "notes_pdf", + "notes_latex", + "figures", + "figure_usage", + "figure_grounding", + "llm_usage", + "page_notes", + "page_notes_markdown", + "weave_report", + "teaching_enrichment", + "ocr_usage", + "vision_usage", +) -def _build_run_summary( - args: argparse.Namespace, - input_path: Path, - output_root: Path, - deck, - modality_report: dict[str, Any], - table_understanding_report: dict[str, Any], - semantic_layout_report: dict[str, Any], - image_importance_report: dict[str, Any] | None, - section_report: dict[str, Any], - deck_brief_report: dict[str, Any] | None, - deck_understanding_report: dict[str, Any] | None, - page_understanding_report: dict[str, Any] | None, - composite_figure_report: dict[str, Any] | None, - figure_report: dict[str, Any] | None, - figure_grounding_report: dict[str, Any] | None, - ocr_report: dict[str, Any] | None, - vision_report: dict[str, Any] | None, - content_guard_report: dict[str, Any] | None, - llm_usage: dict[str, Any] | None, - coverage_report: dict[str, Any], - quality_report: dict[str, Any] | None, - source_map: dict[str, Any], - cache_dirs: dict[str, Path | None], - refresh_slide_ids: set[int], - progress: ProgressReporter, - note_asset_warnings: list[str], - export_report: dict[str, Any] | None = None, - study_pack_report: dict[str, Any] | None = None, - artifact_registry: ArtifactRegistry | None = None, - api_concurrency: dict[str, int] | None = None, -) -> dict[str, Any]: + +def _summary_of(report: dict[str, Any] | None) -> Any: + return report.get("summary") if report else None + + +def _build_run_summary(state: "BuildState") -> dict[str, Any]: + args = state.args + deck = state.deck + notes_result = state.notes_result + coverage_report = state.coverage_report or {} + source_map = state.source_map or {} + llm_usage = notes_result.llm_usage if notes_result else None + try: + text_runtime = resolve_provider_runtime(args.provider, model=args.model, base_url=args.base_url) + except (RuntimeError, ValueError): + # Local builds can run without a configured text model. Preserve any + # explicit values while leaving the unresolved fields empty. + text_runtime = {} pages = deck.pages - images_count = sum(len(page.images) for page in pages) + registered = state.artifacts.as_summary() return { "schema_version": 1, - "source_path": str(input_path), + "source_path": str(state.input_path), "source_type": deck.source_type, - "output_root": str(output_root), + "output_root": str(state.output_root), "run": { "preset": getattr(args, "preset", "lecture"), "provider": getattr(args, "provider", "deepseek"), + "model": text_runtime.get("model") or args.model, + "base_url": text_runtime.get("base_url") or args.base_url, "vision": getattr(args, "vision", "auto"), "speed_mode": args.speed_mode, - "concurrency": max(1, args.concurrency), - "api_concurrency": api_concurrency or _resolve_api_concurrency(args), - "refresh_slide_ids": sorted(refresh_slide_ids), - "cache_dirs": {name: str(path) if path else None for name, path in cache_dirs.items()}, + "concurrency": state.concurrency, + "api_concurrency": state.api_concurrency, + "refresh_slide_ids": sorted(state.refresh_slide_ids), + "cache_dirs": {name: str(path) if path else None for name, path in state.cache_dirs.items()}, "parser": getattr(args, "parser", "auto"), "asset_mode": args.asset_mode, "source_display": args.source_display, @@ -74,7 +97,7 @@ def _build_run_summary( "teaching_enrichment": args.teaching_enrichment, "deck_brief": args.deck_brief, "content_guard": args.content_guard, - "export": parse_export_formats(args.export), + "export": list(state.export_formats), "export_toc": args.export_toc, "weave_dedup": args.weave_dedup, "page_neighborhood": args.page_neighborhood, @@ -92,114 +115,58 @@ def _build_run_summary( "pages": len(pages), "text_blocks": sum(len(page.text_blocks) for page in pages), "tables": sum(len(page.tables) for page in pages), - "images": images_count, + "images": sum(len(page.images) for page in pages), "figure_crops": sum(1 for page in pages for image in page.images if image.role == "figure_crop"), "composite_figures": sum(1 for page in pages for image in page.images if image.role == "composite_figure"), "page_screenshots": sum(1 for page in pages if page.page_screenshot), }, - "composite_figures": composite_figure_report.get("summary") if composite_figure_report else None, - "figure_crop": figure_report.get("summary") if figure_report else None, - "figure_grounding": figure_grounding_report.get("summary") if figure_grounding_report else None, - "page_modalities": modality_report.get("summary") if modality_report else None, - "table_understanding": table_understanding_report.get("summary") if table_understanding_report else None, - "semantic_layout": semantic_layout_report.get("summary") if semantic_layout_report else None, - "image_importance": image_importance_report.get("summary") if image_importance_report else None, - "sections": section_report.get("summary") if section_report else None, - "deck_brief": deck_brief_report.get("summary") if deck_brief_report else None, - "deck_understanding": deck_understanding_report.get("summary") if deck_understanding_report else None, - "page_understanding": page_understanding_report.get("summary") if page_understanding_report else None, - "ocr": ocr_report.get("summary") if ocr_report else None, - "vision": vision_report.get("summary") if vision_report else None, - "content_guard": content_guard_report.get("summary") if content_guard_report else None, - "study_pack": study_pack_report.get("summary") if study_pack_report else None, - "llm": llm_usage.get("summary") if llm_usage else None, - "quality": quality_report.get("summary") if quality_report else None, + "composite_figures": _summary_of(state.composite_figure_report), + "figure_crop": _summary_of(state.figure_report), + "figure_grounding": _summary_of(state.figure_grounding_report), + "page_modalities": _summary_of(state.modality_report), + "table_understanding": _summary_of(state.table_understanding_report), + "semantic_layout": _summary_of(state.semantic_layout_report), + "image_importance": _summary_of(state.image_importance_report), + "sections": _summary_of(state.section_report), + "deck_brief": _summary_of(state.deck_brief_report), + "deck_understanding": _summary_of(state.deck_understanding_report), + "page_understanding": _summary_of(state.page_understanding_report), + "ocr": _summary_of(state.ocr_report), + "vision": _summary_of(state.vision_report), + "content_guard": _summary_of(state.content_guard_report), + "llm": _summary_of(llm_usage), + "quality": _summary_of(state.quality_report), "coverage": { - "total": coverage_report.get("total"), - "covered": coverage_report.get("covered"), - "missing": coverage_report.get("missing"), - "coverage_ratio": coverage_report.get("coverage_ratio"), - "page_coverage": coverage_report.get("page_coverage"), - "trace_coverage": coverage_report.get("trace_coverage"), - "visible_coverage": coverage_report.get("visible_coverage"), - "required_visible_coverage": coverage_report.get("required_visible_coverage"), - "marker_only": coverage_report.get("marker_only"), - "structural_marker_only": coverage_report.get("structural_marker_only"), + key: coverage_report.get(key) + for key in ( + "total", + "covered", + "missing", + "coverage_ratio", + "page_coverage", + "trace_coverage", + "visible_coverage", + "required_visible_coverage", + "marker_only", + "structural_marker_only", + ) }, "source_map": { "note_blocks": len(source_map.get("note_blocks", [])), "default_display_mode": source_map.get("default_display_mode"), }, - "stage_timings": _stage_metrics(progress), + "stage_timings": _stage_metrics(state.progress), "warnings": { - "note_assets": note_asset_warnings, - "content_guard": content_guard_warnings(content_guard_report), - "study_pack": study_pack_report.get("warnings") if study_pack_report else [], - "export": export_warnings(export_report), + "config": list(getattr(args, "_config_warnings", None) or []), + "note_assets": list((notes_result.asset_warnings if notes_result else None) or []), + "notes": list((notes_result.generation_warnings if notes_result else None) or []), + "content_guard": content_guard_warnings(state.content_guard_report), + "export": export_warnings(state.export_report), }, "artifacts": { - "content": "content.json", - "element_ir": "element_ir.json", - "notes": "notes.md", - "note_assets": "notes.assets" if args.asset_mode == "bundle" else None, - "coverage": "coverage.md", - "quality_report": "quality_report.json" if quality_report else None, - "source_map": "source_map.json", - "progress": display_path(progress.path, output_root), - "run_summary": "run_summary.json", - "page_modalities": "page_modalities.json", - "table_understanding": "table_understanding.json", - "semantic_layout": "semantic_layout.json", - "image_importance": "image_importance.json" if image_importance_report else None, - "composite_figures": "composite_figures.json" if composite_figure_report else None, - "sections": "sections.json", - "deck_brief": "deck_brief.json" if deck_brief_report else None, - "deck_brief_markdown": "deck_brief.md" if deck_brief_report else None, - "deck_understanding": "deck_understanding.json" if deck_understanding_report else None, - "page_understanding": "page_understanding.json" if page_understanding_report else None, - "content_guard": "content_guard.json" if content_guard_report else None, - "study_pack": "study_pack.json" if study_pack_report else None, - "review_markdown": "review.md" if study_pack_report and study_pack_report.get("review") else None, - "exam_markdown": "exam.md" if study_pack_report and study_pack_report.get("exam") else None, - "exam_json": "exam.json" if study_pack_report and study_pack_report.get("exam") else None, - "exam_html": "exam.html" if study_pack_report and study_pack_report.get("exam") else None, - "section_study_pack": "section_study_pack.json" if study_pack_report and study_pack_report.get("section_study_pack") else None, - "exam_review_pack": "exam_review_pack.json" if study_pack_report and study_pack_report.get("exam_review_pack") else None, - "final_exam_markdown": "final_exam.md" if study_pack_report and study_pack_report.get("final_exam") else None, - "final_exam_answers": "final_exam.answers.md" if study_pack_report and study_pack_report.get("final_exam") else None, - "wrong_answer_review_prompt": "wrong_answer_review_prompt.md" if study_pack_report and study_pack_report.get("wrong_answer_review") else None, - "export_report": "export_report.json" if export_report else None, - "notes_zip": _export_artifact_path(export_report, "markdown-zip"), - "notes_toc": _export_artifact_path(export_report, "markdown-toc"), - "notes_docx": _export_artifact_path(export_report, "docx"), - "notes_pdf": _export_artifact_path(export_report, "pdf"), - "notes_latex": _export_artifact_path(export_report, "latex"), - "figures": "figures.json" if figure_report else None, - "figure_usage": "figure_usage.json" if figure_report else None, - "figure_grounding": "figure_grounding.json" if figure_grounding_report else None, - "llm_usage": "llm_usage.json" if llm_usage else None, - "page_notes": "page_notes.json" if getattr(args, "note_strategy", "direct") == "lecture-weave" and llm_usage else None, - "page_notes_markdown": "page_notes.md" if getattr(args, "note_strategy", "direct") == "lecture-weave" and llm_usage else None, - "weave_report": "weave_report.json" if getattr(args, "note_strategy", "direct") == "lecture-weave" and llm_usage else None, - "teaching_enrichment": "teaching_enrichment.json" if llm_usage and (llm_usage.get("summary") or {}).get("teaching_enrichment_contexts") else None, - "ocr_usage": "ocr_usage.json" if ocr_report else None, - "vision_usage": "vision_usage.json" if vision_report else None, - "registered": artifact_registry.as_summary() if artifact_registry else {}, + **{key: None for key in RUN_SUMMARY_ARTIFACT_KEYS}, + **registered, + "registered": registered, }, - "progress": progress.snapshot(), + "progress": state.progress.snapshot(), } - - -def _export_artifact_path(export_report: dict[str, Any] | None, fmt: str) -> str | None: - if not export_report: - return None - results = export_report.get("results") - if not isinstance(results, list): - return None - for result in results: - if not isinstance(result, dict): - continue - if result.get("format") == fmt and result.get("status") == "ok": - path = result.get("path") - return str(path) if path else None - return None diff --git a/slidenote/composite_figures.py b/slidenote/composite_figures.py index 92e6c81..b594c38 100644 --- a/slidenote/composite_figures.py +++ b/slidenote/composite_figures.py @@ -8,6 +8,7 @@ from PIL import Image +from slidenote.figures import next_figure_index from slidenote.figure_grounding import normalized_element_bbox, normalized_image_bbox from slidenote.image_assets import image_metadata from slidenote.llm_cache import utc_now_iso @@ -130,7 +131,7 @@ def _process_page( return base_record used_child_ids: set[str] = set() - next_index = _next_figure_index(page) + next_index = next_figure_index(page) try: with Image.open(screenshot_path) as source_image: width, height = source_image.width, source_image.height @@ -347,18 +348,6 @@ def _element_bbox(deck: Deck, page: SlidePage, element: TextBlock | TableBlock) return normalized_element_bbox(deck, page, element) -def _next_figure_index(page: SlidePage) -> int: - next_index = 1 - prefix = f"s{page.slide_id}_fig" - for image in page.images: - if not image.id.startswith(prefix): - continue - suffix = image.id[len(prefix) :] - if suffix.isdigit(): - next_index = max(next_index, int(suffix) + 1) - return next_index - - def _expand_bbox(bbox: list[float], margin: float) -> list[float]: return [ round(max(0.0, bbox[0] - margin), 4), diff --git a/slidenote/content_guard.py b/slidenote/content_guard.py index 6a1b882..10c5d53 100644 --- a/slidenote/content_guard.py +++ b/slidenote/content_guard.py @@ -13,6 +13,8 @@ from slidenote.utils import ( as_float, display_path, + looks_like_outline_page, + parse_json_object, preview, ) @@ -22,6 +24,20 @@ CONTENT_REPAIR_PROMPT_VERSION = "content-repair-v2" REQUIRED_CONFIDENCE_THRESHOLD = 0.7 +_STRUCTURAL_TITLES = {"目录", "课程目录", "本章目录", "章节导航", "contents", "outline", "agenda"} +_STRUCTURAL_LABELS = {"目录", "课程目录", "本章目录", "章节导航", "contents", "outline"} +_COVER_MARKERS = {"讲师", "教师", "教授", "联系邮箱", "邮箱", "主页", "email", "homepage", "http", "www"} +_DEFINITION_SIGNALS = ["定义", "称为", "是指", "definition", "means", "called"] +_CONDITION_SIGNALS = ["如果", "只有", "必须", "条件", "当且仅当", "because", "if ", "when ", "only if", "must"] +# "当" alone matches 当前/相当/应当; require the conditional "当……时" shape. +_CONDITION_PATTERNS = [re.compile(r"(?:^|[，,。；;：:\s])当(?![前今下中年天地然作做成即场])[^，,。；;]{1,40}?时")] +_URL_RE = re.compile(r"(?:https?://|www\.)\S+", flags=re.IGNORECASE) +_HTML_TAG_RE = re.compile(r"]+))*\s*/?>") +_FORMULA_OPERAND = r"(?:\b[A-Za-z]\w{0,2}(?:\([^()]{0,20}\))?|\b\d+(?:\.\d+)?|\))" +_FORMULA_RELATION_RE = re.compile( + rf"{_FORMULA_OPERAND}\s*(?:==|<=|>=|!=|=|<|>)\s*[-+]?(?:[A-Za-z]\w{{0,2}}\b|\d|\()" +) + @dataclass(frozen=True, slots=True) class GuardCandidate: @@ -167,14 +183,15 @@ def structural_slide_ids(report: dict[str, Any] | None) -> set[int] | None: } +def is_required_item(item: dict[str, Any], confidence_threshold: float = REQUIRED_CONFIDENCE_THRESHOLD) -> bool: + """Single rule for "must be explained": flagged must_explain AND confident enough.""" + return bool(item.get("must_explain")) and as_float(item.get("confidence"), 0.0) >= confidence_threshold + + def required_item_ids(report: dict[str, Any] | None, confidence_threshold: float = REQUIRED_CONFIDENCE_THRESHOLD) -> set[str]: if not report: return set() - ids: set[str] = set() - for item in report.get("items", []): - if item.get("must_explain") and as_float(item.get("confidence"), 0.0) >= confidence_threshold: - ids.add(str(item.get("element_id"))) - return ids + return {str(item.get("element_id")) for item in report.get("items", []) if is_required_item(item, confidence_threshold)} def required_items_for_slides( @@ -189,7 +206,7 @@ def required_items_for_slides( slide_id = int(item.get("slide_id") or 0) if slide_ids is not None and slide_id not in slide_ids: continue - if item.get("must_explain") and as_float(item.get("confidence"), 0.0) >= confidence_threshold: + if is_required_item(item, confidence_threshold): result.append(item) return result @@ -289,7 +306,8 @@ def _text_candidate(page: SlidePage, block: TextBlock) -> GuardCandidate | None: text = " ".join(block.content.split()) if not text: return None - lowered = text.lower() + # Keyword signals ignore URLs ("example.com", "?if=") so links are not learning content. + signal_text = _URL_RE.sub(" ", text) role = "concept" confidence = 0.45 must = False @@ -302,17 +320,17 @@ def _text_candidate(page: SlidePage, block: TextBlock) -> GuardCandidate | None: confidence = 0.84 must = True reasons.append("formula_like") - elif _contains_any(text, ["定义", "称为", "是指", "consistency", "definition", "means", "called"]): + elif _contains_any(signal_text, _DEFINITION_SIGNALS): role = "definition" confidence = 0.78 must = True reasons.append("definition_signal") - elif _contains_any(text, ["如果", "当", "只有", "必须", "条件", "because", "if ", "when ", "only if", "must"]): + elif _contains_any(signal_text, _CONDITION_SIGNALS) or any(pattern.search(signal_text) for pattern in _CONDITION_PATTERNS): role = "condition" confidence = 0.74 must = True reasons.append("condition_signal") - elif _contains_any(text, ["例如", "案例", "example", "e.g."]): + elif _contains_any(signal_text, ["例如", "案例", "example", "e.g."]): role = "example" confidence = 0.68 must = True @@ -372,7 +390,7 @@ def _build_local_report(deck: Deck, candidates: list[GuardCandidate], mode: str) pages = [] for index, page in enumerate(deck.pages): page_items = by_slide.get(page.slide_id, []) - structural = _looks_like_structural_page(page, index) + structural = looks_like_structural_page(page, index) has_required = any(item.get("must_explain") for item in page_items) page_role = "mixed" if structural and has_required else "structural" if structural else "content" pages.append( @@ -437,8 +455,8 @@ def _merge_llm_report(deck: Deck, candidates: list[GuardCandidate], parsed: dict pages = [] for index, page in enumerate(deck.pages): page_items = by_slide.get(page.slide_id, []) - local_structural = _looks_like_structural_page(page, index) - has_required = any(item.get("must_explain") and as_float(item.get("confidence"), 0.0) >= REQUIRED_CONFIDENCE_THRESHOLD for item in page_items) + local_structural = looks_like_structural_page(page, index) + has_required = any(is_required_item(item) for item in page_items) role = page_role_by_slide.get(page.slide_id) if role not in {"structural", "content", "mixed"}: role = "mixed" if local_structural and has_required else "structural" if local_structural else "content" @@ -484,16 +502,8 @@ def _classification_prompt(deck: Deck, candidates: list[GuardCandidate]) -> str: def _parse_guard_json(text: str) -> dict[str, Any] | None: - cleaned = text.strip() - if cleaned.startswith("```"): - cleaned = cleaned.strip("`") - if cleaned.lower().startswith("json"): - cleaned = cleaned[4:].strip() - try: - parsed = json.loads(cleaned) - except json.JSONDecodeError: - return None - if not isinstance(parsed, dict) or not isinstance(parsed.get("pages"), list): + parsed = parse_json_object(text) + if parsed is None or not isinstance(parsed.get("pages"), list): return None return parsed @@ -525,7 +535,7 @@ def _item_record(candidate: GuardCandidate) -> dict[str, Any]: def _summary(pages: list[dict[str, Any]], items: list[dict[str, Any]]) -> dict[str, Any]: - required = [item for item in items if item.get("must_explain") and as_float(item.get("confidence"), 0.0) >= REQUIRED_CONFIDENCE_THRESHOLD] + required = [item for item in items if is_required_item(item)] return { "pages_total": len(pages), "structural_pages": sum(1 for page in pages if page.get("page_role") == "structural"), @@ -556,38 +566,38 @@ def _llm_record( "cache_key": cache_key, "cache_file": display_path(cache_path, output_root), "llm_call": llm_call, - "input_tokens": usage.get("input_tokens"), - "output_tokens": usage.get("output_tokens"), - "total_tokens": usage.get("total_tokens"), - "provider_cached_input_tokens": usage.get("provider_cached_input_tokens"), + # Cache hits cost nothing this run; report zero like deck_brief/study_pack. + "input_tokens": usage.get("input_tokens") if llm_call else 0, + "output_tokens": usage.get("output_tokens") if llm_call else 0, + "total_tokens": usage.get("total_tokens") if llm_call else 0, + "provider_cached_input_tokens": usage.get("provider_cached_input_tokens") if llm_call else 0, } -def _looks_like_structural_page(page: SlidePage, index: int) -> bool: +def looks_like_structural_page(page: SlidePage, index: int) -> bool: + """Shared structural-page heuristic (cover, TOC, navigation); also used by coverage.""" title = page.title or "" text = "\n".join([title, *(block.content for block in page.text_blocks)]) - normalized_title = _normalize_text_key(title) - normalized_text = _normalize_text_key(text) - if normalized_title in {"目录", "课程目录", "本章目录", "章节导航", "contents", "outline", "agenda"}: + if normalize_text_key(title) in _STRUCTURAL_TITLES: return True - if index == 0 and any(marker in normalized_text for marker in {"讲师", "教师", "教授", "邮箱", "email", "homepage", "www", "http"}): + if index == 0 and any(marker in normalize_text_key(text) for marker in _COVER_MARKERS): return True - return _looks_like_structural_text(text) + if any(normalize_text_key(line) in _STRUCTURAL_LABELS for line in text.splitlines()[:4]): + return True + return looks_like_outline_page(text) def _looks_like_structural_text(text: str) -> bool: - normalized = _normalize_text_key(text) - if normalized in {"目录", "课程目录", "本章目录", "章节导航", "contents", "outline", "agenda"}: - return True - lines = [line.strip() for line in text.splitlines() if line.strip()] - numbered = [line for line in lines if re.match(r"^\s*(?:\d+|[一二三四五六七八九十])\s*[.、．]", line)] - return len(numbered) >= 3 and sum(len(line) for line in numbered) <= 260 + return normalize_text_key(text) in _STRUCTURAL_TITLES or looks_like_outline_page(text) def _looks_like_formula(text: str) -> bool: - if re.search(r"[=<>≤≥∑∀∃]|\\frac|\\sum|\\forall|O\([^)]+\)", text): + # URLs carry "=" in query strings and HTML tags carry "<>"; neither is math. + text = _HTML_TAG_RE.sub(" ", _URL_RE.sub(" ", text)) + if re.search(r"[≤≥≠≈∑∫∀∃]|\\frac|\\sum|\\forall|\\int|\bO\([^)]+\)", text): return True - return bool(re.search(r"\b[A-Za-z]\w*\s*(?:=|<=|>=|<|>)\s*[\w\d]", text)) + # A relational operator between short math-like operands: "x = 3", "a+b= 0". + return bool(_FORMULA_RELATION_RE.search(text)) def _contains_any(text: str, needles: list[str]) -> bool: @@ -595,7 +605,7 @@ def _contains_any(text: str, needles: list[str]) -> bool: return any(needle.lower() in lowered for needle in needles) -def _normalize_text_key(value: str) -> str: +def normalize_text_key(value: str) -> str: return re.sub(r"[\s:：,，.。;；、\-_（）()<>]+", "", value).lower() diff --git a/slidenote/costing.py b/slidenote/costing.py index 443a112..2157dc7 100644 --- a/slidenote/costing.py +++ b/slidenote/costing.py @@ -6,13 +6,32 @@ from pathlib import Path from typing import Any +# Fallback only; override with "exchange_rates": {"CNY": ...} in the pricing JSON. +DEFAULT_CNY_PER_USD = 7.2 DEFAULT_PRICING = { "currency": "USD", - "exchange_rates": {"USD": 1.0, "CNY": 7.2}, + "exchange_rates": {"USD": 1.0, "CNY": DEFAULT_CNY_PER_USD}, "models": {}, "ocr": {}, } +# Usage files written by stages that report their own call records. +USAGE_FILES = ( + ("llm_usage.json", "llm"), + ("vision_usage.json", "vision"), + ("figure_usage.json", "figure"), + ("ocr_usage.json", "ocr"), +) +# Stage reports that embed model usage in their own shape: +# (file, bucket name, provider/model block, calls key, cache-hits key). +EMBEDDED_USAGE_REPORTS = ( + ("sections.json", "sections", "llm", "llm_call", "local_cache_hits"), + ("deck_brief.json", "deck_brief", "llm", "llm_call", "local_cache_hits"), + ("content_guard.json", "content_guard", "llm", None, None), + ("semantic_layout.json", "semantic_layout", "vision_enhancement", "vision_calls", "vision_cache_hits"), + ("figure_grounding.json", "figure_grounding", "vision_grounding", "vision_calls", "vision_cache_hits"), +) + @dataclass(slots=True) class UsageBucket: name: str @@ -45,15 +64,22 @@ def read_json(path: Path) -> dict[str, Any] | None: def load_pricing(path: Path | None = None) -> dict[str, Any]: - if path and path.exists(): - try: - data = json.loads(path.read_text(encoding="utf-8")) - merged = dict(DEFAULT_PRICING) - merged.update(data) - return merged - except Exception: - return DEFAULT_PRICING - return DEFAULT_PRICING + merged = {key: dict(value) if isinstance(value, dict) else value for key, value in DEFAULT_PRICING.items()} + if not (path and path.exists()): + return merged + try: + data = json.loads(path.read_text(encoding="utf-8")) + except Exception: + return merged + if not isinstance(data, dict): + return merged + for key, value in data.items(): + if key == "exchange_rates" and isinstance(value, dict): + # Merge so a pricing file that only sets EUR keeps the CNY default. + merged["exchange_rates"].update(value) + else: + merged[key] = value + return merged def _int(value: Any) -> int: @@ -147,19 +173,58 @@ def bucket_from_report(name: str, report: dict[str, Any] | None, pricing: dict[s return bucket +def _embedded_usage_report( + report: dict[str, Any] | None, + runtime_key: str, + calls_key: str | None, + hits_key: str | None, +) -> dict[str, Any] | None: + """Normalize a stage report with embedded model usage to the usage-file shape.""" + if not isinstance(report, dict): + return None + runtime = report.get(runtime_key) + if not isinstance(runtime, dict) or not runtime.get("provider"): + return None + if calls_key is None: + # content_guard keeps a single call record in its runtime block. + source = runtime + calls = 1 if runtime.get("llm_call") else 0 + hits = 1 if runtime.get("cache_status") == "local_hit" else 0 + else: + source = report.get("summary") if isinstance(report.get("summary"), dict) else {} + raw_calls = source.get(calls_key) + # sections/deck_brief store a single boolean `llm_call`. + calls = int(raw_calls) if isinstance(raw_calls, bool) else _int(raw_calls) + hits = _int(source.get(hits_key)) if hits_key else 0 + billed = calls > 0 + return { + "provider": runtime.get("provider"), + "model": runtime.get("model"), + "summary": { + "llm_calls": calls, + "local_cache_hits": hits, + # Token counts of cache hits describe the original call, not this run. + "input_tokens": source.get("input_tokens") if billed else 0, + "output_tokens": source.get("output_tokens") if billed else 0, + "total_tokens": source.get("total_tokens") if billed else 0, + "provider_cached_input_tokens": source.get("provider_cached_input_tokens") if billed else 0, + }, + } + + def build_cost_report(output_dir: Path, pricing_path: Path | None = None, currency: str = "USD") -> dict[str, Any]: output_dir = output_dir.resolve() pricing = load_pricing(pricing_path) buckets = [] - for filename, name in [ - ("llm_usage.json", "llm"), - ("vision_usage.json", "vision"), - ("figure_usage.json", "figure"), - ("ocr_usage.json", "ocr"), - ]: + for filename, name in USAGE_FILES: bucket = bucket_from_report(name, read_json(output_dir / filename), pricing) if bucket: buckets.append(bucket) + for filename, name, runtime_key, calls_key, hits_key in EMBEDDED_USAGE_REPORTS: + usage = _embedded_usage_report(read_json(output_dir / filename), runtime_key, calls_key, hits_key) + bucket = bucket_from_report(name, usage, pricing) + if bucket: + buckets.append(bucket) total = UsageBucket(name="total") for bucket in buckets: diff --git a/slidenote/coverage.py b/slidenote/coverage.py index b31c65d..be57a54 100644 --- a/slidenote/coverage.py +++ b/slidenote/coverage.py @@ -3,13 +3,16 @@ import re from dataclasses import dataclass -from slidenote.content_guard import required_item_ids, structural_slide_ids as guard_structural_slide_ids +from slidenote.content_guard import ( + looks_like_structural_page, + required_item_ids, + structural_slide_ids as guard_structural_slide_ids, +) from slidenote.figure_grounding import note_candidate_images from slidenote.ir import iter_expected_source_elements from slidenote.models import Deck, ImageAsset, SlidePage from slidenote.utils import ( escape_md, - looks_like_outline_page, preview, source_tokens, ) @@ -301,65 +304,10 @@ def _structural_slide_ids(deck: Deck) -> set[int]: return { page.slide_id for index, page in enumerate(deck.pages) - if _looks_like_structural_page(page, index) - } - - -def _looks_like_structural_page(page: SlidePage, index: int) -> bool: - title = page.title or "" - text = "\n".join([title, *(block.content for block in page.text_blocks)]) - normalized_title = _normalize_text_key(title) - normalized_text = _normalize_text_key(text) - if _has_structural_title(normalized_title): - return True - if index == 0 and any(marker in normalized_text for marker in _cover_markers()): - return True - if _has_standalone_structural_label(text): - return True - return looks_like_outline_page(text) - - -def _has_structural_title(normalized_title: str) -> bool: - exact_titles = { - "\u76ee\u5f55", - "\u8bfe\u7a0b\u76ee\u5f55", - "\u672c\u7ae0\u76ee\u5f55", - "\u7ae0\u8282\u5bfc\u822a", - "contents", - "outline", - "agenda", - } - return normalized_title in exact_titles - - -def _has_standalone_structural_label(text: str) -> bool: - labels = {"\u76ee\u5f55", "\u8bfe\u7a0b\u76ee\u5f55", "\u672c\u7ae0\u76ee\u5f55", "\u7ae0\u8282\u5bfc\u822a", "contents", "outline"} - for line in text.splitlines()[:4]: - normalized = _normalize_text_key(line) - if normalized in labels: - return True - return False - - -def _cover_markers() -> set[str]: - return { - "\u8bb2\u5e08", - "\u6559\u5e08", - "\u6559\u6388", - "\u8054\u7cfb\u90ae\u7bb1", - "\u90ae\u7bb1", - "\u4e3b\u9875", - "email", - "homepage", - "http", - "www", + if looks_like_structural_page(page, index) } -def _normalize_text_key(value: str) -> str: - return re.sub(r"[\s:\uff1a,\uff0c.\u3002;\uff1b\u3001\-_\uff08\uff09()<>]+", "", value).lower() - - def _figure_coverage(deck: Deck, notes_markdown: str) -> dict[str, object]: image_targets = [target.strip().strip("<>") for target in re.findall(r"!\[[^\]]*]\(([^)]+)\)", notes_markdown)] figures: list[dict[str, object]] = [] diff --git a/slidenote/deck_brief.py b/slidenote/deck_brief.py index 796aeab..00a816e 100644 --- a/slidenote/deck_brief.py +++ b/slidenote/deck_brief.py @@ -10,6 +10,8 @@ from slidenote.models import Deck, SlidePage from slidenote.table_understanding import table_preview from slidenote.utils import ( + int_or_none, + parse_json_object, display_path, str_or_none, ) @@ -121,7 +123,7 @@ def build_deck_brief( warnings=warnings, ) - parsed = _parse_json_object(result_text) + parsed = parse_json_object(result_text) if parsed is None: warnings.append("deck_brief_invalid_json") brief = _empty_brief() @@ -348,31 +350,6 @@ def _page_digest(deck: Deck) -> str: return sha256_text(stable_json([_page_payload(page) for page in deck.pages])) -def _parse_json_object(text: str) -> dict[str, Any] | None: - cleaned = text.strip() - if cleaned.startswith("```"): - cleaned = re.sub(r"^```(?:json)?\s*", "", cleaned, flags=re.IGNORECASE) - cleaned = re.sub(r"\s*```$", "", cleaned) - for candidate in (cleaned, _json_object_slice(cleaned)): - if not candidate: - continue - try: - parsed = json.loads(candidate) - except json.JSONDecodeError: - continue - if isinstance(parsed, dict): - return parsed - return None - - -def _json_object_slice(text: str) -> str | None: - start = text.find("{") - end = text.rfind("}") - if start == -1 or end == -1 or end <= start: - return None - return text[start : end + 1] - - def _normalize_brief(parsed: dict[str, Any], deck: Deck, section_plan: dict[str, Any] | None) -> dict[str, Any]: raw = parsed.get("brief") if isinstance(parsed.get("brief"), dict) else parsed chapter_outline = _dict_list(raw.get("chapter_outline") or raw.get("chapters"), limit=60) @@ -405,7 +382,7 @@ def _normalize_page_roles(raw_roles: Any, deck: Deck) -> list[dict[str, Any]]: normalized: list[dict[str, Any]] = [] seen: set[int] = set() for role in roles: - slide_id = _int_or_none(role.get("slide_id") or role.get("page") or role.get("page_id")) + slide_id = int_or_none(role.get("slide_id") or role.get("page") or role.get("page_id")) if slide_id is None or slide_id not in valid_ids or slide_id in seen: continue role["slide_id"] = slide_id @@ -461,13 +438,13 @@ def _filter_by_slide_ids(items: list[dict[str, Any]], wanted: set[int]) -> list[ def _slide_ids_from_item(item: dict[str, Any]) -> set[int]: ids: set[int] = set() for key in ("slide_id", "first_slide_id", "from_slide_id", "to_slide_id", "start_slide_id", "end_slide_id"): - value = _int_or_none(item.get(key)) + value = int_or_none(item.get(key)) if value is not None: ids.add(value) raw_ids = item.get("slide_ids") or item.get("page_ids") or item.get("pages") if isinstance(raw_ids, list): for raw in raw_ids: - value = _int_or_none(raw) + value = int_or_none(raw) if value is not None: ids.add(value) return ids @@ -515,16 +492,6 @@ def _clean_value(value: Any) -> Any: return value -def _int_or_none(value: Any) -> int | None: - if isinstance(value, bool): - return None - if isinstance(value, int): - return value - if isinstance(value, str) and value.strip().isdigit(): - return int(value.strip()) - return None - - def _truncate(text: str | None, limit: int) -> str: if not text: return "" diff --git a/slidenote/exporting.py b/slidenote/exporting.py index cf6a210..955325e 100644 --- a/slidenote/exporting.py +++ b/slidenote/exporting.py @@ -3,6 +3,7 @@ import re import shutil import subprocess +import tempfile import zipfile from dataclasses import dataclass from pathlib import Path @@ -96,30 +97,21 @@ def build_export_artifacts(notes_markdown: str, output_root: Path, formats: list ) warnings.append("Pandoc was not found on PATH; docx/pdf/latex exports were not generated.") else: - docx_path: Path | None = None - docx_result: dict[str, Any] | None = None - - if "docx" in pandoc_formats or "pdf" in pandoc_formats: + if "docx" in pandoc_formats: docx_result = _run_pandoc(pandoc, source, output_root, "docx") - if docx_result["status"] == "ok": - docx_path = output_root / "notes.docx" - if "docx" in pandoc_formats: - results.append(docx_result) - if docx_result["status"] != "ok": - warnings.append(f"docx export failed: {docx_result.get('reason') or docx_result.get('stderr') or 'unknown error'}") + results.append(docx_result) + if docx_result["status"] != "ok": + warnings.append(f"docx export failed: {docx_result.get('reason') or docx_result.get('stderr') or 'unknown error'}") if "pdf" in pandoc_formats: - if docx_path is None: - pdf_result = { - "format": "pdf", - "status": "failed", - "path": "notes.pdf", - "reason": "docx_required_failed", - "blocking": True, - "dependency": docx_result, - } + if "docx" in pandoc_formats: + pdf_result = _pdf_from_docx_result(docx_result, output_root / "notes.docx", output_root) else: - pdf_result = _run_pdf_from_docx(docx_path, output_root) + # PDF-only: keep the intermediate notes.docx out of the output directory. + with tempfile.TemporaryDirectory(prefix="slidenote-export-") as temp_dir: + temp_docx = Path(temp_dir) / "notes.docx" + intermediate = _run_pandoc(pandoc, source, output_root, "docx", output_path=temp_docx) + pdf_result = _pdf_from_docx_result(intermediate, temp_docx, output_root) results.append(pdf_result) if pdf_result["status"] != "ok": warnings.append(f"pdf export failed: {pdf_result.get('reason') or pdf_result.get('stderr') or 'unknown error'}") @@ -265,15 +257,13 @@ def _write_pandoc_source(markdown: str, output_root: Path) -> Path: def _build_markdown_zip(notes_markdown: str, output_root: Path) -> dict[str, Any]: output_path = output_root / MARKDOWN_ZIP_NAME - notes_path = output_root / "notes.md" try: - if not notes_path.exists(): - write_text(notes_path, notes_markdown) if output_path.exists(): output_path.unlink() asset_files = _markdown_asset_files(output_root) with zipfile.ZipFile(output_path, "w", compression=zipfile.ZIP_DEFLATED) as archive: - archive.write(notes_path, "notes.md") + # Package the markdown we were given, not whatever notes.md is on disk. + archive.writestr("notes.md", notes_markdown) for asset_path in asset_files: archive.write(asset_path, asset_path.relative_to(output_root).as_posix()) archive.writestr( @@ -307,12 +297,14 @@ def _markdown_asset_files(output_root: Path) -> list[Path]: return sorted(path for path in assets_root.rglob("*") if path.is_file()) -def _run_pandoc(pandoc: str, source: Path, output_root: Path, fmt: str) -> dict[str, Any]: +def _run_pandoc(pandoc: str, source: Path, output_root: Path, fmt: str, output_path: Path | None = None) -> dict[str, Any]: output_name = _output_name(fmt) - output_path = output_root / output_name - command = [pandoc, "-f", "markdown-implicit_figures", display_path(source, output_root), "-o", output_name] + target = output_name if output_path is None else str(output_path) + output_path = output_path or output_root / output_name + # pandoc runs from output_root so relative image paths in the notes resolve. + command = [pandoc, "-f", "markdown-implicit_figures", display_path(source, output_root), "-o", target] if fmt == "latex": - command.extend(["--standalone", "--pdf-engine=xelatex", "-V", "documentclass=ctexart", "-V", "geometry:margin=1in"]) + command.extend(["--standalone", "-V", "documentclass=ctexart", "-V", "geometry:margin=1in"]) result: dict[str, Any] = { "format": fmt, @@ -343,10 +335,23 @@ def _run_pandoc(pandoc: str, source: Path, output_root: Path, fmt: str) -> dict[ return result +def _pdf_from_docx_result(docx_result: dict[str, Any], docx_path: Path, output_root: Path) -> dict[str, Any]: + if docx_result["status"] != "ok": + return { + "format": "pdf", + "status": "failed", + "path": "notes.pdf", + "reason": "docx_required_failed", + "blocking": True, + "dependency": docx_result, + } + return _run_pdf_from_docx(docx_path, output_root) + + def _run_pdf_from_docx(docx_path: Path, output_root: Path) -> dict[str, Any]: output_name = "notes.pdf" output_path = output_root / output_name - libreoffice = _find_libreoffice() + libreoffice = find_libreoffice() result: dict[str, Any] = { "format": "pdf", "path": output_name, @@ -396,7 +401,7 @@ def _run_pdf_from_docx(docx_path: Path, output_root: Path) -> dict[str, Any]: return result -def _find_libreoffice() -> str | None: +def find_libreoffice() -> str | None: for executable in _LIBREOFFICE_CANDIDATES: found = shutil.which(executable) if found: diff --git a/slidenote/extractors/pdf.py b/slidenote/extractors/pdf.py index 90fb3a6..0a3b5c1 100644 --- a/slidenote/extractors/pdf.py +++ b/slidenote/extractors/pdf.py @@ -6,7 +6,8 @@ from concurrent.futures import ThreadPoolExecutor, as_completed from pathlib import Path -from slidenote.image_assets import image_metadata, refine_image_role_for_placement +from slidenote.geometry import placement_metrics +from slidenote.image_assets import image_metadata, refine_image_role_for_placement, tiny_image_reason from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock, normalize_rel_path from slidenote.utils import unique_path @@ -159,8 +160,8 @@ def _extract_images(doc: object, page: object, page_index: int, images_dir: Path continue seen.add(xref) bbox = _image_bbox(page, xref) - page_size = _page_size(page) - page_like = _is_page_like_bbox(bbox, page_size) + page_size = _page_size(page) or (None, None) + area_ratio, near_edge, page_like = placement_metrics("pdf", bbox, *page_size) # get_images(full=True) carries the embedded pixel size, so tiny # decorations can be classified without decoding or writing them. prescreen = _prescreen_tiny_image(int(image[2] or 0), int(image[3] or 0)) @@ -192,8 +193,8 @@ def _extract_images(doc: object, page: object, page_index: int, images_dir: Path role, ignored, ignore_reason, - _bbox_area_ratio_xyxy(bbox, page_size), - _bbox_near_page_edge_xyxy(bbox, page_size), + area_ratio, + near_edge, ) images.append( ImageAsset( @@ -220,19 +221,8 @@ def _prescreen_tiny_image(width: int, height: int) -> tuple[str, str] | None: rules; tiny_file (byte size) still requires decoding and is handled by image_metadata afterwards. """ - if width <= 0 or height <= 0: - return None - area = width * height - min_dim = min(width, height) - max_dim = max(width, height) - aspect_ratio = max_dim / max(1, min_dim) - if area < 10_000: - return "decorative", "tiny_area" - if min_dim < 24: - return "decorative", "tiny_dimension" - if aspect_ratio >= 8 and area < 150_000: - return "decorative", "thin_decoration" - return None + reason = tiny_image_reason(width, height) + return ("decorative", reason) if reason else None def _image_bbox(page: object, xref: int) -> list[float] | None: @@ -253,39 +243,6 @@ def _page_size(page: object) -> tuple[float, float] | None: return None -def _is_page_like_bbox(bbox: list[float] | None, page_size: tuple[float, float] | None) -> bool: - if not bbox or not page_size: - return False - width, height = page_size - if width <= 0 or height <= 0: - return False - x1, y1, x2, y2 = bbox - area_ratio = max(0.0, x2 - x1) * max(0.0, y2 - y1) / (width * height) - return area_ratio >= 0.85 - - -def _bbox_area_ratio_xyxy(bbox: list[float] | None, page_size: tuple[float, float] | None) -> float | None: - if not bbox or not page_size: - return None - width, height = page_size - if width <= 0 or height <= 0: - return None - x1, y1, x2, y2 = bbox - return max(0.0, x2 - x1) * max(0.0, y2 - y1) / (width * height) - - -def _bbox_near_page_edge_xyxy(bbox: list[float] | None, page_size: tuple[float, float] | None) -> bool: - if not bbox or not page_size: - return False - width, height = page_size - if width <= 0 or height <= 0: - return False - x1, y1, x2, y2 = bbox - margin_x = width * 0.08 - margin_y = height * 0.08 - return x1 <= margin_x or y1 <= margin_y or x2 >= width - margin_x or y2 >= height - margin_y - - def _render_page(page: object, page_index: int, screenshots_dir: Path, output_root: Path) -> str: try: import fitz diff --git a/slidenote/extractors/pptx.py b/slidenote/extractors/pptx.py index f772aaa..14f9650 100644 --- a/slidenote/extractors/pptx.py +++ b/slidenote/extractors/pptx.py @@ -3,6 +3,7 @@ from pathlib import Path from typing import Iterable +from slidenote.geometry import placement_metrics from slidenote.image_assets import image_metadata, refine_image_role_for_placement from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock, normalize_rel_path from slidenote.rendering import render_pptx_screenshots @@ -201,7 +202,7 @@ def _extract_picture( image_path.write_bytes(image.blob) meta = image_metadata(image_path) bbox = _shape_bbox(shape) - page_like = _is_page_like_shape(bbox, slide_width, slide_height) + area_ratio, near_edge, page_like = placement_metrics("pptx", bbox, slide_width, slide_height) role = "page_image" if page_like else meta["role"] ignored = True if page_like else meta["ignored"] ignore_reason = "full_page_image" if page_like else meta["ignore_reason"] @@ -209,8 +210,8 @@ def _extract_picture( role, ignored, ignore_reason, - _shape_area_ratio(bbox, slide_width, slide_height), - _shape_near_slide_edge(bbox, slide_width, slide_height), + area_ratio, + near_edge, ) return ImageAsset( id=f"s{slide_index}_img{image_index}", @@ -234,29 +235,6 @@ def _shape_bbox(shape: object) -> list[float] | None: return None -def _is_page_like_shape(bbox: list[float] | None, slide_width: float, slide_height: float) -> bool: - if not bbox or slide_width <= 0 or slide_height <= 0: - return False - _, _, width, height = bbox - return max(0.0, width) * max(0.0, height) / (slide_width * slide_height) >= 0.85 - - -def _shape_area_ratio(bbox: list[float] | None, slide_width: float, slide_height: float) -> float | None: - if not bbox or slide_width <= 0 or slide_height <= 0: - return None - _, _, width, height = bbox - return max(0.0, width) * max(0.0, height) / (slide_width * slide_height) - - -def _shape_near_slide_edge(bbox: list[float] | None, slide_width: float, slide_height: float) -> bool: - if not bbox or slide_width <= 0 or slide_height <= 0: - return False - left, top, width, height = bbox - margin_x = slide_width * 0.08 - margin_y = slide_height * 0.08 - return left <= margin_x or top <= margin_y or left + width >= slide_width - margin_x or top + height >= slide_height - margin_y - - def _fallback_title(blocks: list[TextBlock]) -> str | None: if not blocks: return None diff --git a/slidenote/figure_grounding.py b/slidenote/figure_grounding.py index 40293cf..d141243 100644 --- a/slidenote/figure_grounding.py +++ b/slidenote/figure_grounding.py @@ -10,15 +10,14 @@ from slidenote.llm import LLMClient, resolve_provider_runtime from slidenote.llm_cache import LLM_CACHE_SCHEMA_VERSION, LLMCache, make_cache_key, sha256_text from slidenote.llm_cache import utc_now_iso +from slidenote.geometry import normalize_asset_bbox, normalize_page_bbox from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock from slidenote.table_understanding import table_text_for_prompt from slidenote.utils import ( as_float, - clamp_normalized_bbox as _clamp_bbox, cleanup_temp_image, display_path, file_sha256, - looks_normalized, layout_order_from_bbox as _order_from_bbox, parse_json_object, prepare_image_for_api, @@ -29,7 +28,8 @@ FIGURE_GROUNDING_MODES = {"off", "auto", "vision"} FIGURE_PLACEMENT_MODES = {"inline", "page-end"} -FIGURE_AUDIT_MODES = {"off", "local", "llm"} +# Only the local audit is implemented; there is no LLM audit mode. +FIGURE_AUDIT_MODES = {"off", "local"} FIGURE_GROUNDING_PROMPT_VERSION = "figure-grounding-vision-v1" FIGURE_GROUNDING_MIN_CONFIDENCE = 0.55 @@ -159,7 +159,7 @@ def process(index: int, page: SlidePage) -> tuple[int, int, dict[str, Any]]: if image.anchor_element_ids or image.role == "figure_crop": auto_insertable_count += 1 - page_records.append(_image_record(page, image, output_root)) + page_records.append(_image_record(deck, page, image)) page_entries.append( { @@ -272,13 +272,11 @@ def ordered_page_elements( def normalized_image_bbox(deck: Deck, page: SlidePage, image: ImageAsset) -> list[float] | None: - if image.crop_bbox and looks_normalized(image.crop_bbox): - return _clamp_bbox(image.crop_bbox) - return _normalize_bbox(deck.source_type, image.bbox, page) + return normalize_asset_bbox(deck.source_type, page, image) def normalized_element_bbox(deck: Deck, page: SlidePage, element: TextBlock | TableBlock) -> list[float] | None: - return _normalize_bbox(deck.source_type, element.bbox, page) + return normalize_page_bbox(deck.source_type, element.bbox, page) def _layout_elements(deck: Deck, page: SlidePage) -> list[dict[str, Any]]: @@ -421,6 +419,7 @@ def _process_figure_grounding_vision_page( base_url=runtime["base_url"], max_output_tokens=max_output_tokens, temperature=temperature, + for_vision=True, ) llm_result = client.generate_image_with_usage( prepared_path, @@ -809,8 +808,7 @@ def _local_audit_status(image: ImageAsset) -> str: return "ok" -def _image_record(page: SlidePage, image: ImageAsset, output_root: Path) -> dict[str, Any]: - del output_root +def _image_record(deck: Deck, page: SlidePage, image: ImageAsset) -> dict[str, Any]: return { "id": image.id, "path": image.path, @@ -837,29 +835,11 @@ def _image_record(page: SlidePage, image: ImageAsset, output_root: Path) -> dict "crop_warnings": list(image.crop_warnings), "confidence": image.confidence, "bbox": image.bbox, + "bbox_normalized": normalized_image_bbox(deck, page, image), "slide_id": page.slide_id, } -def _normalize_bbox(source_type: str, bbox: list[float] | None, page: SlidePage) -> list[float] | None: - if not bbox or len(bbox) != 4: - return None - if looks_normalized(bbox): - return _clamp_bbox(bbox) - width = page.page_width or 0.0 - height = page.page_height or 0.0 - if width <= 0 or height <= 0: - return None - x1, y1, third, fourth = [float(value) for value in bbox] - if source_type == "pptx": - x2 = x1 + third - y2 = y1 + fourth - else: - x2 = third - y2 = fourth - return _clamp_bbox([x1 / width, y1 / height, x2 / width, y2 / height]) - - def _horizontal_overlap_ratio(a: list[float], b: list[float]) -> float: left = max(a[0], b[0]) right = min(a[2], b[2]) @@ -879,14 +859,19 @@ def _image_sort_key(image: ImageAsset) -> tuple[int, float, str]: def _image_text(image: ImageAsset) -> str: - return " ".join(part for part in [image.caption, image.visual_summary, image.ocr_text] if part) + caption = image.caption if image.caption and not _is_generic_caption(image.caption) else None + return " ".join(part for part in [caption, image.visual_summary, image.ocr_text] if part) def _is_generic_caption(caption: str) -> bool: - return bool(re.fullmatch(r"第\s*\d+\s*页(?:嵌入)?图片\s*\d*|第\s*\d+\s*页图片|图示", caption.strip())) + return bool(re.fullmatch(r"第\s*\d+\s*页(?:嵌入图片|图片|局部图|组合图)\s*\d*|图示", caption.strip())) def _tokens(text: str) -> set[str]: words = {word.lower() for word in re.findall(r"[A-Za-z0-9_]{2,}", text)} - cjk = {char for char in text if "\u4e00" <= char <= "\u9fff"} + # Single CJK characters overlap between almost any two Chinese sentences, so + # compare character bigrams instead. + cjk: set[str] = set() + for run in re.findall(r"[\u4e00-\u9fff]+", text): + cjk.update(run[index : index + 2] for index in range(len(run) - 1)) return words.union(cjk) diff --git a/slidenote/figures.py b/slidenote/figures.py index e057894..567ec07 100644 --- a/slidenote/figures.py +++ b/slidenote/figures.py @@ -1,32 +1,36 @@ from __future__ import annotations import json -from concurrent.futures import ThreadPoolExecutor, as_completed from dataclasses import dataclass from pathlib import Path from typing import Any, Callable -from PIL import Image +from PIL import Image, ImageMath from slidenote.image_assets import image_metadata from slidenote.llm import LLMClient, resolve_provider_runtime from slidenote.llm_cache import LLM_CACHE_SCHEMA_VERSION, LLMCache, make_cache_key, sha256_text, utc_now_iso -from slidenote.modality import page_has_hint +from slidenote.modality import page_has_hint, page_has_manual_modality +from slidenote.geometry import normalize_asset_bbox, normalize_page_bbox from slidenote.models import Deck, ImageAsset, SlidePage, normalize_rel_path from slidenote.utils import ( + advance_progress, + error_summary, + run_target_jobs, as_float, bbox_area, cleanup_temp_image, display_path, file_sha256, page_by_id, + parse_json_object, pixel_box, prepare_image_for_api, sum_int, union_bbox, ) +from slidenote.vision import page_prompt_context from slidenote.semantic_layout import semantic_context_for_page, semantic_layout_for_prompt -from slidenote.table_understanding import table_preview FIGURE_PROMPT_VERSION = "figure-crop-v1" @@ -128,25 +132,14 @@ def process(index: int, target: FigureTarget) -> tuple[int, FigureTarget, dict[s ) return index, target, record, crops - results = [] - if workers == 1: - for index, target in enumerate(targets): - result = process(index, target) - results.append(result) - if progress_callback: - _, completed_target, record, _ = result - progress_callback({"event": "advance", "record": record, "slide_id": completed_target.slide_id}) - else: - with ThreadPoolExecutor(max_workers=workers) as executor: - futures = {executor.submit(process, index, target): (index, target) for index, target in enumerate(targets)} - for future in as_completed(futures): - result = future.result() - results.append(result) - if progress_callback: - _, completed_target, record, _ = result - progress_callback({"event": "advance", "record": record, "slide_id": completed_target.slide_id}) - - for index, target, record, crops in sorted(results, key=lambda item: item[0]): + def failed(index: int, target: FigureTarget, exc: Exception) -> tuple[int, FigureTarget, dict[str, Any], list[ImageAsset]]: + record = _skipped_record(target, "api_error") + record.update({"cache_status": "error", "error": error_summary(exc)}) + return index, target, record, [] + + results = run_target_jobs(targets, process, workers=workers, on_error=failed, on_result=advance_progress(progress_callback)) + + for index, target, record, crops in results: page = page_by_id(deck, target.slide_id) if page is not None: page.images.extend(crops) @@ -161,6 +154,8 @@ def select_figure_targets(deck: Deck, max_targets: int = 80) -> list[FigureTarge for page in deck.pages: if not page.page_screenshot: continue + if page_has_manual_modality(page) and not page_has_hint(page, "crop_figures_from_screenshot"): + continue if _page_has_content_images(page): continue if page_has_hint(page, "crop_figures_from_screenshot"): @@ -193,11 +188,11 @@ def _process_figure_target( ) -> tuple[dict[str, Any], list[ImageAsset]]: source_path = (output_root / target.path).resolve() if not source_path.exists(): - return _skipped_record(target, "missing_file", output_root), [] + return _skipped_record(target, "missing_file"), [] prepared = prepare_image_for_api(source_path, max_edge=max_edge) if prepared is None: - return _skipped_record(target, "unsupported_or_unreadable_image", output_root), [] + return _skipped_record(target, "unsupported_or_unreadable_image"), [] prepared_path, image_meta = prepared try: @@ -248,6 +243,7 @@ def _process_figure_target( base_url=runtime["base_url"], max_output_tokens=max_output_tokens, temperature=temperature, + for_vision=True, ) llm_result = client.generate_image_with_usage(prepared_path, prompt, system_prompt=FIGURE_SYSTEM_PROMPT, image_detail=detail) result_json = llm_result.text @@ -292,9 +288,10 @@ def _process_figure_target( max_crops_per_page=max_crops_per_page, min_confidence=min_confidence, min_area=min_area, - start_index=_next_figure_index(page), + start_index=next_figure_index(page), page=page, source_type=source_type, + model_image_size=_prepared_size(image_meta), ) record["result"] = parsed record["crops"] = crop_records @@ -318,6 +315,7 @@ def _crop_figures( start_index: int = 1, page: SlidePage | None = None, source_type: str | None = None, + model_image_size: tuple[int, int] | None = None, ) -> tuple[list[ImageAsset], list[dict[str, Any]], list[dict[str, Any]]]: crops: list[ImageAsset] = [] crop_records: list[dict[str, Any]] = [] @@ -330,7 +328,7 @@ def _crop_figures( image = image.convert("RGB") normalized_candidates: list[tuple[dict[str, Any], NormalizedFigure]] = [] for candidate in figures: - normalized = _normalize_candidate(candidate, width=width, height=height) + normalized = _normalize_candidate(candidate, *(model_image_size or (width, height))) if normalized is None: skipped.append({"reason": "invalid_bbox", "candidate": candidate}) continue @@ -479,31 +477,14 @@ def _page_layout_boxes(page: SlidePage, source_type: str | None) -> tuple[list[l for image in page.images: if image.ignored or image.role == "page_image": continue - bbox = _normalize_page_bbox(image.crop_bbox or image.bbox, page, source_type) + bbox = normalize_asset_bbox(source_type, page, image) if bbox: visual_boxes.append(bbox) return text_boxes, visual_boxes def _normalize_page_bbox(bbox: list[float] | None, page: SlidePage, source_type: str | None) -> list[float] | None: - if not bbox or len(bbox) != 4: - return None - try: - values = [float(value) for value in bbox] - except (TypeError, ValueError): - return None - if all(-0.001 <= value <= 1.001 for value in values): - return _round_bbox(values) - width = page.page_width or 0.0 - height = page.page_height or 0.0 - if width <= 0 or height <= 0: - return None - x1, y1, third, fourth = values - if source_type == "pptx": - x2, y2 = x1 + third, y1 + fourth - else: - x2, y2 = third, fourth - return _round_bbox([x1 / width, y1 / height, x2 / width, y2 / height]) + return normalize_page_bbox(source_type, bbox, page) def _looks_normalized_bbox(value: object) -> bool: @@ -532,10 +513,13 @@ def _normalize_candidate(candidate: dict[str, Any], width: int, height: int) -> except (TypeError, ValueError): return None + # The prompt asks for 0..1 coordinates. Models sometimes answer in percent or + # in pixels of the image they were shown (the downscaled copy, whose size is + # passed in as width/height). max_value = max(values) if max_value > 100: values = [values[0] / width, values[1] / height, values[2] / width, values[3] / height] - elif max_value > 1.5: + elif max_value > 1.001: values = [value / 100 for value in values] x1, y1, x2, y2 = values @@ -645,19 +629,14 @@ def _foreground_touching_edges(crop: Image.Image) -> list[str]: width, height = crop.width, crop.height if width <= 4 or height <= 4: return [] - background = _estimate_background(crop) - pixels = crop.load() + mask = _foreground_mask(crop, _estimate_background(crop)) margin_x = max(3, int(round(width * 0.018))) margin_y = max(3, int(round(height * 0.018))) center_area = max(1, (width - 2 * margin_x) * (height - 2 * margin_y)) def density(x_start: int, x_end: int, y_start: int, y_end: int) -> float: total = max(1, (x_end - x_start) * (y_end - y_start)) - foreground = 0 - for y in range(y_start, y_end): - for x in range(x_start, x_end): - if _is_foreground_pixel(pixels[x, y], background): - foreground += 1 + foreground = sum(mask[y * width + x_start : y * width + x_end].count(255) for y in range(y_start, y_end)) return foreground / total center_density = density(margin_x, max(margin_x + 1, width - margin_x), margin_y, max(margin_y + 1, height - margin_y)) if center_area else 0.0 @@ -836,14 +815,8 @@ def _foreground_row_bands(crop: Image.Image) -> list[tuple[int, int, float]]: background = _estimate_background(crop) x_step = max(1, width // 650) samples_per_row = max(1, (width + x_step - 1) // x_step) - densities: list[float] = [] - pixels = crop.load() - for y in range(height): - foreground = 0 - for x in range(0, width, x_step): - if _is_foreground_pixel(pixels[x, y], background): - foreground += 1 - densities.append(foreground / samples_per_row) + mask = _foreground_mask(crop, background) + densities = [mask[y * width : (y + 1) * width : x_step].count(255) / samples_per_row for y in range(height)] max_density = max(densities) if densities else 0.0 if max_density < 0.006: return [] @@ -945,6 +918,32 @@ def _is_foreground_pixel(pixel: tuple[int, ...], background: tuple[int, int, int return background_brightness >= 235 and brightness <= 235 and distance >= 24 +def _foreground_mask(crop: Image.Image, background: tuple[int, int, int]) -> bytes: + """Row-major bytes (255 = foreground) equal to ``_is_foreground_pixel`` per pixel.""" + red, green, blue = crop.convert("RGB").split() + bg_red, bg_green, bg_blue = background + light_background = sum(background) / 3 >= 235 + + def expression(args: dict[str, Any]) -> Any: + distance = abs(args["r"] - bg_red) + abs(args["g"] - bg_green) + abs(args["b"] - bg_blue) + mask = distance >= 55 + if light_background: + # brightness <= 235 <=> r + g + b <= 705 + mask = mask | (((args["r"] + args["g"] + args["b"]) <= 705) & (distance >= 24)) + return mask + + if hasattr(ImageMath, "lambda_eval"): + result = ImageMath.lambda_eval(expression, r=red, g=green, b=blue) + else: # Pillow < 10.3 + light = f" | (((r + g + b) <= 705) & ({_distance_expr(background)} >= 24))" if light_background else "" + result = ImageMath.eval(f"({_distance_expr(background)} >= 55){light}", r=red, g=green, b=blue) + return result.convert("L").point(lambda value: 255 if value else 0).tobytes() + + +def _distance_expr(background: tuple[int, int, int]) -> str: + return f"(abs(r - {background[0]}) + abs(g - {background[1]}) + abs(b - {background[2]}))" + + def _is_code_content(content_type: str) -> bool: return content_type.strip().lower() in {"code", "source_code"} @@ -1063,23 +1062,9 @@ def _page_deserves_figure_crop(page: SlidePage) -> bool: def _page_context(page: SlidePage | None, limit: int = 1000) -> str: - if page is None: - return "" - pieces: list[str] = [] - if page.title: - pieces.append(f"标题：{page.title}") - for block in page.text_blocks[:8]: - pieces.append(f"{block.id}({block.type})：{block.content}") - for table in page.tables[:2]: - preview = table_preview(table, limit=260, raw_rows=3) - pieces.append(f"{table.id}(table)：{preview}") - semantic_context = semantic_context_for_page(page, limit=420) - if semantic_context: - pieces.append(f"semantic_layout：{semantic_context}") - text = "\n".join(piece for piece in pieces if piece.strip()) - if len(text) > limit: - return text[: limit - 1] + "…" - return text + semantic_context = semantic_context_for_page(page, limit=420) if page else "" + extra = [f"semantic_layout：{semantic_context}"] if semantic_context else None + return page_prompt_context(page, limit=limit, extra=extra) def _semantic_layout_json_for_prompt(page: SlidePage | None) -> str: @@ -1103,16 +1088,18 @@ def _figure_prompt(target: FigureTarget, page: SlidePage | None) -> str: ) +def _prepared_size(image_meta: dict[str, Any]) -> tuple[int, int] | None: + prepared = image_meta.get("prepared") or {} + width, height = prepared.get("width"), prepared.get("height") + if isinstance(width, int) and isinstance(height, int) and width > 0 and height > 0: + return width, height + return None + + def _parse_figure_json(text: str) -> dict[str, Any]: - cleaned = text.strip() - if cleaned.startswith("```"): - cleaned = cleaned.strip("`") - if cleaned.lower().startswith("json"): - cleaned = cleaned[4:].strip() - try: - parsed = json.loads(cleaned) - except json.JSONDecodeError: - return {"figures": [], "warnings": ["model_output_not_json"], "raw_text": cleaned} + parsed = parse_json_object(text) + if parsed is None: + return {"figures": [], "warnings": ["model_output_not_json"], "raw_text": text.strip()} figures = parsed.get("figures") if not isinstance(figures, list): parsed["figures"] = [] @@ -1141,6 +1128,7 @@ def _build_report( "llm_calls": sum(1 for record in records if record.get("llm_call")), "api_retries": sum(int(record.get("api_retries") or 0) for record in records), "skipped": sum(1 for record in records if record.get("cache_status") == "skipped"), + "failed": sum(1 for record in records if record.get("cache_status") == "error"), "skipped_candidates": sum(len(record.get("skipped_candidates", [])) for record in records), "input_tokens": sum_int(record.get("input_tokens") for record in records), "output_tokens": sum_int(record.get("output_tokens") for record in records), @@ -1181,7 +1169,7 @@ def _base_record(target: FigureTarget, cache_key: str, cache_path: Path, output_ } -def _skipped_record(target: FigureTarget, status: str, output_root: Path) -> dict[str, Any]: +def _skipped_record(target: FigureTarget, status: str) -> dict[str, Any]: return { "slide_id": target.slide_id, "kind": "page_screenshot", @@ -1195,7 +1183,7 @@ def _skipped_record(target: FigureTarget, status: str, output_root: Path) -> dic } -def _next_figure_index(page: SlidePage | None) -> int: +def next_figure_index(page: SlidePage | None) -> int: if page is None: return 1 next_index = 1 diff --git a/slidenote/geometry.py b/slidenote/geometry.py new file mode 100644 index 0000000..c92dc1e --- /dev/null +++ b/slidenote/geometry.py @@ -0,0 +1,123 @@ +"""Bounding-box conventions shared by the parsing and understanding stages. + +Raw ``bbox`` values on ``TextBlock`` / ``TableBlock`` / ``ImageAsset`` are stored in +the parser's native space: + +- ``pptx``: ``[x, y, width, height]`` in EMU. +- ``pdf`` and everything else: ``[x1, y1, x2, y2]`` in points. + +Assets cropped from a page screenshot (``figure_crop``, ``composite_figure``) keep the +pixel rectangle of the screenshot in ``bbox`` and the page-normalized rectangle in +``crop_bbox``. Their ``bbox`` must never be interpreted as page coordinates; use +:func:`asset_source_bbox` to pick the right field. + +Normalized boxes are ``[x1, y1, x2, y2]`` page fractions in ``0..1``. +""" + +from __future__ import annotations + +from typing import Any + +from slidenote.models import ImageAsset, SlidePage + + +def coerce_bbox(value: Any) -> list[float] | None: + if not isinstance(value, (list, tuple)) or len(value) != 4: + return None + try: + return [float(part) for part in value] + except (TypeError, ValueError): + return None + + +def looks_normalized(bbox: list[float]) -> bool: + return len(bbox) == 4 and all(-0.001 <= float(value) <= 1.001 for value in bbox) + + +def clamp_bbox(bbox: list[float], precision: int = 4) -> list[float]: + x1, y1, x2, y2 = [max(0.0, min(1.0, float(value))) for value in bbox] + if x2 < x1: + x1, x2 = x2, x1 + if y2 < y1: + y1, y2 = y2, y1 + return [round(x1, precision), round(y1, precision), round(x2, precision), round(y2, precision)] + + +def normalize_bbox( + source_type: str | None, + bbox: Any, + page_width: float | None, + page_height: float | None, + *, + precision: int = 4, +) -> list[float] | None: + """Convert a native-space bbox (see module docstring) to a normalized xyxy box.""" + values = coerce_bbox(bbox) + if values is None: + return None + if looks_normalized(values): + return clamp_bbox(values, precision) + try: + width = float(page_width or 0.0) + height = float(page_height or 0.0) + except (TypeError, ValueError): + return None + if width <= 0 or height <= 0: + return None + x1, y1, third, fourth = values + if source_type == "pptx": + x2, y2 = x1 + third, y1 + fourth + else: + x2, y2 = third, fourth + return clamp_bbox([x1 / width, y1 / height, x2 / width, y2 / height], precision) + + +def bbox_format(source_type: str | None, bbox: list[float] | None) -> str | None: + if not bbox: + return None + if looks_normalized(bbox): + return "normalized_xyxy" + if source_type == "pptx": + return "source_xywh" + return "source_xyxy" + + +def asset_source_bbox(image: ImageAsset) -> list[float] | None: + """Return the bbox of an image asset in page space (normalized or native). + + Screenshot crops store pixel coordinates in ``bbox``; only their ``crop_bbox`` + describes the position on the page. + """ + if image.crop_bbox: + return coerce_bbox(image.crop_bbox) + if image.crop_source_path: + return None + return coerce_bbox(image.bbox) + + +def normalize_page_bbox(source_type: str | None, bbox: Any, page: SlidePage, *, precision: int = 4) -> list[float] | None: + return normalize_bbox(source_type, bbox, page.page_width, page.page_height, precision=precision) + + +def normalize_asset_bbox(source_type: str | None, page: SlidePage, image: ImageAsset, *, precision: int = 4) -> list[float] | None: + return normalize_page_bbox(source_type, asset_source_bbox(image), page, precision=precision) + + +PAGE_LIKE_AREA_RATIO = 0.85 +PAGE_EDGE_MARGIN = 0.08 + + +def placement_metrics( + source_type: str | None, + bbox: Any, + page_width: float | None, + page_height: float | None, +) -> tuple[float | None, bool, bool]: + """Return ``(area_ratio, near_page_edge, page_like)`` for a native-space bbox.""" + normalized = normalize_bbox(source_type, bbox, page_width, page_height, precision=6) + if normalized is None: + return None, False, False + x1, y1, x2, y2 = normalized + area_ratio = (x2 - x1) * (y2 - y1) + near_edge = x1 <= PAGE_EDGE_MARGIN or y1 <= PAGE_EDGE_MARGIN or x2 >= 1 - PAGE_EDGE_MARGIN or y2 >= 1 - PAGE_EDGE_MARGIN + return area_ratio, near_edge, area_ratio >= PAGE_LIKE_AREA_RATIO diff --git a/slidenote/image_assets.py b/slidenote/image_assets.py index 09c8d2c..4a56f23 100644 --- a/slidenote/image_assets.py +++ b/slidenote/image_assets.py @@ -22,6 +22,13 @@ def image_metadata(path: Path) -> dict[str, Any]: return meta +TINY_FILE_BYTES = 512 +TINY_AREA_PX = 10_000 +TINY_DIMENSION_PX = 24 +THIN_ASPECT_RATIO = 8 +THIN_MAX_AREA_PX = 150_000 + + def classify_image_asset(meta: dict[str, Any]) -> tuple[str, bool, str | None]: width = meta.get("width") height = meta.get("height") @@ -29,22 +36,30 @@ def classify_image_asset(meta: dict[str, Any]) -> tuple[str, bool, str | None]: if not isinstance(width, int) or not isinstance(height, int) or width <= 0 or height <= 0: return "unknown", False, None - area = width * height - min_dim = min(width, height) - max_dim = max(width, height) - aspect_ratio = max_dim / max(1, min_dim) - - if isinstance(file_size, int) and file_size < 512: + if isinstance(file_size, int) and file_size < TINY_FILE_BYTES: return "decorative", True, "tiny_file" - if area < 10_000: - return "decorative", True, "tiny_area" - if min_dim < 24: - return "decorative", True, "tiny_dimension" - if aspect_ratio >= 8 and area < 150_000: - return "decorative", True, "thin_decoration" + reason = tiny_image_reason(width, height) + if reason: + return "decorative", True, reason return "content", False, None +def tiny_image_reason(width: int, height: int) -> str | None: + """Dimension rules for decorative images, usable before an image is decoded.""" + if width <= 0 or height <= 0: + return None + area = width * height + min_dim = min(width, height) + aspect_ratio = max(width, height) / max(1, min_dim) + if area < TINY_AREA_PX: + return "tiny_area" + if min_dim < TINY_DIMENSION_PX: + return "tiny_dimension" + if aspect_ratio >= THIN_ASPECT_RATIO and area < THIN_MAX_AREA_PX: + return "thin_decoration" + return None + + def refine_image_role_for_placement( role: str, ignored: bool, diff --git a/slidenote/image_ranking.py b/slidenote/image_ranking.py index 51610a0..248d65e 100644 --- a/slidenote/image_ranking.py +++ b/slidenote/image_ranking.py @@ -7,6 +7,7 @@ from PIL import Image from slidenote.llm_cache import utc_now_iso +from slidenote.geometry import normalize_asset_bbox from slidenote.models import Deck, ImageAsset, SlidePage @@ -26,8 +27,9 @@ def rank_deck_images(deck: Deck, output_root: Path, mode: str = "local", stage: key=lambda record: (-record["importance_score"], record["image_id"]), ) rank_by_id = {record["image_id"]: index + 1 for index, record in enumerate(ranked)} + images_by_id = {image.id: image for image in page.images} for record in records: - image = next((item for item in page.images if item.id == record["image_id"]), None) + image = images_by_id.get(record["image_id"]) if image is None: continue rank = rank_by_id.get(record["image_id"]) @@ -222,29 +224,7 @@ def _template_like_image(image: ImageAsset, bbox: list[float] | None, output_roo def _normalized_image_bbox(page: SlidePage, image: ImageAsset, source_type: str) -> list[float] | None: - bbox = image.crop_bbox or image.bbox - if not bbox or len(bbox) != 4: - return None - try: - values = [float(value) for value in bbox] - except (TypeError, ValueError): - return None - if all(-0.001 <= value <= 1.001 for value in values): - x1, y1, x2, y2 = values - return [max(0.0, min(1.0, value)) for value in [min(x1, x2), min(y1, y2), max(x1, x2), max(y1, y2)]] - if not page.page_width or not page.page_height: - return None - x1, y1, third, fourth = values - if source_type == "pptx": - x2, y2 = x1 + third, y1 + fourth - else: - x2, y2 = third, fourth - return [ - max(0.0, min(1.0, x1 / page.page_width)), - max(0.0, min(1.0, y1 / page.page_height)), - max(0.0, min(1.0, x2 / page.page_width)), - max(0.0, min(1.0, y2 / page.page_height)), - ] + return normalize_asset_bbox(source_type, page, image) def _record(page: SlidePage, image: ImageAsset, score: float, reasons: list[str], output_root: Path) -> dict[str, Any]: @@ -270,6 +250,8 @@ def _image_dimensions(image: ImageAsset, output_root: Path) -> tuple[int | None, path = output_root / image.path try: with Image.open(path) as opened: - return opened.width, opened.height + # Remember the size so later scoring steps do not reopen the file. + image.width, image.height = opened.width, opened.height except Exception: - return image.width, image.height + pass + return image.width, image.height diff --git a/slidenote/ir.py b/slidenote/ir.py index 9076853..bfc27f4 100644 --- a/slidenote/ir.py +++ b/slidenote/ir.py @@ -2,6 +2,7 @@ from typing import Any, Iterable +from slidenote.geometry import asset_source_bbox from slidenote.ir_context import IRBuildContext from slidenote.ir_projection import source_ref_from_element from slidenote.ir_standard import ( @@ -55,8 +56,8 @@ def build_page_ir( return _build_page_ir(context, page) -def iter_expected_source_elements(deck: Deck) -> Iterable[ElementIR]: - for page in build_deck_ir(deck)["pages"]: +def iter_expected_source_elements(deck: Deck, deck_ir: dict[str, Any] | None = None) -> Iterable[ElementIR]: + for page in (deck_ir or build_deck_ir(deck))["pages"]: for element in page["elements"]: if element.get("kind") == "semantic_group": continue @@ -64,9 +65,9 @@ def iter_expected_source_elements(deck: Deck) -> Iterable[ElementIR]: yield element -def element_index_from_ir(deck: Deck) -> dict[str, dict[str, Any]]: +def element_index_from_ir(deck: Deck, deck_ir: dict[str, Any] | None = None) -> dict[str, dict[str, Any]]: index: dict[str, dict[str, Any]] = {} - for page in build_deck_ir(deck)["pages"]: + for page in (deck_ir or build_deck_ir(deck))["pages"]: slide_id = int(page["slide_id"]) for element in page["elements"]: if element.get("kind") == "semantic_group": @@ -213,7 +214,7 @@ def _image_element( "figure_audit_status": image.figure_audit_status, } source_ids = _unique_ids([image.id, *image.source_element_ids]) - raw_bbox = image.crop_bbox or image.bbox + raw_bbox = asset_source_bbox(image) role = primary_role("image", roles) element = { "element_id": image.id, diff --git a/slidenote/ir_standard.py b/slidenote/ir_standard.py index 8a34226..d31ce75 100644 --- a/slidenote/ir_standard.py +++ b/slidenote/ir_standard.py @@ -2,12 +2,10 @@ from typing import Any +from slidenote.geometry import bbox_format, coerce_bbox, normalize_page_bbox from slidenote.ir_context import IRBuildContext from slidenote.models import SlidePage -from slidenote.utils import ( - as_float, - looks_normalized, -) +from slidenote.utils import as_float def standard_fields( @@ -27,7 +25,7 @@ def standard_fields( layout_order: Any = None, ) -> dict[str, Any]: raw = coerce_bbox(raw_bbox) - normalized = _normalize_bbox(context.deck.source_type, raw, page) + normalized = normalize_page_bbox(context.deck.source_type, raw, page, precision=6) resolved_confidence, confidence_source = _resolve_confidence( semantic=semantic, guard_item=context.guard_item(element_id), @@ -50,7 +48,7 @@ def standard_fields( "role": role, "confidence": resolved_confidence, "confidence_source": confidence_source, - "bbox_format": _bbox_format(context.deck.source_type, raw), + "bbox_format": bbox_format(context.deck.source_type, raw), "bbox_normalized": normalized, "bbox_source": bbox_source if raw else None, "layout_order": resolved_layout_order, @@ -108,15 +106,6 @@ def semantic_value(semantic: dict[str, Any] | None, key: str, default: Any = Non return default if value is None else value -def coerce_bbox(value: Any) -> list[float] | None: - if not isinstance(value, (list, tuple)) or len(value) != 4: - return None - try: - return [float(part) for part in value] - except (TypeError, ValueError): - return None - - def compact(values: dict[str, Any]) -> dict[str, Any]: return {key: value for key, value in values.items() if value is not None} @@ -184,46 +173,6 @@ def _coverage_state( } -def _normalize_bbox(source_type: str, bbox: list[float] | None, page: SlidePage) -> list[float] | None: - if not bbox: - return None - if looks_normalized(bbox): - return _clamp_bbox(bbox) - width = as_float(page.page_width, None) - height = as_float(page.page_height, None) - if not width or not height or width <= 0 or height <= 0: - return None - x1, y1, third, fourth = bbox - if source_type == "pptx": - x2 = x1 + third - y2 = y1 + fourth - else: - x2 = third - y2 = fourth - return _clamp_bbox([x1 / width, y1 / height, x2 / width, y2 / height]) - - -def _bbox_format(source_type: str, bbox: list[float] | None) -> str | None: - if not bbox: - return None - if looks_normalized(bbox): - return "normalized_xyxy" - if source_type == "pptx": - return "source_xywh" - if source_type == "pdf": - return "source_xyxy" - return "source_xyxy" - - -def _clamp_bbox(bbox: list[float]) -> list[float]: - x1, y1, x2, y2 = [max(0.0, min(1.0, float(value))) for value in bbox] - if x2 < x1: - x1, x2 = x2, x1 - if y2 < y1: - y1, y2 = y2, y1 - return [round(x1, 6), round(y1, 6), round(x2, 6), round(y2, 6)] - - def _order_from_bbox(bbox: list[float] | None) -> float | None: if not bbox: return None diff --git a/slidenote/llm.py b/slidenote/llm.py index 107f953..e859a12 100644 --- a/slidenote/llm.py +++ b/slidenote/llm.py @@ -1,8 +1,9 @@ from __future__ import annotations -import json import base64 +import json import mimetypes +import threading import urllib.error import urllib.parse import urllib.request @@ -137,6 +138,14 @@ class LLMResult: "anthropic": "claude", } +HTTP_TIMEOUT_SECONDS = 120 +ANTHROPIC_API_VERSION = "2023-06-01" +# Generic overrides apply to the text-model role only; vision has its own. +TEXT_MODEL_ENV = "SLIDENOTE_MODEL" +TEXT_BASE_URL_ENV = "SLIDENOTE_BASE_URL" +VISION_MODEL_ENV = "SLIDENOTE_VISION_MODEL" +VISION_BASE_URL_ENV = "SLIDENOTE_VISION_BASE_URL" + SYSTEM_PROMPT = ( "你是课程笔记写作助手。输出必须直接进入 Markdown 正文，不要写寒暄、任务复述、JSON 说明或规则遵循说明。" "你要把幻灯片中的零散 bullet 改写成自然、连贯、适合复习的课程笔记，并保留必要来源标记。" @@ -153,13 +162,16 @@ def __init__( base_url: str | None = None, max_output_tokens: int = 4096, temperature: float | None = None, + for_vision: bool = False, ) -> None: self.spec = get_provider_spec(provider) - self.model = _resolve_model(self.spec, model) + self.model = _resolve_model(self.spec, model, for_vision=for_vision) self.api_key = _resolve_api_key(self.spec, api_key) - self.base_url = _resolve_base_url(self.spec, base_url) + self.base_url = _resolve_base_url(self.spec, base_url, for_vision=for_vision) self.max_output_tokens = max_output_tokens self.temperature = temperature + self._openai_client: Any = None + self._openai_client_lock = threading.Lock() @property def provider_name(self) -> str: @@ -208,38 +220,26 @@ def call() -> LLMResult: retry_result = with_api_retries(call) return _with_retry_usage(retry_result.value, retry_result.retries) - def _generate_openai_compatible(self, system_prompt: str, user_prompt: str) -> LLMResult: - try: - from openai import OpenAI - except ImportError as exc: - raise RuntimeError("OpenAI SDK is required for OpenAI-compatible providers. Install with `pip install openai`.") from exc - - client_kwargs: dict[str, Any] = {"api_key": self.api_key} - if self.base_url: - client_kwargs["base_url"] = self.base_url - client = OpenAI(**client_kwargs) + # -- OpenAI-compatible ------------------------------------------------- + + def _openai(self) -> Any: + """Return one SDK client per LLMClient; ``with_api_retries`` owns retries.""" + with self._openai_client_lock: + if self._openai_client is None: + try: + from openai import OpenAI + except ImportError as exc: + raise RuntimeError( + "OpenAI SDK is required for OpenAI-compatible providers. Install with `pip install openai`." + ) from exc + client_kwargs: dict[str, Any] = {"api_key": self.api_key, "max_retries": 0} + if self.base_url: + client_kwargs["base_url"] = self.base_url + self._openai_client = OpenAI(**client_kwargs) + return self._openai_client - request: dict[str, Any] = { - "model": self.model, - "messages": [ - {"role": "system", "content": system_prompt}, - {"role": "user", "content": user_prompt}, - ], - } - if self.max_output_tokens: - request["max_tokens"] = self.max_output_tokens - if self.temperature is not None: - request["temperature"] = self.temperature - - response = client.chat.completions.create(**request) - content = response.choices[0].message.content - return LLMResult( - text=content.strip() if content else "", - usage=_with_finish_reason( - _normalize_openai_usage(getattr(response, "usage", None)), - getattr(response.choices[0], "finish_reason", None), - ), - ) + def _generate_openai_compatible(self, system_prompt: str, user_prompt: str) -> LLMResult: + return self._openai_chat(system_prompt, user_prompt) def _generate_openai_image( self, @@ -249,34 +249,28 @@ def _generate_openai_image( mime_type: str, image_detail: str, ) -> LLMResult: - try: - from openai import OpenAI - except ImportError as exc: - raise RuntimeError("OpenAI SDK is required for OpenAI-compatible providers. Install with `pip install openai`.") from exc - - client_kwargs: dict[str, Any] = {"api_key": self.api_key} - if self.base_url: - client_kwargs["base_url"] = self.base_url - client = OpenAI(**client_kwargs) - data_url = f"data:{mime_type};base64,{base64.b64encode(image_bytes).decode('ascii')}" + data_url = f"data:{mime_type};base64,{_b64(image_bytes)}" + return self._openai_chat( + system_prompt, + [ + {"type": "text", "text": user_prompt}, + {"type": "image_url", "image_url": {"url": data_url, "detail": image_detail}}, + ], + ) + + def _openai_chat(self, system_prompt: str, user_content: str | list[dict[str, Any]]) -> LLMResult: request: dict[str, Any] = { "model": self.model, "messages": [ {"role": "system", "content": system_prompt}, - { - "role": "user", - "content": [ - {"type": "text", "text": user_prompt}, - {"type": "image_url", "image_url": {"url": data_url, "detail": image_detail}}, - ], - }, + {"role": "user", "content": user_content}, ], } if self.max_output_tokens: request["max_tokens"] = self.max_output_tokens if self.temperature is not None: request["temperature"] = self.temperature - response = client.chat.completions.create(**request) + response = self._openai().chat.completions.create(**request) content = response.choices[0].message.content return LLMResult( text=content.strip() if content else "", @@ -286,52 +280,23 @@ def _generate_openai_image( ), ) + # -- Gemini ------------------------------------------------------------ + def _generate_gemini(self, system_prompt: str, user_prompt: str) -> LLMResult: - model = self.model.removeprefix("models/") - endpoint = f"{self.base_url.rstrip('/')}/models/{urllib.parse.quote(model, safe='')}:generateContent" - payload: dict[str, Any] = { - "system_instruction": {"parts": [{"text": system_prompt}]}, - "contents": [{"role": "user", "parts": [{"text": user_prompt}]}], - } - generation_config: dict[str, Any] = {} - if self.max_output_tokens: - generation_config["maxOutputTokens"] = self.max_output_tokens - if self.temperature is not None: - generation_config["temperature"] = self.temperature - if generation_config: - payload["generationConfig"] = generation_config + return self._gemini_generate(system_prompt, [{"text": user_prompt}]) - data = _post_json(endpoint, payload, {"x-goog-api-key": self.api_key}) - candidates = data.get("candidates") or [] - if not candidates: - raise RuntimeError(f"Gemini returned no candidates: {data}") - parts = candidates[0].get("content", {}).get("parts", []) - return LLMResult( - text="".join(part.get("text", "") for part in parts).strip(), - usage=_with_finish_reason( - _normalize_gemini_usage(data.get("usageMetadata")), candidates[0].get("finishReason") - ), + def _generate_gemini_image(self, system_prompt: str, user_prompt: str, image_bytes: bytes, mime_type: str) -> LLMResult: + return self._gemini_generate( + system_prompt, + [{"inline_data": {"mime_type": mime_type, "data": _b64(image_bytes)}}, {"text": user_prompt}], ) - def _generate_gemini_image(self, system_prompt: str, user_prompt: str, image_bytes: bytes, mime_type: str) -> LLMResult: + def _gemini_generate(self, system_prompt: str, parts: list[dict[str, Any]]) -> LLMResult: model = self.model.removeprefix("models/") endpoint = f"{self.base_url.rstrip('/')}/models/{urllib.parse.quote(model, safe='')}:generateContent" payload: dict[str, Any] = { "system_instruction": {"parts": [{"text": system_prompt}]}, - "contents": [ - { - "role": "user", - "parts": [ - { - "inline_data": { - "mime_type": mime_type, - "data": base64.b64encode(image_bytes).decode("ascii"), - } - }, - {"text": user_prompt}, - ], - } - ], + "contents": [{"role": "user", "parts": parts}], } generation_config: dict[str, Any] = {} if self.max_output_tokens: @@ -344,71 +309,42 @@ def _generate_gemini_image(self, system_prompt: str, user_prompt: str, image_byt candidates = data.get("candidates") or [] if not candidates: raise RuntimeError(f"Gemini returned no candidates: {data}") - parts = candidates[0].get("content", {}).get("parts", []) + response_parts = candidates[0].get("content", {}).get("parts", []) return LLMResult( - text="".join(part.get("text", "") for part in parts).strip(), + text="".join(part.get("text", "") for part in response_parts).strip(), usage=_with_finish_reason( _normalize_gemini_usage(data.get("usageMetadata")), candidates[0].get("finishReason") ), ) + # -- Claude ------------------------------------------------------------ + def _generate_claude(self, system_prompt: str, user_prompt: str) -> LLMResult: - endpoint = f"{self.base_url.rstrip('/')}/v1/messages" - payload: dict[str, Any] = { - "model": self.model, - "max_tokens": self.max_output_tokens, - "system": system_prompt, - "messages": [{"role": "user", "content": user_prompt}], - } - if self.temperature is not None: - payload["temperature"] = self.temperature + return self._claude_messages(system_prompt, user_prompt) - data = _post_json( - endpoint, - payload, - { - "x-api-key": self.api_key, - "anthropic-version": "2023-06-01", - }, - ) - blocks = data.get("content") or [] - return LLMResult( - text="".join(block.get("text", "") for block in blocks if block.get("type") == "text").strip(), - usage=_with_finish_reason(_normalize_claude_usage(data.get("usage")), data.get("stop_reason")), + def _generate_claude_image(self, system_prompt: str, user_prompt: str, image_bytes: bytes, mime_type: str) -> LLMResult: + return self._claude_messages( + system_prompt, + [ + {"type": "image", "source": {"type": "base64", "media_type": mime_type, "data": _b64(image_bytes)}}, + {"type": "text", "text": user_prompt}, + ], ) - def _generate_claude_image(self, system_prompt: str, user_prompt: str, image_bytes: bytes, mime_type: str) -> LLMResult: + def _claude_messages(self, system_prompt: str, user_content: str | list[dict[str, Any]]) -> LLMResult: endpoint = f"{self.base_url.rstrip('/')}/v1/messages" payload: dict[str, Any] = { "model": self.model, "max_tokens": self.max_output_tokens, "system": system_prompt, - "messages": [ - { - "role": "user", - "content": [ - { - "type": "image", - "source": { - "type": "base64", - "media_type": mime_type, - "data": base64.b64encode(image_bytes).decode("ascii"), - }, - }, - {"type": "text", "text": user_prompt}, - ], - } - ], + "messages": [{"role": "user", "content": user_content}], } if self.temperature is not None: payload["temperature"] = self.temperature data = _post_json( endpoint, payload, - { - "x-api-key": self.api_key, - "anthropic-version": "2023-06-01", - }, + {"x-api-key": self.api_key, "anthropic-version": ANTHROPIC_API_VERSION}, ) blocks = data.get("content") or [] return LLMResult( @@ -447,19 +383,19 @@ def resolve_provider_runtime(provider: str, model: str | None = None, base_url: return { "provider": spec.canonical_name, "model": _resolve_model(spec, model, for_vision=for_vision), - "base_url": _resolve_base_url(spec, base_url), + "base_url": _resolve_base_url(spec, base_url, for_vision=for_vision), "supports_image_input": spec.supports_image_input, } def _resolve_model(spec: ProviderSpec, explicit_model: str | None, for_vision: bool = False) -> str: if for_vision: - model = explicit_model or first_env(("SLIDENOTE_VISION_MODEL",) + spec.vision_model_envs) or spec.default_vision_model + model = explicit_model or first_env((VISION_MODEL_ENV,) + spec.vision_model_envs) or spec.default_vision_model else: - model = explicit_model or first_env(("SLIDENOTE_MODEL",) + spec.model_envs) or spec.default_model + model = explicit_model or first_env((TEXT_MODEL_ENV,) + spec.model_envs) or spec.default_model if not model: model_envs = spec.vision_model_envs if for_vision else spec.model_envs - generic_env = "SLIDENOTE_VISION_MODEL" if for_vision else "SLIDENOTE_MODEL" + generic_env = VISION_MODEL_ENV if for_vision else TEXT_MODEL_ENV raise RuntimeError( f"`{spec.canonical_name}` requires a model name. Pass `--model ...` or set one of: " f"{', '.join(model_envs) or generic_env}" @@ -477,8 +413,9 @@ def _resolve_api_key(spec: ProviderSpec, explicit_api_key: str | None) -> str: return key -def _resolve_base_url(spec: ProviderSpec, explicit_base_url: str | None) -> str | None: - return explicit_base_url or first_env(("SLIDENOTE_BASE_URL",) + spec.base_url_envs) or spec.base_url +def _resolve_base_url(spec: ProviderSpec, explicit_base_url: str | None, for_vision: bool = False) -> str | None: + generic_env = VISION_BASE_URL_ENV if for_vision else TEXT_BASE_URL_ENV + return explicit_base_url or first_env((generic_env,) + spec.base_url_envs) or spec.base_url def _post_json(url: str, payload: dict[str, Any], headers: dict[str, str]) -> dict[str, Any]: @@ -493,7 +430,7 @@ def _post_json(url: str, payload: dict[str, Any], headers: dict[str, str]) -> di }, ) try: - with urllib.request.urlopen(request, timeout=120) as response: + with urllib.request.urlopen(request, timeout=HTTP_TIMEOUT_SECONDS) as response: response_body = response.read().decode("utf-8") except urllib.error.HTTPError as exc: error_body = exc.read().decode("utf-8", errors="replace") @@ -503,6 +440,10 @@ def _post_json(url: str, payload: dict[str, Any], headers: dict[str, str]) -> di return json.loads(response_body) +def _b64(data: bytes) -> str: + return base64.b64encode(data).decode("ascii") + + def _guess_mime_type(path: Path) -> str: mime_type, _ = mimetypes.guess_type(path.name) if mime_type in {"image/jpeg", "image/png", "image/gif", "image/webp"}: diff --git a/slidenote/llm_cache.py b/slidenote/llm_cache.py index 0d801ef..ca28e23 100644 --- a/slidenote/llm_cache.py +++ b/slidenote/llm_cache.py @@ -2,11 +2,15 @@ import hashlib import json +import os +import tempfile +import time from datetime import datetime, timezone from pathlib import Path from typing import Any LLM_CACHE_SCHEMA_VERSION = 1 +_REPLACE_RETRY_DELAYS = (0.05, 0.1, 0.2, 0.4) def utc_now_iso() -> str: @@ -25,6 +29,34 @@ def make_cache_key(data: Any) -> str: return sha256_text(stable_json(data)) +def atomic_write_text(path: Path, content: str) -> None: + """Write via a temp file in the same directory, then ``os.replace``. + + Readers (GUI pollers, concurrent cache lookups) never observe a partially + written file. On Windows ``os.replace`` fails with PermissionError while + another process holds the target open, so it is retried briefly. + """ + path.parent.mkdir(parents=True, exist_ok=True) + fd, tmp_name = tempfile.mkstemp(dir=path.parent, prefix=f".{path.name}.", suffix=".tmp") + try: + with os.fdopen(fd, "w", encoding="utf-8") as handle: + handle.write(content) + for delay in (*_REPLACE_RETRY_DELAYS, None): + try: + os.replace(tmp_name, path) + return + except PermissionError: + if delay is None: + raise + time.sleep(delay) + except BaseException: + try: + os.unlink(tmp_name) + except OSError: + pass + raise + + class LLMCache: def __init__(self, cache_dir: Path, mode: str = "on") -> None: if mode not in {"on", "off", "refresh"}: @@ -69,6 +101,6 @@ def write(self, cache_key: str, entry: dict[str, Any]) -> Path | None: "created_at": utc_now_iso(), **entry, } - path.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8") + atomic_write_text(path, json.dumps(payload, ensure_ascii=False, indent=2)) return path diff --git a/slidenote/modality.py b/slidenote/modality.py index 6c735a2..995cb6e 100644 --- a/slidenote/modality.py +++ b/slidenote/modality.py @@ -1,13 +1,18 @@ from __future__ import annotations +import json +import os from collections import Counter from dataclasses import dataclass +from pathlib import Path from typing import Any from slidenote.llm_cache import utc_now_iso from slidenote.models import Deck, SlidePage +from slidenote.utils import file_sha256 PAGE_MODALITIES = {"native_text", "mixed", "image_only", "shape_diagram", "decorative"} +OVERRIDE_MODALITIES = PAGE_MODALITIES | {"unknown"} @dataclass(frozen=True, slots=True) @@ -30,6 +35,81 @@ def enrich_deck_with_modalities(deck: Deck) -> dict[str, Any]: return build_modality_report(deck, results) +def apply_modality_overrides(deck: Deck, report: dict[str, Any], manifest_path: Path | str) -> dict[str, Any]: + """Apply reviewer page labels after local classification, before visual processing. + + The GUI's version 1 manifest stores overrides under string slide IDs. Invalid or + stale entries are reported and ignored so a handwritten manifest cannot break a + build. A source hash, when provided, takes precedence over the upload path. + """ + path = Path(manifest_path) + if not path.is_file(): + return report + + warnings: list[str] = [] + try: + manifest = json.loads(path.read_text(encoding="utf-8")) + except (OSError, UnicodeError, json.JSONDecodeError) as exc: + warnings.append(f"Could not read modality override manifest: {exc}") + return _record_override_result(report, path, [], warnings) + + if not isinstance(manifest, dict) or type(manifest.get("schema_version")) is not int or manifest["schema_version"] != 1: + warnings.append("Modality override manifest must be a version 1 JSON object.") + return _record_override_result(report, path, [], warnings) + if not _manifest_matches_source(manifest, deck.source_path, warnings): + return _record_override_result(report, path, [], warnings) + + overrides = manifest.get("pages") + if not isinstance(overrides, dict): + warnings.append("Modality override manifest.pages must be an object keyed by slide ID.") + return _record_override_result(report, path, [], warnings) + + pages_by_id = {page.slide_id: page for page in deck.pages} + report_pages = { + item.get("slide_id"): item + for item in report.get("pages", []) + if isinstance(item, dict) + } + applied: list[dict[str, Any]] = [] + for raw_slide_id, entry in overrides.items(): + if not isinstance(raw_slide_id, str) or not raw_slide_id.isdecimal() or int(raw_slide_id) < 1 or str(int(raw_slide_id)) != raw_slide_id: + warnings.append(f"Invalid modality override slide ID: {raw_slide_id!r}.") + continue + slide_id = int(raw_slide_id) + page = pages_by_id.get(slide_id) + report_page = report_pages.get(slide_id) + if page is None or report_page is None: + warnings.append(f"Modality override slide {slide_id} is not in this deck.") + continue + if not isinstance(entry, dict) or not isinstance(entry.get("modality"), str) or entry["modality"] not in OVERRIDE_MODALITIES: + warnings.append(f"Invalid modality override for slide {slide_id}.") + continue + + modality = entry["modality"] + previous_modality = page.page_modality + note = entry.get("note") if isinstance(entry.get("note"), str) else "" + updated_at = entry.get("updated_at") if isinstance(entry.get("updated_at"), str) else None + hints = list(page.processing_hints) if modality == "unknown" else _override_processing_hints(page, modality) + confidence = 0.0 if modality == "unknown" else 1.0 + page.page_modality = modality + page.modality_confidence = confidence + page.modality_reasons = ["manual_override"] + page.processing_hints = hints + report_page.update( + modality=modality, + confidence=confidence, + reasons=list(page.modality_reasons), + processing_hints=list(hints), + classifier_modality=previous_modality, + manual_override={"note": note, "updated_at": updated_at}, + ) + applied.append({"slide_id": slide_id, "modality": modality, "previous_modality": previous_modality}) + + if applied: + report["summary"] = _modality_summary(report["pages"]) + return _record_override_result(report, path, applied, warnings) + + def classify_page_modality(page: SlidePage) -> PageModalityResult: stats = _page_stats(page) text_len = int(stats["text_chars"]) @@ -83,33 +163,24 @@ def classify_page_modality(page: SlidePage) -> PageModalityResult: def build_modality_report(deck: Deck, results: list[PageModalityResult]) -> dict[str, Any]: - counts = Counter(result.modality for result in results) + pages = [ + { + "slide_id": result.slide_id, + "modality": result.modality, + "confidence": result.confidence, + "reasons": result.reasons, + "processing_hints": result.processing_hints, + "stats": result.stats, + } + for result in results + ] return { "schema_version": 1, "generated_at": utc_now_iso(), "source_path": deck.source_path, "source_type": deck.source_type, - "summary": { - "pages_total": len(results), - "modalities": dict(sorted(counts.items())), - "image_driven_pages": sum(1 for result in results if result.modality in {"image_only", "shape_diagram"}), - "embedded_image_pages": sum(1 for result in results if "use_embedded_images" in result.processing_hints), - "ocr_recommended_pages": sum(1 for result in results if "ocr_page_screenshot" in result.processing_hints), - "figure_crop_recommended_pages": sum( - 1 for result in results if "crop_figures_from_screenshot" in result.processing_hints - ), - }, - "pages": [ - { - "slide_id": result.slide_id, - "modality": result.modality, - "confidence": result.confidence, - "reasons": result.reasons, - "processing_hints": result.processing_hints, - "stats": result.stats, - } - for result in results - ], + "summary": _modality_summary(pages), + "pages": pages, } @@ -117,6 +188,98 @@ def page_has_hint(page: SlidePage, hint: str) -> bool: return hint in (page.processing_hints or []) +def page_has_manual_modality(page: SlidePage) -> bool: + return page.page_modality != "unknown" and "manual_override" in (page.modality_reasons or []) + + +def _override_processing_hints(page: SlidePage, modality: str) -> list[str]: + has_text = bool(page.text_blocks or page.tables) + has_images = any(not image.ignored and image.role != "page_image" for image in page.images) + has_ocr_images = any(not image.ignored for image in page.images) + has_screenshot = bool(page.page_screenshot) + if modality == "native_text": + return ["use_extracted_text"] + if modality == "mixed": + hints = ["use_extracted_text"] if has_text else [] + if has_images: + hints.extend(["use_embedded_images", "vision_large_images"]) + elif has_screenshot: + hints.append("vision_page_screenshot") + if has_screenshot and sum(len(block.content.strip()) for block in page.text_blocks) < 80: + hints.append("ocr_page_screenshot") + return hints + if modality == "image_only": + hints = ["ocr_page_screenshot"] if has_screenshot or has_ocr_images else [] + if has_screenshot: + hints.extend(["crop_figures_from_screenshot", "vision_page_screenshot"]) + if has_images: + hints.extend(["use_embedded_images", "vision_large_images"]) + return hints + if modality == "shape_diagram": + hints = ["use_extracted_text"] if has_text else [] + if has_screenshot: + hints.extend(["crop_figures_from_screenshot", "vision_page_screenshot"]) + if has_images: + hints.extend(["use_embedded_images", "vision_large_images"]) + return hints + if modality == "decorative": + return ["low_priority"] + return [] + + +def _modality_summary(pages: list[dict[str, Any]]) -> dict[str, Any]: + counts = Counter(str(page["modality"]) for page in pages) + return { + "pages_total": len(pages), + "modalities": dict(sorted(counts.items())), + "image_driven_pages": sum(1 for page in pages if page["modality"] in {"image_only", "shape_diagram"}), + "embedded_image_pages": sum(1 for page in pages if "use_embedded_images" in page["processing_hints"]), + "ocr_recommended_pages": sum(1 for page in pages if "ocr_page_screenshot" in page["processing_hints"]), + "figure_crop_recommended_pages": sum(1 for page in pages if "crop_figures_from_screenshot" in page["processing_hints"]), + } + + +def _record_override_result( + report: dict[str, Any], path: Path, applied: list[dict[str, Any]], warnings: list[str] +) -> dict[str, Any]: + report.setdefault("summary", {})["override_pages"] = len(applied) + report["overrides"] = { + "manifest_path": str(path), + "applied": applied, + "warnings": warnings, + } + return report + + +def _manifest_matches_source(manifest: dict[str, Any], source_path: str, warnings: list[str]) -> bool: + source_hash = manifest.get("source_sha256") + if source_hash is not None: + if not isinstance(source_hash, str) or len(source_hash) != 64 or any(char not in "0123456789abcdefABCDEF" for char in source_hash): + warnings.append("Modality override source_sha256 must be a 64-character hex digest.") + return False + try: + digest = file_sha256(Path(source_path)).removeprefix("sha256:") + except OSError as exc: + warnings.append(f"Could not verify modality override source: {exc}") + return False + if digest != source_hash.lower(): + warnings.append("Modality override source hash does not match this deck.") + return False + return True + + manifest_source_path = manifest.get("source_path") + if manifest_source_path is not None: + if not isinstance(manifest_source_path, str) or not manifest_source_path: + warnings.append("Modality override source_path must be a non-empty string.") + return False + expected = os.path.normcase(os.path.normpath(str(Path(manifest_source_path).resolve()))) + actual = os.path.normcase(os.path.normpath(str(Path(source_path).resolve()))) + if expected != actual: + warnings.append("Modality override source path does not match this deck.") + return False + return True + + def _page_stats(page: SlidePage) -> dict[str, Any]: text_chars = sum(len(block.content.strip()) for block in page.text_blocks) content_images = sum(1 for image in page.images if not image.ignored and image.role != "page_image") diff --git a/slidenote/notes/__init__.py b/slidenote/notes/__init__.py index 8a6fe72..62ff57a 100644 --- a/slidenote/notes/__init__.py +++ b/slidenote/notes/__init__.py @@ -7,14 +7,11 @@ from slidenote.figure_grounding import FIGURE_PLACEMENT_MODES from slidenote.models import Deck -from .assembly import ( - _prepare_note_assets, - _select_note_contexts, - _validate_markdown_image_links, -) +from .assets import _prepare_note_assets, _validate_markdown_image_links +from .contexts import _select_note_contexts from .local import _generate_notes_locally from .options import NOTE_PROFILES, TEACHING_ENRICHMENT_MODES, resolve_note_depth, should_run_teaching_enrichment -from .orchestrator import _generate_notes_with_llm +from .direct import _generate_notes_with_llm # --------------------------------------------------------------------------- # Constants @@ -41,11 +38,11 @@ class NoteOptions: """All note-generation configuration, passed down the pipeline as one object. Replaces the ~31-parameter call signatures that used to thread through - notes/__init__ -> orchestrator -> direct/lecture_weave -> llm_calls. + notes/__init__ -> direct/lecture_weave -> llm_calls. """ use_llm: bool = False - provider: str = "openai" + provider: str = "deepseek" model: str | None = None api_key: str | None = None base_url: str | None = None @@ -84,6 +81,7 @@ class NoteGenerationResult: page_notes_markdown: str | None = None weave_report: dict[str, Any] | None = None teaching_report: dict[str, Any] | None = None + generation_warnings: list[str] | None = None # --------------------------------------------------------------------------- diff --git a/slidenote/notes/assembly.py b/slidenote/notes/assembly.py deleted file mode 100644 index 43fded6..0000000 --- a/slidenote/notes/assembly.py +++ /dev/null @@ -1,1345 +0,0 @@ -from __future__ import annotations - -import base64 -import html -import mimetypes -import re -import shutil -from dataclasses import dataclass -from pathlib import Path -from typing import Any - -from slidenote.figure_grounding import note_candidate_images -from slidenote.image_ranking import sorted_images_by_importance -from slidenote.models import Deck, ImageAsset, SlidePage, TextBlock -from slidenote.utils import ( - context_title, - display_path, - looks_like_outline_page, - looks_like_section_title_page, - source_tokens, - sum_int, -) -from .versions import PAGE_LECTURE_PROMPT_VERSION, TEACHING_ENRICHMENT_PROMPT_VERSION, WEAVE_PROMPT_VERSION - -SOURCE_COMMENT_PREFIX = "slidenote-source:" -_CSS_HEX_COLOR_RE = re.compile(r"^#[0-9A-Fa-f]{6}$") - - -@dataclass(frozen=True, slots=True) -class NoteContext: - id: str - kind: str - title: str - pages: list[SlidePage] - - -# --------------------------------------------------------------------------- -# Figure grounding & image helpers -# --------------------------------------------------------------------------- - -def _ensure_grounded_figures( - markdown: str, - deck: Deck, - asset_map: dict[str, str], - source_display: str, - figure_placement: str, -) -> str: - current = markdown.rstrip() - frontmatter_slide_ids = _leading_frontmatter_slide_ids(deck.pages) - for page in deck.pages: - if page.slide_id in frontmatter_slide_ids: - continue - for image in note_candidate_images(page): - image_path = _asset_display_path(image.path, asset_map) - block = "\n".join(_render_image(page, image, asset_map=asset_map, source_display=source_display)).strip() - if not block: - continue - if figure_placement == "inline": - current = _remove_existing_image_block(current, image_path, image) - current = _insert_figure_block(current, page, image, block, figure_placement) - continue - if _image_markdown_present(current, image_path): - if image.id not in source_tokens(current): - current = _ensure_image_source_marker(current, page, image, image_path, source_display) - continue - current = _insert_figure_block(current, page, image, block, figure_placement) - return current.rstrip() + "\n" - - -def _image_markdown_present(markdown: str, image_path: str) -> bool: - if not image_path: - return False - escaped = re.escape(image_path.strip()) - return bool(re.search(rf"!\[[^\]]*]\({escaped}\)", markdown)) or image_path in markdown - - -def _ensure_image_source_marker( - markdown: str, - page: SlidePage, - image: ImageAsset, - image_path: str, - source_display: str, -) -> str: - marker = _source_marker(page.slide_id, _image_source_ids(image), source_display) - if not marker: - return markdown - lines = markdown.splitlines() - for index, line in enumerate(lines): - if image_path in line and line.lstrip().startswith("!["): - if marker in line or (index + 1 < len(lines) and marker in lines[index + 1]): - return markdown - new_lines = list(lines) - new_lines.insert(index + 1, marker) - return "\n".join(new_lines).rstrip() + "\n" - return markdown - - -def _remove_existing_image_block(markdown: str, image_path: str, image: ImageAsset) -> str: - if not image_path: - return markdown - lines = markdown.splitlines() - remove: set[int] = set() - source_ids = set(_image_source_ids(image)) - for index, line in enumerate(lines): - if not _line_has_image_target(line, image_path): - continue - remove.update(_image_block_indexes_to_remove(lines, index, image, source_ids)) - if not remove: - return markdown - kept = [line for index, line in enumerate(lines) if index not in remove] - return _collapse_blank_lines(kept).rstrip() + "\n" - - -def _image_block_indexes_to_remove(lines: list[str], image_index: int, image: ImageAsset, source_ids: set[str]) -> set[int]: - remove = {image_index} - before = image_index - 1 - while before >= 0 and not lines[before].strip(): - remove.add(before) - before -= 1 - if before >= 0 and _is_marker_only_for_ids(lines[before], source_ids): - remove.add(before) - caption = before - 1 - while caption >= 0 and not lines[caption].strip(): - remove.add(caption) - caption -= 1 - if caption >= 0 and _is_image_caption_line(lines[caption], image): - remove.add(caption) - - after = image_index + 1 - while after < len(lines) and not lines[after].strip(): - remove.add(after) - after += 1 - if after < len(lines) and _is_marker_only_for_ids(lines[after], source_ids): - remove.add(after) - return remove - - -def _is_image_caption_line(line: str, image: ImageAsset) -> bool: - stripped = line.strip() - caption = (image.caption or "").strip() - if caption and stripped in {caption, f"{caption}\u3002"}: - return True - return bool(re.fullmatch(r"\u7b2c\s*\d+\s*\u9875(?:\u56fe\u7247|\u56fe\u793a|\u622a\u56fe).*[\u3002.]?", stripped)) - - -def _line_has_image_target(line: str, image_path: str) -> bool: - normalized_path = image_path.strip().strip("<>").replace("\\", "/") - for target in re.findall(r"!\[[^\]]*]\(([^)]+)\)", line): - normalized_target = target.strip().strip("<>").replace("\\", "/") - if normalized_target == normalized_path: - return True - return False - - -def _is_marker_only_for_ids(line: str, source_ids: set[str]) -> bool: - stripped = line.strip() - if not stripped or SOURCE_COMMENT_PREFIX not in stripped: - return False - if not re.fullmatch(r"", stripped): - return False - return bool(source_ids.intersection(source_tokens(stripped))) - - -def _collapse_blank_lines(lines: list[str]) -> str: - collapsed: list[str] = [] - blank = False - for line in lines: - is_blank = not line.strip() - if is_blank and blank: - continue - collapsed.append(line) - blank = is_blank - return "\n".join(collapsed) - - -def _insert_figure_block(markdown: str, page: SlidePage, image: ImageAsset, block: str, figure_placement: str) -> str: - if figure_placement == "inline": - inserted = _insert_after_anchor_source(markdown, image.anchor_element_ids, block) - if inserted != markdown: - return inserted - inserted = _insert_after_page_source(markdown, page.slide_id, block) - if inserted != markdown: - return inserted - fallback_heading = f"### \u7b2c {page.slide_id} \u9875\u56fe\u793a" - return f"{markdown.rstrip()}\n\n{fallback_heading}\n\n{block}" - - -def _insert_after_anchor_source(markdown: str, anchor_ids: list[str], block: str) -> str: - if not anchor_ids: - return markdown - lines = markdown.splitlines() - for index, line in enumerate(lines): - if SOURCE_COMMENT_PREFIX not in line: - continue - if not any(anchor_id in line for anchor_id in anchor_ids): - continue - insert_at = _paragraph_end_after(lines, index) - return _insert_lines(lines, insert_at, block) - return markdown - - -def _insert_after_page_source(markdown: str, slide_id: int, block: str) -> str: - lines = markdown.splitlines() - marker = f"p{slide_id}:" - candidate_index: int | None = None - for index, line in enumerate(lines): - if SOURCE_COMMENT_PREFIX in line and marker in line: - candidate_index = index - if candidate_index is None: - return markdown - insert_at = _paragraph_end_after(lines, candidate_index) - return _insert_lines(lines, insert_at, block) - - -def _paragraph_end_after(lines: list[str], index: int) -> int: - cursor = index + 1 - while cursor < len(lines) and lines[cursor].strip(): - cursor += 1 - while cursor < len(lines) and not lines[cursor].strip(): - cursor += 1 - return cursor - - -def _insert_lines(lines: list[str], index: int, block: str) -> str: - new_lines = list(lines) - insert = ["", *block.splitlines(), ""] - new_lines[index:index] = insert - return "\n".join(new_lines).rstrip() + "\n" - - -# --------------------------------------------------------------------------- -# Image rendering (moved from local to break circular dependency) -# --------------------------------------------------------------------------- - -def _render_image(page: SlidePage, image: ImageAsset, asset_map: dict[str, str], source_display: str) -> list[str]: - caption = image.caption or f"\u7b2c {page.slide_id} \u9875\u56fe\u7247" - lines = [ - f"{caption}\u3002", - _source_marker(page.slide_id, _image_source_ids(image), source_display), - "", - ] - explanation = image.figure_explanation or image.visual_summary - if explanation: - label = "\u56fe\u793a\u8bf4\u660e" if image.figure_explanation else "\u56fe\u7247\u89c6\u89c9\u89e3\u6790" - lines.append(f"{label}\uff1a{_ensure_sentence(explanation)}") - if _should_render_image_ocr(image, explanation): - if explanation: - lines.append("") - lines.append("\u56fe\u7247 OCR \u6587\u5b57\uff1a") - lines.extend(_quote_multiline(image.ocr_text)) - if explanation or _should_render_image_ocr(image, explanation): - lines.append("") - lines.append(f"![{caption}]({_asset_display_path(image.path, asset_map)})") - return lines - - -def _should_render_image_ocr(image: ImageAsset, explanation: str | None) -> bool: - if not image.ocr_text: - return False - if image.figure_explanation_status == "ocr_text": - return False - return not bool(explanation) - - -def _ensure_sentence(text: str) -> str: - value = " ".join(text.split()).strip() - if value and value[-1] not in "\u3002.!!\uff1f?\uff1a:": - value += "\u3002" - return value - - -def _quote_multiline(text: str) -> list[str]: - lines = [line.strip() for line in text.splitlines() if line.strip()] - return [f"> {line}" for line in lines] - - -def _styled_block_text(block: TextBlock) -> str | None: - if not block.style_runs: - return None - pieces: list[str] = [] - has_visible_style = False - for run in block.style_runs: - text = str(run.get("text") or "") - if not text: - continue - escaped = html.escape(text).replace("\n", "
") - css: list[str] = [] - color = _safe_css_color(run.get("color")) - if color: - css.append(f"color:{color}") - if run.get("bold") is True: - css.append("font-weight:700") - if run.get("italic") is True: - css.append("font-style:italic") - if css: - has_visible_style = True - pieces.append(f'{escaped}') - else: - pieces.append(escaped) - rendered = "".join(pieces).strip() - return rendered if has_visible_style and rendered else None - - -def _safe_css_color(value: object) -> str | None: - color = str(value or "").strip() - return color.upper() if _CSS_HEX_COLOR_RE.fullmatch(color) else None - - -# --------------------------------------------------------------------------- -# Composition / final markdown assembly -# --------------------------------------------------------------------------- - -def _compose_final_markdown( - deck: Deck, - contexts: list[NoteContext], - final_chunks: dict[str, str], - section_plan: dict[str, Any] | None, - source_display: str, -) -> str: - del source_display - lines = [f"# {_document_title(deck)}", ""] - add_context_headings = _should_add_context_headings(contexts) - leading_frontmatter_slide_ids = _leading_frontmatter_slide_ids(deck.pages) - section_number = 1 - for context in contexts: - content = final_chunks.get(context.id, "").strip() - if not content: - continue - if add_context_headings: - heading_title = _context_heading_title(context, section_plan) - if _is_frontmatter_heading(heading_title, context) and len(contexts) > 1: - content = _frontmatter_source_markers(context.pages) - else: - lines.append(_context_heading(context, heading_title, section_number)) - lines.append("") - section_number += 1 - content = _prepare_context_chunk(content, heading_title, add_outer_heading=True) - content = _strip_leading_frontmatter_content(content, context, leading_frontmatter_slide_ids) - content = _number_subsection_headings(content) - else: - content = _prepare_context_chunk(content, context.title, add_outer_heading=False) - if content: - lines.append(content) - lines.append("") - return "\n".join(lines).rstrip() + "\n" - - -# --------------------------------------------------------------------------- -# Heading / title utilities -# --------------------------------------------------------------------------- - -def _document_title(deck: Deck) -> str: - stem = Path(deck.source_path).stem - for page in deck.pages[:3]: - title = (page.title or "").strip() - if title and not _is_generic_heading_text(title): - return f"{stem}\uff1a{title}" if title != stem else stem - return stem - - -def _should_add_context_headings(contexts: list[NoteContext]) -> bool: - if not contexts: - return False - if len(contexts) > 1: - return True - return contexts[0].kind == "section" - - -def _context_heading(context: NoteContext, title: str, section_number: int) -> str: - if context.kind == "page": - slide_id = context.pages[0].slide_id if context.pages else section_number - return f"## \u7b2c {slide_id} \u9875\uff1a{title}" - return f"## {_chinese_ordinal(section_number)}\u3001{title}" - - -def _context_heading_title(context: NoteContext, section_plan: dict[str, Any] | None) -> str: - planned_title = _planned_context_title(context, section_plan) - title = planned_title or context.title or "" - title = _clean_heading_text(title) - if title: - return title - if context.kind == "page" and context.pages: - return context.pages[0].title or f"\u7b2c {context.pages[0].slide_id} \u9875" - return "\u672c\u8282\u5185\u5bb9" - - -def _planned_context_title(context: NoteContext, section_plan: dict[str, Any] | None) -> str | None: - if not section_plan: - return None - sections = section_plan.get("sections") - if not isinstance(sections, list): - return None - slide_ids = [page.slide_id for page in context.pages] - for section in sections: - if not isinstance(section, dict): - continue - if section.get("section_id") == context.id or section.get("slide_ids") == slide_ids: - title = str(section.get("title") or "").strip() - return title or None - return None - - -def _prepare_context_chunk(markdown: str, section_title: str, add_outer_heading: bool) -> str: - text = _remove_generation_info_sections(markdown) - text = _drop_redundant_leading_headings(text, section_title) if add_outer_heading else text - text = _demote_chunk_headings(text, minimum_level=3 if add_outer_heading else 2) - text = _remove_empty_sections(text) - return text.strip() - - -def _drop_redundant_leading_headings(markdown: str, section_title: str) -> str: - lines = markdown.splitlines() - while True: - first_index = next((index for index, line in enumerate(lines) if line.strip()), None) - if first_index is None: - return "" - match = re.match(r"^(#{1,6})\s+(.*)$", lines[first_index].strip()) - if not match: - return "\n".join(lines).strip() - heading_text = _clean_heading_text(match.group(2)) - if not _is_redundant_context_heading(heading_text, section_title): - return "\n".join(lines).strip() - del lines[first_index] - while first_index < len(lines) and not lines[first_index].strip(): - del lines[first_index] - - -def _is_redundant_context_heading(heading_text: str, section_title: str) -> bool: - heading_norm = _normalize_title_key(heading_text) - context_norm = _normalize_title_key(section_title) - if not heading_norm: - return True - if _is_generic_heading_text(heading_text): - return True - return bool(context_norm and (heading_norm == context_norm or heading_norm in context_norm or context_norm in heading_norm)) - - -def _demote_chunk_headings(markdown: str, minimum_level: int) -> str: - lines: list[str] = [] - for line in markdown.splitlines(): - match = re.match(r"^(#{1,6})\s+(.*)$", line) - if not match: - lines.append(line) - continue - text = _clean_heading_text(match.group(2)) - if not text or _is_generic_heading_text(text): - continue - level = max(minimum_level, len(match.group(1))) - lines.append("#" * min(level, 6) + " " + text) - return "\n".join(lines) - - -def _strip_leading_frontmatter_content(markdown: str, context: NoteContext, leading_frontmatter_slide_ids: set[int]) -> str: - frontmatter_slide_ids = {page.slide_id for page in context.pages if page.slide_id in leading_frontmatter_slide_ids} - if not frontmatter_slide_ids: - return markdown - blocks = re.split(r"\n\s*\n", markdown.strip()) - kept: list[str] = [] - dropping = True - for block in blocks: - stripped = block.strip() - if not stripped: - continue - if dropping and _is_horizontal_rule(stripped): - continue - if dropping and _is_droppable_frontmatter_block(stripped, frontmatter_slide_ids): - continue - dropping = False - kept.append(stripped) - marker = _frontmatter_source_markers([page for page in context.pages if page.slide_id in frontmatter_slide_ids]) - if not marker: - return "\n\n".join(kept).strip() - body = "\n\n".join(kept).strip() - return f"{marker}\n\n{body}".strip() if body else marker - - -def _is_droppable_frontmatter_block(block: str, frontmatter_slide_ids: set[int]) -> bool: - match = re.match(r"^(#{1,6})\s+(.*)$", block) - if match: - return _is_generic_heading_text(match.group(2)) or _looks_like_frontmatter_text(match.group(2)) - slide_ids = _source_slide_ids(block) - if slide_ids and slide_ids.issubset(frontmatter_slide_ids): - return True - return _looks_like_frontmatter_text(block) - - -def _is_horizontal_rule(block: str) -> bool: - return bool(re.fullmatch(r"[-*_]{3,}", block.strip())) - - -def _number_subsection_headings(markdown: str) -> str: - counters: list[int] = [] - base_level: int | None = None - lines: list[str] = [] - for line in markdown.splitlines(): - match = re.match(r"^(#{1,6})\s+(.*)$", line) - if not match: - lines.append(line) - continue - level = len(match.group(1)) - title = _strip_heading_number(match.group(2).strip()) - if level < 3: - counters = [] - base_level = None - lines.append(line) - continue - if base_level is None or level < base_level: - base_level = level - counters = [] - depth = max(0, level - base_level) - while len(counters) <= depth: - counters.append(0) - counters = counters[: depth + 1] - counters[depth] += 1 - prefix = ".".join(str(value) for value in counters) - separator = ". " if len(counters) == 1 else " " - lines.append(f"{match.group(1)} {prefix}{separator}{title}") - return "\n".join(lines) - - -def _strip_heading_number(title: str) -> str: - return re.sub( - r"^\s*(?:\d+(?:\.\d+)*\.?|[\u4e00\u4e8c\u4e09\u56db\u4e94\u516d\u4e03\u516b\u4e5d\u5341]+[\u3001.])\s*", - "", - title, - ).strip() - - -def _remove_generation_info_sections(markdown: str) -> str: - lines = markdown.splitlines() - kept: list[str] = [] - skipping = False - skip_level = 0 - for line in lines: - match = re.match(r"^(#{1,6})\s+(.*)$", line) - if match: - level = len(match.group(1)) - heading = _normalize_title_key(match.group(2)) - if heading in {"\u751f\u6210\u4fe1\u606f", "generationinfo", "generationmetadata"}: - skipping = True - skip_level = level - continue - if skipping and level <= skip_level: - skipping = False - if not skipping: - kept.append(line) - return "\n".join(kept) - - -def _remove_empty_sections(markdown: str) -> str: - lines = markdown.splitlines() - cleaned: list[str] = [] - previous_blank = False - for line in lines: - blank = not line.strip() - if blank and previous_blank: - continue - cleaned.append(line) - previous_blank = blank - return "\n".join(cleaned).strip() - - -def _clean_heading_text(value: str) -> str: - text = re.sub(r"", "", value).strip() - text = re.sub(r"^\u8bfe\u7a0b\u7b14\u8bb0[\uff1a:\s-]*", "", text).strip() - text = re.sub(r"^\s*[\uff08(]?\s*(?:\d+|[\u4e00\u4e8c\u4e09\u56db\u4e94\u516d\u4e03\u516b\u4e5d\u5341]+)\s*[)\uff09.\u3001]\s*", "", text).strip() - return text.strip("\uff1a: -") - - -def _normalize_title_key(value: str) -> str: - return re.sub(r"[\s:\uff1a,\uff0c.\u3002;\uff1b\u3001\-_\u2014\uff08\uff09()\u300a\u300b<>]+", "", _clean_heading_text(value)).lower() - - -def _is_generic_heading_text(value: str) -> bool: - normalized = _normalize_title_key(value) - return normalized in { - "", - "\u8bfe\u7a0b\u7b14\u8bb0", - "\u7b14\u8bb0", - "\u8bb2\u4e49", - "\u751f\u6210\u4fe1\u606f", - "\u89e3\u6790\u63d0\u9192", - "\u76ee\u5f55", - "contents", - "overview", - } - - -def _is_frontmatter_heading(title: str, context: NoteContext) -> bool: - normalized = _normalize_title_key(title) - if normalized in {"\u76ee\u5f55", "contents", "\u8bfe\u7a0b\u6982\u89c8", "overview"}: - return True - if len(context.pages) <= 2 and all(_normalize_title_key(page.title or "") in {"\u76ee\u5f55", "contents"} for page in context.pages): - return True - return False - - -def _leading_frontmatter_slide_ids(pages: list[SlidePage]) -> set[int]: - slide_ids: set[int] = set() - for index, page in enumerate(pages): - if not _is_frontmatter_page(page, index): - break - slide_ids.add(page.slide_id) - return slide_ids - - -def _is_frontmatter_page(page: SlidePage, index: int) -> bool: - title = page.title or "" - normalized_title = _normalize_title_key(title) - if normalized_title in {"\u76ee\u5f55", "contents", "outline", "agenda"}: - return True - text = "\n".join([title, *(block.content for block in page.text_blocks)]) - if "\u76ee\u5f55" in text or "Contents" in text: - return True - if index == 0 and _looks_like_cover_page(text): - return True - return index <= 3 and looks_like_outline_page(text) - - -def _looks_like_cover_page(text: str) -> bool: - normalized = _normalize_title_key(text) - cover_markers = { - "\u8bb2\u5e08", - "\u6559\u5e08", - "\u6559\u6388", - "\u8054\u7cfb\u90ae\u7bb1", - "\u90ae\u7bb1", - "\u4e3b\u9875", - "email", - "homepage", - "http", - "www", - } - return any(marker in normalized for marker in cover_markers) - - -def _looks_like_frontmatter_text(text: str) -> bool: - normalized = _normalize_title_key(text) - markers = { - "\u76ee\u5f55", - "\u8bfe\u7a0b\u76ee\u5f55", - "\u672c\u7ae0\u76ee\u5f55", - "\u4e3b\u6807\u9898", - "\u526f\u6807\u9898", - "\u8bb2\u5e08", - "\u6559\u6388", - "\u8054\u7cfb\u90ae\u7bb1", - "\u4e3b\u9875", - "contents", - "overview", - } - return any(marker in normalized for marker in markers) - - -def _frontmatter_source_markers(pages: list[SlidePage]) -> str: - markers = [_source_marker(page.slide_id, _page_source_ids(page), "hidden") for page in pages] - return "\n".join(marker for marker in markers if marker) - - -def _chinese_ordinal(index: int) -> str: - numerals = ["\u4e00", "\u4e8c", "\u4e09", "\u56db", "\u4e94", "\u516d", "\u4e03", "\u516b", "\u4e5d", "\u5341"] - if 1 <= index <= 10: - return numerals[index - 1] - if 11 <= index <= 19: - return "\u5341" + numerals[index - 11] - if index == 20: - return "\u4e8c\u5341" - return str(index) - - -# --------------------------------------------------------------------------- -# Source markers -# --------------------------------------------------------------------------- - -def _source_marker(slide_id: int, element_ids: list[str], source_display: str) -> str: - ids = [element_id for element_id in element_ids if element_id] - comment = f"" if ids else "" - if source_display == "hidden": - return comment - if source_display == "footnote": - return f"\uff08PPT \u7b2c {slide_id} \u9875\uff09 {comment}".rstrip() - detail = "\u3001".join(ids) - return f"\u3010\u5bf9\u5e94 PPT\uff1a\u7b2c {slide_id} \u9875\uff0c\u5143\u7d20 {detail}\u3011 {comment}".rstrip() - - -def _image_source_ids(image: ImageAsset) -> list[str]: - ids: list[str] = [] - seen: set[str] = set() - for element_id in [image.id, *image.source_element_ids]: - if element_id and element_id not in seen: - ids.append(element_id) - seen.add(element_id) - return ids - - -def _page_element_ids(page: SlidePage) -> list[str]: - ids = [block.id for block in page.text_blocks] - ids.extend(table.id for table in page.tables) - return ids - - -def _page_source_ids(page: SlidePage) -> list[str]: - ids = _page_element_ids(page) - ids.extend(image.id for image in page.images if not image.ignored) - return ids - - -# --------------------------------------------------------------------------- -# Asset management -# --------------------------------------------------------------------------- - -def _prepare_note_assets(deck: Deck, output_root: Path, asset_mode: str, screenshot_policy: str) -> tuple[dict[str, str], list[str]]: - asset_map: dict[str, str] = {} - warnings: list[str] = [] - seen_destinations: set[Path] = set() - for rel_path, kind in _iter_note_asset_paths(deck, screenshot_policy=screenshot_policy): - if rel_path in asset_map: - continue - source_path = _resolve_output_asset(output_root, rel_path) - if not source_path.exists(): - warnings.append(f"Missing note asset: {rel_path}") - continue - if asset_mode == "absolute": - asset_map[rel_path] = source_path.as_posix() - elif asset_mode == "embed": - embedded = _embed_asset(source_path) - if embedded: - asset_map[rel_path] = embedded - else: - warnings.append(f"Could not embed note asset: {rel_path}") - else: - destination = _bundled_asset_destination(output_root, rel_path, kind, seen_destinations) - destination.parent.mkdir(parents=True, exist_ok=True) - shutil.copy2(source_path, destination) - asset_map[rel_path] = destination.relative_to(output_root).as_posix() - seen_destinations.add(destination) - return asset_map, warnings - - -def _iter_note_asset_paths(deck: Deck, screenshot_policy: str) -> list[tuple[str, str]]: - paths: list[tuple[str, str]] = [] - for page in deck.pages: - if _should_render_screenshot(page, screenshot_policy): - paths.append((page.page_screenshot, "screenshots")) - for image in sorted_images_by_importance(page.images): - if not image.ignored: - kind = "figures" if image.role in {"figure_crop", "composite_figure"} else "images" - paths.append((image.path, kind)) - return paths - - -def _resolve_output_asset(output_root: Path, path: str) -> Path: - asset_path = Path(path) - if asset_path.is_absolute(): - return asset_path - return (output_root / asset_path).resolve() - - -def _bundled_asset_destination(output_root: Path, rel_path: str, kind: str, seen_destinations: set[Path]) -> Path: - source = Path(rel_path) - subdir = "screenshots" if kind == "screenshots" else "figures" if kind == "figures" else "images" - stem = source.stem or "asset" - suffix = source.suffix or ".png" - destination = output_root / "notes.assets" / subdir / f"{stem}{suffix}" - counter = 2 - while destination in seen_destinations: - destination = output_root / "notes.assets" / subdir / f"{stem}-{counter}{suffix}" - counter += 1 - return destination - - -def _embed_asset(source_path: Path) -> str | None: - try: - data = source_path.read_bytes() - except OSError: - return None - mime_type, _ = mimetypes.guess_type(source_path.name) - if not mime_type: - mime_type = "application/octet-stream" - return f"data:{mime_type};base64,{base64.b64encode(data).decode('ascii')}" - - -def _asset_display_path(path: str, asset_map: dict[str, str]) -> str: - return asset_map.get(path, path) - - -def _repair_markdown_image_links(markdown: str, output_root: Path, asset_map: dict[str, str]) -> str: - if not markdown or not asset_map: - return markdown - exact, by_name = _asset_link_rewrite_maps(asset_map) - - def replace(match: re.Match[str]) -> str: - alt = match.group(1) - target = match.group(2) - cleaned = _normalize_image_target(target) - if not cleaned or cleaned.startswith(("data:", "http://", "https://")): - return match.group(0) - replacement = exact.get(cleaned) or exact.get(cleaned.lstrip("./")) - if replacement and replacement != cleaned: - return f"![{alt}]({replacement})" - if _image_target_exists(cleaned, output_root): - return match.group(0) - if replacement is None: - replacement = by_name.get(_path_name(cleaned)) - if not replacement: - return match.group(0) - return f"![{alt}]({replacement})" - - return re.sub(r"!\[([^\]]*)]\(([^)]+)\)", replace, markdown) - - -def _asset_link_rewrite_maps(asset_map: dict[str, str]) -> tuple[dict[str, str], dict[str, str]]: - exact: dict[str, str] = {} - by_name_values: dict[str, set[str]] = {} - for raw_path, display_path in asset_map.items(): - display = _normalize_image_target(display_path) - if not display: - continue - for candidate in {raw_path, display_path, _normalize_image_target(raw_path), display}: - key = _normalize_image_target(candidate) - if key and not key.startswith(("data:", "http://", "https://")): - exact[key] = display - exact[key.lstrip("./")] = display - by_name_values.setdefault(_path_name(raw_path), set()).add(display) - by_name_values.setdefault(_path_name(display_path), set()).add(display) - by_name = {name: next(iter(values)) for name, values in by_name_values.items() if name and len(values) == 1} - return exact, by_name - - -def _normalize_image_target(target: object) -> str: - return str(target or "").strip().strip("<>").replace("\\", "/") - - -def _path_name(path: object) -> str: - return _normalize_image_target(path).rstrip("/").rsplit("/", 1)[-1] - - -def _image_target_exists(target: str, output_root: Path) -> bool: - path = Path(target) - candidate = path if path.is_absolute() else output_root / path - return candidate.exists() - - -def _should_render_screenshot(page: SlidePage, screenshot_policy: str) -> bool: - if not page.page_screenshot: - return False - if screenshot_policy == "always": - return True - if screenshot_policy == "never": - return False - return not any(not image.ignored and image.role != "page_image" for image in page.images) - - -def _validate_markdown_image_links(markdown: str, output_root: Path) -> list[str]: - warnings: list[str] = [] - for target in re.findall(r"!\[[^\]]*]\(([^)]+)\)", markdown): - cleaned = target.strip().strip("<>") - if not cleaned or cleaned.startswith(("data:", "http://", "https://")): - continue - path = Path(cleaned) - if path.is_absolute(): - candidate = path - else: - candidate = output_root / path - if not candidate.exists(): - warnings.append(f"Markdown image link target is missing: {cleaned}") - for target in re.findall(r"`(!\[[^\]]*]\([^)]+\))`", markdown): - warnings.append(f"Markdown image is wrapped as code and will not render: {target}") - return warnings - - -# --------------------------------------------------------------------------- -# Context selection -# --------------------------------------------------------------------------- - -def _select_note_contexts(deck: Deck, requested: str, section_plan: dict[str, Any] | None = None) -> list[NoteContext]: - resolved = _resolved_context_mode(deck, requested) - if resolved == "document": - return [NoteContext(id="doc", kind="document", title=Path(deck.source_path).stem, pages=list(deck.pages))] - if resolved == "page": - return [ - NoteContext(id=f"p{page.slide_id}", kind="page", title=page.title or f"\u7b2c {page.slide_id} \u9875", pages=[page]) - for page in deck.pages - ] - return _section_contexts(deck, section_plan=section_plan) - - -def _resolved_context_mode(deck: Deck, requested: str) -> str: - if requested != "auto": - return requested - if len(deck.pages) <= 12 and _structured_char_count(deck) <= 16_000: - return "document" - return "section" - - -def _structured_char_count(deck: Deck) -> int: - total = 0 - for page in deck.pages: - total += sum(len(block.content) for block in page.text_blocks) - total += sum(len(cell) for table in page.tables for row in table.rows for cell in row) - total += len(page.page_ocr_text or "") + len(page.page_visual_summary or "") - total += sum(len(image.ocr_text or "") + len(image.visual_summary or "") for image in page.images) - return total - - -def _section_contexts(deck: Deck, section_plan: dict[str, Any] | None = None) -> list[NoteContext]: - if not deck.pages: - return [] - if section_plan: - planned_contexts = _section_contexts_from_plan(deck, section_plan) - if planned_contexts: - return planned_contexts - boundaries = _section_boundaries(deck) - if len(boundaries) <= 1: - boundaries = [deck.pages[index].slide_id for index in range(0, len(deck.pages), 8)] - contexts: list[NoteContext] = [] - slide_to_index = {page.slide_id: index for index, page in enumerate(deck.pages)} - boundary_indexes = sorted({slide_to_index[slide_id] for slide_id in boundaries if slide_id in slide_to_index}) - if not boundary_indexes or boundary_indexes[0] != 0: - boundary_indexes.insert(0, 0) - for position, start_index in enumerate(boundary_indexes): - end_index = boundary_indexes[position + 1] if position + 1 < len(boundary_indexes) else len(deck.pages) - pages = deck.pages[start_index:end_index] - if not pages: - continue - title = context_title(pages, position + 1) - contexts.append(NoteContext(id=f"sec{position + 1}", kind="section", title=title, pages=pages)) - return contexts - - -def _section_contexts_from_plan(deck: Deck, section_plan: dict[str, Any]) -> list[NoteContext]: - pages_by_id = {page.slide_id: page for page in deck.pages} - contexts: list[NoteContext] = [] - sections = section_plan.get("sections") - if not isinstance(sections, list): - return [] - for index, section in enumerate(sections, start=1): - if not isinstance(section, dict): - continue - raw_ids = section.get("slide_ids") - if not isinstance(raw_ids, list): - continue - pages = [pages_by_id[slide_id] for slide_id in raw_ids if isinstance(slide_id, int) and slide_id in pages_by_id] - if not pages: - continue - context_id = str(section.get("section_id") or f"sec{index}") - title = str(section.get("title") or context_title(pages, index)).strip() or context_title(pages, index) - contexts.append(NoteContext(id=context_id, kind="section", title=title, pages=pages)) - return contexts - - -def _section_boundaries(deck: Deck) -> list[int]: - outline_titles = _outline_titles(deck) - boundaries = [deck.pages[0].slide_id] - for page in deck.pages[1:]: - title = _normalize_heading_text(page.title or "") - if not title or "\u76ee\u5f55" in title or title.lower() == "contents": - continue - if any(title == outline or title in outline or outline in title for outline in outline_titles): - boundaries.append(page.slide_id) - elif not outline_titles and looks_like_section_title_page(page): - boundaries.append(page.slide_id) - return sorted(set(boundaries)) - - -def _outline_titles(deck: Deck) -> set[str]: - titles: set[str] = set() - for page in deck.pages: - page_text = "\n".join(block.content for block in page.text_blocks) - if "\u76ee\u5f55" not in page_text and "Contents" not in page_text: - continue - for line in page_text.splitlines(): - normalized = _normalize_heading_text(line) - if not normalized or normalized.lower() in {"\u76ee\u5f55", "contents"}: - continue - if len(normalized) >= 4: - titles.add(normalized) - return titles - - -def _normalize_heading_text(value: str) -> str: - value = re.sub(r"^\s*(?:\d+|[\u4e00\u4e8c\u4e09\u56db\u4e94\u516d\u4e03\u516b\u4e5d\u5341]+)(?:[.\u3001\s-]+)", "", value.strip()) - return re.sub(r"\s+", "", value).strip("\uff1a:") - - -# --------------------------------------------------------------------------- -# Postprocessing -# --------------------------------------------------------------------------- - -def _postprocess_llm_markdown(markdown: str, source_display: str) -> str: - text = _unwrap_code_images(markdown) - text = _fill_empty_image_alts(text) - text = _remove_meta_paragraphs(text) - text = _normalize_chunk_headings(text) - text = _convert_visible_sources(text, source_display) - return text.strip() - - -def _unwrap_code_images(markdown: str) -> str: - return re.sub(r"`(!\[[^\]]*]\([^)]+\))`", r"\1", markdown) - - -def _fill_empty_image_alts(markdown: str) -> str: - return re.sub(r"!\[\s*]\(", "![\u56fe\u793a](", markdown) - - -def _remove_meta_paragraphs(markdown: str) -> str: - paragraphs = re.split(r"\n\s*\n", markdown) - kept = [paragraph.strip() for paragraph in paragraphs if paragraph.strip() and not _is_meta_paragraph(paragraph)] - return "\n\n".join(kept) - - -def _is_meta_paragraph(paragraph: str) -> bool: - normalized = " ".join(line.strip() for line in paragraph.splitlines() if line.strip()) - banned_patterns = [ - "\u597d\u7684\uff0c\u8fd9\u662f", - "\u597d\u7684\uff0c\u6211\u5c06", - "\u4ee5\u4e0b\u662f\u6839\u636e", - "\u4e0b\u9762\u662f\u4f9d\u636e", - "\u6839\u636e\u60a8\u63d0\u4f9b\u7684 JSON", - "\u6839\u636e\u4f60\u63d0\u4f9b\u7684 JSON", - "\u8bfe\u7a0b\u6750\u6599 JSON", - "\u7b14\u8bb0\u5df2\u4e25\u683c\u9075\u5faa", - "\u4e25\u683c\u9075\u5faa\u5168\u90e8\u786c\u6027\u8981\u6c42", - "\u8986\u76d6\u4e86\u6240\u6709\u6587\u672c\u5757", - "\u8986\u76d6\u6bcf\u4e00\u4e2a\u6587\u672c\u5757", - "\u6bcf\u6bb5\u5747\u6807\u6ce8", - "\u6bcf\u4e00\u6bb5\u90fd\u6807\u6ce8", - "\u672a\u63d0\u4f9b\u56fe\u7247\u50cf\u7d20", - "\u672a\u63d0\u4f9b\u56fe\u50cf\u50cf\u7d20", - "\u672a\u63d0\u4f9b\u56fe\u7247\u7684 OCR", - "\u672a\u63d0\u4f9b\u8be5\u622a\u56fe\u7684 OCR", - "\u672a\u8fdb\u884c\u89c6\u89c9\u89e3\u6790", - "\u65e0\u6cd5\u8fdb\u884c\u5177\u4f53\u63cf\u8ff0", - "\u65e0\u6cd5\u8fdb\u4e00\u6b65\u8bf4\u660e", - "\u65e0\u6cd5\u5bf9\u622a\u56fe\u5185\u5bb9", - "\u5efa\u8bae\u5728\u539f\u59cb\u5e7b\u706f\u7247", - "\u82e5\u9700\u4e86\u89e3\u56fe\u7247\u5177\u4f53\u5185\u5bb9", - "\u56fe\u7247\u7559\u4f5c\u539f\u59cb\u8bc1\u636e", - "\u4ec5\u4f5c\u4e3a\u8bc1\u636e\u4fdd\u7559", - ] - if any(pattern in normalized for pattern in banned_patterns): - return True - structure_only_patterns = [ - "\u5e7b\u706f\u7247\u9996\u5148\u63d0\u51fa", - "\u8fd9\u4e00\u9875\u5728\u4e0a\u4e00\u9875\u7684\u57fa\u7840\u4e0a", - "\u4e0a\u4e00\u9875\u4ecb\u7ecd\u4e86", - "\u4e0b\u4e00\u9875\u5c06", - "\u672c\u9875\u4e3b\u8981\u8bb2\u89e3", - "\u672c\u9875\u4ecb\u7ecd\u4e86", - "\u8fd9\u9875\u5c55\u793a", - "\u6b64\u9875\u5185\u5bb9", - "\u6b64\u5e7b\u706f\u7247", - "\u8fd9\u5f20\u5e7b\u706f\u7247", - "\u8fd9\u7ec4\u5e7b\u706f\u7247", - "\u8be5\u5e7b\u706f\u7247", - "\u5f53\u524d\u5e7b\u706f\u7247", - ] - if SOURCE_COMMENT_PREFIX in normalized or len(normalized) > 80: - return False - return any(normalized.startswith(pattern) for pattern in structure_only_patterns) - - -def _normalize_chunk_headings(markdown: str) -> str: - lines: list[str] = [] - for line in markdown.splitlines(): - match = re.match(r"^(#{1,6})\s+(.*)$", line) - if not match: - lines.append(line) - continue - text = re.sub(r"^\u8bfe\u7a0b\u7b14\u8bb0[\uff1a:\s-]*", "", match.group(2).strip()) - if not text: - continue - level = max(2, len(match.group(1))) - lines.append("#" * level + " " + text) - return "\n".join(lines) - - -def _convert_visible_sources(markdown: str, source_display: str) -> str: - if source_display == "inline": - return _ensure_source_comments_for_inline(markdown) - - def replace(match: re.Match[str]) -> str: - citation = match.group(0) - element_ids = re.findall(r"\bs\d+_(?:t|tbl|img|fig)\d+\b", citation) - slide_match = re.search(r"\u7b2c\s*(\d+)\s*\u9875", citation) - if not slide_match: - return "" - slide_id = int(slide_match.group(1)) - if source_display == "footnote": - return _source_marker(slide_id, element_ids, "footnote") - return _source_marker(slide_id, element_ids, "hidden") - - return re.sub(r"\u3010[^\u3011]*?PPT[^\u3011]*?\u3011", replace, markdown) - - -def _ensure_source_comments_for_inline(markdown: str) -> str: - def replace(match: re.Match[str]) -> str: - citation = match.group(0) - if SOURCE_COMMENT_PREFIX in citation: - return citation - element_ids = re.findall(r"\bs\d+_(?:t|tbl|img|fig)\d+\b", citation) - slide_match = re.search(r"\u7b2c\s*(\d+)\s*\u9875", citation) - if not slide_match or not element_ids: - return citation - return f"{citation} {_source_marker(int(slide_match.group(1)), element_ids, 'hidden')}" - - return re.sub(r"\u3010[^\u3011]*?PPT[^\u3011]*?\u3011", replace, markdown) - - -# --------------------------------------------------------------------------- -# Reports -# --------------------------------------------------------------------------- - -def _build_page_notes_report( - deck: Deck, - output_root: Path, - provider: str, - model: str, - base_url: str | None, - note_depth: str, - note_language: str, - term_policy: str, - page_neighborhood: int, - pages: list[NoteContext], - page_markdown_by_slide: dict[int, str], - page_records: list[dict[str, Any]], - deck_brief: dict[str, Any] | None = None, -) -> dict[str, Any]: - from slidenote.llm_cache import utc_now_iso - from .prompts import _prompt_deck_brief, _prompt_brief_hash - prompt_brief = _prompt_deck_brief(deck_brief) - record_by_slide = {record.get("slide_id"): record for record in page_records} - page_entries: list[dict[str, Any]] = [] - for context in pages: - page = context.pages[0] - record = record_by_slide.get(page.slide_id, {}) - markdown = page_markdown_by_slide.get(page.slide_id, "") - page_entries.append( - { - "slide_id": page.slide_id, - "title": page.title, - "markdown": markdown, - "source_ids": sorted(source_tokens(markdown)), - "cache_status": record.get("cache_status"), - "llm_call": record.get("llm_call"), - "cache_file": record.get("cache_file"), - "input_tokens": record.get("input_tokens"), - "output_tokens": record.get("output_tokens"), - "total_tokens": record.get("total_tokens"), - } - ) - return { - "schema_version": 1, - "generated_at": utc_now_iso(), - "source_path": deck.source_path, - "source_type": deck.source_type, - "provider": provider, - "model": model, - "base_url": base_url, - "prompt_version": PAGE_LECTURE_PROMPT_VERSION, - "request": { - "note_depth": note_depth, - "note_language": note_language, - "term_policy": term_policy, - "page_neighborhood": page_neighborhood, - "deck_brief_used": bool(prompt_brief), - "deck_brief_hash": _prompt_brief_hash(prompt_brief), - }, - "summary": { - "pages_total": len(page_entries), - "llm_calls": sum(1 for record in page_records if record.get("llm_call")), - "local_cache_hits": sum(1 for record in page_records if record.get("cache_status") == "local_hit"), - "input_tokens": sum_int(record.get("input_tokens") for record in page_records), - "output_tokens": sum_int(record.get("output_tokens") for record in page_records), - "total_tokens": sum_int(record.get("total_tokens") for record in page_records), - }, - "pages": page_entries, - } - - -def _render_page_notes_markdown(deck: Deck, page_notes: dict[str, Any]) -> str: - lines = [f"# {Path(deck.source_path).stem} Page Notes", ""] - for page in page_notes.get("pages", []): - title = page.get("title") or f"\u7b2c {page.get('slide_id')} \u9875" - lines.append(f"## \u7b2c {page.get('slide_id')} \u9875\uff1a{title}") - lines.append("") - markdown = str(page.get("markdown") or "").strip() - if markdown: - lines.append(markdown) - lines.append("") - return "\n".join(lines).rstrip() + "\n" - - -def _build_weave_report( - deck: Deck, - output_root: Path, - note_context: str, - note_depth: str, - note_language: str, - term_policy: str, - weave_dedup: str, - contexts: list[NoteContext], - final_chunks: dict[str, str], - page_markdown_by_slide: dict[int, str], - weave_records: list[dict[str, Any]], - deck_brief: dict[str, Any] | None = None, -) -> dict[str, Any]: - from slidenote.llm_cache import utc_now_iso - from .prompts import _prompt_deck_brief, _prompt_brief_hash - prompt_brief = _prompt_deck_brief(deck_brief) - record_by_context = {record.get("context_id"): record for record in weave_records} - context_entries: list[dict[str, Any]] = [] - for context in contexts: - markdown = final_chunks.get(context.id, "") - final_tokens = source_tokens(markdown) - input_tokens: set[str] = set() - pages: list[dict[str, Any]] = [] - for page in context.pages: - page_tokens = source_tokens(page_markdown_by_slide.get(page.slide_id, "")) - input_tokens.update(page_tokens) - pages.append( - { - "slide_id": page.slide_id, - "title": page.title, - "page_note_source_ids": sorted(page_tokens), - "retained_source_ids": sorted(page_tokens.intersection(final_tokens)), - "possibly_compressed_source_ids": sorted(page_tokens - final_tokens), - } - ) - record = record_by_context.get(f"weave_{context.id}", {}) - context_entries.append( - { - "context_id": context.id, - "context_title": context.title, - "slide_ids": [page.slide_id for page in context.pages], - "input_source_ids": sorted(input_tokens), - "final_source_ids": sorted(final_tokens), - "possibly_compressed_source_ids": sorted(input_tokens - final_tokens), - "cache_status": record.get("cache_status"), - "llm_call": record.get("llm_call"), - "cache_file": record.get("cache_file"), - "pages": pages, - } - ) - return { - "schema_version": 1, - "generated_at": utc_now_iso(), - "source_path": deck.source_path, - "source_type": deck.source_type, - "prompt_version": WEAVE_PROMPT_VERSION, - "request": { - "note_context": note_context, - "note_depth": note_depth, - "note_language": note_language, - "term_policy": term_policy, - "weave_dedup": weave_dedup, - "deck_brief_used": bool(prompt_brief), - "deck_brief_hash": _prompt_brief_hash(prompt_brief), - }, - "summary": { - "contexts_total": len(context_entries), - "llm_calls": sum(1 for record in weave_records if record.get("llm_call")), - "local_cache_hits": sum(1 for record in weave_records if record.get("cache_status") == "local_hit"), - "input_tokens": sum_int(record.get("input_tokens") for record in weave_records), - "output_tokens": sum_int(record.get("output_tokens") for record in weave_records), - "total_tokens": sum_int(record.get("total_tokens") for record in weave_records), - }, - "contexts": context_entries, - } - - -def _build_teaching_enrichment_report( - deck: Deck, - output_root: Path, - note_context: str, - note_profile: str, - note_depth: str, - note_language: str, - term_policy: str, - contexts: list[NoteContext], - final_chunks: dict[str, str], - page_markdown_by_slide: dict[int, str], - teaching_records: list[dict[str, Any]], - deck_brief: dict[str, Any] | None = None, -) -> dict[str, Any]: - from slidenote.llm_cache import utc_now_iso - from .prompts import _prompt_deck_brief, _prompt_brief_hash - - prompt_brief = _prompt_deck_brief(deck_brief) - record_by_context = {record.get("context_id"): record for record in teaching_records} - context_entries: list[dict[str, Any]] = [] - for context in contexts: - markdown = final_chunks.get(context.id, "") - final_tokens = source_tokens(markdown) - input_tokens: set[str] = set() - for page in context.pages: - input_tokens.update(source_tokens(page_markdown_by_slide.get(page.slide_id, ""))) - record = record_by_context.get(f"teaching_{context.id}", {}) - context_entries.append( - { - "context_id": context.id, - "context_title": context.title, - "slide_ids": [page.slide_id for page in context.pages], - "input_source_ids": sorted(input_tokens), - "final_source_ids": sorted(final_tokens), - "possibly_added_source_ids": sorted(final_tokens - input_tokens), - "possibly_dropped_source_ids": sorted(input_tokens - final_tokens), - "cache_status": record.get("cache_status"), - "llm_call": record.get("llm_call"), - "cache_file": record.get("cache_file"), - "input_tokens": record.get("input_tokens"), - "output_tokens": record.get("output_tokens"), - "total_tokens": record.get("total_tokens"), - } - ) - return { - "schema_version": 1, - "generated_at": utc_now_iso(), - "source_path": deck.source_path, - "source_type": deck.source_type, - "prompt_version": TEACHING_ENRICHMENT_PROMPT_VERSION, - "request": { - "note_context": note_context, - "note_profile": note_profile, - "note_depth": note_depth, - "note_language": note_language, - "term_policy": term_policy, - "deck_brief_used": bool(prompt_brief), - "deck_brief_hash": _prompt_brief_hash(prompt_brief), - }, - "summary": { - "contexts_total": len(context_entries), - "llm_calls": sum(1 for record in teaching_records if record.get("llm_call")), - "local_cache_hits": sum(1 for record in teaching_records if record.get("cache_status") == "local_hit"), - "input_tokens": sum_int(record.get("input_tokens") for record in teaching_records), - "output_tokens": sum_int(record.get("output_tokens") for record in teaching_records), - "total_tokens": sum_int(record.get("total_tokens") for record in teaching_records), - }, - "contexts": context_entries, - } - - -def _source_slide_ids(markdown: str) -> set[int]: - return {int(match) for match in re.findall(r"\bp(\d+):", markdown)} diff --git a/slidenote/notes/assets.py b/slidenote/notes/assets.py new file mode 100644 index 0000000..83adffd --- /dev/null +++ b/slidenote/notes/assets.py @@ -0,0 +1,179 @@ +"""Copy, embed and relink image assets referenced by the notes.""" + +from __future__ import annotations + +import base64 +import mimetypes +import re +import shutil +from pathlib import Path + +from slidenote.image_ranking import sorted_images_by_importance +from slidenote.models import Deck, SlidePage + + +def _prepare_note_assets(deck: Deck, output_root: Path, asset_mode: str, screenshot_policy: str) -> tuple[dict[str, str], list[str]]: + asset_map: dict[str, str] = {} + warnings: list[str] = [] + seen_destinations: set[Path] = set() + for rel_path, kind in _iter_note_asset_paths(deck, screenshot_policy=screenshot_policy): + if rel_path in asset_map: + continue + source_path = _resolve_output_asset(output_root, rel_path) + if not source_path.exists(): + warnings.append(f"Missing note asset: {rel_path}") + continue + if asset_mode == "absolute": + asset_map[rel_path] = source_path.as_posix() + elif asset_mode == "embed": + embedded = _embed_asset(source_path) + if embedded: + asset_map[rel_path] = embedded + else: + warnings.append(f"Could not embed note asset: {rel_path}") + else: + destination = _bundled_asset_destination(output_root, rel_path, kind, seen_destinations) + destination.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(source_path, destination) + asset_map[rel_path] = destination.relative_to(output_root).as_posix() + seen_destinations.add(destination) + return asset_map, warnings + + +def _iter_note_asset_paths(deck: Deck, screenshot_policy: str) -> list[tuple[str, str]]: + paths: list[tuple[str, str]] = [] + for page in deck.pages: + if _should_render_screenshot(page, screenshot_policy): + paths.append((page.page_screenshot, "screenshots")) + for image in sorted_images_by_importance(page.images): + if not image.ignored: + kind = "figures" if image.role in {"figure_crop", "composite_figure"} else "images" + paths.append((image.path, kind)) + return paths + + +def _resolve_output_asset(output_root: Path, path: str) -> Path: + asset_path = Path(path) + if asset_path.is_absolute(): + return asset_path + return (output_root / asset_path).resolve() + + +def _bundled_asset_destination(output_root: Path, rel_path: str, kind: str, seen_destinations: set[Path]) -> Path: + source = Path(rel_path) + subdir = "screenshots" if kind == "screenshots" else "figures" if kind == "figures" else "images" + stem = source.stem or "asset" + suffix = source.suffix or ".png" + destination = output_root / "notes.assets" / subdir / f"{stem}{suffix}" + counter = 2 + while destination in seen_destinations: + destination = output_root / "notes.assets" / subdir / f"{stem}-{counter}{suffix}" + counter += 1 + return destination + + +def _embed_asset(source_path: Path) -> str | None: + try: + data = source_path.read_bytes() + except OSError: + return None + mime_type, _ = mimetypes.guess_type(source_path.name) + if not mime_type: + mime_type = "application/octet-stream" + return f"data:{mime_type};base64,{base64.b64encode(data).decode('ascii')}" + + +def _asset_display_path(path: str, asset_map: dict[str, str]) -> str: + return asset_map.get(path, path) + + +def _repair_markdown_image_links(markdown: str, output_root: Path, asset_map: dict[str, str]) -> str: + if not markdown or not asset_map: + return markdown + exact, by_name = _asset_link_rewrite_maps(asset_map) + + def replace(match: re.Match[str]) -> str: + alt = match.group(1) + target = match.group(2) + cleaned = _normalize_image_target(target) + if not cleaned or cleaned.startswith(("data:", "http://", "https://")): + return match.group(0) + replacement = exact.get(cleaned) or exact.get(_strip_current_dir_prefix(cleaned)) + if replacement and replacement != cleaned: + return f"![{alt}]({replacement})" + if _image_target_exists(cleaned, output_root): + return match.group(0) + if replacement is None: + replacement = by_name.get(_path_name(cleaned)) + if not replacement: + return match.group(0) + return f"![{alt}]({replacement})" + + return re.sub(r"!\[([^\]]*)]\(([^)]+)\)", replace, markdown) + + +def _asset_link_rewrite_maps(asset_map: dict[str, str]) -> tuple[dict[str, str], dict[str, str]]: + exact: dict[str, str] = {} + by_name_values: dict[str, set[str]] = {} + for raw_path, display_path in asset_map.items(): + display = _normalize_image_target(display_path) + if not display: + continue + for candidate in {raw_path, display_path, _normalize_image_target(raw_path), display}: + key = _normalize_image_target(candidate) + if key and not key.startswith(("data:", "http://", "https://")): + exact[key] = display + exact[_strip_current_dir_prefix(key)] = display + by_name_values.setdefault(_path_name(raw_path), set()).add(display) + by_name_values.setdefault(_path_name(display_path), set()).add(display) + by_name = {name: next(iter(values)) for name, values in by_name_values.items() if name and len(values) == 1} + return exact, by_name + + +def _strip_current_dir_prefix(path: str) -> str: + """Remove leading ``./`` segments only; ``../`` must survive (unlike ``lstrip``).""" + while path.startswith("./"): + path = path[2:] + return path + + +def _normalize_image_target(target: object) -> str: + return str(target or "").strip().strip("<>").replace("\\", "/") + + +def _path_name(path: object) -> str: + return _normalize_image_target(path).rstrip("/").rsplit("/", 1)[-1] + + +def _image_target_exists(target: str, output_root: Path) -> bool: + path = Path(target) + candidate = path if path.is_absolute() else output_root / path + return candidate.exists() + + +def _should_render_screenshot(page: SlidePage, screenshot_policy: str) -> bool: + if not page.page_screenshot: + return False + if screenshot_policy == "always": + return True + if screenshot_policy == "never": + return False + return not any(not image.ignored and image.role != "page_image" for image in page.images) + + +def _validate_markdown_image_links(markdown: str, output_root: Path) -> list[str]: + warnings: list[str] = [] + for target in re.findall(r"!\[[^\]]*]\(([^)]+)\)", markdown): + cleaned = target.strip().strip("<>") + if not cleaned or cleaned.startswith(("data:", "http://", "https://")): + continue + path = Path(cleaned) + if path.is_absolute(): + candidate = path + else: + candidate = output_root / path + if not candidate.exists(): + warnings.append(f"Markdown image link target is missing: {cleaned}") + for target in re.findall(r"`(!\[[^\]]*]\([^)]+\))`", markdown): + warnings.append(f"Markdown image is wrapped as code and will not render: {target}") + return warnings diff --git a/slidenote/notes/compose.py b/slidenote/notes/compose.py new file mode 100644 index 0000000..8bba465 --- /dev/null +++ b/slidenote/notes/compose.py @@ -0,0 +1,264 @@ +"""Compose per-context Markdown into the final notes document.""" + +from __future__ import annotations + +import re +from pathlib import Path +from typing import Any + +from slidenote.models import Deck +from .contexts import NoteContext +from .frontmatter import ( + _clean_heading_text, + _frontmatter_source_markers, + _is_frontmatter_heading, + _is_generic_heading_text, + _leading_frontmatter_slide_ids, + _looks_like_frontmatter_text, + _normalize_title_key, +) +from .sources import _collapse_blank_lines, _source_slide_ids + + +def _compose_final_markdown( + deck: Deck, + contexts: list[NoteContext], + final_chunks: dict[str, str], + section_plan: dict[str, Any] | None, + source_display: str, +) -> str: + del source_display + lines = [f"# {_document_title(deck)}", ""] + add_context_headings = _should_add_context_headings(contexts) + leading_frontmatter_slide_ids = _leading_frontmatter_slide_ids(deck.pages) + section_number = 1 + for context in contexts: + content = final_chunks.get(context.id, "").strip() + if not content: + continue + if add_context_headings: + heading_title = _context_heading_title(context, section_plan) + if _is_frontmatter_heading(heading_title, context) and len(contexts) > 1: + content = _frontmatter_source_markers(context.pages) + else: + lines.append(_context_heading(context, heading_title, section_number)) + lines.append("") + section_number += 1 + content = _prepare_context_chunk(content, heading_title, add_outer_heading=True) + content = _strip_leading_frontmatter_content(content, context, leading_frontmatter_slide_ids) + content = _number_subsection_headings(content) + else: + content = _prepare_context_chunk(content, context.title, add_outer_heading=False) + if content: + lines.append(content) + lines.append("") + return "\n".join(lines).rstrip() + "\n" + + +def _document_title(deck: Deck) -> str: + stem = Path(deck.source_path).stem + for page in deck.pages[:3]: + title = (page.title or "").strip() + if title and not _is_generic_heading_text(title): + return f"{stem}\uff1a{title}" if title != stem else stem + return stem + + +def _should_add_context_headings(contexts: list[NoteContext]) -> bool: + if not contexts: + return False + if len(contexts) > 1: + return True + return contexts[0].kind == "section" + + +def _context_heading(context: NoteContext, title: str, section_number: int) -> str: + if context.kind == "page": + slide_id = context.pages[0].slide_id if context.pages else section_number + return f"## \u7b2c {slide_id} \u9875\uff1a{title}" + return f"## {_chinese_ordinal(section_number)}\u3001{title}" + + +def _context_heading_title(context: NoteContext, section_plan: dict[str, Any] | None) -> str: + planned_title = _planned_context_title(context, section_plan) + title = planned_title or context.title or "" + title = _clean_heading_text(title) + if title: + return title + if context.kind == "page" and context.pages: + return context.pages[0].title or f"\u7b2c {context.pages[0].slide_id} \u9875" + return "\u672c\u8282\u5185\u5bb9" + + +def _planned_context_title(context: NoteContext, section_plan: dict[str, Any] | None) -> str | None: + if not section_plan: + return None + sections = section_plan.get("sections") + if not isinstance(sections, list): + return None + slide_ids = [page.slide_id for page in context.pages] + for section in sections: + if not isinstance(section, dict): + continue + if section.get("section_id") == context.id or section.get("slide_ids") == slide_ids: + title = str(section.get("title") or "").strip() + return title or None + return None + + +def _prepare_context_chunk(markdown: str, section_title: str, add_outer_heading: bool) -> str: + text = _remove_generation_info_sections(markdown) + text = _drop_redundant_leading_headings(text, section_title) if add_outer_heading else text + text = _demote_chunk_headings(text, minimum_level=3 if add_outer_heading else 2) + text = _collapse_blank_lines(text.splitlines()) + return text.strip() + + +def _drop_redundant_leading_headings(markdown: str, section_title: str) -> str: + lines = markdown.splitlines() + while True: + first_index = next((index for index, line in enumerate(lines) if line.strip()), None) + if first_index is None: + return "" + match = re.match(r"^(#{1,6})\s+(.*)$", lines[first_index].strip()) + if not match: + return "\n".join(lines).strip() + heading_text = _clean_heading_text(match.group(2)) + if not _is_redundant_context_heading(heading_text, section_title): + return "\n".join(lines).strip() + del lines[first_index] + while first_index < len(lines) and not lines[first_index].strip(): + del lines[first_index] + + +def _is_redundant_context_heading(heading_text: str, section_title: str) -> bool: + heading_norm = _normalize_title_key(heading_text) + context_norm = _normalize_title_key(section_title) + if not heading_norm: + return True + if _is_generic_heading_text(heading_text): + return True + return bool(context_norm and (heading_norm == context_norm or heading_norm in context_norm or context_norm in heading_norm)) + + +def _demote_chunk_headings(markdown: str, minimum_level: int) -> str: + lines: list[str] = [] + for line in markdown.splitlines(): + match = re.match(r"^(#{1,6})\s+(.*)$", line) + if not match: + lines.append(line) + continue + text = _clean_heading_text(match.group(2)) + if not text or _is_generic_heading_text(text): + continue + level = max(minimum_level, len(match.group(1))) + lines.append("#" * min(level, 6) + " " + text) + return "\n".join(lines) + + +def _strip_leading_frontmatter_content(markdown: str, context: NoteContext, leading_frontmatter_slide_ids: set[int]) -> str: + frontmatter_slide_ids = {page.slide_id for page in context.pages if page.slide_id in leading_frontmatter_slide_ids} + if not frontmatter_slide_ids: + return markdown + blocks = re.split(r"\n\s*\n", markdown.strip()) + kept: list[str] = [] + dropping = True + for block in blocks: + stripped = block.strip() + if not stripped: + continue + if dropping and _is_horizontal_rule(stripped): + continue + if dropping and _is_droppable_frontmatter_block(stripped, frontmatter_slide_ids): + continue + dropping = False + kept.append(stripped) + marker = _frontmatter_source_markers([page for page in context.pages if page.slide_id in frontmatter_slide_ids]) + if not marker: + return "\n\n".join(kept).strip() + body = "\n\n".join(kept).strip() + return f"{marker}\n\n{body}".strip() if body else marker + + +def _is_droppable_frontmatter_block(block: str, frontmatter_slide_ids: set[int]) -> bool: + match = re.match(r"^(#{1,6})\s+(.*)$", block) + if match: + return _is_generic_heading_text(match.group(2)) or _looks_like_frontmatter_text(match.group(2)) + slide_ids = _source_slide_ids(block) + if slide_ids and slide_ids.issubset(frontmatter_slide_ids): + return True + return _looks_like_frontmatter_text(block) + + +def _is_horizontal_rule(block: str) -> bool: + return bool(re.fullmatch(r"[-*_]{3,}", block.strip())) + + +def _number_subsection_headings(markdown: str) -> str: + counters: list[int] = [] + base_level: int | None = None + lines: list[str] = [] + for line in markdown.splitlines(): + match = re.match(r"^(#{1,6})\s+(.*)$", line) + if not match: + lines.append(line) + continue + level = len(match.group(1)) + title = _strip_heading_number(match.group(2).strip()) + if level < 3: + counters = [] + base_level = None + lines.append(line) + continue + if base_level is None or level < base_level: + base_level = level + counters = [] + depth = max(0, level - base_level) + while len(counters) <= depth: + counters.append(0) + counters = counters[: depth + 1] + counters[depth] += 1 + prefix = ".".join(str(value) for value in counters) + separator = ". " if len(counters) == 1 else " " + lines.append(f"{match.group(1)} {prefix}{separator}{title}") + return "\n".join(lines) + + +def _strip_heading_number(title: str) -> str: + return re.sub( + r"^\s*(?:\d+(?:\.\d+)*\.?|[\u4e00\u4e8c\u4e09\u56db\u4e94\u516d\u4e03\u516b\u4e5d\u5341]+[\u3001.])\s*", + "", + title, + ).strip() + + +def _remove_generation_info_sections(markdown: str) -> str: + lines = markdown.splitlines() + kept: list[str] = [] + skipping = False + skip_level = 0 + for line in lines: + match = re.match(r"^(#{1,6})\s+(.*)$", line) + if match: + level = len(match.group(1)) + heading = _normalize_title_key(match.group(2)) + if heading in {"\u751f\u6210\u4fe1\u606f", "generationinfo", "generationmetadata"}: + skipping = True + skip_level = level + continue + if skipping and level <= skip_level: + skipping = False + if not skipping: + kept.append(line) + return "\n".join(kept) + + +def _chinese_ordinal(index: int) -> str: + numerals = ["\u4e00", "\u4e8c", "\u4e09", "\u56db", "\u4e94", "\u516d", "\u4e03", "\u516b", "\u4e5d", "\u5341"] + if 1 <= index <= 10: + return numerals[index - 1] + if 11 <= index <= 19: + return "\u5341" + numerals[index - 11] + if index == 20: + return "\u4e8c\u5341" + return str(index) diff --git a/slidenote/notes/context_runner.py b/slidenote/notes/context_runner.py new file mode 100644 index 0000000..c065c42 --- /dev/null +++ b/slidenote/notes/context_runner.py @@ -0,0 +1,84 @@ +from __future__ import annotations + +from concurrent.futures import ThreadPoolExecutor, as_completed +from typing import Any, Callable + +from .contexts import NoteContext + +ContextResult = tuple[str, dict[str, Any]] +ContextFallback = Callable[[NoteContext, Exception], ContextResult] + + +def _run_note_contexts( + contexts: list[NoteContext], + process: Callable[[NoteContext], ContextResult], + *, + workers: int, + progress_callback: Callable[[dict[str, Any]], None] | None = None, + fallback: ContextFallback | None = None, +) -> dict[str, ContextResult]: + """Run ``process`` for each context serially or in a thread pool. + + A failing context is replaced by ``fallback`` so one bad model call does + not discard the rest of the deck. When every context fails (e.g. a + configuration error such as a missing API key) the first error is raised. + """ + results: dict[str, ContextResult] = {} + failures: list[tuple[NoteContext, Exception]] = [] + + def record(context: NoteContext, result: ContextResult) -> None: + results[context.id] = result + if progress_callback: + progress_callback(result[1]) + + if max(1, workers) == 1: + for context in contexts: + try: + result = process(context) + except Exception as exc: + failures.append((context, exc)) + continue + record(context, result) + else: + with ThreadPoolExecutor(max_workers=workers) as executor: + futures = {executor.submit(process, context): context for context in contexts} + for future in as_completed(futures): + context = futures[future] + try: + result = future.result() + except Exception as exc: + failures.append((context, exc)) + continue + record(context, result) + + if failures: + if fallback is None or len(failures) == len(contexts): + raise failures[0][1] + for context, exc in failures: + record(context, fallback(context, exc)) + return results + + +def _failed_context_record(context: NoteContext, exc: Exception, *, generation_stage: str, fallback: str) -> dict[str, Any]: + # Provider error text can contain credentials, so only the type is kept. + return { + "context_id": context.id, + "context_kind": context.kind, + "context_title": context.title, + "slide_id": context.pages[0].slide_id if context.pages else None, + "slide_ids": [page.slide_id for page in context.pages], + "generation_stage": generation_stage, + "cache_status": "failed", + "llm_call": False, + "error_type": type(exc).__name__, + "fallback": fallback, + } + + +def _context_failure_warnings(records: list[dict[str, Any]]) -> list[str]: + return [ + f"note_context_failed:{record.get('generation_stage')}:{record.get('context_id')}:" + f"{record.get('error_type')} (used {record.get('fallback')} fallback)" + for record in records + if record.get("cache_status") == "failed" + ] diff --git a/slidenote/notes/contexts.py b/slidenote/notes/contexts.py new file mode 100644 index 0000000..f5febec --- /dev/null +++ b/slidenote/notes/contexts.py @@ -0,0 +1,130 @@ +"""Note context selection: whole document, sections, or single pages.""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from slidenote.models import Deck, SlidePage +from slidenote.sections import _normalize_heading_text +from slidenote.utils import context_title, looks_like_section_title_page + + +DOCUMENT_CONTEXT_MAX_PAGES = 12 +DOCUMENT_CONTEXT_MAX_CHARS = 16_000 +# Section size used when no outline or section plan gives boundaries. +FALLBACK_SECTION_PAGES = 8 + + +@dataclass(frozen=True, slots=True) +class NoteContext: + id: str + kind: str + title: str + pages: list[SlidePage] + + +def _select_note_contexts(deck: Deck, requested: str, section_plan: dict[str, Any] | None = None) -> list[NoteContext]: + resolved = _resolved_context_mode(deck, requested) + if resolved == "document": + return [NoteContext(id="doc", kind="document", title=Path(deck.source_path).stem, pages=list(deck.pages))] + if resolved == "page": + return [ + NoteContext(id=f"p{page.slide_id}", kind="page", title=page.title or f"\u7b2c {page.slide_id} \u9875", pages=[page]) + for page in deck.pages + ] + return _section_contexts(deck, section_plan=section_plan) + + +def _resolved_context_mode(deck: Deck, requested: str) -> str: + if requested != "auto": + return requested + if len(deck.pages) <= DOCUMENT_CONTEXT_MAX_PAGES and _structured_char_count(deck) <= DOCUMENT_CONTEXT_MAX_CHARS: + return "document" + return "section" + + +def _structured_char_count(deck: Deck) -> int: + total = 0 + for page in deck.pages: + total += sum(len(block.content) for block in page.text_blocks) + total += sum(len(cell) for table in page.tables for row in table.rows for cell in row) + total += len(page.page_ocr_text or "") + len(page.page_visual_summary or "") + total += sum(len(image.ocr_text or "") + len(image.visual_summary or "") for image in page.images) + return total + + +def _section_contexts(deck: Deck, section_plan: dict[str, Any] | None = None) -> list[NoteContext]: + if not deck.pages: + return [] + if section_plan: + planned_contexts = _section_contexts_from_plan(deck, section_plan) + if planned_contexts: + return planned_contexts + boundaries = _section_boundaries(deck) + if len(boundaries) <= 1: + boundaries = [deck.pages[index].slide_id for index in range(0, len(deck.pages), FALLBACK_SECTION_PAGES)] + contexts: list[NoteContext] = [] + slide_to_index = {page.slide_id: index for index, page in enumerate(deck.pages)} + boundary_indexes = sorted({slide_to_index[slide_id] for slide_id in boundaries if slide_id in slide_to_index}) + if not boundary_indexes or boundary_indexes[0] != 0: + boundary_indexes.insert(0, 0) + for position, start_index in enumerate(boundary_indexes): + end_index = boundary_indexes[position + 1] if position + 1 < len(boundary_indexes) else len(deck.pages) + pages = deck.pages[start_index:end_index] + if not pages: + continue + title = context_title(pages, position + 1) + contexts.append(NoteContext(id=f"sec{position + 1}", kind="section", title=title, pages=pages)) + return contexts + + +def _section_contexts_from_plan(deck: Deck, section_plan: dict[str, Any]) -> list[NoteContext]: + pages_by_id = {page.slide_id: page for page in deck.pages} + contexts: list[NoteContext] = [] + sections = section_plan.get("sections") + if not isinstance(sections, list): + return [] + for index, section in enumerate(sections, start=1): + if not isinstance(section, dict): + continue + raw_ids = section.get("slide_ids") + if not isinstance(raw_ids, list): + continue + pages = [pages_by_id[slide_id] for slide_id in raw_ids if isinstance(slide_id, int) and slide_id in pages_by_id] + if not pages: + continue + context_id = str(section.get("section_id") or f"sec{index}") + title = str(section.get("title") or context_title(pages, index)).strip() or context_title(pages, index) + contexts.append(NoteContext(id=context_id, kind="section", title=title, pages=pages)) + return contexts + + +def _section_boundaries(deck: Deck) -> list[int]: + outline_titles = _outline_titles(deck) + boundaries = [deck.pages[0].slide_id] + for page in deck.pages[1:]: + title = _normalize_heading_text(page.title or "") + if not title or "\u76ee\u5f55" in title or title.lower() == "contents": + continue + if any(title == outline or title in outline or outline in title for outline in outline_titles): + boundaries.append(page.slide_id) + elif not outline_titles and looks_like_section_title_page(page): + boundaries.append(page.slide_id) + return sorted(set(boundaries)) + + +def _outline_titles(deck: Deck) -> set[str]: + titles: set[str] = set() + for page in deck.pages: + page_text = "\n".join(block.content for block in page.text_blocks) + if "\u76ee\u5f55" not in page_text and "contents" not in page_text.lower(): + continue + for line in page_text.splitlines(): + normalized = _normalize_heading_text(line) + if not normalized or normalized.lower() in {"\u76ee\u5f55", "contents"}: + continue + if len(normalized) >= 4: + titles.add(normalized) + return titles diff --git a/slidenote/notes/direct.py b/slidenote/notes/direct.py index 0cc7518..95a444c 100644 --- a/slidenote/notes/direct.py +++ b/slidenote/notes/direct.py @@ -1,28 +1,22 @@ from __future__ import annotations -from concurrent.futures import ThreadPoolExecutor, as_completed from dataclasses import replace from pathlib import Path from typing import Any -from slidenote.content_guard import record_repair from slidenote.llm import resolve_provider_runtime from slidenote.llm_cache import LLMCache from slidenote.models import Deck -from .assembly import ( - NoteContext, - _compose_final_markdown, - _ensure_grounded_figures, - _postprocess_llm_markdown, - _repair_markdown_image_links, - _resolved_context_mode, - _select_note_contexts, -) +from .contexts import NoteContext, _resolved_context_mode, _select_note_contexts +from .postprocess import _postprocess_llm_markdown +from .context_runner import _context_failure_warnings, _failed_context_record, _run_note_contexts +from .finalize import _finalize_notes_markdown from .lecture_weave import _generate_notes_with_lecture_weave from .llm_calls import _generate_llm_context -from .repair import _repair_required_markdown_once +from .local import _render_local_context from .usage import _build_usage_report +from .versions import NOTE_PROMPT_VERSION def _generate_notes_with_llm( @@ -36,46 +30,42 @@ def _generate_notes_with_llm( from . import NoteGenerationResult runtime = resolve_provider_runtime(options.provider, model=options.model, base_url=options.base_url) - resolved_provider = str(runtime["provider"]) - resolved_model = str(runtime["model"]) - resolved_base_url = runtime["base_url"] supports_image_input = bool(runtime["supports_image_input"]) resolved_cache_dir = (options.cache_dir or (output_root / ".cache" / "llm")).resolve() - runtime_options = replace( + # Resolve provider runtime defaults once so calls, cache keys and reports + # all use the same canonical provider/model/base URL/cache directory. + options = replace( options, - provider=resolved_provider, - model=resolved_model, - base_url=resolved_base_url, + provider=str(runtime["provider"]), + model=str(runtime["model"]), + base_url=runtime["base_url"], cache_dir=resolved_cache_dir, ) - cache = LLMCache(resolved_cache_dir, mode=runtime_options.cache_mode) - if runtime_options.note_strategy == "lecture-weave": + cache = LLMCache(resolved_cache_dir, mode=options.cache_mode) + if options.note_strategy == "lecture-weave": return _generate_notes_with_lecture_weave( deck=deck, output_root=output_root, - options=runtime_options, + options=options, note_depth=note_depth, asset_map=asset_map, cache=cache, supports_image_input=supports_image_input, ) - options = runtime_options contexts = _select_note_contexts(deck, options.note_context, section_plan=options.section_plan) resolved_note_context = _resolved_context_mode(deck, options.note_context) refresh_ids = options.refresh_slide_ids or set() - workers = max(1, int(options.concurrency or 1)) - context_results: dict[str, tuple[str, dict[str, Any]]] = {} - def process(context: NoteContext) -> tuple[str, str, dict[str, Any]]: + def process(context: NoteContext) -> tuple[str, dict[str, Any]]: content, context_record = _generate_llm_context( context=context, output_root=output_root, cache=cache, options=options, - provider=resolved_provider, - model=resolved_model, - base_url=resolved_base_url, + provider=options.provider, + model=options.model, + base_url=options.base_url, supports_image_input=supports_image_input, force_refresh=bool(refresh_ids.intersection({page.slide_id for page in context.pages})), asset_map=asset_map, @@ -83,63 +73,48 @@ def process(context: NoteContext) -> tuple[str, str, dict[str, Any]]: note_depth=note_depth, source_type=deck.source_type, ) - content = _postprocess_llm_markdown(content, source_display=options.source_display) - return context.id, content, context_record - - if workers == 1: - for context in contexts: - context_id, content, context_record = process(context) - context_results[context_id] = (content, context_record) - if options.progress_callback: - options.progress_callback(context_record) - else: - with ThreadPoolExecutor(max_workers=workers) as executor: - futures = {executor.submit(process, context): context for context in contexts} - for future in as_completed(futures): - context_id, content, context_record = future.result() - context_results[context_id] = (content, context_record) - if options.progress_callback: - options.progress_callback(context_record) + return _postprocess_llm_markdown(content, source_display=options.source_display), context_record - usage_contexts: list[dict[str, Any]] = [] - final_chunks: dict[str, str] = {} - for context in contexts: - content, context_record = context_results[context.id] - usage_contexts.append(context_record) - final_chunks[context.id] = content + def local_fallback(context: NoteContext, exc: Exception) -> tuple[str, dict[str, Any]]: + content = _render_local_context( + context, + asset_map=asset_map, + source_display=options.source_display, + note_style=options.note_style, + screenshot_policy=options.screenshot_policy, + figure_placement=options.figure_placement, + ) + return content, _failed_context_record(context, exc, generation_stage="note_context", fallback="local") - markdown = _compose_final_markdown( - deck=deck, - contexts=contexts, - final_chunks=final_chunks, - section_plan=options.section_plan, - source_display=options.source_display, + context_results = _run_note_contexts( + contexts, + process, + workers=max(1, int(options.concurrency or 1)), + progress_callback=options.progress_callback, + fallback=local_fallback, ) - markdown = _repair_markdown_image_links(markdown, output_root, asset_map) - markdown = _ensure_grounded_figures(markdown, deck, asset_map, options.source_display, options.figure_placement) - repair_context_records: list[dict[str, Any]] = [] - markdown, repair_record = _repair_required_markdown_once( - deck=deck, - context=NoteContext(id="final", kind="final", title="final", pages=deck.pages), - markdown=markdown, + usage_contexts = [context_results[context.id][1] for context in contexts] + final_chunks = {context.id: context_results[context.id][0] for context in contexts} + + markdown, repair_context_records = _finalize_notes_markdown( + deck, + contexts, + final_chunks, output_root=output_root, cache=cache, options=options, + asset_map=asset_map, stage="final", ) - if repair_record is not None: - record_repair(options.content_guard, repair_record) - if isinstance(repair_record.get("llm"), dict): - repair_context_records.append(repair_record["llm"]) - markdown = _repair_markdown_image_links(markdown, output_root, asset_map) - markdown = _ensure_grounded_figures(markdown, deck, asset_map, options.source_display, options.figure_placement) - markdown = _repair_markdown_image_links(markdown, output_root, asset_map) + warnings = _context_failure_warnings(usage_contexts) usage_report = _build_usage_report( deck=deck, output_root=output_root, options=options, contexts=usage_contexts + repair_context_records, note_strategy=options.note_strategy, + prompt_version=NOTE_PROMPT_VERSION, repair_contexts=repair_context_records, + warnings=warnings, ) - return NoteGenerationResult(markdown=markdown, llm_usage=usage_report) + return NoteGenerationResult(markdown=markdown, llm_usage=usage_report, generation_warnings=warnings) diff --git a/slidenote/notes/figure_blocks.py b/slidenote/notes/figure_blocks.py new file mode 100644 index 0000000..b108ad0 --- /dev/null +++ b/slidenote/notes/figure_blocks.py @@ -0,0 +1,191 @@ +"""Insert grounded figures next to the text they explain.""" + +from __future__ import annotations + +import re + +from slidenote.figure_grounding import note_candidate_images +from slidenote.models import Deck, ImageAsset, SlidePage +from slidenote.utils import source_tokens +from .assets import _asset_display_path +from .frontmatter import _leading_frontmatter_slide_ids +from .render_blocks import _render_image +from .sources import SOURCE_COMMENT_PREFIX, _collapse_blank_lines, _image_source_ids, _source_marker + + +def _ensure_grounded_figures( + markdown: str, + deck: Deck, + asset_map: dict[str, str], + source_display: str, + figure_placement: str, +) -> str: + current = markdown.rstrip() + frontmatter_slide_ids = _leading_frontmatter_slide_ids(deck.pages) + for page in deck.pages: + if page.slide_id in frontmatter_slide_ids: + continue + for image in note_candidate_images(page): + image_path = _asset_display_path(image.path, asset_map) + block = "\n".join(_render_image(page, image, asset_map=asset_map, source_display=source_display)).strip() + if not block: + continue + if figure_placement == "inline": + current = _remove_existing_image_block(current, image_path, image) + current = _insert_figure_block(current, page, image, block, figure_placement) + continue + if _image_markdown_present(current, image_path): + if image.id not in source_tokens(current): + current = _ensure_image_source_marker(current, page, image, image_path, source_display) + continue + current = _insert_figure_block(current, page, image, block, figure_placement) + return current.rstrip() + "\n" + + +def _image_markdown_present(markdown: str, image_path: str) -> bool: + if not image_path: + return False + escaped = re.escape(image_path.strip()) + return bool(re.search(rf"!\[[^\]]*]\({escaped}\)", markdown)) or image_path in markdown + + +def _ensure_image_source_marker( + markdown: str, + page: SlidePage, + image: ImageAsset, + image_path: str, + source_display: str, +) -> str: + marker = _source_marker(page.slide_id, _image_source_ids(image), source_display) + if not marker: + return markdown + lines = markdown.splitlines() + for index, line in enumerate(lines): + if image_path in line and line.lstrip().startswith("!["): + if marker in line or (index + 1 < len(lines) and marker in lines[index + 1]): + return markdown + new_lines = list(lines) + new_lines.insert(index + 1, marker) + return "\n".join(new_lines).rstrip() + "\n" + return markdown + + +def _remove_existing_image_block(markdown: str, image_path: str, image: ImageAsset) -> str: + if not image_path: + return markdown + lines = markdown.splitlines() + remove: set[int] = set() + source_ids = set(_image_source_ids(image)) + for index, line in enumerate(lines): + if not _line_has_image_target(line, image_path): + continue + remove.update(_image_block_indexes_to_remove(lines, index, image, source_ids)) + if not remove: + return markdown + kept = [line for index, line in enumerate(lines) if index not in remove] + return _collapse_blank_lines(kept).rstrip() + "\n" + + +def _image_block_indexes_to_remove(lines: list[str], image_index: int, image: ImageAsset, source_ids: set[str]) -> set[int]: + remove = {image_index} + before = image_index - 1 + while before >= 0 and not lines[before].strip(): + remove.add(before) + before -= 1 + if before >= 0 and _is_marker_only_for_ids(lines[before], source_ids): + remove.add(before) + caption = before - 1 + while caption >= 0 and not lines[caption].strip(): + remove.add(caption) + caption -= 1 + if caption >= 0 and _is_image_caption_line(lines[caption], image): + remove.add(caption) + + after = image_index + 1 + while after < len(lines) and not lines[after].strip(): + remove.add(after) + after += 1 + if after < len(lines) and _is_marker_only_for_ids(lines[after], source_ids): + remove.add(after) + return remove + + +def _is_image_caption_line(line: str, image: ImageAsset) -> bool: + stripped = line.strip() + caption = (image.caption or "").strip() + if caption and stripped in {caption, f"{caption}\u3002"}: + return True + return bool(re.fullmatch(r"\u7b2c\s*\d+\s*\u9875(?:\u56fe\u7247|\u56fe\u793a|\u622a\u56fe).*[\u3002.]?", stripped)) + + +def _line_has_image_target(line: str, image_path: str) -> bool: + normalized_path = image_path.strip().strip("<>").replace("\\", "/") + for target in re.findall(r"!\[[^\]]*]\(([^)]+)\)", line): + normalized_target = target.strip().strip("<>").replace("\\", "/") + if normalized_target == normalized_path: + return True + return False + + +def _is_marker_only_for_ids(line: str, source_ids: set[str]) -> bool: + stripped = line.strip() + if not stripped or SOURCE_COMMENT_PREFIX not in stripped: + return False + if not re.fullmatch(r"", stripped): + return False + return bool(source_ids.intersection(source_tokens(stripped))) + + +def _insert_figure_block(markdown: str, page: SlidePage, image: ImageAsset, block: str, figure_placement: str) -> str: + if figure_placement == "inline": + inserted = _insert_after_anchor_source(markdown, image.anchor_element_ids, block) + if inserted != markdown: + return inserted + inserted = _insert_after_page_source(markdown, page.slide_id, block) + if inserted != markdown: + return inserted + fallback_heading = f"### \u7b2c {page.slide_id} \u9875\u56fe\u793a" + return f"{markdown.rstrip()}\n\n{fallback_heading}\n\n{block}" + + +def _insert_after_anchor_source(markdown: str, anchor_ids: list[str], block: str) -> str: + if not anchor_ids: + return markdown + lines = markdown.splitlines() + for index, line in enumerate(lines): + if SOURCE_COMMENT_PREFIX not in line: + continue + if not any(anchor_id in line for anchor_id in anchor_ids): + continue + insert_at = _paragraph_end_after(lines, index) + return _insert_lines(lines, insert_at, block) + return markdown + + +def _insert_after_page_source(markdown: str, slide_id: int, block: str) -> str: + lines = markdown.splitlines() + marker = f"p{slide_id}:" + candidate_index: int | None = None + for index, line in enumerate(lines): + if SOURCE_COMMENT_PREFIX in line and marker in line: + candidate_index = index + if candidate_index is None: + return markdown + insert_at = _paragraph_end_after(lines, candidate_index) + return _insert_lines(lines, insert_at, block) + + +def _paragraph_end_after(lines: list[str], index: int) -> int: + cursor = index + 1 + while cursor < len(lines) and lines[cursor].strip(): + cursor += 1 + while cursor < len(lines) and not lines[cursor].strip(): + cursor += 1 + return cursor + + +def _insert_lines(lines: list[str], index: int, block: str) -> str: + new_lines = list(lines) + insert = ["", *block.splitlines(), ""] + new_lines[index:index] = insert + return "\n".join(new_lines).rstrip() + "\n" diff --git a/slidenote/notes/finalize.py b/slidenote/notes/finalize.py new file mode 100644 index 0000000..3efaea7 --- /dev/null +++ b/slidenote/notes/finalize.py @@ -0,0 +1,81 @@ +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from slidenote.content_guard import missing_required_items, record_repair +from slidenote.coverage import analyze_coverage +from slidenote.llm_cache import LLMCache +from slidenote.models import Deck + +from .assets import _repair_markdown_image_links +from .compose import _compose_final_markdown +from .contexts import NoteContext +from .figure_blocks import _ensure_grounded_figures +from .repair import _repair_required_markdown_once + + +def _finalize_notes_markdown( + deck: Deck, + contexts: list[NoteContext], + final_chunks: dict[str, str], + *, + output_root: Path, + cache: LLMCache, + options: "NoteOptions", + asset_map: dict[str, str], + stage: str, +) -> tuple[str, list[dict[str, Any]]]: + """Compose the final document and repair contexts that miss required items. + + Repairs run per context (never on the whole deck) so the rewrite fits the + output-token budget; accepted repairs replace the chunk in ``final_chunks``. + Returns the final markdown and the LLM usage records of repair calls. + """ + + def compose() -> str: + markdown = _compose_final_markdown( + deck=deck, + contexts=contexts, + final_chunks=final_chunks, + section_plan=options.section_plan, + source_display=options.source_display, + ) + markdown = _repair_markdown_image_links(markdown, output_root, asset_map) + return _ensure_grounded_figures(markdown, deck, asset_map, options.source_display, options.figure_placement) + + markdown = compose() + content_guard = options.content_guard + if not content_guard: + return markdown, [] + coverage = analyze_coverage(deck, markdown, content_guard=content_guard) + missing_slide_ids = {item.get("slide_id") for item in missing_required_items(content_guard, coverage)} + if not missing_slide_ids: + return markdown, [] + + refresh_ids = options.refresh_slide_ids or set() + repair_usage: list[dict[str, Any]] = [] + changed = False + for context in contexts: + slide_ids = {page.slide_id for page in context.pages} + if not slide_ids & missing_slide_ids or not final_chunks.get(context.id, "").strip(): + continue + repaired, record = _repair_required_markdown_once( + deck=Deck(source_path=deck.source_path, source_type=deck.source_type, pages=list(context.pages)), + context=context, + markdown=final_chunks[context.id], + output_root=output_root, + cache=cache, + options=options, + stage=stage, + force_refresh=bool(refresh_ids & slide_ids), + ) + if record is None: + continue + record_repair(content_guard, record) + if isinstance(record.get("llm"), dict): + repair_usage.append(record["llm"]) + if record.get("accepted"): + final_chunks[context.id] = repaired + changed = True + return (compose() if changed else markdown), repair_usage diff --git a/slidenote/notes/frontmatter.py b/slidenote/notes/frontmatter.py new file mode 100644 index 0000000..d8f2f97 --- /dev/null +++ b/slidenote/notes/frontmatter.py @@ -0,0 +1,107 @@ +"""Detect cover/agenda front matter and generic headings.""" + +from __future__ import annotations + +import re + +from slidenote.models import SlidePage +from slidenote.utils import looks_like_outline_page +from .contexts import NoteContext +from .sources import _page_source_ids, _source_marker + + +def _clean_heading_text(value: str) -> str: + text = re.sub(r"", "", value).strip() + text = re.sub(r"^\u8bfe\u7a0b\u7b14\u8bb0[\uff1a:\s-]*", "", text).strip() + text = re.sub(r"^\s*[\uff08(]?\s*(?:\d+|[\u4e00\u4e8c\u4e09\u56db\u4e94\u516d\u4e03\u516b\u4e5d\u5341]+)\s*[)\uff09.\u3001]\s*", "", text).strip() + return text.strip("\uff1a: -") + + +def _normalize_title_key(value: str) -> str: + return re.sub(r"[\s:\uff1a,\uff0c.\u3002;\uff1b\u3001\-_\u2014\uff08\uff09()\u300a\u300b<>]+", "", _clean_heading_text(value)).lower() + + +def _is_generic_heading_text(value: str) -> bool: + normalized = _normalize_title_key(value) + return normalized in { + "", + "\u8bfe\u7a0b\u7b14\u8bb0", + "\u7b14\u8bb0", + "\u8bb2\u4e49", + "\u751f\u6210\u4fe1\u606f", + "\u89e3\u6790\u63d0\u9192", + "\u76ee\u5f55", + "contents", + "overview", + } + + +def _is_frontmatter_heading(title: str, context: NoteContext) -> bool: + normalized = _normalize_title_key(title) + if normalized in {"\u76ee\u5f55", "contents", "\u8bfe\u7a0b\u6982\u89c8", "overview"}: + return True + if len(context.pages) <= 2 and all(_normalize_title_key(page.title or "") in {"\u76ee\u5f55", "contents"} for page in context.pages): + return True + return False + + +def _leading_frontmatter_slide_ids(pages: list[SlidePage]) -> set[int]: + slide_ids: set[int] = set() + for index, page in enumerate(pages): + if not _is_frontmatter_page(page, index): + break + slide_ids.add(page.slide_id) + return slide_ids + + +def _is_frontmatter_page(page: SlidePage, index: int) -> bool: + title = page.title or "" + normalized_title = _normalize_title_key(title) + if normalized_title in {"\u76ee\u5f55", "contents", "outline", "agenda"}: + return True + text = "\n".join([title, *(block.content for block in page.text_blocks)]) + if "\u76ee\u5f55" in text or "Contents" in text: + return True + if index == 0 and _looks_like_cover_page(text): + return True + return index <= 3 and looks_like_outline_page(text) + + +def _looks_like_cover_page(text: str) -> bool: + normalized = _normalize_title_key(text) + cover_markers = { + "\u8bb2\u5e08", + "\u6559\u5e08", + "\u6559\u6388", + "\u8054\u7cfb\u90ae\u7bb1", + "\u90ae\u7bb1", + "\u4e3b\u9875", + "email", + "homepage", + "http", + "www", + } + return any(marker in normalized for marker in cover_markers) + + +def _looks_like_frontmatter_text(text: str) -> bool: + normalized = _normalize_title_key(text) + markers = { + "\u76ee\u5f55", + "\u8bfe\u7a0b\u76ee\u5f55", + "\u672c\u7ae0\u76ee\u5f55", + "\u4e3b\u6807\u9898", + "\u526f\u6807\u9898", + "\u8bb2\u5e08", + "\u6559\u6388", + "\u8054\u7cfb\u90ae\u7bb1", + "\u4e3b\u9875", + "contents", + "overview", + } + return any(marker in normalized for marker in markers) + + +def _frontmatter_source_markers(pages: list[SlidePage]) -> str: + markers = [_source_marker(page.slide_id, _page_source_ids(page), "hidden") for page in pages] + return "\n".join(marker for marker in markers if marker) diff --git a/slidenote/notes/lecture_weave.py b/slidenote/notes/lecture_weave.py index be1c89f..2d23924 100644 --- a/slidenote/notes/lecture_weave.py +++ b/slidenote/notes/lecture_weave.py @@ -1,6 +1,5 @@ from __future__ import annotations -from concurrent.futures import ThreadPoolExecutor, as_completed from pathlib import Path from typing import Any @@ -8,24 +7,18 @@ from slidenote.llm_cache import LLMCache from slidenote.models import Deck -from .assembly import ( - NoteContext, - _build_page_notes_report, - _build_teaching_enrichment_report, - _build_weave_report, - _compose_final_markdown, - _ensure_grounded_figures, - _postprocess_llm_markdown, - _repair_markdown_image_links, - _render_page_notes_markdown, - _resolved_context_mode, - _select_note_contexts, -) +from .contexts import NoteContext, _resolved_context_mode, _select_note_contexts +from .postprocess import _postprocess_llm_markdown +from .reports import _build_page_notes_report, _build_teaching_enrichment_report, _build_weave_report, _render_page_notes_markdown +from .context_runner import _context_failure_warnings, _failed_context_record, _run_note_contexts +from .finalize import _finalize_notes_markdown from .llm_calls import _generate_page_lecture_context, _generate_teaching_enrichment_context, _generate_weave_context -from .options import should_run_teaching_enrichment +from .local import _render_local_context +from .options import needs_teaching_enrichment, should_run_teaching_enrichment from .prompt_payload import _section_title_by_slide from .repair import _repair_required_markdown_once from .usage import _build_usage_report +from .versions import WEAVE_PROMPT_VERSION def _generate_notes_with_lecture_weave( @@ -43,61 +36,150 @@ def _generate_notes_with_lecture_weave( # ``direct`` resolves provider runtime defaults once and stores them back # into the immutable copy passed here, keeping calls, cache keys and reports # on the same canonical provider/model/base URL/cache directory. - provider = options.provider - model = options.model - api_key = options.api_key - base_url = options.base_url - max_output_tokens = options.max_output_tokens - temperature = options.temperature - cache_mode = options.cache_mode - cache_dir = options.cache_dir - concurrency = options.concurrency - refresh_slide_ids = options.refresh_slide_ids - progress_callback = options.progress_callback - asset_mode = options.asset_mode - source_display = options.source_display - note_context = options.note_context - note_style = options.note_style - note_profile = options.note_profile - note_language = options.note_language - term_policy = options.term_policy - teaching_enrichment = options.teaching_enrichment - weave_dedup = options.weave_dedup - page_neighborhood = options.page_neighborhood - screenshot_policy = options.screenshot_policy - figure_placement = options.figure_placement - section_plan = options.section_plan - deck_brief = options.deck_brief - content_guard = options.content_guard - - refresh_ids = refresh_slide_ids or set() - workers = max(1, int(concurrency or 1)) page_contexts = [ - NoteContext(id=f"p{page.slide_id}", kind="page_note", title=page.title or f"\u7b2c {page.slide_id} \u9875", pages=[page]) + NoteContext(id=f"p{page.slide_id}", kind="page_note", title=page.title or f"第 {page.slide_id} 页", pages=[page]) for page in deck.pages ] - section_titles = _section_title_by_slide(deck, section_plan=section_plan) - page_results: dict[str, tuple[str, dict[str, Any]]] = {} + page_markdown_by_slide, page_records, repair_context_records = _generate_page_notes( + deck, page_contexts, output_root, options, note_depth=note_depth, asset_map=asset_map, + cache=cache, supports_image_input=supports_image_input, + ) + + resolved_note_context = _resolved_context_mode(deck, options.note_context) + weave_contexts = _select_note_contexts(deck, options.note_context, section_plan=options.section_plan) + final_chunks, weave_records = _weave_contexts( + weave_contexts, page_markdown_by_slide, output_root, options, cache=cache, + note_context=resolved_note_context, note_depth=note_depth, + ) + teaching_contexts = _teaching_contexts(weave_contexts, final_chunks, options) + if options.progress_callback and options.teaching_enrichment == "auto": + options.progress_callback({"event": "total", "total": len(page_contexts) + len(weave_contexts) + len(teaching_contexts)}) + teaching_records = _enrich_teaching_contexts( + teaching_contexts, final_chunks, page_markdown_by_slide, output_root, options, cache=cache, + note_context=resolved_note_context, note_depth=note_depth, + ) + + markdown, final_repair_records = _finalize_notes_markdown( + deck, + weave_contexts, + final_chunks, + output_root=output_root, + cache=cache, + options=options, + asset_map=asset_map, + stage="weave", + ) + repair_context_records.extend(final_repair_records) + warnings = _context_failure_warnings(page_records + weave_records + teaching_records) + usage_report = _build_usage_report( + deck=deck, + output_root=output_root, + options=options, + contexts=page_records + weave_records + teaching_records + repair_context_records, + note_strategy="lecture-weave", + prompt_version=WEAVE_PROMPT_VERSION, + page_contexts=page_records, + weave_contexts=weave_records, + teaching_enrichment_contexts=teaching_records, + repair_contexts=repair_context_records, + warnings=warnings, + ) + + page_notes = _build_page_notes_report( + deck=deck, + provider=options.provider, + model=options.model, + base_url=options.base_url, + note_depth=note_depth, + note_language=options.note_language, + term_policy=options.term_policy, + page_neighborhood=options.page_neighborhood, + pages=page_contexts, + page_markdown_by_slide=page_markdown_by_slide, + page_records=page_records, + deck_brief=options.deck_brief, + ) + weave_report = _build_weave_report( + deck=deck, + note_context=resolved_note_context, + note_depth=note_depth, + note_language=options.note_language, + term_policy=options.term_policy, + weave_dedup=options.weave_dedup, + contexts=weave_contexts, + final_chunks=final_chunks, + page_markdown_by_slide=page_markdown_by_slide, + weave_records=weave_records, + deck_brief=options.deck_brief, + ) + teaching_report = None + if teaching_records: + teaching_report = _build_teaching_enrichment_report( + deck=deck, + note_context=resolved_note_context, + note_profile=options.note_profile, + note_depth=note_depth, + note_language=options.note_language, + term_policy=options.term_policy, + contexts=teaching_contexts, + final_chunks=final_chunks, + page_markdown_by_slide=page_markdown_by_slide, + teaching_records=teaching_records, + deck_brief=options.deck_brief, + ) + return NoteGenerationResult( + markdown=markdown, + llm_usage=usage_report, + page_notes=page_notes, + page_notes_markdown=_render_page_notes_markdown(deck, page_notes), + weave_report=weave_report, + teaching_report=teaching_report, + generation_warnings=warnings, + ) + + +def _refresh_requested(options: "NoteOptions", context: NoteContext) -> bool: + return bool((options.refresh_slide_ids or set()).intersection(page.slide_id for page in context.pages)) - def process_page(context: NoteContext) -> tuple[str, str, dict[str, Any]]: + +def _workers(options: "NoteOptions") -> int: + return max(1, int(options.concurrency or 1)) + + +def _generate_page_notes( + deck: Deck, + page_contexts: list[NoteContext], + output_root: Path, + options: "NoteOptions", + *, + note_depth: str, + asset_map: dict[str, str], + cache: LLMCache, + supports_image_input: bool, +) -> tuple[dict[int, str], list[dict[str, Any]], list[dict[str, Any]]]: + """Stage 1: one lecture-style note per page, each repaired for required items.""" + section_titles = _section_title_by_slide(deck, section_plan=options.section_plan) + + def process_page(context: NoteContext) -> tuple[str, dict[str, Any]]: page = context.pages[0] + force_refresh = _refresh_requested(options, context) content, record = _generate_page_lecture_context( deck=deck, context=context, output_root=output_root, cache=cache, options=options, - provider=provider, - model=model, - base_url=base_url, + provider=options.provider, + model=options.model, + base_url=options.base_url, supports_image_input=supports_image_input, - force_refresh=page.slide_id in refresh_ids, + force_refresh=force_refresh, asset_map=asset_map, note_depth=note_depth, - page_neighborhood=page_neighborhood, + page_neighborhood=options.page_neighborhood, section_title=section_titles.get(page.slide_id), ) - content = _postprocess_llm_markdown(content, source_display=source_display) + content = _postprocess_llm_markdown(content, source_display=options.source_display) page_deck = Deck(source_path=deck.source_path, source_type=deck.source_type, pages=[page]) content, repair_record = _repair_required_markdown_once( deck=page_deck, @@ -107,26 +189,27 @@ def process_page(context: NoteContext) -> tuple[str, str, dict[str, Any]]: cache=cache, options=options, stage="page_note", + force_refresh=force_refresh, ) if repair_record is not None: record["content_guard_repair"] = repair_record - return context.id, content, record + return content, record - if workers == 1: - for context in page_contexts: - context_id, content, record = process_page(context) - page_results[context_id] = (content, record) - if progress_callback: - progress_callback(record) - else: - with ThreadPoolExecutor(max_workers=workers) as executor: - futures = {executor.submit(process_page, context): context for context in page_contexts} - for future in as_completed(futures): - context_id, content, record = future.result() - page_results[context_id] = (content, record) - if progress_callback: - progress_callback(record) + def local_fallback(context: NoteContext, exc: Exception) -> tuple[str, dict[str, Any]]: + content = _render_local_context( + context, + asset_map=asset_map, + source_display=options.source_display, + note_style=options.note_style, + screenshot_policy=options.screenshot_policy, + figure_placement=options.figure_placement, + ) + return content, _failed_context_record(context, exc, generation_stage="page_note", fallback="local") + page_results = _run_note_contexts( + page_contexts, process_page, workers=_workers(options), + progress_callback=options.progress_callback, fallback=local_fallback, + ) page_markdown_by_slide: dict[int, str] = {} page_records: list[dict[str, Any]] = [] repair_context_records: list[dict[str, Any]] = [] @@ -136,181 +219,112 @@ def process_page(context: NoteContext) -> tuple[str, str, dict[str, Any]]: page_records.append(record) repair_record = record.get("content_guard_repair") if isinstance(repair_record, dict): - record_repair(content_guard, repair_record) + record_repair(options.content_guard, repair_record) if isinstance(repair_record.get("llm"), dict): repair_context_records.append(repair_record["llm"]) + return page_markdown_by_slide, page_records, repair_context_records - resolved_note_context = _resolved_context_mode(deck, note_context) - weave_contexts = _select_note_contexts(deck, note_context, section_plan=section_plan) - weave_results: dict[str, tuple[str, dict[str, Any]]] = {} - def process_weave(context: NoteContext) -> tuple[str, str, dict[str, Any]]: +def _weave_contexts( + weave_contexts: list[NoteContext], + page_markdown_by_slide: dict[int, str], + output_root: Path, + options: "NoteOptions", + *, + cache: LLMCache, + note_context: str, + note_depth: str, +) -> tuple[dict[str, str], list[dict[str, Any]]]: + """Stage 2: weave page notes of each context into one coherent chunk.""" + + def process_weave(context: NoteContext) -> tuple[str, dict[str, Any]]: content, record = _generate_weave_context( context=context, page_markdown_by_slide=page_markdown_by_slide, output_root=output_root, cache=cache, options=options, - provider=provider, - model=model, - base_url=base_url, - note_context=resolved_note_context, + provider=options.provider, + model=options.model, + base_url=options.base_url, + note_context=note_context, note_depth=note_depth, - force_refresh=bool(refresh_ids.intersection({page.slide_id for page in context.pages})), + force_refresh=_refresh_requested(options, context), ) - return context.id, _postprocess_llm_markdown(content, source_display=source_display), record + return _postprocess_llm_markdown(content, source_display=options.source_display), record - if workers == 1: - for context in weave_contexts: - context_id, content, record = process_weave(context) - weave_results[context_id] = (content, record) - if progress_callback: - progress_callback(record) - else: - with ThreadPoolExecutor(max_workers=workers) as executor: - futures = {executor.submit(process_weave, context): context for context in weave_contexts} - for future in as_completed(futures): - context_id, content, record = future.result() - weave_results[context_id] = (content, record) - if progress_callback: - progress_callback(record) - - weave_records: list[dict[str, Any]] = [] - final_chunks: dict[str, str] = {} - for context in weave_contexts: - content, record = weave_results[context.id] - final_chunks[context.id] = content - weave_records.append(record) + def page_notes_fallback(context: NoteContext, exc: Exception) -> tuple[str, dict[str, Any]]: + content = "\n\n".join(page_markdown_by_slide.get(page.slide_id, "") for page in context.pages).strip() + weave_context = NoteContext(id=f"weave_{context.id}", kind=f"weave_{context.kind}", title=context.title, pages=context.pages) + return content, _failed_context_record(weave_context, exc, generation_stage="weave", fallback="page_notes") - teaching_records: list[dict[str, Any]] = [] - teaching_report: dict[str, Any] | None = None - if should_run_teaching_enrichment(note_profile, teaching_enrichment, "lecture-weave"): - teaching_results: dict[str, tuple[str, dict[str, Any]]] = {} + weave_results = _run_note_contexts( + weave_contexts, process_weave, workers=_workers(options), + progress_callback=options.progress_callback, fallback=page_notes_fallback, + ) + final_chunks = {context.id: weave_results[context.id][0] for context in weave_contexts} + weave_records = [weave_results[context.id][1] for context in weave_contexts] + return final_chunks, weave_records - def process_teaching(context: NoteContext) -> tuple[str, str, dict[str, Any]]: - content, record = _generate_teaching_enrichment_context( - context=context, - woven_markdown=final_chunks.get(context.id, ""), - page_markdown_by_slide=page_markdown_by_slide, - output_root=output_root, - cache=cache, - options=options, - provider=provider, - model=model, - base_url=base_url, - note_context=resolved_note_context, - note_depth=note_depth, - force_refresh=bool(refresh_ids.intersection({page.slide_id for page in context.pages})), - ) - return context.id, _postprocess_llm_markdown(content, source_display=source_display), record - if workers == 1: - for context in weave_contexts: - context_id, content, record = process_teaching(context) - teaching_results[context_id] = (content, record) - if progress_callback: - progress_callback(record) - else: - with ThreadPoolExecutor(max_workers=workers) as executor: - futures = {executor.submit(process_teaching, context): context for context in weave_contexts} - for future in as_completed(futures): - context_id, content, record = future.result() - teaching_results[context_id] = (content, record) - if progress_callback: - progress_callback(record) +def _teaching_contexts( + weave_contexts: list[NoteContext], + final_chunks: dict[str, str], + options: "NoteOptions", +) -> list[NoteContext]: + if not should_run_teaching_enrichment(options.note_profile, options.teaching_enrichment, "lecture-weave"): + return [] + return [ + context for context in weave_contexts + if options.teaching_enrichment == "force" + or needs_teaching_enrichment(final_chunks.get(context.id, ""), len(context.pages)) + ] - for context in weave_contexts: - content, record = teaching_results[context.id] - final_chunks[context.id] = content - teaching_records.append(record) - markdown = _compose_final_markdown( - deck=deck, - contexts=weave_contexts, - final_chunks=final_chunks, - section_plan=section_plan, - source_display=source_display, - ) - markdown = _repair_markdown_image_links(markdown, output_root, asset_map) - markdown = _ensure_grounded_figures(markdown, deck, asset_map, source_display, figure_placement) - markdown, final_repair_record = _repair_required_markdown_once( - deck=deck, - context=NoteContext(id="final", kind="final", title="final", pages=deck.pages), - markdown=markdown, - output_root=output_root, - cache=cache, - options=options, - stage="weave", - ) - if final_repair_record is not None: - record_repair(content_guard, final_repair_record) - if isinstance(final_repair_record.get("llm"), dict): - repair_context_records.append(final_repair_record["llm"]) - markdown = _repair_markdown_image_links(markdown, output_root, asset_map) - markdown = _ensure_grounded_figures(markdown, deck, asset_map, source_display, figure_placement) - markdown = _repair_markdown_image_links(markdown, output_root, asset_map) - all_context_records = page_records + weave_records + teaching_records + repair_context_records - usage_report = _build_usage_report( - deck=deck, - output_root=output_root, - options=options, - contexts=all_context_records, - note_strategy="lecture-weave", - page_contexts=page_records, - weave_contexts=weave_records, - teaching_enrichment_contexts=teaching_records, - repair_contexts=repair_context_records, - ) +def _enrich_teaching_contexts( + teaching_contexts: list[NoteContext], + final_chunks: dict[str, str], + page_markdown_by_slide: dict[int, str], + output_root: Path, + options: "NoteOptions", + *, + cache: LLMCache, + note_context: str, + note_depth: str, +) -> list[dict[str, Any]]: + """Stage 3 (optional): add examples, pitfalls and self-checks; updates ``final_chunks``.""" + if not teaching_contexts: + return [] - page_notes = _build_page_notes_report( - deck=deck, - provider=provider, - model=model, - base_url=base_url, - output_root=output_root, - note_depth=note_depth, - note_language=note_language, - term_policy=term_policy, - page_neighborhood=page_neighborhood, - pages=page_contexts, - page_markdown_by_slide=page_markdown_by_slide, - page_records=page_records, - deck_brief=deck_brief, - ) - weave_report = _build_weave_report( - deck=deck, - output_root=output_root, - note_context=resolved_note_context, - note_depth=note_depth, - note_language=note_language, - term_policy=term_policy, - weave_dedup=weave_dedup, - contexts=weave_contexts, - final_chunks=final_chunks, - page_markdown_by_slide=page_markdown_by_slide, - weave_records=weave_records, - deck_brief=deck_brief, - ) - if teaching_records: - teaching_report = _build_teaching_enrichment_report( - deck=deck, + def process_teaching(context: NoteContext) -> tuple[str, dict[str, Any]]: + content, record = _generate_teaching_enrichment_context( + context=context, + woven_markdown=final_chunks.get(context.id, ""), + page_markdown_by_slide=page_markdown_by_slide, output_root=output_root, - note_context=resolved_note_context, - note_profile=note_profile, + cache=cache, + options=options, + provider=options.provider, + model=options.model, + base_url=options.base_url, + note_context=note_context, note_depth=note_depth, - note_language=note_language, - term_policy=term_policy, - contexts=weave_contexts, - final_chunks=final_chunks, - page_markdown_by_slide=page_markdown_by_slide, - teaching_records=teaching_records, - deck_brief=deck_brief, + force_refresh=_refresh_requested(options, context), ) - return NoteGenerationResult( - markdown=markdown, - llm_usage=usage_report, - page_notes=page_notes, - page_notes_markdown=_render_page_notes_markdown(deck, page_notes), - weave_report=weave_report, - teaching_report=teaching_report, + return _postprocess_llm_markdown(content, source_display=options.source_display), record + + def keep_woven_fallback(context: NoteContext, exc: Exception) -> tuple[str, dict[str, Any]]: + teaching_context = NoteContext(id=f"teaching_{context.id}", kind=f"teaching_{context.kind}", title=context.title, pages=context.pages) + record = _failed_context_record(teaching_context, exc, generation_stage="teaching_enrichment", fallback="woven_notes") + return final_chunks.get(context.id, ""), record + + teaching_results = _run_note_contexts( + teaching_contexts, process_teaching, workers=_workers(options), + progress_callback=options.progress_callback, fallback=keep_woven_fallback, ) + teaching_records: list[dict[str, Any]] = [] + for context in teaching_contexts: + content, record = teaching_results[context.id] + final_chunks[context.id] = content + teaching_records.append(record) + return teaching_records diff --git a/slidenote/notes/llm_calls.py b/slidenote/notes/llm_calls.py index 4b41fde..5b1c090 100644 --- a/slidenote/notes/llm_calls.py +++ b/slidenote/notes/llm_calls.py @@ -1,18 +1,21 @@ from __future__ import annotations from pathlib import Path -from typing import Any +from typing import TYPE_CHECKING, Any from slidenote.llm import LLMClient as _DefaultLLMClient, SYSTEM_PROMPT from slidenote.llm_cache import LLM_CACHE_SCHEMA_VERSION, LLMCache, make_cache_key, sha256_text, stable_json from slidenote.models import Deck from slidenote.utils import display_path -from .assembly import NoteContext +from .contexts import NoteContext from .prompt_payload import _prompt_brief_hash, _prompt_deck_brief, _prompt_slide_scope from .prompt_templates import _llm_context_prompt, _llm_page_lecture_prompt, _llm_teaching_enrichment_prompt, _llm_weave_prompt from .versions import NOTE_PROMPT_VERSION, PAGE_LECTURE_PROMPT_VERSION, TEACHING_ENRICHMENT_PROMPT_VERSION, WEAVE_PROMPT_VERSION +if TYPE_CHECKING: + from . import NoteOptions + LLMClient = _DefaultLLMClient @@ -36,145 +39,56 @@ def _generate_llm_context( source_type: str, force_refresh: bool = False, ) -> tuple[str, dict[str, Any]]: - cache_mode = options.cache_mode - api_key = options.api_key - max_output_tokens = options.max_output_tokens - temperature = options.temperature - asset_mode = options.asset_mode - source_display = options.source_display - note_style = options.note_style - note_profile = options.note_profile - note_language = options.note_language - term_policy = options.term_policy - screenshot_policy = options.screenshot_policy - figure_placement = options.figure_placement - deck_brief = options.deck_brief - content_guard = options.content_guard user_prompt = _llm_context_prompt( context, supports_image_input=supports_image_input, asset_map=asset_map, - source_display=source_display, + source_display=options.source_display, note_context=note_context, - note_style=note_style, - note_profile=note_profile, + note_style=options.note_style, + note_profile=options.note_profile, note_depth=note_depth, - note_language=note_language, - term_policy=term_policy, - screenshot_policy=screenshot_policy, - figure_placement=figure_placement, + note_language=options.note_language, + term_policy=options.term_policy, + screenshot_policy=options.screenshot_policy, + figure_placement=options.figure_placement, source_type=source_type, - deck_brief=deck_brief, - content_guard=content_guard, + deck_brief=options.deck_brief, + content_guard=options.content_guard, ) - prompt_brief = _prompt_deck_brief(deck_brief, [page.slide_id for page in context.pages]) - cache_key_payload = { - "schema_version": LLM_CACHE_SCHEMA_VERSION, - "prompt_version": NOTE_PROMPT_VERSION, - "provider": provider, - "model": model, - "base_url": base_url, - "temperature": temperature, - "max_output_tokens": max_output_tokens, - "asset_mode": asset_mode, - "source_display": source_display, - "note_context": note_context, - "note_style": note_style, - "note_profile": note_profile, - "note_depth": note_depth, - "note_language": note_language, - "term_policy": term_policy, - "screenshot_policy": screenshot_policy, - "figure_placement": figure_placement, - "deck_brief_hash": _prompt_brief_hash(prompt_brief), - "content_guard_used": bool(content_guard), - "system_prompt_hash": sha256_text(SYSTEM_PROMPT), - "user_prompt_hash": sha256_text(user_prompt), - "user_prompt": user_prompt, - } - cache_key = make_cache_key(cache_key_payload) - cache_path = cache.path_for(cache_key) - prompt_hash = sha256_text(stable_json(cache_key_payload)) - cached = None if force_refresh else cache.read(cache_key) - context_record = _base_usage_context_record( + prompt_brief = _prompt_deck_brief(options.deck_brief, [page.slide_id for page in context.pages]) + return _generate_cached_llm_text( context=context, - cache_key=cache_key, - cache_path=cache_path, output_root=output_root, - prompt_hash=prompt_hash, + cache=cache, + cache_mode=options.cache_mode, + provider=provider, + model=model, + api_key=options.api_key, + base_url=base_url, + max_output_tokens=options.max_output_tokens, + temperature=options.temperature, + user_prompt=user_prompt, + prompt_version=NOTE_PROMPT_VERSION, + generation_stage="context_note", + force_refresh=force_refresh, + request_options={ + "asset_mode": options.asset_mode, + "source_display": options.source_display, + "note_context": note_context, + "note_style": options.note_style, + "note_profile": options.note_profile, + "note_depth": note_depth, + "note_language": options.note_language, + "term_policy": options.term_policy, + "screenshot_policy": options.screenshot_policy, + "figure_placement": options.figure_placement, + "deck_brief_used": bool(prompt_brief), + "deck_brief_hash": _prompt_brief_hash(prompt_brief), + "content_guard_used": bool(options.content_guard), + }, ) - if cached: - content = cached["output_text"] - cached_usage = cached.get("response_usage") or {} - context_record.update( - { - "cache_status": "local_hit", - "llm_call": False, - "input_tokens": 0, - "output_tokens": 0, - "total_tokens": 0, - "provider_cached_input_tokens": 0, - "cached_entry_usage": cached_usage, - "cached_at": cached.get("created_at"), - } - ) - else: - client = _make_llm_client( - provider=provider, - model=model, - api_key=api_key, - base_url=base_url, - max_output_tokens=max_output_tokens, - temperature=temperature, - ) - llm_result = client.generate_with_usage(user_prompt) - content = llm_result.text - response_usage = llm_result.usage or {} - cache_status = "disabled" if cache_mode == "off" else "refresh" if cache_mode == "refresh" or force_refresh else "miss" - written_path = cache.write( - cache_key, - { - "provider": provider, - "model": model, - "base_url": base_url, - "prompt_version": NOTE_PROMPT_VERSION, - "context_id": context.id, - "context_kind": context.kind, - "slide_ids": [page.slide_id for page in context.pages], - "request": { - "temperature": temperature, - "max_output_tokens": max_output_tokens, - "note_profile": note_profile, - "note_depth": note_depth, - "note_language": note_language, - "term_policy": term_policy, - "deck_brief_used": bool(prompt_brief), - "content_guard_used": bool(content_guard), - }, - "prompt_hash": prompt_hash, - "output_text": content, - "response_usage": response_usage, - }, - ) - if written_path is not None: - cache_path = written_path - context_record.update( - { - "cache_status": cache_status, - "llm_call": True, - "api_retries": response_usage.get("retries", 0), - "input_tokens": response_usage.get("input_tokens"), - "output_tokens": response_usage.get("output_tokens"), - "total_tokens": response_usage.get("total_tokens"), - "provider_cached_input_tokens": response_usage.get("provider_cached_input_tokens"), - "provider_usage": response_usage, - } - ) - - context_record["note_chars"] = len(content) - context_record["cache_file"] = display_path(cache_path, output_root) - return content, context_record def _generate_page_lecture_context( deck: Deck, @@ -193,7 +107,6 @@ def _generate_page_lecture_context( section_title: str | None, force_refresh: bool = False, ) -> tuple[str, dict[str, Any]]: - from .prompts import _prompt_slide_scope cache_mode = options.cache_mode api_key = options.api_key max_output_tokens = options.max_output_tokens diff --git a/slidenote/notes/local.py b/slidenote/notes/local.py index f81defb..fe6ca8a 100644 --- a/slidenote/notes/local.py +++ b/slidenote/notes/local.py @@ -8,23 +8,12 @@ from slidenote.models import Deck, SlidePage, TableBlock, TextBlock from slidenote.utils import escape_md -from .assembly import ( - NoteContext, - _asset_display_path, - _context_heading, - _context_heading_title, - _document_title, - _ensure_sentence, - _is_frontmatter_heading, - _page_element_ids, - _quote_multiline, - _render_image, - _section_contexts, - _should_add_context_headings, - _should_render_screenshot, - _source_marker, - _styled_block_text, -) +from .assets import _asset_display_path, _should_render_screenshot +from .compose import _context_heading, _context_heading_title, _document_title, _should_add_context_headings +from .contexts import NoteContext, _section_contexts +from .frontmatter import _is_frontmatter_heading +from .render_blocks import _ensure_sentence, _quote_multiline, _render_image, _styled_block_text +from .sources import _page_element_ids, _source_marker def _generate_notes_locally( @@ -80,6 +69,31 @@ def _generate_notes_locally( return "\n".join(lines).rstrip() + "\n" +def _render_local_context( + context: NoteContext, + asset_map: dict[str, str], + source_display: str, + note_style: str, + screenshot_policy: str, + figure_placement: str, +) -> str: + """Rule-based markdown for one context, used when its LLM call fails.""" + lines: list[str] = [] + for page in context.pages: + lines.extend( + _render_local_page( + page, + asset_map=asset_map, + source_display=source_display, + note_style=note_style, + screenshot_policy=screenshot_policy, + figure_placement=figure_placement, + heading_level="##", + ) + ) + return "\n".join(lines).strip() + + def _render_local_page( page: SlidePage, asset_map: dict[str, str], diff --git a/slidenote/notes/options.py b/slidenote/notes/options.py index 77b5bbe..a64a7d1 100644 --- a/slidenote/notes/options.py +++ b/slidenote/notes/options.py @@ -1,5 +1,7 @@ from __future__ import annotations +import re + NOTE_PROFILES = {"auto", "lecture-notes", "study-guide"} TEACHING_ENRICHMENT_MODES = {"auto", "off", "force"} @@ -20,3 +22,17 @@ def should_run_teaching_enrichment(note_profile: str, teaching_enrichment: str, if teaching_enrichment == "force": return True return note_profile in {"lecture-notes", "study-guide"} + + +def needs_teaching_enrichment(markdown: str, page_count: int) -> bool: + """Skip the extra model pass when the woven draft already has teaching signals.""" + visible = re.sub(r"", "", markdown, flags=re.DOTALL) + body_chars = sum(char.isalnum() for char in visible) + if body_chars < max(100, 60 * page_count): + return True + teaching_signals = ( + r"例如|例子|比如|类比|for example|e\.g\.|analogy", + r"易错|误解|陷阱|常见错误|pitfall|misconception|common mistake", + r"自测|思考题|练习题|检查自己|self[- ]?check|quiz|review question", + ) + return not all(re.search(pattern, visible, flags=re.IGNORECASE) for pattern in teaching_signals) diff --git a/slidenote/notes/orchestrator.py b/slidenote/notes/orchestrator.py deleted file mode 100644 index f324fe2..0000000 --- a/slidenote/notes/orchestrator.py +++ /dev/null @@ -1,27 +0,0 @@ -from __future__ import annotations - -from .direct import _generate_notes_with_llm -from .lecture_weave import _generate_notes_with_lecture_weave -from .llm_calls import ( - _base_usage_context_record, - _generate_cached_llm_text, - _generate_llm_context, - _generate_page_lecture_context, - _generate_weave_context, -) -from .repair import _repair_required_markdown_once -from .versions import NOTE_PROMPT_VERSION, PAGE_LECTURE_PROMPT_VERSION, WEAVE_PROMPT_VERSION - -__all__ = [ - "NOTE_PROMPT_VERSION", - "PAGE_LECTURE_PROMPT_VERSION", - "WEAVE_PROMPT_VERSION", - "_base_usage_context_record", - "_generate_cached_llm_text", - "_generate_llm_context", - "_generate_notes_with_lecture_weave", - "_generate_notes_with_llm", - "_generate_page_lecture_context", - "_generate_weave_context", - "_repair_required_markdown_once", -] diff --git a/slidenote/notes/postprocess.py b/slidenote/notes/postprocess.py new file mode 100644 index 0000000..4508d5f --- /dev/null +++ b/slidenote/notes/postprocess.py @@ -0,0 +1,128 @@ +"""Clean up raw LLM Markdown before composition.""" + +from __future__ import annotations + +import re + +from .sources import SOURCE_COMMENT_PREFIX, _source_marker + + +def _postprocess_llm_markdown(markdown: str, source_display: str) -> str: + text = _unwrap_code_images(markdown) + text = _fill_empty_image_alts(text) + text = _remove_meta_paragraphs(text) + text = _normalize_chunk_headings(text) + text = _convert_visible_sources(text, source_display) + return text.strip() + + +def _unwrap_code_images(markdown: str) -> str: + return re.sub(r"`(!\[[^\]]*]\([^)]+\))`", r"\1", markdown) + + +def _fill_empty_image_alts(markdown: str) -> str: + return re.sub(r"!\[\s*]\(", "![\u56fe\u793a](", markdown) + + +def _remove_meta_paragraphs(markdown: str) -> str: + paragraphs = re.split(r"\n\s*\n", markdown) + kept = [paragraph.strip() for paragraph in paragraphs if paragraph.strip() and not _is_meta_paragraph(paragraph)] + return "\n\n".join(kept) + + +def _is_meta_paragraph(paragraph: str) -> bool: + normalized = " ".join(line.strip() for line in paragraph.splitlines() if line.strip()) + banned_patterns = [ + "\u597d\u7684\uff0c\u8fd9\u662f", + "\u597d\u7684\uff0c\u6211\u5c06", + "\u4ee5\u4e0b\u662f\u6839\u636e", + "\u4e0b\u9762\u662f\u4f9d\u636e", + "\u6839\u636e\u60a8\u63d0\u4f9b\u7684 JSON", + "\u6839\u636e\u4f60\u63d0\u4f9b\u7684 JSON", + "\u8bfe\u7a0b\u6750\u6599 JSON", + "\u7b14\u8bb0\u5df2\u4e25\u683c\u9075\u5faa", + "\u4e25\u683c\u9075\u5faa\u5168\u90e8\u786c\u6027\u8981\u6c42", + "\u8986\u76d6\u4e86\u6240\u6709\u6587\u672c\u5757", + "\u8986\u76d6\u6bcf\u4e00\u4e2a\u6587\u672c\u5757", + "\u6bcf\u6bb5\u5747\u6807\u6ce8", + "\u6bcf\u4e00\u6bb5\u90fd\u6807\u6ce8", + "\u672a\u63d0\u4f9b\u56fe\u7247\u50cf\u7d20", + "\u672a\u63d0\u4f9b\u56fe\u50cf\u50cf\u7d20", + "\u672a\u63d0\u4f9b\u56fe\u7247\u7684 OCR", + "\u672a\u63d0\u4f9b\u8be5\u622a\u56fe\u7684 OCR", + "\u672a\u8fdb\u884c\u89c6\u89c9\u89e3\u6790", + "\u65e0\u6cd5\u8fdb\u884c\u5177\u4f53\u63cf\u8ff0", + "\u65e0\u6cd5\u8fdb\u4e00\u6b65\u8bf4\u660e", + "\u65e0\u6cd5\u5bf9\u622a\u56fe\u5185\u5bb9", + "\u5efa\u8bae\u5728\u539f\u59cb\u5e7b\u706f\u7247", + "\u82e5\u9700\u4e86\u89e3\u56fe\u7247\u5177\u4f53\u5185\u5bb9", + "\u56fe\u7247\u7559\u4f5c\u539f\u59cb\u8bc1\u636e", + "\u4ec5\u4f5c\u4e3a\u8bc1\u636e\u4fdd\u7559", + ] + if any(pattern in normalized for pattern in banned_patterns): + return True + structure_only_patterns = [ + "\u5e7b\u706f\u7247\u9996\u5148\u63d0\u51fa", + "\u8fd9\u4e00\u9875\u5728\u4e0a\u4e00\u9875\u7684\u57fa\u7840\u4e0a", + "\u4e0a\u4e00\u9875\u4ecb\u7ecd\u4e86", + "\u4e0b\u4e00\u9875\u5c06", + "\u672c\u9875\u4e3b\u8981\u8bb2\u89e3", + "\u672c\u9875\u4ecb\u7ecd\u4e86", + "\u8fd9\u9875\u5c55\u793a", + "\u6b64\u9875\u5185\u5bb9", + "\u6b64\u5e7b\u706f\u7247", + "\u8fd9\u5f20\u5e7b\u706f\u7247", + "\u8fd9\u7ec4\u5e7b\u706f\u7247", + "\u8be5\u5e7b\u706f\u7247", + "\u5f53\u524d\u5e7b\u706f\u7247", + ] + if SOURCE_COMMENT_PREFIX in normalized or len(normalized) > 80: + return False + return any(normalized.startswith(pattern) for pattern in structure_only_patterns) + + +def _normalize_chunk_headings(markdown: str) -> str: + lines: list[str] = [] + for line in markdown.splitlines(): + match = re.match(r"^(#{1,6})\s+(.*)$", line) + if not match: + lines.append(line) + continue + text = re.sub(r"^\u8bfe\u7a0b\u7b14\u8bb0[\uff1a:\s-]*", "", match.group(2).strip()) + if not text: + continue + level = max(2, len(match.group(1))) + lines.append("#" * level + " " + text) + return "\n".join(lines) + + +def _convert_visible_sources(markdown: str, source_display: str) -> str: + if source_display == "inline": + return _ensure_source_comments_for_inline(markdown) + + def replace(match: re.Match[str]) -> str: + citation = match.group(0) + element_ids = re.findall(r"\bs\d+_(?:t|tbl|img|fig)\d+\b", citation) + slide_match = re.search(r"\u7b2c\s*(\d+)\s*\u9875", citation) + if not slide_match: + return "" + slide_id = int(slide_match.group(1)) + if source_display == "footnote": + return _source_marker(slide_id, element_ids, "footnote") + return _source_marker(slide_id, element_ids, "hidden") + + return re.sub(r"\u3010[^\u3011]*?PPT[^\u3011]*?\u3011", replace, markdown) + + +def _ensure_source_comments_for_inline(markdown: str) -> str: + def replace(match: re.Match[str]) -> str: + citation = match.group(0) + if SOURCE_COMMENT_PREFIX in citation: + return citation + element_ids = re.findall(r"\bs\d+_(?:t|tbl|img|fig)\d+\b", citation) + slide_match = re.search(r"\u7b2c\s*(\d+)\s*\u9875", citation) + if not slide_match or not element_ids: + return citation + return f"{citation} {_source_marker(int(slide_match.group(1)), element_ids, 'hidden')}" + + return re.sub(r"\u3010[^\u3011]*?PPT[^\u3011]*?\u3011", replace, markdown) diff --git a/slidenote/notes/prompt_payload.py b/slidenote/notes/prompt_payload.py index f2d4d3d..a63f5db 100644 --- a/slidenote/notes/prompt_payload.py +++ b/slidenote/notes/prompt_payload.py @@ -13,7 +13,8 @@ from slidenote.semantic_layout import semantic_layout_for_prompt from slidenote.table_understanding import table_preview -from .assembly import _asset_display_path, _section_contexts, _should_render_screenshot +from .assets import _asset_display_path, _should_render_screenshot +from .contexts import _section_contexts def _page_payload_for_prompt( diff --git a/slidenote/notes/prompt_templates.py b/slidenote/notes/prompt_templates.py index 145f521..b3d3a74 100644 --- a/slidenote/notes/prompt_templates.py +++ b/slidenote/notes/prompt_templates.py @@ -5,9 +5,8 @@ from typing import Any from slidenote.content_guard import learning_items_for_page -from slidenote.models import Deck, SlidePage +from slidenote.models import Deck -from .assembly import NoteContext from .prompt_payload import ( _nearby_page_payloads, _page_payload_for_prompt, @@ -23,25 +22,6 @@ ) -def _llm_page_prompt(page: SlidePage, supports_image_input: bool = False) -> str: - context = NoteContext(id=f"p{page.slide_id}", kind="page", title=page.title or f"\u7b2c {page.slide_id} \u9875", pages=[page]) - return _llm_context_prompt( - context, - supports_image_input=supports_image_input, - asset_map={}, - source_display="hidden", - note_context="page", - note_style="article", - note_profile="auto", - note_depth="detailed", - note_language="zh", - term_policy="bilingual", - screenshot_policy="fallback", - figure_placement="inline", - source_type="pdf", - ) - - def _llm_context_prompt( context, supports_image_input: bool, diff --git a/slidenote/notes/prompts.py b/slidenote/notes/prompts.py deleted file mode 100644 index de59bef..0000000 --- a/slidenote/notes/prompts.py +++ /dev/null @@ -1,32 +0,0 @@ -from __future__ import annotations - -from .prompt_payload import ( - _nearby_page_payloads, - _page_brief, - _page_payload_for_prompt, - _prompt_brief_hash, - _prompt_deck_brief, - _prompt_slide_scope, - _section_title_by_slide, -) -from .prompt_rules import _language_prompt_rule, _note_depth_rule, _source_prompt_rule, _term_policy_prompt_rule -from .prompt_templates import _llm_context_prompt, _llm_page_lecture_prompt, _llm_page_prompt, _llm_repair_prompt, _llm_weave_prompt - -__all__ = [ - "_language_prompt_rule", - "_llm_context_prompt", - "_llm_page_lecture_prompt", - "_llm_page_prompt", - "_llm_repair_prompt", - "_llm_weave_prompt", - "_nearby_page_payloads", - "_note_depth_rule", - "_page_brief", - "_page_payload_for_prompt", - "_prompt_brief_hash", - "_prompt_deck_brief", - "_prompt_slide_scope", - "_section_title_by_slide", - "_source_prompt_rule", - "_term_policy_prompt_rule", -] diff --git a/slidenote/notes/quality.py b/slidenote/notes/quality.py index 81899a8..f54bc09 100644 --- a/slidenote/notes/quality.py +++ b/slidenote/notes/quality.py @@ -5,7 +5,6 @@ from slidenote.llm_cache import utc_now_iso from slidenote.models import Deck -from slidenote.study_pack import build_question_quality_report from slidenote.utils import round_score @@ -17,7 +16,6 @@ def build_note_quality_report( note_context: str, note_strategy: str, note_depth: str, - study_pack_report: dict[str, Any] | None = None, ) -> dict[str, Any]: paragraphs = _paragraphs(notes_markdown) heading_count = len(re.findall(r"(?m)^#{2,4}\s+", notes_markdown)) @@ -52,7 +50,6 @@ def build_note_quality_report( pitfall_score=pitfall_score, image_total=image_total, ) - question_quality = build_question_quality_report(study_pack_report) if study_pack_report else None return { "schema_version": 1, "generated_at": utc_now_iso(), @@ -71,8 +68,9 @@ def build_note_quality_report( "mechanical_page_listing_score": mechanical_page_listing_score, "self_test_score": self_test_score, "pitfall_score": pitfall_score, - "question_quality": question_quality, - "question_quality_score": question_quality.get("overall_score") if question_quality else None, + # Question quality belongs to `study-pack`; build keeps the documented null keys. + "question_quality": None, + "question_quality_score": None, "hallucination_risk": hallucination_risk, "suggested_repairs": suggested_repairs, "summary": { @@ -83,7 +81,7 @@ def build_note_quality_report( "source_images": image_total, "coverage_missing": coverage_report.get("missing") if coverage_report else None, "required_visible_missing": (coverage_report.get("required_visible_coverage") or {}).get("missing") if coverage_report else None, - "question_quality_score": question_quality.get("overall_score") if question_quality else None, + "question_quality_score": None, "suggested_repairs": len(suggested_repairs), }, } diff --git a/slidenote/notes/render_blocks.py b/slidenote/notes/render_blocks.py new file mode 100644 index 0000000..95f20d6 --- /dev/null +++ b/slidenote/notes/render_blocks.py @@ -0,0 +1,87 @@ +"""Render images and styled text blocks as Markdown.""" + +from __future__ import annotations + +import html +import re + +from slidenote.models import ImageAsset, SlidePage, TextBlock +from .assets import _asset_display_path +from .sources import _image_source_ids, _source_marker + + +_CSS_HEX_COLOR_RE = re.compile(r"^#[0-9A-Fa-f]{6}$") + + +def _render_image(page: SlidePage, image: ImageAsset, asset_map: dict[str, str], source_display: str) -> list[str]: + caption = image.caption or f"\u7b2c {page.slide_id} \u9875\u56fe\u7247" + lines = [ + f"{caption}\u3002", + _source_marker(page.slide_id, _image_source_ids(image), source_display), + "", + ] + explanation = image.figure_explanation or image.visual_summary + if explanation: + label = "\u56fe\u793a\u8bf4\u660e" if image.figure_explanation else "\u56fe\u7247\u89c6\u89c9\u89e3\u6790" + lines.append(f"{label}\uff1a{_ensure_sentence(explanation)}") + if _should_render_image_ocr(image, explanation): + if explanation: + lines.append("") + lines.append("\u56fe\u7247 OCR \u6587\u5b57\uff1a") + lines.extend(_quote_multiline(image.ocr_text)) + if explanation or _should_render_image_ocr(image, explanation): + lines.append("") + lines.append(f"![{caption}]({_asset_display_path(image.path, asset_map)})") + return lines + + +def _should_render_image_ocr(image: ImageAsset, explanation: str | None) -> bool: + if not image.ocr_text: + return False + if image.figure_explanation_status == "ocr_text": + return False + return not bool(explanation) + + +def _ensure_sentence(text: str) -> str: + value = " ".join(text.split()).strip() + if value and value[-1] not in "\u3002.!!\uff1f?\uff1a:": + value += "\u3002" + return value + + +def _quote_multiline(text: str) -> list[str]: + lines = [line.strip() for line in text.splitlines() if line.strip()] + return [f"> {line}" for line in lines] + + +def _styled_block_text(block: TextBlock) -> str | None: + if not block.style_runs: + return None + pieces: list[str] = [] + has_visible_style = False + for run in block.style_runs: + text = str(run.get("text") or "") + if not text: + continue + escaped = html.escape(text).replace("\n", "
") + css: list[str] = [] + color = _safe_css_color(run.get("color")) + if color: + css.append(f"color:{color}") + if run.get("bold") is True: + css.append("font-weight:700") + if run.get("italic") is True: + css.append("font-style:italic") + if css: + has_visible_style = True + pieces.append(f'{escaped}') + else: + pieces.append(escaped) + rendered = "".join(pieces).strip() + return rendered if has_visible_style and rendered else None + + +def _safe_css_color(value: object) -> str | None: + color = str(value or "").strip() + return color.upper() if _CSS_HEX_COLOR_RE.fullmatch(color) else None diff --git a/slidenote/notes/repair.py b/slidenote/notes/repair.py index c94bd27..5062c7a 100644 --- a/slidenote/notes/repair.py +++ b/slidenote/notes/repair.py @@ -9,14 +9,19 @@ from slidenote.llm_cache import LLMCache from slidenote.models import Deck -from .assembly import NoteContext, _postprocess_llm_markdown +from .contexts import NoteContext +from .postprocess import _postprocess_llm_markdown from .llm_calls import _generate_cached_llm_text from .prompt_templates import _llm_repair_prompt _IMAGE_LINK = re.compile(r"!\[[^\]]*]\(([^)]+)\)") # Repair adds missing explanations; substantial compression is unsafe here. -_MIN_BODY_RETENTION = 0.8 +MIN_BODY_RETENTION = 0.8 +# A repair must re-emit the whole input, so skip inputs whose estimated output +# would not fit the output-token budget (CJK text is roughly 0.5-1 token/char). +REPAIR_ESTIMATED_TOKENS_PER_CHAR = 0.7 +REPAIR_OUTPUT_BUDGET_HEADROOM = 0.85 _INCOMPLETE_FINISH_REASONS = { "length", "max_tokens", "content_filter", "safety", "recitation", "blocklist", "prohibited_content", "spii", "malformed_function_call", @@ -55,14 +60,8 @@ def _repair_required_markdown_once( options: "NoteOptions", *, stage: str, + force_refresh: bool = False, ) -> tuple[str, dict[str, Any] | None]: - cache_mode = options.cache_mode - provider = options.provider - model = options.model - api_key = options.api_key - base_url = options.base_url - max_output_tokens = options.max_output_tokens - temperature = options.temperature source_display = options.source_display note_language = options.note_language term_policy = options.term_policy @@ -85,6 +84,11 @@ def _repair_required_markdown_once( "unresolved_items": missing_before, "llm": None, } + if not _fits_output_budget(markdown, options.max_output_tokens): + # Sending an over-long draft would truncate the rewrite; keep the draft. + record["rejection_reasons"] = ["input_too_long_for_output_budget"] + record["input_chars"] = len(markdown) + return markdown, record prompt = _llm_repair_prompt( markdown=markdown, missing_items=missing_before, @@ -103,13 +107,13 @@ def _repair_required_markdown_once( ), output_root=output_root, cache=cache, - cache_mode=cache_mode, - provider=provider, - model=model, - api_key=api_key, - base_url=base_url, - max_output_tokens=max_output_tokens, - temperature=temperature, + cache_mode=options.cache_mode, + provider=options.provider, + model=options.model, + api_key=options.api_key, + base_url=options.base_url, + max_output_tokens=options.max_output_tokens, + temperature=options.temperature, user_prompt=prompt, prompt_version=CONTENT_REPAIR_PROMPT_VERSION, generation_stage=f"content_repair_{stage}", @@ -119,7 +123,7 @@ def _repair_required_markdown_once( "term_policy": term_policy, "missing_item_ids": [str(item.get("element_id")) for item in missing_before], }, - force_refresh=False, + force_refresh=force_refresh, ) except Exception as exc: # This optional repair must not discard an already generated draft. @@ -145,7 +149,7 @@ def _repair_required_markdown_once( reasons.append("coverage_regression") if lost_images: reasons.append("missing_images") - if candidate_chars < original_chars * _MIN_BODY_RETENTION: + if candidate_chars < original_chars * MIN_BODY_RETENTION: reasons.append("body_truncated") if not before_ids - unresolved_ids: reasons.append("no_coverage_improvement") @@ -172,3 +176,10 @@ def _repair_required_markdown_once( record["resolved_items"] = sorted(before_ids - unresolved_ids) record["unresolved_items"] = unresolved return repaired, record + + +def _fits_output_budget(markdown: str, max_output_tokens: int | None) -> bool: + if not max_output_tokens: + return True + estimated_tokens = len(markdown) * REPAIR_ESTIMATED_TOKENS_PER_CHAR + return estimated_tokens <= max_output_tokens * REPAIR_OUTPUT_BUDGET_HEADROOM diff --git a/slidenote/notes/reports.py b/slidenote/notes/reports.py new file mode 100644 index 0000000..b119478 --- /dev/null +++ b/slidenote/notes/reports.py @@ -0,0 +1,241 @@ +"""Build page-note, weave and teaching-enrichment reports.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from slidenote.models import Deck +from slidenote.utils import source_tokens, sum_int +from .versions import ( + PAGE_LECTURE_PROMPT_VERSION, + TEACHING_ENRICHMENT_PROMPT_VERSION, + WEAVE_PROMPT_VERSION, +) +from .contexts import NoteContext + + +def _build_page_notes_report( + deck: Deck, + provider: str, + model: str, + base_url: str | None, + note_depth: str, + note_language: str, + term_policy: str, + page_neighborhood: int, + pages: list[NoteContext], + page_markdown_by_slide: dict[int, str], + page_records: list[dict[str, Any]], + deck_brief: dict[str, Any] | None = None, +) -> dict[str, Any]: + from slidenote.llm_cache import utc_now_iso + from .prompt_payload import _prompt_brief_hash, _prompt_deck_brief + prompt_brief = _prompt_deck_brief(deck_brief) + record_by_slide = {record.get("slide_id"): record for record in page_records} + page_entries: list[dict[str, Any]] = [] + for context in pages: + page = context.pages[0] + record = record_by_slide.get(page.slide_id, {}) + markdown = page_markdown_by_slide.get(page.slide_id, "") + page_entries.append( + { + "slide_id": page.slide_id, + "title": page.title, + "markdown": markdown, + "source_ids": sorted(source_tokens(markdown)), + "cache_status": record.get("cache_status"), + "llm_call": record.get("llm_call"), + "cache_file": record.get("cache_file"), + "input_tokens": record.get("input_tokens"), + "output_tokens": record.get("output_tokens"), + "total_tokens": record.get("total_tokens"), + } + ) + return { + "schema_version": 1, + "generated_at": utc_now_iso(), + "source_path": deck.source_path, + "source_type": deck.source_type, + "provider": provider, + "model": model, + "base_url": base_url, + "prompt_version": PAGE_LECTURE_PROMPT_VERSION, + "request": { + "note_depth": note_depth, + "note_language": note_language, + "term_policy": term_policy, + "page_neighborhood": page_neighborhood, + "deck_brief_used": bool(prompt_brief), + "deck_brief_hash": _prompt_brief_hash(prompt_brief), + }, + "summary": { + "pages_total": len(page_entries), + "llm_calls": sum(1 for record in page_records if record.get("llm_call")), + "local_cache_hits": sum(1 for record in page_records if record.get("cache_status") == "local_hit"), + "input_tokens": sum_int(record.get("input_tokens") for record in page_records), + "output_tokens": sum_int(record.get("output_tokens") for record in page_records), + "total_tokens": sum_int(record.get("total_tokens") for record in page_records), + }, + "pages": page_entries, + } + + +def _render_page_notes_markdown(deck: Deck, page_notes: dict[str, Any]) -> str: + lines = [f"# {Path(deck.source_path).stem} Page Notes", ""] + for page in page_notes.get("pages", []): + title = page.get("title") or f"\u7b2c {page.get('slide_id')} \u9875" + lines.append(f"## \u7b2c {page.get('slide_id')} \u9875\uff1a{title}") + lines.append("") + markdown = str(page.get("markdown") or "").strip() + if markdown: + lines.append(markdown) + lines.append("") + return "\n".join(lines).rstrip() + "\n" + + +def _build_weave_report( + deck: Deck, + note_context: str, + note_depth: str, + note_language: str, + term_policy: str, + weave_dedup: str, + contexts: list[NoteContext], + final_chunks: dict[str, str], + page_markdown_by_slide: dict[int, str], + weave_records: list[dict[str, Any]], + deck_brief: dict[str, Any] | None = None, +) -> dict[str, Any]: + from slidenote.llm_cache import utc_now_iso + from .prompt_payload import _prompt_brief_hash, _prompt_deck_brief + prompt_brief = _prompt_deck_brief(deck_brief) + record_by_context = {record.get("context_id"): record for record in weave_records} + context_entries: list[dict[str, Any]] = [] + for context in contexts: + markdown = final_chunks.get(context.id, "") + final_tokens = source_tokens(markdown) + input_tokens: set[str] = set() + pages: list[dict[str, Any]] = [] + for page in context.pages: + page_tokens = source_tokens(page_markdown_by_slide.get(page.slide_id, "")) + input_tokens.update(page_tokens) + pages.append( + { + "slide_id": page.slide_id, + "title": page.title, + "page_note_source_ids": sorted(page_tokens), + "retained_source_ids": sorted(page_tokens.intersection(final_tokens)), + "possibly_compressed_source_ids": sorted(page_tokens - final_tokens), + } + ) + record = record_by_context.get(f"weave_{context.id}", {}) + context_entries.append( + { + "context_id": context.id, + "context_title": context.title, + "slide_ids": [page.slide_id for page in context.pages], + "input_source_ids": sorted(input_tokens), + "final_source_ids": sorted(final_tokens), + "possibly_compressed_source_ids": sorted(input_tokens - final_tokens), + "cache_status": record.get("cache_status"), + "llm_call": record.get("llm_call"), + "cache_file": record.get("cache_file"), + "pages": pages, + } + ) + return { + "schema_version": 1, + "generated_at": utc_now_iso(), + "source_path": deck.source_path, + "source_type": deck.source_type, + "prompt_version": WEAVE_PROMPT_VERSION, + "request": { + "note_context": note_context, + "note_depth": note_depth, + "note_language": note_language, + "term_policy": term_policy, + "weave_dedup": weave_dedup, + "deck_brief_used": bool(prompt_brief), + "deck_brief_hash": _prompt_brief_hash(prompt_brief), + }, + "summary": { + "contexts_total": len(context_entries), + "llm_calls": sum(1 for record in weave_records if record.get("llm_call")), + "local_cache_hits": sum(1 for record in weave_records if record.get("cache_status") == "local_hit"), + "input_tokens": sum_int(record.get("input_tokens") for record in weave_records), + "output_tokens": sum_int(record.get("output_tokens") for record in weave_records), + "total_tokens": sum_int(record.get("total_tokens") for record in weave_records), + }, + "contexts": context_entries, + } + + +def _build_teaching_enrichment_report( + deck: Deck, + note_context: str, + note_profile: str, + note_depth: str, + note_language: str, + term_policy: str, + contexts: list[NoteContext], + final_chunks: dict[str, str], + page_markdown_by_slide: dict[int, str], + teaching_records: list[dict[str, Any]], + deck_brief: dict[str, Any] | None = None, +) -> dict[str, Any]: + from slidenote.llm_cache import utc_now_iso + from .prompt_payload import _prompt_brief_hash, _prompt_deck_brief + + prompt_brief = _prompt_deck_brief(deck_brief) + record_by_context = {record.get("context_id"): record for record in teaching_records} + context_entries: list[dict[str, Any]] = [] + for context in contexts: + markdown = final_chunks.get(context.id, "") + final_tokens = source_tokens(markdown) + input_tokens: set[str] = set() + for page in context.pages: + input_tokens.update(source_tokens(page_markdown_by_slide.get(page.slide_id, ""))) + record = record_by_context.get(f"teaching_{context.id}", {}) + context_entries.append( + { + "context_id": context.id, + "context_title": context.title, + "slide_ids": [page.slide_id for page in context.pages], + "input_source_ids": sorted(input_tokens), + "final_source_ids": sorted(final_tokens), + "possibly_added_source_ids": sorted(final_tokens - input_tokens), + "possibly_dropped_source_ids": sorted(input_tokens - final_tokens), + "cache_status": record.get("cache_status"), + "llm_call": record.get("llm_call"), + "cache_file": record.get("cache_file"), + "input_tokens": record.get("input_tokens"), + "output_tokens": record.get("output_tokens"), + "total_tokens": record.get("total_tokens"), + } + ) + return { + "schema_version": 1, + "generated_at": utc_now_iso(), + "source_path": deck.source_path, + "source_type": deck.source_type, + "prompt_version": TEACHING_ENRICHMENT_PROMPT_VERSION, + "request": { + "note_context": note_context, + "note_profile": note_profile, + "note_depth": note_depth, + "note_language": note_language, + "term_policy": term_policy, + "deck_brief_used": bool(prompt_brief), + "deck_brief_hash": _prompt_brief_hash(prompt_brief), + }, + "summary": { + "contexts_total": len(context_entries), + "llm_calls": sum(1 for record in teaching_records if record.get("llm_call")), + "local_cache_hits": sum(1 for record in teaching_records if record.get("cache_status") == "local_hit"), + "input_tokens": sum_int(record.get("input_tokens") for record in teaching_records), + "output_tokens": sum_int(record.get("output_tokens") for record in teaching_records), + "total_tokens": sum_int(record.get("total_tokens") for record in teaching_records), + }, + "contexts": context_entries, + } diff --git a/slidenote/notes/sources.py b/slidenote/notes/sources.py new file mode 100644 index 0000000..5803cf4 --- /dev/null +++ b/slidenote/notes/sources.py @@ -0,0 +1,59 @@ +"""Source markers and shared markdown helpers used across note assembly.""" + +from __future__ import annotations + +import re + +from slidenote.models import ImageAsset, SlidePage + + +SOURCE_COMMENT_PREFIX = "slidenote-source:" + + +def _collapse_blank_lines(lines: list[str]) -> str: + collapsed: list[str] = [] + blank = False + for line in lines: + is_blank = not line.strip() + if is_blank and blank: + continue + collapsed.append(line) + blank = is_blank + return "\n".join(collapsed) + + +def _source_marker(slide_id: int, element_ids: list[str], source_display: str) -> str: + ids = [element_id for element_id in element_ids if element_id] + comment = f"" if ids else "" + if source_display == "hidden": + return comment + if source_display == "footnote": + return f"\uff08PPT \u7b2c {slide_id} \u9875\uff09 {comment}".rstrip() + detail = "\u3001".join(ids) + return f"\u3010\u5bf9\u5e94 PPT\uff1a\u7b2c {slide_id} \u9875\uff0c\u5143\u7d20 {detail}\u3011 {comment}".rstrip() + + +def _image_source_ids(image: ImageAsset) -> list[str]: + ids: list[str] = [] + seen: set[str] = set() + for element_id in [image.id, *image.source_element_ids]: + if element_id and element_id not in seen: + ids.append(element_id) + seen.add(element_id) + return ids + + +def _page_element_ids(page: SlidePage) -> list[str]: + ids = [block.id for block in page.text_blocks] + ids.extend(table.id for table in page.tables) + return ids + + +def _page_source_ids(page: SlidePage) -> list[str]: + ids = _page_element_ids(page) + ids.extend(image.id for image in page.images if not image.ignored) + return ids + + +def _source_slide_ids(markdown: str) -> set[int]: + return {int(match) for match in re.findall(r"\bp(\d+):", markdown)} diff --git a/slidenote/notes/usage.py b/slidenote/notes/usage.py index 4865c78..578b092 100644 --- a/slidenote/notes/usage.py +++ b/slidenote/notes/usage.py @@ -8,7 +8,6 @@ from slidenote.utils import display_path, sum_int from .prompt_payload import _prompt_brief_hash, _prompt_deck_brief -from .versions import NOTE_PROMPT_VERSION def _build_usage_report( @@ -18,34 +17,14 @@ def _build_usage_report( options: "NoteOptions", contexts: list[dict[str, Any]], note_strategy: str, + prompt_version: str, page_contexts: list[dict[str, Any]] | None = None, weave_contexts: list[dict[str, Any]] | None = None, teaching_enrichment_contexts: list[dict[str, Any]] | None = None, repair_contexts: list[dict[str, Any]] | None = None, + warnings: list[str] | None = None, ) -> dict[str, Any]: - provider = options.provider - model = options.model - base_url = options.base_url - cache_mode = options.cache_mode - cache_dir = options.cache_dir - max_output_tokens = options.max_output_tokens - temperature = options.temperature - note_context = options.note_context - source_display = options.source_display - note_style = options.note_style - note_profile = options.note_profile - note_depth = options.note_depth - note_language = options.note_language - term_policy = options.term_policy - teaching_enrichment = options.teaching_enrichment - weave_dedup = options.weave_dedup - page_neighborhood = options.page_neighborhood - asset_mode = options.asset_mode - screenshot_policy = options.screenshot_policy - figure_placement = options.figure_placement - deck_brief = options.deck_brief - content_guard = options.content_guard - prompt_brief = _prompt_deck_brief(deck_brief) + prompt_brief = _prompt_deck_brief(options.deck_brief) summary = { "pages_total": len(deck.pages), "contexts_total": len(contexts), @@ -60,6 +39,7 @@ def _build_usage_report( "local_cache_misses": sum(1 for context in contexts if context.get("cache_status") == "miss"), "local_cache_refreshes": sum(1 for context in contexts if context.get("cache_status") == "refresh"), "cache_disabled_calls": sum(1 for context in contexts if context.get("cache_status") == "disabled"), + "failed_contexts": sum(1 for context in contexts if context.get("cache_status") == "failed"), "llm_calls": sum(1 for context in contexts if context.get("llm_call")), "api_retries": sum(int(context.get("api_retries") or 0) for context in contexts), "input_tokens": sum_int(context.get("input_tokens") for context in contexts), @@ -72,41 +52,40 @@ def _build_usage_report( "generated_at": utc_now_iso(), "source_path": deck.source_path, "source_type": deck.source_type, - "provider": provider, - "model": model, - "base_url": base_url, - "prompt_version": NOTE_PROMPT_VERSION, + "provider": options.provider, + "model": options.model, + "base_url": options.base_url, + "prompt_version": prompt_version, "cache": { - "mode": cache_mode, - "dir": display_path(cache_dir, output_root), + "mode": options.cache_mode, + "dir": display_path(options.cache_dir, output_root), }, "request": { - "temperature": temperature, - "max_output_tokens": max_output_tokens, - "note_context": note_context, + "temperature": options.temperature, + "max_output_tokens": options.max_output_tokens, + "note_context": options.note_context, "note_strategy": note_strategy, - "note_depth": note_depth, - "note_profile": note_profile, - "note_language": note_language, - "term_policy": term_policy, - "teaching_enrichment": teaching_enrichment, - "weave_dedup": weave_dedup, - "page_neighborhood": page_neighborhood, - "source_display": source_display, - "note_style": note_style, - "asset_mode": asset_mode, - "screenshot_policy": screenshot_policy, - "figure_placement": figure_placement, + "note_depth": options.note_depth, + "note_profile": options.note_profile, + "note_language": options.note_language, + "term_policy": options.term_policy, + "teaching_enrichment": options.teaching_enrichment, + "weave_dedup": options.weave_dedup, + "page_neighborhood": options.page_neighborhood, + "source_display": options.source_display, + "note_style": options.note_style, + "asset_mode": options.asset_mode, + "screenshot_policy": options.screenshot_policy, + "figure_placement": options.figure_placement, "deck_brief_used": bool(prompt_brief), "deck_brief_hash": _prompt_brief_hash(prompt_brief), - "content_guard_used": bool(content_guard), + "content_guard_used": bool(options.content_guard), }, "summary": summary, - "pages": contexts, + "warnings": list(warnings or []), "contexts": contexts, "page_contexts": page_contexts or [], "weave_contexts": weave_contexts or [], "teaching_enrichment_contexts": teaching_enrichment_contexts or [], "repair_contexts": repair_contexts or [], } - diff --git a/slidenote/ocr.py b/slidenote/ocr.py index f30c0a7..5cf243d 100644 --- a/slidenote/ocr.py +++ b/slidenote/ocr.py @@ -2,27 +2,28 @@ import base64 import json -import tempfile +import threading import urllib.error import urllib.parse import urllib.request -from concurrent.futures import ThreadPoolExecutor, as_completed from dataclasses import asdict, dataclass from pathlib import Path from typing import Any, Callable -from PIL import Image - from slidenote.api_retry import with_api_retries from slidenote.llm_cache import LLMCache, make_cache_key, utc_now_iso -from slidenote.modality import page_has_hint +from slidenote.modality import page_has_hint, page_has_manual_modality from slidenote.models import Deck, SlidePage from slidenote.utils import ( + advance_progress, + error_summary, + run_target_jobs, cleanup_temp_image, display_path, file_sha256, first_env, image_area, + prepare_image_for_api, preview, ) @@ -172,6 +173,15 @@ def enrich_deck_with_ocr( if progress_callback: progress_callback({"event": "start", "total": len(targets)}) + client_lock = threading.Lock() + shared_client: list[OCRClient] = [] + + def client_for_run() -> OCRClient: + with client_lock: + if not shared_client: + shared_client.append(OCRClient(provider=provider_name, api_key=api_key, secret_key=secret_key, endpoint=endpoint, language=language)) + return shared_client[0] + def process(index: int, target: OCRTarget) -> tuple[int, OCRTarget, dict[str, Any], str | None, str]: return ( index, @@ -182,8 +192,7 @@ def process(index: int, target: OCRTarget) -> tuple[int, OCRTarget, dict[str, An cache=cache, cache_mode=cache_mode, provider_name=provider_name, - api_key=api_key, - secret_key=secret_key, + client_for_run=client_for_run, endpoint=endpoint, language=language, max_edge=max_edge, @@ -191,25 +200,14 @@ def process(index: int, target: OCRTarget) -> tuple[int, OCRTarget, dict[str, An ), ) - results = [] - if workers == 1: - for index, target in enumerate(targets): - result = process(index, target) - results.append(result) - if progress_callback: - _, completed_target, record, _, _ = result - progress_callback({"event": "advance", "record": record, "slide_id": completed_target.slide_id}) - else: - with ThreadPoolExecutor(max_workers=workers) as executor: - futures = {executor.submit(process, index, target): (index, target) for index, target in enumerate(targets)} - for future in as_completed(futures): - result = future.result() - results.append(result) - if progress_callback: - _, completed_target, record, _, _ = result - progress_callback({"event": "advance", "record": record, "slide_id": completed_target.slide_id}) - - for index, target, record, text, status in sorted(results, key=lambda item: item[0]): + def failed(index: int, target: OCRTarget, exc: Exception) -> tuple[int, OCRTarget, dict[str, Any], str | None, str]: + record = _skipped_record(target, "api_error") + record.update({"cache_status": "error", "error": error_summary(exc)}) + return index, target, record, None, "failed" + + results = run_target_jobs(targets, process, workers=workers, on_error=failed, on_result=advance_progress(progress_callback)) + + for index, target, record, text, status in results: _apply_ocr_result(deck, target, ocr_text=text, ocr_status=status) records_by_index[index] = record @@ -223,8 +221,7 @@ def _process_ocr_target( cache: LLMCache, cache_mode: str, provider_name: str, - api_key: str | None, - secret_key: str | None, + client_for_run: Callable[[], "OCRClient"], endpoint: str | None, language: str, max_edge: int, @@ -233,7 +230,7 @@ def _process_ocr_target( source_path = (output_root / target.path).resolve() if not source_path.exists(): return _skipped_record(target, "missing_file"), None, "missing_file" - prepared = _prepare_image_for_ocr(source_path, max_edge=max_edge) + prepared = prepare_image_for_ocr(source_path, max_edge=max_edge) if prepared is None: return _skipped_record(target, "unsupported_or_unreadable_image"), None, "unsupported_or_unreadable_image" @@ -267,7 +264,7 @@ def _process_ocr_target( } ) else: - client = OCRClient(provider=provider_name, api_key=api_key, secret_key=secret_key, endpoint=endpoint, language=language) + client = client_for_run() retry_result = with_api_retries(lambda: client.recognize(prepared_path)) result = retry_result.value text = result.text @@ -319,7 +316,9 @@ def select_ocr_targets( targets: list[OCRTarget] = [] for page in deck.pages: text_len = sum(len(block.content.strip()) for block in page.text_blocks) - needs_page_ocr = page_has_hint(page, "ocr_page_screenshot") or text_len < min_text_chars or bool(page.warnings) + needs_page_ocr = page_has_hint(page, "ocr_page_screenshot") + if not page_has_manual_modality(page): + needs_page_ocr = needs_page_ocr or text_len < min_text_chars or bool(page.warnings) if mode == "all" or needs_page_ocr: if page.page_screenshot: reason = "all_page_screenshot" if mode == "all" else page.page_modality or "low_extracted_text" @@ -360,28 +359,26 @@ def _apply_ocr_result(deck: Deck, target: OCRTarget, ocr_text: str | None = None image.ocr_status = ocr_status or image.ocr_status -def _prepare_image_for_ocr(path: Path, max_edge: int) -> tuple[Path, dict[str, Any]] | None: - try: - with Image.open(path) as image: - original = {"width": image.width, "height": image.height, "mode": image.mode, "format": image.format} - image = image.convert("RGB") - scale = min(1.0, max_edge / max(image.width, image.height)) - if scale < 1.0: - image = image.resize((max(1, int(image.width * scale)), max(1, int(image.height * scale)))) - tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".jpg") - tmp_path = Path(tmp.name) - tmp.close() - image.save(tmp_path, format="JPEG", quality=90, optimize=True) - meta = { - "original": original, - "prepared": {"width": image.width, "height": image.height, "mime_type": "image/jpeg", "bytes": tmp_path.stat().st_size}, - } - return tmp_path, meta - except Exception: - return None +def prepare_image_for_ocr(path: Path, max_edge: int) -> tuple[Path, dict[str, Any]] | None: + return prepare_image_for_api(path, max_edge=max_edge, quality=90) + + +_BAIDU_TOKENS: dict[tuple[str, str], str] = {} +_BAIDU_TOKEN_LOCK = threading.Lock() def _baidu_access_token(api_key: str, secret_key: str) -> str: + # Tokens are valid for 30 days; one per process avoids an extra request per image. + with _BAIDU_TOKEN_LOCK: + cached = _BAIDU_TOKENS.get((api_key, secret_key)) + if cached: + return cached + token = _fetch_baidu_access_token(api_key, secret_key) + _BAIDU_TOKENS[(api_key, secret_key)] = token + return token + + +def _fetch_baidu_access_token(api_key: str, secret_key: str) -> str: query = urllib.parse.urlencode({"grant_type": "client_credentials", "client_id": api_key, "client_secret": secret_key}) data = _get_json(f"https://aip.baidubce.com/oauth/2.0/token?{query}") token = data.get("access_token") @@ -435,6 +432,7 @@ def _build_report( "api_calls": sum(1 for record in records if record.get("api_call")), "api_retries": sum(int(record.get("api_retries") or 0) for record in records), "skipped": sum(1 for record in records if record.get("cache_status") == "skipped"), + "failed": sum(1 for record in records if record.get("cache_status") == "error"), "text_chars": sum(record.get("text_chars", 0) for record in records if isinstance(record.get("text_chars"), int)), } return { diff --git a/slidenote/parser_adapters.py b/slidenote/parser_adapters.py index 5446eff..18ed45f 100644 --- a/slidenote/parser_adapters.py +++ b/slidenote/parser_adapters.py @@ -3,6 +3,7 @@ import json import os import re +import shlex import subprocess from dataclasses import dataclass, fields from pathlib import Path @@ -234,13 +235,30 @@ def _run_command(command: list[str]) -> subprocess.CompletedProcess[str]: def _run_shell_template(command_template: str, input_path: Path, out_dir: Path) -> subprocess.CompletedProcess[str]: - command = command_template.format( - input=str(input_path), - out=str(out_dir), - output=str(out_dir), - stem=input_path.stem, - ) - return subprocess.run(command, check=False, text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True) + return _run_command(_command_from_template(command_template, input_path, out_dir)) + + +def _command_from_template(command_template: str, input_path: Path, out_dir: Path) -> list[str]: + """Split a user command template into argv without invoking a shell. + + Placeholders are substituted per argument, so paths with spaces stay one + argument and other braces in the template (e.g. JSON options) are left alone. + """ + placeholders = { + "{input}": str(input_path), + "{out}": str(out_dir), + "{output}": str(out_dir), + "{stem}": input_path.stem, + } + posix = os.name != "nt" + command: list[str] = [] + for token in shlex.split(command_template, posix=posix): + if not posix and len(token) >= 2 and token[0] == token[-1] and token[0] in "'" + '"': + token = token[1:-1] + for placeholder, value in placeholders.items(): + token = token.replace(placeholder, value) + command.append(token) + return command def _external_parser_error( @@ -440,13 +458,11 @@ def _find_pages(data: dict[str, Any] | list[Any]) -> list[Any]: def _generic_text_blocks(raw_page: dict[str, Any], slide_id: int) -> list[TextBlock]: explicit_blocks = raw_page.get("text_blocks") or raw_page.get("blocks") or raw_page.get("children") or [] blocks: list[TextBlock] = [] - for item in _iter_dicts(explicit_blocks): + for item in _iter_dicts(explicit_blocks, stop=_is_text_leaf): text = _text_from_node(item) - if not text: - continue - kind = str(item.get("type") or item.get("label") or item.get("block_type") or "paragraph").lower() - if "table" in kind or _table_rows(item): + if not text or _is_table_node(item) or _is_image_node(item): continue + kind = _node_kind(item) or "paragraph" blocks.append(TextBlock(id=str(item.get("id") or f"s{slide_id}_t{len(blocks) + 1}"), type=_text_type(kind, text), content=text, bbox=_bbox_or_none(item.get("bbox")))) if blocks: return blocks @@ -458,11 +474,10 @@ def _generic_text_blocks(raw_page: dict[str, Any], slide_id: int) -> list[TextBl def _generic_tables(raw_page: dict[str, Any], slide_id: int) -> list[TableBlock]: tables: list[TableBlock] = [] - for item in _iter_dicts(raw_page.get("tables") or raw_page.get("blocks") or raw_page.get("children") or []): - kind = str(item.get("type") or item.get("label") or item.get("block_type") or "").lower() - rows = _table_rows(item) - if not rows and "table" not in kind: + for item in _iter_dicts(raw_page.get("tables") or raw_page.get("blocks") or raw_page.get("children") or [], stop=_is_table_node): + if not _is_table_node(item): continue + rows = _table_rows(item) tables.append( TableBlock( id=str(item.get("id") or f"s{slide_id}_tbl{len(tables) + 1}"), @@ -475,11 +490,10 @@ def _generic_tables(raw_page: dict[str, Any], slide_id: int) -> list[TableBlock] def _generic_images(raw_page: dict[str, Any], slide_id: int, output_root: Path, asset_root: Path) -> list[ImageAsset]: images: list[ImageAsset] = [] - for item in _iter_dicts(raw_page.get("images") or raw_page.get("pictures") or raw_page.get("blocks") or raw_page.get("children") or []): - path = str_or_none(item.get("path") or item.get("image_path") or item.get("uri") or item.get("src")) - kind = str(item.get("type") or item.get("label") or item.get("block_type") or "").lower() - if not path and "image" not in kind and "picture" not in kind: + for item in _iter_dicts(raw_page.get("images") or raw_page.get("pictures") or raw_page.get("blocks") or raw_page.get("children") or [], stop=_is_image_node): + if not _is_image_node(item): continue + path = str_or_none(item.get("path") or item.get("image_path") or item.get("uri") or item.get("src")) images.append( ImageAsset( id=str(item.get("id") or item.get("image_id") or f"s{slide_id}_img{len(images) + 1}"), @@ -523,18 +537,40 @@ def _markdown_title(markdown: str) -> str | None: return preview(first, 140) if first else None -def _iter_dicts(value: object) -> list[dict[str, Any]]: +def _iter_dicts(value: object, stop: Callable[[dict[str, Any]], bool] | None = None) -> list[dict[str, Any]]: + """Flatten nested nodes; children of nodes matching ``stop`` are not visited.""" result: list[dict[str, Any]] = [] if isinstance(value, dict): result.append(value) - for child in value.values(): - result.extend(_iter_dicts(child)) + if stop is None or not stop(value): + for child in value.values(): + result.extend(_iter_dicts(child, stop)) elif isinstance(value, list): for item in value: - result.extend(_iter_dicts(item)) + result.extend(_iter_dicts(item, stop)) return result +def _node_kind(node: dict[str, Any]) -> str: + return str(node.get("type") or node.get("label") or node.get("block_type") or "").lower() + + +def _is_table_node(node: dict[str, Any]) -> bool: + return "table" in _node_kind(node) or bool(_table_rows(node)) + + +def _is_image_node(node: dict[str, Any]) -> bool: + kind = _node_kind(node) + if "image" in kind or "picture" in kind or "figure" in kind: + return True + return bool(node.get("path") or node.get("image_path") or node.get("uri") or node.get("src")) + + +def _is_text_leaf(node: dict[str, Any]) -> bool: + # A node with its own text already contains its children's text. + return bool(_text_from_node(node)) or _is_table_node(node) or _is_image_node(node) + + def _text_from_node(node: dict[str, Any]) -> str: for key in ("content", "text", "markdown", "html"): value = node.get(key) diff --git a/slidenote/pipeline.py b/slidenote/pipeline.py index 6457403..e2b8810 100644 --- a/slidenote/pipeline.py +++ b/slidenote/pipeline.py @@ -1,56 +1,43 @@ from __future__ import annotations -from dataclasses import dataclass, field +from dataclasses import dataclass from pathlib import Path -from typing import Any, Callable, Protocol +from typing import Any, Callable -from slidenote.models import Deck from slidenote.utils import display_path, write_json, write_text -@dataclass(slots=True) -class StageResult: +@dataclass(frozen=True, slots=True) +class BuildStep: name: str - status: str = "ok" - report: dict[str, Any] | None = None - artifacts: dict[str, str] = field(default_factory=dict) - warnings: list[str] = field(default_factory=list) + runner: Callable[[Any], None] + enabled: Callable[[Any], bool] | None = None + tracks_progress: bool = True -@dataclass(slots=True) -class BuildContext: - args: Any - input_path: Path - output_root: Path - progress: Any - cache_dirs: dict[str, Path | None] = field(default_factory=dict) - refresh_slide_ids: set[int] = field(default_factory=set) - concurrency: int = 1 - artifacts: "ArtifactRegistry" | None = None - reports: dict[str, StageResult] = field(default_factory=dict) - - -class Stage(Protocol): - name: str - dependencies: list[str] - artifacts: list[str] - - def run(self, deck: Deck, context: BuildContext) -> StageResult: - ... - - -@dataclass(slots=True) -class FunctionStage: +@dataclass(frozen=True, slots=True) +class BuildPhase: name: str - runner: Callable[[Deck, BuildContext], StageResult | dict[str, Any] | None] - dependencies: list[str] = field(default_factory=list) - artifacts: list[str] = field(default_factory=list) - - def run(self, deck: Deck, context: BuildContext) -> StageResult: - result = self.runner(deck, context) - if isinstance(result, StageResult): - return result - return StageResult(name=self.name, report=result) + steps: tuple[BuildStep, ...] + + +def run_build_plan(state: Any, phases: tuple[BuildPhase, ...]) -> None: + """Run one explicit plan; disabled steps never enter progress accounting.""" + planned = [ + (phase.name, step) + for phase in phases + for step in phase.steps + if step.enabled is None or step.enabled(state) + ] + state.progress.set_plan([step.name for _, step in planned if step.tracks_progress]) + for phase in phases: + phase_steps = [step for phase_name, step in planned if phase_name == phase.name] + if not phase_steps: + continue + state.progress.set_phase(phase.name) + for step in phase_steps: + step.runner(state) + state.progress.set_phase(None) class ArtifactRegistry: @@ -79,17 +66,5 @@ def register(self, name: str, path: str | Path | None) -> None: def get(self, name: str) -> str | None: return self._artifacts.get(name) - def relative_path(self, name: str) -> str | None: - return self.get(name) - def as_summary(self) -> dict[str, str]: return dict(sorted(self._artifacts.items())) - - -def run_stage(deck: Deck, context: BuildContext, stage: Stage) -> StageResult: - for dependency in stage.dependencies: - if dependency not in context.reports: - raise RuntimeError(f"Stage `{stage.name}` depends on missing stage `{dependency}`.") - result = stage.run(deck, context) - context.reports[stage.name] = result - return result diff --git a/slidenote/progress.py b/slidenote/progress.py index dfd69fa..e6a82b3 100644 --- a/slidenote/progress.py +++ b/slidenote/progress.py @@ -1,12 +1,12 @@ from __future__ import annotations +import json import time from dataclasses import dataclass from pathlib import Path from typing import Any -from slidenote.llm_cache import utc_now_iso -from slidenote.utils import write_json +from slidenote.llm_cache import atomic_write_text, utc_now_iso @dataclass(slots=True) @@ -31,10 +31,20 @@ def __init__(self, path: Path, quiet: bool = False) -> None: self._stage_started = self._run_started self.current_stage: StageRecord | None = None self.stages: list[StageRecord] = [] + self.current_phase: str | None = None + self.planned_stages: list[str] = [] self.status = "running" self.message = "" self.write() + def set_plan(self, stage_names: list[str]) -> None: + self.planned_stages = list(stage_names) + self.write() + + def set_phase(self, phase: str | None) -> None: + self.current_phase = phase + self.write() + def start_stage(self, name: str, total: int | None = None, message: str | None = None) -> None: self._stage_started = time.perf_counter() self.current_stage = StageRecord(name=name, started_at=utc_now_iso(), total=total) @@ -91,7 +101,7 @@ def fail(self, message: str) -> None: self.write() def write(self) -> None: - write_json(self.path, self.snapshot()) + atomic_write_text(self.path, json.dumps(self.snapshot(), ensure_ascii=False, indent=2)) def snapshot(self) -> dict[str, Any]: current = _stage_to_dict(self.current_stage) if self.current_stage else None @@ -104,6 +114,8 @@ def snapshot(self) -> dict[str, Any]: "updated_at": utc_now_iso(), "elapsed_seconds": round(elapsed, 3), "current_stage": current, + "current_phase": self.current_phase, + "planned_stages": list(self.planned_stages), "stages": [_stage_to_dict(stage) for stage in self.stages], } diff --git a/slidenote/rendering.py b/slidenote/rendering.py index b3d3409..427777d 100644 --- a/slidenote/rendering.py +++ b/slidenote/rendering.py @@ -96,6 +96,9 @@ def _render_with_powerpoint(input_path: Path, screenshots_dir: Path, output_root except Exception: return {} + # PowerPoint is a single-instance COM server: Dispatch attaches to a copy the + # user already has open. Only quit it if we started it and nothing else is open. + was_running = _powerpoint_is_running(win32com.client) powerpoint = None presentation = None try: @@ -115,9 +118,18 @@ def _render_with_powerpoint(input_path: Path, screenshots_dir: Path, output_root presentation.Close() except Exception: pass - if powerpoint is not None: + if powerpoint is not None and not was_running: try: - powerpoint.Quit() + if powerpoint.Presentations.Count == 0: + powerpoint.Quit() except Exception: pass + +def _powerpoint_is_running(win32com_client) -> bool: + try: + win32com_client.GetActiveObject("PowerPoint.Application") + except Exception: + return False + return True + diff --git a/slidenote/sections.py b/slidenote/sections.py index 5db1888..2614a06 100644 --- a/slidenote/sections.py +++ b/slidenote/sections.py @@ -9,6 +9,8 @@ from slidenote.llm_cache import LLM_CACHE_SCHEMA_VERSION, LLMCache, make_cache_key, sha256_text, stable_json, utc_now_iso from slidenote.models import Deck, SlidePage from slidenote.utils import ( + int_or_none, + error_summary, context_title, display_path, looks_like_section_title_page, @@ -82,7 +84,22 @@ def build_section_plan( max_output_tokens=max_output_tokens, temperature=temperature, ) - result = client.generate_with_usage(prompt, system_prompt=SECTION_SYSTEM_PROMPT) + try: + result = client.generate_with_usage(prompt, system_prompt=SECTION_SYSTEM_PROMPT) + except Exception as exc: # noqa: BLE001 - section planning must not abort the build + plan = dict(local_plan) + plan["warnings"] = [*local_plan.get("warnings", []), "llm_section_call_failed"] + plan["method"] = "local_fallback" + plan["requested_mode"] = mode + plan["prompt_version"] = SECTION_PROMPT_VERSION + plan["llm"] = { + "provider": runtime["provider"], + "model": runtime["model"], + "base_url": runtime["base_url"], + "llm_call": True, + "error": error_summary(exc), + } + return plan result_text = result.text usage = result.usage or {} cache_status = "disabled" if cache_mode == "off" else "refresh" if cache_mode == "refresh" else "miss" @@ -236,8 +253,8 @@ def _normalize_model_plan(deck: Deck, parsed: dict[str, Any] | None, local_plan: for raw in raw_sections: if not isinstance(raw, dict): continue - start = raw.get("start_slide_id") - if not isinstance(start, int) or start not in valid_ids: + start = int_or_none(raw.get("start_slide_id")) + if start is None or start not in valid_ids: continue if start in starts: continue @@ -336,16 +353,12 @@ def _section_boundaries(deck: Deck) -> tuple[list[int], dict[int, str]]: return sorted(set(boundaries)), reasons -def _outline_titles(deck: Deck) -> set[str]: - return {item["normalized_title"] for item in _extract_outline_items(deck)} - - def _extract_outline_items(deck: Deck) -> list[dict[str, Any]]: items: list[dict[str, Any]] = [] seen: set[str] = set() for page in deck.pages: page_text = "\n".join(block.content for block in page.text_blocks) - if "目录" not in page_text and "Contents" not in page_text: + if "目录" not in page_text and "contents" not in page_text.lower(): continue pending_number: str | None = None for line in page_text.splitlines(): diff --git a/slidenote/semantic_layout.py b/slidenote/semantic_layout.py index 85b30e2..3ea9e13 100644 --- a/slidenote/semantic_layout.py +++ b/slidenote/semantic_layout.py @@ -8,12 +8,12 @@ from slidenote.llm import LLMClient, resolve_provider_runtime from slidenote.llm_cache import LLM_CACHE_SCHEMA_VERSION, LLMCache, make_cache_key, sha256_text, utc_now_iso -from slidenote.modality import page_has_hint +from slidenote.modality import page_has_hint, page_has_manual_modality +from slidenote.geometry import normalize_asset_bbox, normalize_page_bbox from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock from slidenote.table_understanding import table_preview from slidenote.utils import ( as_float, - clamp_normalized_bbox as _clamp_bbox, cleanup_temp_image, display_path, file_sha256, @@ -135,7 +135,9 @@ def process(index: int, page: SlidePage) -> tuple[int, int, dict[str, Any]]: if vision_record.get("status") == "applied": result = vision_record["result"] page_method = "vision_enhanced_v1" - confidence = float(vision_record.get("confidence") or confidence) + model_confidence = as_float(vision_record.get("confidence"), None) + if model_confidence is not None: + confidence = round(max(0.0, min(1.0, model_confidence)), 3) vision_pages += 1 else: fallback_pages += 1 @@ -231,7 +233,7 @@ def analyze_page_semantic_layout(deck: Deck, page: SlidePage) -> dict[str, Any]: blocks.append(_image_block_record(deck, page, image)) blocks = sorted(blocks, key=lambda block: (_layout_order(block), str(block["id"]))) groups = _semantic_groups(page, blocks) - relations = _semantic_relations(blocks, groups) + relations = _semantic_relations(blocks) return {"blocks": blocks, "groups": groups, "relations": relations} @@ -333,6 +335,7 @@ def _process_semantic_layout_vision_page( base_url=runtime["base_url"], max_output_tokens=max_output_tokens, temperature=temperature, + for_vision=True, ) llm_result = client.generate_image_with_usage( prepared_path, @@ -584,6 +587,8 @@ def _validated_vision_layout( def _page_needs_vision_enhancement(page: SlidePage, local_result: dict[str, Any]) -> bool: if not page.page_screenshot: return False + if page_has_manual_modality(page) and page.page_modality in {"native_text", "decorative"}: + return False if page_has_hint(page, "vision_page_screenshot") or page_has_hint(page, "crop_figures_from_screenshot"): return True if page.page_modality in {"mixed", "image_only", "shape_diagram"}: @@ -695,7 +700,7 @@ def _text_block_record(deck: Deck, page: SlidePage, block: TextBlock) -> dict[st block_type = _classify_text_block(block) learning_role = _learning_role_for_block(block_type, block.content) must_explain = learning_role not in {"structural", "decorative"} - bbox = _normalize_bbox(deck.source_type, block.bbox, (page.page_width, page.page_height)) + bbox = normalize_page_bbox(deck.source_type, block.bbox, page) return { "id": block.id, "kind": "text", @@ -713,7 +718,7 @@ def _text_block_record(deck: Deck, page: SlidePage, block: TextBlock) -> dict[st def _table_block_record(deck: Deck, page: SlidePage, table: TableBlock) -> dict[str, Any]: - bbox = _normalize_bbox(deck.source_type, table.bbox, (page.page_width, page.page_height)) + bbox = normalize_page_bbox(deck.source_type, table.bbox, page) return { "id": table.id, "kind": "table", @@ -731,7 +736,7 @@ def _table_block_record(deck: Deck, page: SlidePage, table: TableBlock) -> dict[ def _image_block_record(deck: Deck, page: SlidePage, image: ImageAsset) -> dict[str, Any]: - bbox = _normalize_bbox(deck.source_type, image.crop_bbox or image.bbox, (page.page_width, page.page_height)) + bbox = normalize_asset_bbox(deck.source_type, page, image) block_type = "figure" if image.role in {"figure_crop", "composite_figure"} else image.role or "image" return { "id": image.id, @@ -800,8 +805,7 @@ def _cluster_key_blocks(blocks: list[dict[str, Any]]) -> list[list[dict[str, Any return clusters -def _semantic_relations(blocks: list[dict[str, Any]], groups: list[dict[str, Any]]) -> list[dict[str, Any]]: - del groups +def _semantic_relations(blocks: list[dict[str, Any]]) -> list[dict[str, Any]]: relations: list[dict[str, Any]] = [] code_blocks = [block for block in blocks if block.get("block_type") == "code"] output_blocks = [block for block in blocks if block.get("block_type") == "output"] @@ -856,40 +860,43 @@ def _classify_text_block(block: TextBlock) -> str: return "explanation" +_CODE_PATTERNS = [ + r"#\s*include\b", + r"\busing\s+namespace\b", + r"\bmain\s*\(", + r"\bstd::", + r"\bcout\s*<<", + r"\bcin\s*>>", + r"\bgetline\s*\(", + r"\bprintf\s*\(", + r"\bdef\s+\w+\s*\(", + r"\breturn\b[^\n]*;", + r"\b(?:int|char|string|float|double|bool|void|long)\s+[A-Za-z_]\w*\s*(?:[;=,(\[]|$)", +] + + def _looks_like_code(text: str) -> bool: - signals = [ - "#include", - "using namespace", - "main()", - "cout", - "cin", - "getline", - "std::", - "return ", - "char ", - "string ", - "int ", - ] - lowered = text.lower() - if any(signal in lowered for signal in signals): + if any(re.search(pattern, text, re.IGNORECASE | re.MULTILINE) for pattern in _CODE_PATTERNS): return True code_chars = sum(text.count(char) for char in "{};<>=") - return code_chars >= 4 and bool(re.search(r"\b(if|for|while|void|int|char|string|cout|cin)\b", lowered)) + return code_chars >= 4 and bool(re.search(r"\b(if|for|while|void|int|char|string|cout|cin)\b", text, re.IGNORECASE)) def _looks_like_output(text: str) -> bool: - lowered = text.lower() - if re.search(r"\benter\s+(your|student)", lowered): + # Console transcripts: prompts such as "Enter your name:" or labelled I/O lines. + if re.search(r"^\s*(?:please\s+)?(?:enter|input|type)\s+[^\n:]{1,40}:", text, re.IGNORECASE | re.MULTILINE): return True - return any(signal in text for signal in ["Data Entered", "Student Number", "Student Name", "Hello John", "输入", "输出"]) and ":" in text + return bool(re.search(r"^\s*(?:输入|输出|运行结果|output|input)\s*[:：]", text, re.IGNORECASE | re.MULTILINE)) def _contains_cause_signal(text: str) -> bool: - return bool(re.search(r"因为|由于|导致|所以|因此|异常|留在|依然|被接下来|流提取|换行符|空白字符|缓冲|before|after", text, re.IGNORECASE)) + return bool(re.search(r"因为|由于|导致|造成|之所以|原因|\bbecause\b|\bdue to\b|\bcaused by\b", text, re.IGNORECASE)) def _contains_fix_signal(text: str) -> bool: - return bool(re.search(r"cin\.ignore|ignore\(|需要|必须|清空|清除|丢弃|解决|修复|避免|之前|之后", text, re.IGNORECASE)) + if re.search(r"(?:需要|应该|应当|要先)\s*(?:调用|使用)?\s*[A-Za-z_][\w.:]*\s*\(", text): + return True + return bool(re.search(r"解决|修复|避免|清空|清除|丢弃|改为|改成|\bfix(?:es|ed)?\b|\bto avoid\b|\bsolution\b|\bworkaround\b", text, re.IGNORECASE)) def _contains_visual_annotation(text: str) -> bool: @@ -973,8 +980,9 @@ def _learning_goal(blocks: list[dict[str, Any]], scene_type: str) -> str: previews = [str(block.get("preview") or "") for block in blocks] text = " ".join(previews) if scene_type == "code_causal_explanation": - if re.search(r"getline|cin|换行符|空白字符|ignore", text, re.IGNORECASE): - return "讲清 cin 提取运算符与 getline 混用时的换行符残留问题、现象和修复方法。" + cause = next((str(block.get("preview") or "") for block in blocks if block.get("learning_role") == "cause"), "") + if cause: + return f"讲清代码示例的运行现象、原因和修复方法：{preview(cause, 100)}" return "讲清代码示例的运行现象、原因和修复方法。" if scene_type == "code_example_with_output": return "把代码与运行输出对应起来，说明示例验证了什么行为。" @@ -985,28 +993,6 @@ def _learning_goal(blocks: list[dict[str, Any]], scene_type: str) -> str: return preview(text, 140) or "讲解本组核心概念。" -def _normalize_bbox(source_type: str, bbox: list[float] | None, page_size: tuple[float | None, float | None] | None) -> list[float] | None: - if not bbox or len(bbox) != 4: - return None - if all(-0.001 <= float(value) <= 1.001 for value in bbox): - return _clamp_bbox(bbox) - width, height = page_size or (None, None) - if not width or not height: - return None - x1, y1, third, fourth = [float(value) for value in bbox] - if source_type == "pptx": - x2, y2 = x1 + third, y1 + fourth - else: - x2, y2 = third, fourth - return _clamp_bbox([x1 / width, y1 / height, x2 / width, y2 / height]) - - -def _page_size_for_bbox(deck: Deck, page: SlidePage | None) -> tuple[float | None, float | None] | None: - if page is not None: - return page.page_width, page.page_height - return None - - def _layout_order(block: dict[str, Any]) -> float: try: return float(block.get("layout_order")) diff --git a/slidenote/source_map.py b/slidenote/source_map.py index f0187e2..ce6d369 100644 --- a/slidenote/source_map.py +++ b/slidenote/source_map.py @@ -5,19 +5,27 @@ from pathlib import Path from typing import Any -from slidenote.ir import build_page_ir, element_index_from_ir +from slidenote.ir import build_deck_ir, element_index_from_ir from slidenote.llm_cache import utc_now_iso -from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock -from slidenote.table_understanding import table_preview +from slidenote.models import Deck, SlidePage from slidenote.utils import preview ELEMENT_PATTERN = re.compile(r"\bs\d+_(?:t|tbl|img|fig)\d+\b") -def build_source_map(deck: Deck, notes_markdown: str, output_root: Path) -> dict[str, Any]: +def build_source_map( + deck: Deck, + notes_markdown: str, + output_root: Path, + *, + content_guard: dict[str, Any] | None = None, + coverage_report: dict[str, Any] | None = None, +) -> dict[str, Any]: blocks = _note_blocks(notes_markdown) - element_index = _element_index(deck) + deck_ir = build_deck_ir(deck, content_guard=content_guard, coverage_report=coverage_report) + page_irs = {int(page_ir["slide_id"]): page_ir for page_ir in deck_ir["pages"]} + element_index = element_index_from_ir(deck, deck_ir) image_path_index = _image_path_index(deck) note_blocks: list[dict[str, Any]] = [] used_block_ids: dict[str, int] = {} @@ -56,7 +64,7 @@ def build_source_map(deck: Deck, notes_markdown: str, output_root: Path) -> dict "source_type": deck.source_type, "display_modes": ["hidden", "footnote", "inline"], "default_display_mode": "hidden", - "pages": [_page_sources(page) for page in deck.pages], + "pages": [_page_sources(page, page_irs.get(page.slide_id)) for page in deck.pages], "note_blocks": note_blocks, "artifacts": { "notes": "notes.md", @@ -207,11 +215,7 @@ def _strip_html_comments(text: str) -> str: return re.sub(r"", "", text, flags=re.DOTALL) -def _element_index(deck: Deck) -> dict[str, dict[str, Any]]: - return element_index_from_ir(deck) - - -def _page_sources(page: SlidePage) -> dict[str, Any]: +def _page_sources(page: SlidePage, page_ir: dict[str, Any] | None) -> dict[str, Any]: return { "slide_id": page.slide_id, "title": page.title, @@ -249,60 +253,5 @@ def _page_sources(page: SlidePage) -> dict[str, Any]: ], "semantic_groups": page.semantic_groups, "semantic_relations": page.semantic_relations, - "element_ir": build_page_ir(Deck(source_path="", source_type="", pages=[page]), page), - } - - -def _text_ref(deck: Deck, page: SlidePage, block: TextBlock) -> dict[str, Any]: - return { - "type": "text", - "source_path": deck.source_path, - "slide_id": page.slide_id, - "element_id": block.id, - "element_type": block.type, - "preview": preview(block.content), - } - - -def _table_ref(deck: Deck, page: SlidePage, table: TableBlock) -> dict[str, Any]: - return { - "type": "table", - "source_path": deck.source_path, - "slide_id": page.slide_id, - "element_id": table.id, - "preview": table_preview(table), - "table_summary": table.table_summary, - "table_conclusion": table.table_conclusion, - "key_rows": table.key_rows, - } - - -def _image_ref(deck: Deck, page: SlidePage, image: ImageAsset) -> dict[str, Any]: - return { - "type": "image", - "source_path": deck.source_path, - "slide_id": page.slide_id, - "element_id": image.id, - "path": image.path, - "role": image.role, - "width": image.width, - "height": image.height, - "crop_source_path": image.crop_source_path, - "crop_bbox": image.crop_bbox, - "crop_method": image.crop_method, - "crop_quality": image.crop_quality, - "crop_warnings": list(image.crop_warnings), - "confidence": image.confidence, - "importance_score": image.importance_score, - "importance_rank": image.importance_rank, - "importance_reason": image.importance_reason, - "layout_order": image.layout_order, - "source_element_ids": list(image.source_element_ids), - "anchor_element_ids": image.anchor_element_ids, - "anchor_reason": image.anchor_reason, - "grounding_confidence": image.grounding_confidence, - "figure_explanation": image.figure_explanation, - "figure_explanation_status": image.figure_explanation_status, - "figure_audit_status": image.figure_audit_status, - "preview": image.caption or image.path, + "element_ir": page_ir, } diff --git a/slidenote/study_pack/__init__.py b/slidenote/study_pack/__init__.py index 924bfe9..cb534e2 100644 --- a/slidenote/study_pack/__init__.py +++ b/slidenote/study_pack/__init__.py @@ -5,12 +5,12 @@ from pathlib import Path from typing import Any -from slidenote.content_guard import REQUIRED_CONFIDENCE_THRESHOLD +from slidenote.content_guard import is_required_item from slidenote.exporting import clean_markdown_for_export from slidenote.llm import LLMClient, resolve_provider_runtime from slidenote.llm_cache import LLM_CACHE_SCHEMA_VERSION, LLMCache, make_cache_key, sha256_text, stable_json, utc_now_iso from slidenote.models import Deck, TableBlock, TextBlock -from slidenote.utils import as_float, display_path +from slidenote.utils import as_float, display_path, parse_json_object from .common import ( IMPORTANCE_LABELS, @@ -91,18 +91,18 @@ def _build_local_report_item( point: str, explanation: str, slide_id: int | None, - importance: str = "key", role: str | None = None, image_refs: list[dict[str, Any]] | None = None, ) -> dict[str, Any]: clean_point = _clean_inline(point) clean_explanation = _clean_inline(explanation) or clean_point + importance = _infer_importance(clean_point + " " + clean_explanation, role) return { "section": section or "核心知识点", - "importance": _infer_importance(clean_point + " " + clean_explanation, role), + "importance": importance, "point": clean_point or "知识点", "explanation": clean_explanation or "需要结合原始笔记复习。", - "why": _local_why(_infer_importance(clean_point + " " + clean_explanation, role)), + "why": _local_why(importance), "pitfall": _local_pitfall(clean_point, role), "source_refs": [f"P{slide_id}"] if slide_id else [], "image_refs": image_refs or [], @@ -118,6 +118,12 @@ def _collect_study_items( items: list[dict[str, Any]] = [] seen: set[str] = set() + def add(item: dict[str, Any]) -> bool: + """Add ``item`` if new; return True once the limit is reached.""" + _add_unique(items, item, seen) + return len(items) >= limit + + # Priority order: required guard items, then per-page tables, figures, text, then note headings. for guard_item in _guard_items(content_guard): element_id = str(guard_item.get("element_id") or "") element = lookup.get(element_id, {}) @@ -132,18 +138,15 @@ def _collect_study_items( slide_id=_as_int(guard_item.get("slide_id") or element.get("slide_id"), 0) or None, role=str(guard_item.get("learning_role") or element.get("kind") or ""), ) - if _add_unique(items, item, seen): - continue - if len(items) >= limit: + if add(item): return items for page in deck.pages: section = page.title or f"第 {page.slide_id} 页" for table in page.tables: text = table.table_conclusion or table.table_summary or _table_text(table) - if text: - item = _build_local_report_item(section, _point_from_text(text), text, page.slide_id, role="table_conclusion") - _add_unique(items, item, seen) + if text and add(_build_local_report_item(section, _point_from_text(text), text, page.slide_id, role="table_conclusion")): + return items for image in page.images: if image.ignored: continue @@ -157,21 +160,18 @@ def _collect_study_items( role="figure_explanation", image_refs=[{"id": image.id, "title": image.caption or f"P{page.slide_id} 图示", "path": image.path, "source_ref": f"P{page.slide_id}"}], ) - _add_unique(items, item, seen) + if add(item): + return items for block in page.text_blocks: text = _text_block_text(block) if _skip_text(text): continue - item = _build_local_report_item(section, _point_from_text(text), text, page.slide_id, role=block.type) - _add_unique(items, item, seen) - if len(items) >= limit: + if add(_build_local_report_item(section, _point_from_text(text), text, page.slide_id, role=block.type)): return items for heading in _headings_from_notes(notes_markdown): - item = _build_local_report_item("笔记结构", heading, heading, None, role="heading") - _add_unique(items, item, seen) - if len(items) >= limit: - break + if add(_build_local_report_item("笔记结构", heading, heading, None, role="heading")): + return items if not items: items.append( { @@ -184,7 +184,7 @@ def _collect_study_items( "source_refs": [], } ) - return items[:limit] + return items def _content_guard_for_prompt(content_guard: dict[str, Any] | None) -> list[dict[str, Any]]: if not content_guard: @@ -192,7 +192,7 @@ def _content_guard_for_prompt(content_guard: dict[str, Any] | None) -> list[dict result = [] for item in _dict_list(content_guard.get("items"), limit=300): confidence = as_float(item.get("confidence"), 0.0) - if item.get("must_explain") or confidence >= REQUIRED_CONFIDENCE_THRESHOLD: + if is_required_item(item): result.append( { "slide_id": item.get("slide_id"), @@ -367,7 +367,6 @@ def _generate_llm_data( "content_guard_hash": sha256_text(stable_json(content_guard or {})), "system_prompt_hash": sha256_text(STUDY_PACK_SYSTEM_PROMPT), "user_prompt_hash": sha256_text(prompt), - "user_prompt": prompt, } cache_key = make_cache_key(cache_key_payload) cache_path = cache.path_for(cache_key) @@ -423,7 +422,7 @@ def _generate_llm_data( warnings.append(f"study_pack_llm_failed:{type(exc).__name__}:{exc}") return None, _llm_record(runtime, cache_key, cache_path, output_root, prompt_hash, "error", llm_call, usage), warnings - parsed = _parse_json_object(raw_text) + parsed = parse_json_object(raw_text) if parsed is None: warnings.append("study_pack_invalid_json") return None, _llm_record(runtime, cache_key, cache_path, output_root, prompt_hash, cache_status, llm_call, usage), warnings @@ -435,11 +434,7 @@ def _guard_items(content_guard: dict[str, Any] | None) -> list[dict[str, Any]]: if not content_guard: return [] items = _dict_list(content_guard.get("items"), limit=1000) - result = [] - for item in items: - if item.get("must_explain") or as_float(item.get("confidence"), 0.0) >= REQUIRED_CONFIDENCE_THRESHOLD: - result.append(item) - return result + return [item for item in items if is_required_item(item)] def _llm_record( runtime: dict[str, Any], @@ -494,6 +489,7 @@ def _normalize_exam(raw: dict[str, Any] | None, fallback: dict[str, Any], questi options = _string_list(question.get("options"), limit=8) if len(options) < 2: normalized["type"] = "short" + normalized["points"] = _as_int(question.get("points"), _default_points("short")) normalized["answer"] = _clean_inline(question.get("answer")) or normalized["explanation"] else: normalized["options"] = options @@ -521,6 +517,7 @@ def _normalize_review(raw: dict[str, Any] | None, fallback: dict[str, Any]) -> d "why": _clean_inline(item.get("why")), "pitfall": _clean_inline(item.get("pitfall")), "source_refs": _string_list(item.get("source_refs"), limit=12), + "image_refs": _normalize_image_refs(item.get("image_refs")), } ) if not checklist: @@ -540,24 +537,6 @@ def _normalize_study_data(raw: dict[str, Any], fallback: dict[str, Any], questio normalized_exam = _normalize_exam(exam, fallback.get("exam") or {}, question_count) return {"review": normalized_review, "exam": normalized_exam} -def _parse_json_object(text: str) -> dict[str, Any] | None: - raw = (text or "").strip() - if raw.startswith("```"): - raw = re.sub(r"^```(?:json)?\s*", "", raw) - raw = re.sub(r"\s*```$", "", raw) - try: - value = json.loads(raw) - except json.JSONDecodeError: - start = raw.find("{") - end = raw.rfind("}") - if start < 0 or end <= start: - return None - try: - value = json.loads(raw[start : end + 1]) - except json.JSONDecodeError: - return None - return value if isinstance(value, dict) else None - def _skip_text(text: str) -> bool: clean = _clean_inline(text) if len(clean) < 3: diff --git a/slidenote/study_pack/common.py b/slidenote/study_pack/common.py index c419ac6..21d392e 100644 --- a/slidenote/study_pack/common.py +++ b/slidenote/study_pack/common.py @@ -1,5 +1,7 @@ from pathlib import Path from typing import Any + +from slidenote.utils import as_int import re IMPORTANCE_LABELS = { @@ -17,16 +19,24 @@ "comprehensive": "综合题", } -def _as_int(value: Any, default: int) -> int: - try: - return int(value) - except (TypeError, ValueError): - return default +_as_int = as_int + +# Only strip real HTML markup (a tag name followed by name=value attributes), so +# inline math such as "a < b and c > d" or "ad" survives. +_HTML_COMMENT_RE = re.compile(r"", flags=re.DOTALL) +_HTML_TAG_RE = re.compile( + r"]+))*" + r"\s*/?>" +) + + +def _strip_html(value: str) -> str: + return _HTML_TAG_RE.sub("", _HTML_COMMENT_RE.sub("", value)) + def _clean_inline(value: Any) -> str: - text = str(value or "") - text = re.sub(r"<[^>]+>", "", text) - text = re.sub(r"", "", text, flags=re.DOTALL) + text = _strip_html(str(value or "")) return " ".join(text.split()).strip() def _dict_list(value: Any, limit: int = 100) -> list[dict[str, Any]]: diff --git a/slidenote/study_pack/exam.py b/slidenote/study_pack/exam.py index 01bec52..35522d7 100644 --- a/slidenote/study_pack/exam.py +++ b/slidenote/study_pack/exam.py @@ -39,14 +39,25 @@ def _attach_inline_figure_refs(exam: dict[str, Any], figure_table_notes: list[di for note in matched[:2] ] +MINUTES_PER_FINAL_QUESTION = 6 + + +def _question_points(question: dict[str, Any]) -> int: + qtype = _normalize_question_type(question.get("type")) + return _as_int(question.get("points"), _default_points(qtype)) + + +def _final_exam_duration(question_count: int) -> int: + return max(30, min(180, question_count * MINUTES_PER_FINAL_QUESTION)) + + def _build_final_exam(exam: dict[str, Any]) -> dict[str, Any]: questions = _dict_list(exam.get("questions"), limit=200) - total_points = sum(_as_int(question.get("points"), _default_points(_normalize_question_type(question.get("type")))) for question in questions) return { "title": f"{_clean_inline(exam.get('title')) or '课程'} - 期末模拟卷", "mode": "mock_final", - "duration_minutes": max(30, min(180, len(questions) * 6)), - "total_points": total_points, + "duration_minutes": _final_exam_duration(len(questions)), + "total_points": sum(_question_points(question) for question in questions), "instructions": "先独立完成，再核对 final_exam.answers.md；错题回到来源页和 review.md 对应章节复盘。", "questions": questions, } @@ -345,39 +356,32 @@ def render_exam_html(report: dict[str, Any]) -> str: """ -def render_exam_markdown(report: dict[str, Any]) -> str: - exam = report.get("exam") if isinstance(report.get("exam"), dict) else {} - title = _clean_inline(exam.get("title")) or _source_title(report) - questions = _dict_list(exam.get("questions"), limit=200) - lines = [f"# {title} - 自测题", ""] - subtitle = _clean_inline(exam.get("subtitle")) - if subtitle: - lines.extend([subtitle, ""]) - lines.extend(["## 题目", ""]) +def _question_lines(questions: list[dict[str, Any]]) -> list[str]: + lines: list[str] = [] for index, question in enumerate(questions, start=1): qtype = _normalize_question_type(question.get("type")) label = QUESTION_TYPE_LABELS.get(qtype, "题目") - points = _as_int(question.get("points"), _default_points(qtype)) - lines.append(f"{index}. 【{label} · {points} 分】{_clean_inline(question.get('question'))}") + lines.append(f"{index}. 【{label} · {_question_points(question)} 分】{_clean_inline(question.get('question'))}") if qtype == "choice": for option_index, option in enumerate(_string_list(question.get("options"), limit=8)): lines.append(f" {chr(65 + option_index)}. {option}") for image_ref in _dict_list(question.get("image_refs"), limit=3): path = _clean_inline(image_ref.get("path")) - title = _clean_inline(image_ref.get("title")) or "题目图" + image_title = _clean_inline(image_ref.get("title")) or "题目图" if path: - lines.append(f" ![{title}]({path})") + lines.append(f" ![{image_title}]({path})") if qtype in {"short", "essay", "comprehensive"}: - lines.append("") - lines.append(" 答：") + lines.extend(["", " 答："]) lines.append("") + return lines - lines.extend(["## 答案与解析", ""]) + +def _answer_lines(questions: list[dict[str, Any]]) -> list[str]: + lines: list[str] = [] for index, question in enumerate(questions, start=1): qtype = _normalize_question_type(question.get("type")) - answer = _answer_text(question, qtype) explanation = _clean_inline(question.get("explanation")) or "复习对应知识点后再核对答案。" - lines.append(f"{index}. **答案**：{answer}") + lines.append(f"{index}. **答案**：{_answer_text(question, qtype)}") lines.append(f" **解析**：{explanation}") pitfall = _clean_inline(question.get("pitfall")) if pitfall: @@ -386,54 +390,41 @@ def render_exam_markdown(report: dict[str, Any]) -> str: if source_refs: lines.append(f" **来源**：{', '.join(source_refs)}") lines.append("") + return lines + + +def render_exam_markdown(report: dict[str, Any]) -> str: + exam = report.get("exam") if isinstance(report.get("exam"), dict) else {} + title = _clean_inline(exam.get("title")) or _source_title(report) + questions = _dict_list(exam.get("questions"), limit=200) + lines = [f"# {title} - 自测题", ""] + subtitle = _clean_inline(exam.get("subtitle")) + if subtitle: + lines.extend([subtitle, ""]) + lines.extend(["## 题目", "", *_question_lines(questions)]) + lines.extend(["## 答案与解析", "", *_answer_lines(questions)]) return "\n".join(lines).rstrip() + "\n" + def render_final_exam_answers_markdown(report: dict[str, Any]) -> str: final_exam = report.get("final_exam") if isinstance(report.get("final_exam"), dict) else {} title = _clean_inline(final_exam.get("title")) or _source_title(report) questions = _dict_list(final_exam.get("questions"), limit=200) - lines = [f"# {title} - 答案与评分提示", ""] - for index, question in enumerate(questions, start=1): - qtype = _normalize_question_type(question.get("type")) - answer = _answer_text(question, qtype) - explanation = _clean_inline(question.get("explanation")) or "复习对应知识点后再核对答案。" - lines.append(f"{index}. **答案**：{answer}") - lines.append(f" **解析**：{explanation}") - pitfall = _clean_inline(question.get("pitfall")) - if pitfall: - lines.append(f" **易错提醒**：{pitfall}") - source_refs = _string_list(question.get("source_refs"), limit=8) - if source_refs: - lines.append(f" **来源**：{', '.join(source_refs)}") - lines.append("") + lines = [f"# {title} - 答案与评分提示", "", *_answer_lines(questions)] return "\n".join(lines).rstrip() + "\n" + def render_final_exam_markdown(report: dict[str, Any]) -> str: final_exam = report.get("final_exam") if isinstance(report.get("final_exam"), dict) else {} title = _clean_inline(final_exam.get("title")) or _source_title(report) questions = _dict_list(final_exam.get("questions"), limit=200) - total_points = sum(_as_int(question.get("points"), 0) for question in questions) - duration = _as_int(final_exam.get("duration_minutes"), max(30, len(questions) * 5)) + total_points = sum(_question_points(question) for question in questions) + duration = _as_int(final_exam.get("duration_minutes"), _final_exam_duration(len(questions))) lines = [f"# {title}", "", f"- 建议时长：{duration} 分钟", f"- 总分：{total_points} 分", ""] - lines.extend(["## 试题", ""]) - for index, question in enumerate(questions, start=1): - qtype = _normalize_question_type(question.get("type")) - label = QUESTION_TYPE_LABELS.get(qtype, "题目") - points = _as_int(question.get("points"), _default_points(qtype)) - lines.append(f"{index}. 【{label} · {points} 分】{_clean_inline(question.get('question'))}") - if qtype == "choice": - for option_index, option in enumerate(_string_list(question.get("options"), limit=8)): - lines.append(f" {chr(65 + option_index)}. {option}") - for image_ref in _dict_list(question.get("image_refs"), limit=3): - path = _clean_inline(image_ref.get("path")) - title_text = _clean_inline(image_ref.get("title")) or "题目图" - if path: - lines.append(f" ![{title_text}]({path})") - if qtype in {"short", "essay", "comprehensive"}: - lines.extend(["", " 答："]) - lines.append("") + lines.extend(["## 试题", "", *_question_lines(questions)]) return "\n".join(lines).rstrip() + "\n" + def render_wrong_answer_review_prompt(report: dict[str, Any]) -> str: wrong_review = report.get("wrong_answer_review") if isinstance(report.get("wrong_answer_review"), dict) else {} title = _clean_inline(wrong_review.get("title")) or "错题复盘 Prompt" diff --git a/slidenote/study_pack/questions.py b/slidenote/study_pack/questions.py index e9636bb..fd41305 100644 --- a/slidenote/study_pack/questions.py +++ b/slidenote/study_pack/questions.py @@ -1,6 +1,8 @@ from slidenote.models import Deck from slidenote.utils import round_score from typing import Any +import hashlib +import random import re from .common import ( _as_int, @@ -76,13 +78,15 @@ def _local_choice_options(item: dict[str, Any], items: list[dict[str, Any]]) -> continue distractors.append(f"把「{other_point}」的作用误当成「{point}」的主要含义。") if other_explanation: - distractors.append(f"只记住相邻结论“{_trim_text(other_explanation, 52)}”，但忽略它和「{point}」的适用条件。") + distractors.append(f"只记住相邻结论“{_shorten(other_explanation, 52)}”，但忽略它和「{point}」的适用条件。") if len(distractors) >= 3: break + # Generic but plausible misconceptions (over-generalisation, reversed causality, + # isolation from neighbouring concepts); avoid trivially eliminable options. fallback = [ - f"只背「{point}」这个名称，但不能说明它解决的问题和限制。", - f"把「{point}」理解成任何场景都成立的结论，忽略材料给出的条件。", - f"只记住最终结论，却不能解释「{point}」与前后概念的关系。", + f"把「{point}」理解成任何场景都成立的结论，忽略材料给出的前提条件。", + f"把「{point}」中的因果关系颠倒，把结果当成了原因。", + f"认为「{point}」与前后概念相互独立，彼此之间没有依赖关系。", ] for option in fallback: if len(distractors) >= 3: @@ -136,6 +140,37 @@ def _local_pitfall(point: str, role: str | None = None) -> str: return "不要只看图名，要能沿箭头或结构关系讲出因果链。" return "不要只背关键词，要能说明它解决的问题和使用场景。" +def _seeded_rng(*parts: Any) -> random.Random: + """Deterministic RNG so local study packs are reproducible across runs.""" + digest = hashlib.sha256("|".join(str(part) for part in parts).encode("utf-8")).digest() + return random.Random(int.from_bytes(digest[:8], "big")) + + +def _local_true_false(item: dict[str, Any], items: list[dict[str, Any]], index: int) -> dict[str, Any]: + point = _clean_inline(item.get("point")) or "知识点" + explanation = _clean_inline(item.get("explanation")) or point + other_explanations = [ + text + for other in items + if (text := _clean_inline(other.get("explanation"))) and text != explanation and _clean_inline(other.get("point")) != point + ] + rng = _seeded_rng("true_false", point, index) + if not other_explanations or rng.random() < 0.5: + return { + "question": f"判断：根据材料，「{point}」可以概括为：{_shorten(explanation, 120)}", + "answer": True, + "explanation": f"正确。材料原意：{explanation}", + "pitfall": "判断题要逐句核对条件和对象，不要只凭关键词作答。", + } + swapped = rng.choice(other_explanations) + return { + "question": f"判断：根据材料，「{point}」可以概括为：{_shorten(swapped, 120)}", + "answer": False, + "explanation": f"错误。这句描述对应的是另一个知识点；「{point}」的原意是：{explanation}", + "pitfall": "相邻知识点的描述容易张冠李戴，注意核对说明对应的对象。", + } + + def _local_questions(items: list[dict[str, Any]], question_count: int) -> list[dict[str, Any]]: questions: list[dict[str, Any]] = [] if not items: @@ -148,6 +183,8 @@ def _local_questions(items: list[dict[str, Any]], question_count: int) -> list[d image_refs = _normalize_image_refs(item.get("image_refs")) if index % 4 == 1: options = _local_choice_options(item, items) + correct = options[0] + _seeded_rng("choice", point, index).shuffle(options) questions.append( { "id": f"q{index}", @@ -155,7 +192,7 @@ def _local_questions(items: list[dict[str, Any]], question_count: int) -> list[d "points": 2, "question": f"关于「{point}」，哪一项最符合材料中的含义？", "options": options, - "answer": 0, + "answer": options.index(correct), "explanation": explanation, "pitfall": _clean_inline(item.get("pitfall")), "source_refs": source_refs, @@ -168,10 +205,7 @@ def _local_questions(items: list[dict[str, Any]], question_count: int) -> list[d "id": f"q{index}", "type": "true_false", "points": 1, - "question": f"判断：「{point}」只要背下名称即可，不需要理解它解决的问题或使用场景。", - "answer": False, - "explanation": f"错误。复习时应说明它的含义、作用和易错点：{explanation}", - "pitfall": "把概念当成孤立名词，是短期备考最常见的失分方式。", + **_local_true_false(item, items, index), "source_refs": source_refs, "image_refs": image_refs, } @@ -311,6 +345,10 @@ def _question_quality_flags( flags.append("questions_too_definition_like") return flags +def _shorten(text: str, limit: int) -> str: + """Truncate inline text for question/option display (no prompt-budget marker).""" + return text if len(text) <= limit else text[: limit - 1].rstrip() + "…" + def _trim_text(text: str, limit: int) -> str: if len(text) <= limit: return text diff --git a/slidenote/study_pack/review.py b/slidenote/study_pack/review.py index 51b3fbb..cd19ba0 100644 --- a/slidenote/study_pack/review.py +++ b/slidenote/study_pack/review.py @@ -6,6 +6,7 @@ _dict_list, _source_title, _string_list, + _strip_html, ) def _headings_from_notes(markdown: str) -> list[str]: @@ -13,7 +14,7 @@ def _headings_from_notes(markdown: str) -> list[str]: for line in markdown.splitlines(): match = re.match(r"^#{2,4}\s+(.+?)\s*$", line.strip()) if match: - heading = re.sub(r"<[^>]+>", "", match.group(1)).strip() + heading = _strip_html(match.group(1)).strip() if heading: headings.append(heading) return headings[:20] diff --git a/slidenote/study_pack_runner.py b/slidenote/study_pack_runner.py index 8bfda2d..a7ea918 100644 --- a/slidenote/study_pack_runner.py +++ b/slidenote/study_pack_runner.py @@ -3,6 +3,7 @@ import argparse import json import os +import sys from pathlib import Path from typing import Any @@ -20,6 +21,10 @@ ) from slidenote.utils import write_json, write_text +DEFAULT_STUDY_PACK_PROVIDER = "deepseek" +# Review + exam JSON for up to 60 questions needs a large completion budget. +STUDY_PACK_MAX_OUTPUT_TOKENS = 12000 + def run_study_pack(args: argparse.Namespace) -> int: output_root = args.build_out_dir.resolve() @@ -35,8 +40,17 @@ def run_study_pack(args: argparse.Namespace) -> int: notes_markdown = notes_path.read_text(encoding="utf-8") run_summary = _read_optional_json(output_root / "run_summary.json") or {} run_config = run_summary.get("run") if isinstance(run_summary.get("run"), dict) else {} - provider = str(run_config.get("provider") or "deepseek") - use_llm = _provider_can_run(provider) + warnings: list[str] = [] + provider = str(run_config.get("provider") or "") + if not provider: + provider = DEFAULT_STUDY_PACK_PROVIDER + warnings.append( + f"study_pack_provider_fallback:{provider} (run_summary.json has no run.provider; " + "the original build's provider/model could not be reused)" + ) + model = str(run_config.get("model") or "") or None + base_url = str(run_config.get("base_url") or "") or None + use_llm = _provider_can_run(provider, model=model) report = build_study_pack( deck=deck, @@ -47,9 +61,11 @@ def run_study_pack(args: argparse.Namespace) -> int: question_count=max(1, int(args.question_count or 12)), use_llm=use_llm, provider=provider, + model=model, + base_url=base_url, cache_mode="on", cache_dir=output_root / ".cache" / "llm", - max_output_tokens=12000, + max_output_tokens=STUDY_PACK_MAX_OUTPUT_TOKENS, temperature=0.0, note_language=str(run_config.get("note_language") or "zh"), section_plan=_read_optional_json(output_root / "sections.json"), @@ -60,6 +76,10 @@ def run_study_pack(args: argparse.Namespace) -> int: ) if report is None: raise UserFacingConfigError("study-pack did not produce any output.") + if warnings: + report["warnings"] = [*warnings, *(report.get("warnings") or [])] + for warning in warnings: + print(f"Warning: {warning}", file=sys.stderr) _write_study_pack_outputs(output_root, report) if not args.quiet: @@ -90,13 +110,13 @@ def _write_study_pack_outputs(output_root: Path, report: dict[str, Any]) -> None write_text(output_root / "wrong_answer_review_prompt.md", render_wrong_answer_review_prompt(report)) -def _provider_can_run(provider: str) -> bool: +def _provider_can_run(provider: str, model: str | None = None) -> bool: try: spec = get_provider_spec(provider) except ValueError: return False has_key = any(os.getenv(name) for name in spec.api_key_envs) - has_model = bool(os.getenv("SLIDENOTE_MODEL") or any(os.getenv(name) for name in spec.model_envs) or spec.default_model) + has_model = bool(model or os.getenv("SLIDENOTE_MODEL") or any(os.getenv(name) for name in spec.model_envs) or spec.default_model) return has_key and has_model diff --git a/slidenote/table_understanding.py b/slidenote/table_understanding.py index 536e112..54d2fca 100644 --- a/slidenote/table_understanding.py +++ b/slidenote/table_understanding.py @@ -217,20 +217,21 @@ def _table_summary(headers: list[str], data_rows: list[tuple[int, list[str]]], w dimensions = _join_terms(headers[:4]) suffix = f"等 {len(headers)} 个维度" if len(headers) > 4 else "这些维度" count_text = f"{data_count} 条记录" if data_count else "表头维度" - return f"表格围绕「{dimensions}」{suffix}组织，主要用于对比或归纳 {count_text}。" + return f"表格按「{dimensions}」{suffix}列出 {count_text}。" row_count = data_count - return f"表格包含 {row_count} 行、{width} 列信息，主要用于并列展示相关条目。" + return f"表格包含 {row_count} 行、{width} 列信息。" def _table_conclusion(key_rows: list[dict[str, Any]], headers: list[str], has_header: bool) -> str | None: if not key_rows: return None + # Describe what the table lists; interpreting it is left to the note writer. if has_header and len(headers) >= 2: row_summaries = [_key_row_sentence(row) for row in key_rows[:3]] dimensions = _join_terms(headers[1:4]) - return f"表格重点比较 {_join_terms([str(row.get('label') or '') for row in key_rows[:3]])}，差异主要落在「{dimensions}」等维度；{_join_terms(row_summaries, sep='；')}。" + return f"表格列出了 {_join_terms([str(row.get('label') or '') for row in key_rows[:3]])} 在「{dimensions}」等维度上的取值；{_join_terms(row_summaries, sep='；')}。" labels = [str(row.get("label") or "") for row in key_rows[:3]] - return f"表格中的 {_join_terms(labels)} 是需要优先关注的代表性条目。" + return f"表格中的代表性条目包括 {_join_terms(labels)}。" def _key_row_sentence(row: dict[str, Any]) -> str: diff --git a/slidenote/textbook.py b/slidenote/textbook.py index 2eeb933..bd33749 100644 --- a/slidenote/textbook.py +++ b/slidenote/textbook.py @@ -7,11 +7,13 @@ from typing import Any from slidenote.llm_cache import LLMCache, make_cache_key, sha256_text, utc_now_iso -from slidenote.ocr import OCRClient, _prepare_image_for_ocr +from slidenote.api_retry import with_api_retries +from slidenote.ocr import OCRClient, prepare_image_for_ocr from slidenote.utils import ( cleanup_temp_image, display_path, ensure_clean_dir, + file_sha256, write_json, write_text, ) @@ -52,7 +54,7 @@ def build_textbook_index( raise FileNotFoundError(input_path) ensure_clean_dir(output_root) - source_hash = _sha256_file(input_path) + source_hash = file_sha256(input_path) pages, metadata = extract_textbook_pages(input_path) ocr_report = None if ocr != "off": @@ -307,7 +309,7 @@ def _recognize_textbook_page( "provider": provider, "source_hash": source_hash, "physical_page": page_number, - "rendered_hash": _sha256_file(rendered_path), + "rendered_hash": file_sha256(rendered_path), } ) cache_path = cache.path_for(cache_key) @@ -325,18 +327,20 @@ def _recognize_textbook_page( record["text_chars"] = len(text) return record, text - prepared = _prepare_image_for_ocr(rendered_path, max_edge=2200) + prepared = prepare_image_for_ocr(rendered_path, max_edge=2200) if prepared is None: record.update({"cache_status": "skipped", "skip_reason": "unreadable_rendered_page"}) return record, "" prepared_path, image_meta = prepared try: client = OCRClient(provider=provider, api_key=api_key, secret_key=secret_key) - result = client.recognize(prepared_path) + retry_result = with_api_retries(lambda: client.recognize(prepared_path)) + result = retry_result.value text = result.text.strip() record.update( { "api_call": True, + "api_retries": retry_result.retries, "text_chars": len(text), "provider_usage": result.usage, "image_meta": image_meta, @@ -772,13 +776,3 @@ def _title_hint(text: str) -> str | None: return _strip_toc_leader(line)[:140] first = next(iter(_clean_lines(text)), "") return first[:140] if first else None - - -def _sha256_file(path: Path) -> str: - import hashlib - - digest = hashlib.sha256() - with path.open("rb") as handle: - for chunk in iter(lambda: handle.read(1024 * 1024), b""): - digest.update(chunk) - return "sha256:" + digest.hexdigest() diff --git a/slidenote/understanding.py b/slidenote/understanding.py index 38580e5..521fbae 100644 --- a/slidenote/understanding.py +++ b/slidenote/understanding.py @@ -5,10 +5,12 @@ from typing import Any from slidenote.content_guard import learning_items_for_page +from slidenote.geometry import normalize_asset_bbox from slidenote.llm_cache import utc_now_iso from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock from slidenote.table_understanding import table_preview from slidenote.utils import ( + int_or_none, as_float, preview, str_or_none, @@ -138,7 +140,7 @@ def _page_understanding( ) -> dict[str, Any]: role = _page_role(page, role_record) tables = [_table_record(table) for table in page.tables] - figures = [_figure_record(page, image) for image in _page_figures(page)] + figures = [_figure_record(deck, page, image) for image in _page_figures(page)] semantic = { "blocks": _semantic_blocks(page.semantic_blocks), "groups": page.semantic_groups[:12], @@ -260,7 +262,7 @@ def _brief(deck_brief_report: dict[str, Any] | None) -> dict[str, Any]: def _page_roles_from_brief(brief: dict[str, Any]) -> dict[int, dict[str, Any]]: roles: dict[int, dict[str, Any]] = {} for item in _dict_list(brief.get("page_roles"), limit=1000): - slide_id = _int_or_none(item.get("slide_id")) + slide_id = int_or_none(item.get("slide_id")) if slide_id is not None: roles[slide_id] = item return roles @@ -312,7 +314,7 @@ def _page_figures(page: SlidePage) -> list[ImageAsset]: ) -def _figure_record(page: SlidePage, image: ImageAsset) -> dict[str, Any]: +def _figure_record(deck: Deck, page: SlidePage, image: ImageAsset) -> dict[str, Any]: return { "id": image.id, "slide_id": page.slide_id, @@ -331,6 +333,7 @@ def _figure_record(page: SlidePage, image: ImageAsset) -> dict[str, Any]: "anchor_reason": image.anchor_reason, "grounding_confidence": image.grounding_confidence, "bbox": image.bbox, + "bbox_normalized": normalize_asset_bbox(deck.source_type, page, image), "crop_source_path": image.crop_source_path, "crop_bbox": image.crop_bbox, "crop_quality": image.crop_quality, @@ -431,12 +434,3 @@ def _dedupe(values: list[str]) -> list[str]: result.append(normalized) seen.add(normalized) return result - - -def _int_or_none(value: Any) -> int | None: - if isinstance(value, bool): - return None - try: - return int(value) - except (TypeError, ValueError): - return None diff --git a/slidenote/utils.py b/slidenote/utils.py index 85136fe..4d46301 100644 --- a/slidenote/utils.py +++ b/slidenote/utils.py @@ -8,10 +8,12 @@ import subprocess import tempfile from pathlib import Path -from typing import Any, Iterable +from concurrent.futures import ThreadPoolExecutor, as_completed +from typing import Any, Callable, Iterable, Sequence, TypeVar from PIL import Image +from slidenote import geometry from slidenote.models import Deck, SlidePage @@ -86,6 +88,24 @@ def display_path(path: Path | None, output_root: Path | None) -> str | None: return str(path) +def as_int(value: Any, default: int = 0) -> int: + try: + return int(value) + except (TypeError, ValueError): + return default + + +def int_or_none(value: Any) -> int | None: + """Strict integer parsing for model output: ints and digit strings only (no bools/floats).""" + if isinstance(value, bool): + return None + if isinstance(value, int): + return value + if isinstance(value, str) and value.strip().isdigit(): + return int(value.strip()) + return None + + def as_float(value: Any, default: float | None = None) -> float | None: try: return float(value) @@ -149,20 +169,32 @@ def context_title(pages: list[SlidePage], index: int) -> str: def parse_json_object(text: str) -> dict[str, Any] | None: - cleaned = text.strip() + """Parse a model reply that should be a JSON object. + + Accepts fenced code blocks and replies with prose around the object; returns + ``None`` for anything that is not a JSON object (lists, strings, invalid JSON). + """ + cleaned = str(text or "").strip() if cleaned.startswith("```"): - cleaned = cleaned.strip("`") - if cleaned.lower().startswith("json"): - cleaned = cleaned[4:].strip() - try: - parsed = json.loads(cleaned) - except json.JSONDecodeError: - return None - return parsed if isinstance(parsed, dict) else None + cleaned = re.sub(r"^```(?:json)?\s*", "", cleaned, flags=re.IGNORECASE) + cleaned = re.sub(r"\s*```$", "", cleaned) + start = cleaned.find("{") + end = cleaned.rfind("}") + sliced = cleaned[start : end + 1] if 0 <= start < end else None + for candidate in (cleaned, sliced): + if not candidate: + continue + try: + parsed = json.loads(candidate) + except json.JSONDecodeError: + continue + if isinstance(parsed, dict): + return parsed + return None def looks_normalized(bbox: list[float]) -> bool: - return len(bbox) == 4 and all(-0.001 <= float(value) <= 1.001 for value in bbox) + return geometry.looks_normalized(bbox) def union_bbox(boxes: list[list[float]]) -> list[float]: @@ -189,12 +221,7 @@ def bbox_area(bbox: list[float]) -> float: def clamp_normalized_bbox(bbox: list[float]) -> list[float]: - x1, y1, x2, y2 = [max(0.0, min(1.0, float(value))) for value in bbox] - if x2 < x1: - x1, x2 = x2, x1 - if y2 < y1: - y1, y2 = y2, y1 - return [round(x1, 4), round(y1, 4), round(x2, 4), round(y2, 4)] + return geometry.clamp_bbox(bbox, 4) def layout_order_from_bbox(bbox: list[float] | None) -> float: @@ -204,7 +231,11 @@ def layout_order_from_bbox(bbox: list[float] | None) -> float: def file_sha256(path: Path) -> str: - return "sha256:" + hashlib.sha256(path.read_bytes().hex().encode("utf-8")).hexdigest() + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return "sha256:" + digest.hexdigest() def cleanup_temp_image(path: Path) -> None: @@ -232,7 +263,7 @@ def image_area(path: Path) -> int | None: return None -def prepare_image_for_api(path: Path, max_edge: int) -> tuple[Path, dict[str, Any]] | None: +def prepare_image_for_api(path: Path, max_edge: int, quality: int = 85) -> tuple[Path, dict[str, Any]] | None: try: with Image.open(path) as image: original = {"width": image.width, "height": image.height, "mode": image.mode, "format": image.format} @@ -243,7 +274,7 @@ def prepare_image_for_api(path: Path, max_edge: int) -> tuple[Path, dict[str, An tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".jpg") tmp_path = Path(tmp.name) tmp.close() - image.save(tmp_path, format="JPEG", quality=85, optimize=True) + image.save(tmp_path, format="JPEG", quality=quality, optimize=True) meta = { "original": original, "prepared": {"width": image.width, "height": image.height, "mime_type": "image/jpeg", "bytes": tmp_path.stat().st_size}, @@ -263,3 +294,64 @@ def page_by_id(deck: Deck, slide_id: int) -> SlidePage | None: def source_tokens(markdown: str) -> set[str]: return set(re.findall(r"\bs\d+_(?:t|tbl|img|fig)\d+\b", markdown)) + + +T = TypeVar("T") +R = TypeVar("R") + + +def run_target_jobs( + targets: Sequence[T], + process: Callable[[int, T], R], + *, + workers: int, + on_error: Callable[[int, T, Exception], R], + on_result: Callable[[R], None] | None = None, +) -> list[R]: + """Run ``process(index, target)`` for every target, serially or in a thread pool. + + A failing target is converted into a result by ``on_error`` so one bad image does + not abort the whole stage. If every target fails, the first error is re-raised so + configuration problems (missing keys, unreachable endpoint) stay visible. + """ + results: list[tuple[int, R]] = [] + errors: list[Exception] = [] + + def finish(index: int, target: T, run: Callable[[], R]) -> None: + try: + result = run() + except Exception as exc: # noqa: BLE001 - per-target isolation + errors.append(exc) + result = on_error(index, target, exc) + results.append((index, result)) + if on_result: + on_result(result) + + if workers <= 1: + for index, target in enumerate(targets): + finish(index, target, lambda index=index, target=target: process(index, target)) + else: + with ThreadPoolExecutor(max_workers=workers) as executor: + futures = {executor.submit(process, index, target): (index, target) for index, target in enumerate(targets)} + for future in as_completed(futures): + index, target = futures[future] + finish(index, target, future.result) + if targets and len(errors) == len(targets): + raise errors[0] + return [result for _, result in sorted(results, key=lambda item: item[0])] + + +def error_summary(exc: Exception) -> str: + return preview(f"{type(exc).__name__}: {exc}", 300) + + +def advance_progress(progress_callback: Callable[[dict[str, Any]], None] | None) -> Callable[[tuple[Any, ...]], None] | None: + """``on_result`` hook for run_target_jobs results shaped ``(index, target, record, ...)``.""" + if progress_callback is None: + return None + + def advance(result: tuple[Any, ...]) -> None: + _, target, record = result[:3] + progress_callback({"event": "advance", "record": record, "slide_id": target.slide_id}) + + return advance diff --git a/slidenote/vision.py b/slidenote/vision.py index dd9345e..77cd189 100644 --- a/slidenote/vision.py +++ b/slidenote/vision.py @@ -1,7 +1,5 @@ from __future__ import annotations -import json -from concurrent.futures import ThreadPoolExecutor, as_completed from dataclasses import asdict, dataclass from pathlib import Path from typing import Any, Callable @@ -9,10 +7,14 @@ from slidenote.image_ranking import sorted_images_by_importance from slidenote.llm import LLMClient, resolve_provider_runtime from slidenote.llm_cache import LLM_CACHE_SCHEMA_VERSION, LLMCache, make_cache_key, sha256_text, utc_now_iso -from slidenote.modality import page_has_hint +from slidenote.modality import page_has_hint, page_has_manual_modality from slidenote.models import Deck, SlidePage from slidenote.table_understanding import table_preview from slidenote.utils import ( + advance_progress, + parse_json_object, + error_summary, + run_target_jobs, cleanup_temp_image, display_path, file_sha256, @@ -97,25 +99,14 @@ def process(index: int, target: VisionTarget) -> tuple[int, VisionTarget, dict[s ) return index, target, record, parsed - results = [] - if workers == 1: - for index, target in enumerate(targets): - result = process(index, target) - results.append(result) - if progress_callback: - _, completed_target, record, _ = result - progress_callback({"event": "advance", "record": record, "slide_id": completed_target.slide_id}) - else: - with ThreadPoolExecutor(max_workers=workers) as executor: - futures = {executor.submit(process, index, target): (index, target) for index, target in enumerate(targets)} - for future in as_completed(futures): - result = future.result() - results.append(result) - if progress_callback: - _, completed_target, record, _ = result - progress_callback({"event": "advance", "record": record, "slide_id": completed_target.slide_id}) - - for index, target, record, parsed in sorted(results, key=lambda item: item[0]): + def failed(index: int, target: VisionTarget, exc: Exception) -> tuple[int, VisionTarget, dict[str, Any], dict[str, Any]]: + record = _skipped_record(target, "api_error") + record.update({"cache_status": "error", "error": error_summary(exc), "visual_status": "failed"}) + return index, target, record, {} + + results = run_target_jobs(targets, process, workers=workers, on_error=failed, on_result=advance_progress(progress_callback)) + + for index, target, record, parsed in results: _apply_visual_result( deck, target, @@ -145,12 +136,12 @@ def _process_visual_target( ) -> tuple[dict[str, Any], dict[str, Any]]: source_path = (output_root / target.path).resolve() if not source_path.exists(): - record = _skipped_record(target, "missing_file", output_root) + record = _skipped_record(target, "missing_file") record["visual_status"] = "missing_file" return record, {} prepared = prepare_image_for_api(source_path, max_edge=max_edge) if prepared is None: - record = _skipped_record(target, "unsupported_or_unreadable_image", output_root) + record = _skipped_record(target, "unsupported_or_unreadable_image") record["visual_status"] = "unsupported_or_unreadable_image" return record, {} @@ -199,6 +190,7 @@ def _process_visual_target( base_url=runtime["base_url"], max_output_tokens=max_output_tokens, temperature=temperature, + for_vision=True, ) llm_result = client.generate_image_with_usage(prepared_path, prompt, system_prompt=VISION_SYSTEM_PROMPT, image_detail=detail) result_json = llm_result.text @@ -251,6 +243,12 @@ def select_vision_targets( targets: list[VisionTarget] = [] for page in deck.pages: if mode == "auto": + if page_has_manual_modality(page): + if page.page_screenshot and page_has_hint(page, "vision_page_screenshot"): + targets.append(VisionTarget(page.slide_id, "page_screenshot", page.page_screenshot, reason="manual_modality")) + elif page_has_hint(page, "vision_large_images"): + targets.extend(_large_image_targets(page, output_root, min_area=min_area, first_only=True)) + continue figure_targets = _role_image_targets(page, output_root, role="figure_crop", min_area=0, first_only=False) if figure_targets: targets.extend(figure_targets) @@ -344,6 +342,12 @@ def _vision_prompt(target: VisionTarget, page: SlidePage | None = None) -> str: def _page_context(page: SlidePage | None, limit: int = 1200) -> str: + extra = [f"page_ocr_text：{page.page_ocr_text[:600]}"] if page and page.page_ocr_text and page.page_ocr_status else [] + return page_prompt_context(page, limit=limit, extra=extra) + + +def page_prompt_context(page: SlidePage | None, *, limit: int, extra: list[str] | None = None) -> str: + """Compact page text (title, first text blocks, tables) for vision prompts.""" if page is None: return "" pieces: list[str] = [] @@ -354,8 +358,7 @@ def _page_context(page: SlidePage | None, limit: int = 1200) -> str: for table in page.tables[:2]: preview = table_preview(table, limit=260, raw_rows=3) pieces.append(f"{table.id}(table)：{preview}") - if page.page_ocr_text and page.page_ocr_status: - pieces.append(f"page_ocr_text：{page.page_ocr_text[:600]}") + pieces.extend(extra or []) text = "\n".join(piece for piece in pieces if piece.strip()) if len(text) > limit: return text[: limit - 1] + "…" @@ -363,15 +366,9 @@ def _page_context(page: SlidePage | None, limit: int = 1200) -> str: def _parse_visual_json(text: str) -> dict[str, Any]: - cleaned = text.strip() - if cleaned.startswith("```"): - cleaned = cleaned.strip("`") - if cleaned.lower().startswith("json"): - cleaned = cleaned[4:].strip() - try: - parsed = json.loads(cleaned) - except json.JSONDecodeError: - parsed = {"ocr_text": "", "visual_summary": cleaned, "content_type": "unknown", "confidence": None, "warnings": ["model_output_not_json"]} + parsed = parse_json_object(text) + if parsed is None: + parsed = {"ocr_text": "", "visual_summary": text.strip(), "content_type": "unknown", "confidence": None, "warnings": ["model_output_not_json"]} parsed.setdefault("ocr_text", "") parsed.setdefault("visual_summary", "") parsed.setdefault("warnings", []) @@ -419,6 +416,7 @@ def _build_report( "llm_calls": sum(1 for record in records if record.get("llm_call")), "api_retries": sum(int(record.get("api_retries") or 0) for record in records), "skipped": sum(1 for record in records if record.get("cache_status") == "skipped"), + "failed": sum(1 for record in records if record.get("cache_status") == "error"), "input_tokens": sum_int(record.get("input_tokens") for record in records), "output_tokens": sum_int(record.get("output_tokens") for record in records), "total_tokens": sum_int(record.get("total_tokens") for record in records), @@ -455,7 +453,7 @@ def _base_record(target: VisionTarget, cache_key: str, cache_path: Path, output_ } -def _skipped_record(target: VisionTarget, status: str, output_root: Path) -> dict[str, Any]: +def _skipped_record(target: VisionTarget, status: str) -> dict[str, Any]: return { "slide_id": target.slide_id, "kind": target.kind, diff --git a/tests/test_build_artifacts.py b/tests/test_build_artifacts.py new file mode 100644 index 0000000..c81f136 --- /dev/null +++ b/tests/test_build_artifacts.py @@ -0,0 +1,53 @@ +import json + +from slidenote.build.artifacts import remove_stale_build_artifacts + + +def test_stale_artifacts_are_removed_but_cache_overrides_and_input_are_kept(tmp_path): + (tmp_path / "run_summary.json").write_text( + json.dumps({"schema_version": 1, "source_path": "lecture.pdf", "source_type": "pdf", "artifacts": {"content": "content.json"}}), + encoding="utf-8", + ) + (tmp_path / "vision_usage.json").write_text("{}", encoding="utf-8") + (tmp_path / "notes.assets").mkdir() + (tmp_path / "notes.assets" / "old.png").write_bytes(b"x") + (tmp_path / ".cache").mkdir() + (tmp_path / "page_modalities.overrides.json").write_text("{}", encoding="utf-8") + (tmp_path / "my_notes.txt").write_text("keep", encoding="utf-8") + images = tmp_path / "images" + images.mkdir() + lecture = images / "lecture.pdf" + lecture.write_bytes(b"%PDF") + + removed = remove_stale_build_artifacts(tmp_path, keep=(lecture,)) + + assert {"run_summary.json", "vision_usage.json", "notes.assets"} <= set(removed) + assert not (tmp_path / "vision_usage.json").exists() + assert (tmp_path / ".cache").is_dir() + assert (tmp_path / "page_modalities.overrides.json").exists() + assert (tmp_path / "my_notes.txt").exists() + assert lecture.exists() + + +def test_unrelated_output_directory_is_never_cleaned(tmp_path): + (tmp_path / "progress.json").write_text('{"status": "failed"}', encoding="utf-8") + (tmp_path / "run_summary.json").write_text("{}", encoding="utf-8") + (tmp_path / "notes.md").write_text("my own notes", encoding="utf-8") + (tmp_path / "images").mkdir() + (tmp_path / "images" / "photo.png").write_bytes(b"x") + + assert remove_stale_build_artifacts(tmp_path) == [] + assert (tmp_path / "notes.md").read_text(encoding="utf-8") == "my own notes" + assert (tmp_path / "images" / "photo.png").exists() + + +def test_partial_build_content_marker_is_cleaned(tmp_path): + (tmp_path / "content.json").write_text( + json.dumps({"source_path": "lecture.pdf", "source_type": "pdf", "pages": [{"slide_id": 1}]}), + encoding="utf-8", + ) + (tmp_path / "vision_usage.json").write_text("{}", encoding="utf-8") + + removed = remove_stale_build_artifacts(tmp_path) + + assert set(removed) == {"content.json", "vision_usage.json"} diff --git a/tests/test_build_robustness.py b/tests/test_build_robustness.py new file mode 100644 index 0000000..8a8318a --- /dev/null +++ b/tests/test_build_robustness.py @@ -0,0 +1,431 @@ +from __future__ import annotations + +import json +import sys +import urllib.error +from pathlib import Path +from types import SimpleNamespace + +import fitz +import pytest + +from slidenote.api_retry import is_transient_api_error +from slidenote.build.config import _apply_build_preset_defaults +from slidenote.cli import _build_parser, _explicit_cli_options, main +from slidenote.llm import LLMClient, resolve_provider_runtime +from slidenote.llm_cache import LLMCache, atomic_write_text +from slidenote.models import Deck, SlidePage, TextBlock +from slidenote.notes import generate_notes_result +from slidenote.notes.assets import _repair_markdown_image_links +from slidenote.notes.contexts import NoteContext +from slidenote.notes.repair import _repair_required_markdown_once + + +def _pdf(tmp_path: Path) -> Path: + source = tmp_path / "lecture.pdf" + doc = fitz.open() + page = doc.new_page() + page.insert_text((72, 72), "Transport Layer") + doc.save(source) + doc.close() + return source + + +def _preset_args(argv: list[str]): + args = _build_parser().parse_args(argv) + args._explicit_options = _explicit_cli_options(argv) + _apply_build_preset_defaults(args) + return args + + +# -- bug 1: stale artifacts ------------------------------------------------- + + +def test_build_removes_stale_generated_artifacts_but_keeps_cache_and_overrides(tmp_path): + source = _pdf(tmp_path) + out = tmp_path / "out" + (out / "notes.assets" / "images").mkdir(parents=True) + (out / "notes.assets" / "images" / "old.png").write_bytes(b"old") + (out / ".cache" / "llm").mkdir(parents=True) + (out / ".cache" / "llm" / "entry.json").write_text("{}", encoding="utf-8") + (out / "run_summary.json").write_text( + json.dumps({"schema_version": 1, "source_path": str(source), "source_type": "pdf", "artifacts": {"content": "content.json"}}), + encoding="utf-8", + ) + (out / "vision_usage.json").write_text('{"summary": {"api_calls": 9}}', encoding="utf-8") + (out / "cost_report.json").write_text("{}", encoding="utf-8") + (out / "page_modalities.overrides.json").write_text('{"schema_version": 1, "pages": {}}', encoding="utf-8") + (out / "my_notes.txt").write_text("user file", encoding="utf-8") + + assert main(["build", str(source), "--out", str(out), "--quiet", "--preset", "local"]) == 0 + + assert not (out / "vision_usage.json").exists() + assert not (out / "cost_report.json").exists() + assert not (out / "notes.assets" / "images" / "old.png").exists() + assert (out / ".cache" / "llm" / "entry.json").exists() + assert (out / "page_modalities.overrides.json").exists() + assert (out / "my_notes.txt").read_text(encoding="utf-8") == "user file" + assert (out / "notes.md").exists() + + +def test_stale_artifact_cleanup_never_removes_the_input_file(tmp_path): + out = tmp_path / "out" + (out / "images").mkdir(parents=True) + source = _pdf(out / "images") + + assert main(["build", str(source), "--out", str(out), "--quiet", "--preset", "local"]) == 0 + assert source.exists() + + +# -- bug 2: retry classification --------------------------------------------- + + +@pytest.mark.parametrize(("code", "transient"), [(400, False), (401, False), (403, False), (404, False), (429, True), (503, True)]) +def test_http_error_status_decides_retry(code, transient): + error = urllib.error.HTTPError("https://api.test", code, "msg", {}, None) + assert is_transient_api_error(error) is transient + wrapped = RuntimeError(f"LLM request failed with HTTP {code}: body") + wrapped.__cause__ = error + assert is_transient_api_error(wrapped) is transient + + +def test_wrapped_connection_error_is_transient(): + wrapped = RuntimeError("LLM request failed: ") + wrapped.__cause__ = urllib.error.URLError("refused") + assert is_transient_api_error(wrapped) is True + + +@pytest.mark.parametrize( + "message", + ["prompt has 500 tokens over the limit", "invalid model gpt-4o-2024-05-13 (id 503)", "max_tokens must be <= 429"], +) +def test_bare_numbers_in_messages_are_not_status_codes(message): + assert is_transient_api_error(ValueError(message)) is False + + +def test_status_in_message_requires_status_context(): + assert is_transient_api_error(RuntimeError("HTTP 502 bad gateway")) is True + assert is_transient_api_error(RuntimeError("status code: 401 unauthorized")) is False + + +# -- bug 3: one SDK client, SDK retries disabled ------------------------------- + + +def test_openai_client_is_reused_and_sdk_retries_disabled(monkeypatch): + constructed: list[dict] = [] + response = SimpleNamespace( + choices=[SimpleNamespace(message=SimpleNamespace(content="ok"), finish_reason="stop")], + usage={"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}, + ) + + def fake_openai(**kwargs): + constructed.append(kwargs) + return SimpleNamespace(chat=SimpleNamespace(completions=SimpleNamespace(create=lambda **_: response))) + + monkeypatch.setitem(sys.modules, "openai", SimpleNamespace(OpenAI=fake_openai)) + client = LLMClient(provider="openai", model="m", api_key="k") + client.generate_with_usage("a") + client.generate_with_usage("b") + + assert len(constructed) == 1 + assert constructed[0]["max_retries"] == 0 + + +# -- bug 4: generic overrides apply to the text role only ---------------------- + + +def test_generic_model_and_base_url_overrides_do_not_leak_into_vision(monkeypatch): + monkeypatch.setenv("SLIDENOTE_MODEL", "deepseek-text") + monkeypatch.setenv("SLIDENOTE_BASE_URL", "https://text.example") + for name in ("QWEN_VISION_MODEL", "DASHSCOPE_VISION_MODEL", "SLIDENOTE_VISION_MODEL", "QWEN_BASE_URL", "DASHSCOPE_BASE_URL"): + monkeypatch.delenv(name, raising=False) + + vision = resolve_provider_runtime("qwen", for_vision=True) + text = resolve_provider_runtime("qwen") + + assert vision["model"] == "qwen-vl-plus" + assert vision["base_url"] == "https://dashscope.aliyuncs.com/compatible-mode/v1" + assert text["model"] == "deepseek-text" + assert text["base_url"] == "https://text.example" + client = LLMClient(provider="openai", model="gpt-4.1-mini", api_key="k", for_vision=True) + assert client.base_url is None + + +# -- bug 5 / 6: local preset conflicts and --vision off -------------------------- + + +def test_local_preset_warns_about_conflicting_explicit_flags(): + args = _preset_args(["build", "lecture.pdf", "--preset", "local", "--vision", "auto", "--ocr", "all"]) + + assert args.vision == "off" + assert args.ocr == "off" + assert any("--vision" in warning for warning in args._config_warnings) + assert any("--ocr" in warning for warning in args._config_warnings) + + +def test_local_preset_without_conflict_has_no_warning(): + args = _preset_args(["build", "lecture.pdf", "--preset", "local", "--vision", "off"]) + assert args._config_warnings == [] + + +def test_local_preset_conflict_is_printed_and_recorded(tmp_path, capsys): + source = _pdf(tmp_path) + out = tmp_path / "out" + + assert main(["build", str(source), "--out", str(out), "--quiet", "--preset", "local", "--vision", "auto"]) == 0 + + assert "--vision" in capsys.readouterr().err + run_summary = json.loads((out / "run_summary.json").read_text(encoding="utf-8")) + assert any("--vision" in warning for warning in run_summary["warnings"]["config"]) + + +def test_vision_off_downgrades_vision_dependent_modes(): + from slidenote.build.config import BUILD_PRESET_DEFAULTS + + lecture = BUILD_PRESET_DEFAULTS["lecture"] + original = dict(lecture) + try: + lecture.update(figure_grounding="vision", semantic_layout="vision", figure_crop="vision") + args = _preset_args(["build", "lecture.pdf", "--vision", "off"]) + finally: + lecture.clear() + lecture.update(original) + + assert args.figure_grounding == "auto" + assert args.semantic_layout == "local" + assert args.figure_crop == "off" + + +def test_vision_stage_is_not_planned_when_vision_is_off(): + from slidenote.build.stages import BUILD_PHASES + + vision_step = next(step for phase in BUILD_PHASES for step in phase.steps if step.name == "vision") + state = SimpleNamespace(args=SimpleNamespace(vision="off", figure_grounding="vision")) + assert vision_step.enabled(state) is False + + +# -- bug 7: ./ prefix handling ------------------------------------------------- + + +def test_image_link_repair_does_not_strip_parent_directory(tmp_path): + asset_map = {"images/x.png": "notes.assets/images/x.png"} + markdown = "![a](../x.png)\n\n![b](./images/x.png)\n" + + repaired = _repair_markdown_image_links(markdown, tmp_path, asset_map) + + assert "![b](notes.assets/images/x.png)" in repaired + # `../x.png` is a different file; it may only be rewritten via the by-name + # fallback, never by treating it as `x.png` through character stripping. + assert "](../x.png)" in repaired or "](notes.assets/images/x.png)" in repaired + from slidenote.notes.assets import _asset_link_rewrite_maps + + exact, _ = _asset_link_rewrite_maps({"../shared/x.png": "notes.assets/images/x.png"}) + assert "shared/x.png" not in exact + assert "../shared/x.png" in exact + + +# -- bug 8: repair refresh and per-context final repair ------------------------ + + +def _guard(items: list[tuple[str, int]]) -> dict: + return { + "required_confidence_threshold": 0.7, + "summary": {"repair_attempts": 0, "required_missing": 0, "residual_risks": 0}, + "pages": [], + "items": [ + {"element_id": element_id, "slide_id": slide_id, "learning_role": "definition", + "must_explain": True, "confidence": 0.95, "reason": "learning content"} + for element_id, slide_id in items + ], + "repairs": [], + } + + +TEXTS = { + "s1_t1": "Alpha replicas acknowledge each committed write before the client proceeds.", + "s2_t1": "Beta quorums overlap so that every read observes the latest committed write.", + "s2_t2": "Gamma majority writes prevent two conflicting values from being accepted together.", +} + + +def _para(element_id: str) -> str: + slide = element_id[1] + return f"{TEXTS[element_id]} " + + +def _two_page_deck() -> Deck: + return Deck( + source_path="lecture.pdf", + source_type="pdf", + pages=[ + SlidePage(slide_id=1, title="Alpha", text_blocks=[TextBlock(id="s1_t1", type="paragraph", content=TEXTS["s1_t1"])]), + SlidePage(slide_id=2, title="Beta", text_blocks=[ + TextBlock(id="s2_t1", type="paragraph", content=TEXTS["s2_t1"]), + TextBlock(id="s2_t2", type="paragraph", content=TEXTS["s2_t2"]), + ]), + ], + ) + + +def test_repair_propagates_force_refresh(monkeypatch, tmp_path): + deck = _two_page_deck() + captured = {} + + def fake_generate(**kwargs): + captured.update(kwargs) + return _para("s2_t1") + "\n\n" + _para("s2_t2"), {"llm_call": True} + + monkeypatch.setattr("slidenote.notes.repair._generate_cached_llm_text", fake_generate) + from slidenote.notes import NoteOptions + + _repair_required_markdown_once( + deck=Deck(source_path="lecture.pdf", source_type="pdf", pages=[deck.pages[1]]), + context=NoteContext(id="p2", kind="page", title="Beta", pages=[deck.pages[1]]), + markdown=_para("s2_t1"), + output_root=tmp_path, + cache=LLMCache(tmp_path / "cache", mode="off"), + options=NoteOptions(content_guard=_guard([("s2_t2", 2)]), cache_mode="off"), + stage="final", + force_refresh=True, + ) + + assert captured["force_refresh"] is True + + +def test_repair_skips_input_that_cannot_fit_output_budget(monkeypatch, tmp_path): + deck = _two_page_deck() + monkeypatch.setattr( + "slidenote.notes.repair._generate_cached_llm_text", + lambda **kwargs: pytest.fail("over-long repair must not be sent"), + ) + from slidenote.notes import NoteOptions + + original = _para("s2_t1") + "\n\n" + ("filler text " * 400) + markdown, record = _repair_required_markdown_once( + deck=Deck(source_path="lecture.pdf", source_type="pdf", pages=[deck.pages[1]]), + context=NoteContext(id="final", kind="final", title="final", pages=[deck.pages[1]]), + markdown=original, + output_root=tmp_path, + cache=LLMCache(tmp_path / "cache", mode="off"), + options=NoteOptions(content_guard=_guard([("s2_t2", 2)]), cache_mode="off", max_output_tokens=500), + stage="final", + ) + + assert markdown == original + assert record["accepted"] is False + assert record["rejection_reasons"] == ["input_too_long_for_output_budget"] + + +def test_final_repair_only_sends_the_context_with_missing_items(monkeypatch, tmp_path): + repair_prompts: list[str] = [] + + class FakeClient: + def __init__(self, **kwargs): + pass + + def generate_with_usage(self, prompt): + if '"task": "repair_required_learning_coverage"' in prompt: + repair_prompts.append(prompt) + text = _para("s2_t1") + "\n\n" + _para("s2_t2") + elif "s2_t1" in prompt: + text = _para("s2_t1") + else: + text = _para("s1_t1") + return SimpleNamespace(text=text, usage={"input_tokens": 1, "output_tokens": 1, "total_tokens": 2}) + + monkeypatch.setattr("slidenote.notes.llm_calls.LLMClient", FakeClient) + guard = _guard([("s1_t1", 1), ("s2_t2", 2)]) + result = generate_notes_result( + _two_page_deck(), tmp_path, use_llm=True, provider="openai", api_key="test", + note_strategy="direct", note_context="page", content_guard=guard, cache_mode="off", + ) + + assert len(repair_prompts) == 1 + assert "Alpha replicas" not in repair_prompts[0] + assert TEXTS["s2_t2"] in result.markdown + assert TEXTS["s1_t1"] in result.markdown + assert guard["repairs"][0]["accepted"] is True + assert guard["repairs"][0]["slide_ids"] == [2] + + +# -- bug 9: setup errors --------------------------------------------------------- + + +def test_missing_input_is_a_friendly_error_recorded_in_progress(tmp_path, capsys): + out = tmp_path / "out" + + assert main(["build", str(tmp_path / "missing.pdf"), "--out", str(out), "--quiet"]) == 2 + + assert "Input file not found" in capsys.readouterr().err + progress = json.loads((out / "progress.json").read_text(encoding="utf-8")) + assert progress["status"] == "failed" + assert "Input file not found" in progress["message"] + + +# -- bug 15: atomic writes and per-context failure fallback ----------------------- + + +def test_atomic_write_replaces_content_without_leaving_temp_files(tmp_path): + target = tmp_path / "sub" / "progress.json" + atomic_write_text(target, "first") + atomic_write_text(target, "second") + + assert target.read_text(encoding="utf-8") == "second" + assert [path.name for path in target.parent.iterdir()] == ["progress.json"] + + +def test_atomic_write_retries_windows_permission_error(tmp_path, monkeypatch): + import os + + calls = {"count": 0} + real_replace = os.replace + + def flaky_replace(src, dst): + calls["count"] += 1 + if calls["count"] == 1: + raise PermissionError("locked by reader") + return real_replace(src, dst) + + monkeypatch.setattr("slidenote.llm_cache.os.replace", flaky_replace) + monkeypatch.setattr("slidenote.llm_cache.time.sleep", lambda _seconds: None) + atomic_write_text(tmp_path / "x.json", "ok") + + assert (tmp_path / "x.json").read_text(encoding="utf-8") == "ok" + assert calls["count"] == 2 + + +@pytest.mark.parametrize("concurrency", [1, 2]) +def test_failed_context_falls_back_to_local_notes(monkeypatch, tmp_path, concurrency): + class FlakyClient: + def __init__(self, **kwargs): + pass + + def generate_with_usage(self, prompt): + if "s2_t1" in prompt: + raise RuntimeError("HTTP 500 upstream exploded") + return SimpleNamespace(text=_para("s1_t1"), usage={"total_tokens": 2}) + + monkeypatch.setattr("slidenote.notes.llm_calls.LLMClient", FlakyClient) + result = generate_notes_result( + _two_page_deck(), tmp_path, use_llm=True, provider="openai", api_key="test", + note_strategy="direct", note_context="page", cache_mode="off", concurrency=concurrency, + ) + + assert TEXTS["s1_t1"] in result.markdown + assert TEXTS["s2_t1"] in result.markdown + assert result.llm_usage["summary"]["failed_contexts"] == 1 + assert result.generation_warnings and "p2" in result.generation_warnings[0] + assert "upstream exploded" not in json.dumps(result.llm_usage) + + +def test_all_contexts_failing_still_raises(monkeypatch, tmp_path): + class BrokenClient: + def __init__(self, **kwargs): + raise RuntimeError("Missing API key for provider `openai`.") + + monkeypatch.setattr("slidenote.notes.llm_calls.LLMClient", BrokenClient) + with pytest.raises(RuntimeError, match="Missing API key"): + generate_notes_result( + _two_page_deck(), tmp_path, use_llm=True, provider="openai", api_key="test", + note_strategy="direct", note_context="page", cache_mode="off", + ) diff --git a/tests/test_cli.py b/tests/test_cli.py index 0a4b04e..b75ba6e 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -1,7 +1,7 @@ +import hashlib import json import zipfile from argparse import Namespace -from pathlib import Path import fitz import pytest @@ -72,6 +72,9 @@ def test_build_writes_progress_and_run_summary(tmp_path): composite_figures = json.loads((out / "composite_figures.json").read_text(encoding="utf-8")) figure_grounding = json.loads((out / "figure_grounding.json").read_text(encoding="utf-8")) assert progress["status"] == "complete" + assert progress["planned_stages"][0] == "parse" + assert "ocr" not in progress["planned_stages"] + assert "vision" not in progress["planned_stages"] assert run_summary["counts"]["pages"] == 1 assert run_summary["artifacts"]["progress"] == "progress.json" assert run_summary["artifacts"]["source_map"] == "source_map.json" @@ -138,6 +141,30 @@ def test_build_writes_progress_and_run_summary(tmp_path): assert not (out / "export_report.json").exists() +def test_build_applies_saved_page_modality_correction(tmp_path): + source = tmp_path / "lecture.pdf" + doc = fitz.open() + page = doc.new_page() + page.insert_text((72, 72), "A short native-text page") + doc.save(source) + doc.close() + out = tmp_path / "out" + out.mkdir() + (out / "page_modalities.overrides.json").write_text(json.dumps({ + "schema_version": 1, + "source_sha256": hashlib.sha256(source.read_bytes()).hexdigest(), + "pages": {"1": {"modality": "image_only", "note": "Check screenshot"}}, + }), encoding="utf-8") + + assert main(["build", str(source), "--out", str(out), "--quiet", "--preset", "local"]) == 0 + + modalities = json.loads((out / "page_modalities.json").read_text(encoding="utf-8")) + content = json.loads((out / "content.json").read_text(encoding="utf-8")) + assert modalities["summary"]["override_pages"] == 1 + assert modalities["pages"][0]["modality"] == "image_only" + assert content["pages"][0]["page_modality"] == "image_only" + + def test_internal_quality_concurrency_is_wired_to_build_stages(tmp_path, monkeypatch): source = tmp_path / "lecture.pdf" doc = fitz.open() @@ -605,7 +632,7 @@ def test_study_pack_command_generates_local_review_and_exam_pack(tmp_path): assert (out / "final_exam.answers.md").exists() assert (out / "wrong_answer_review_prompt.md").exists() assert "review_mode" not in run_summary["run"] - assert run_summary["artifacts"]["study_pack"] is None + assert run_summary["artifacts"].get("study_pack") is None assert study_pack["summary"]["questions_total"] == 4 assert study_pack["question_quality"]["overall_score"] >= 0 diff --git a/tests/test_content_guard.py b/tests/test_content_guard.py index cf97696..45c1f89 100644 --- a/tests/test_content_guard.py +++ b/tests/test_content_guard.py @@ -1,5 +1,3 @@ -import json - from slidenote.content_guard import build_content_guard, missing_required_items from slidenote.coverage import analyze_coverage from slidenote.models import Deck, SlidePage, TextBlock @@ -92,3 +90,36 @@ def test_missing_required_items_merges_guard_metadata(): assert missing[0]["element_id"] == "s1_t1" assert missing[0]["learning_role"] == "definition" assert missing[0]["coverage"]["marker_only"] is True + + +def _guard_role(content: str) -> str | None: + from slidenote.content_guard import _text_candidate + + page = SlidePage(slide_id=1) + candidate = _text_candidate(page, TextBlock(id="s1_t1", type="paragraph", content=content)) + return candidate.local_role if candidate else None + + +def test_condition_and_formula_heuristics_ignore_common_false_positives(): + assert _guard_role("当前我们先介绍背景") is None + assert _guard_role("这两种方法相当接近") is None + assert _guard_role("详见 https://example.com/page?id=3&x=1") is None + assert _guard_role("重点 内容") is None + assert _guard_role("系统的 consistency 很重要") is None + assert _guard_role("当队列为空时，消费者阻塞。") == "condition" + assert _guard_role("x = 3") == "formula" + assert _guard_role("f(x) >= 0") == "formula" + assert _guard_role("TCP 是指传输控制协议") == "definition" + + +def test_record_repair_counts_attempts_rejections_and_residual_risks(): + from slidenote.content_guard import content_guard_warnings, record_repair + + report = {"summary": {"residual_risks": 1}} + record_repair(report, {"accepted": True, "unresolved_items": []}) + record_repair(report, {"accepted": False, "unresolved_items": ["s1_t1", "s2_t1"]}) + + assert report["summary"]["repair_attempts"] == 2 + assert report["summary"]["repair_rejections"] == 1 + assert report["summary"]["residual_risks"] == 3 + assert "content_guard_repair_rejected:1" in content_guard_warnings(report) diff --git a/tests/test_costing.py b/tests/test_costing.py index e3d5173..f22e575 100644 --- a/tests/test_costing.py +++ b/tests/test_costing.py @@ -60,3 +60,56 @@ def test_write_cost_report_outputs_three_files(tmp_path: Path): assert (out / "cost_dashboard.html").exists() assert report["summary"]["calls"] == 3 assert report["summary"]["estimated_cost_usd"] == 0.603 + + +def test_cost_report_includes_stage_reports_with_embedded_usage(tmp_path: Path): + out = tmp_path / "out" + out.mkdir() + reports = { + "sections.json": { + "llm": {"provider": "deepseek", "model": "deepseek-v4-flash"}, + "summary": {"llm_call": True, "local_cache_hits": 0, "input_tokens": 100, "output_tokens": 10, "total_tokens": 110}, + }, + "deck_brief.json": { + "llm": {"provider": "deepseek", "model": "deepseek-v4-flash"}, + "summary": {"llm_call": False, "local_cache_hits": 1, "input_tokens": 0, "output_tokens": 0, "total_tokens": 0}, + }, + "content_guard.json": { + "llm": {"provider": "deepseek", "model": "deepseek-v4-flash", "llm_call": True, "cache_status": "miss", + "input_tokens": 200, "output_tokens": 20, "total_tokens": 220}, + }, + "semantic_layout.json": { + "vision_enhancement": {"provider": "qwen", "model": "qwen-vl-plus"}, + "summary": {"vision_calls": 2, "vision_cache_hits": 1, "input_tokens": 300, "output_tokens": 30, "total_tokens": 330}, + }, + "figure_grounding.json": { + "vision_grounding": {"provider": None, "model": None}, + "summary": {"vision_calls": 0, "input_tokens": 0}, + }, + } + for name, data in reports.items(): + (out / name).write_text(json.dumps(data), encoding="utf-8") + + report = build_cost_report(out) + + stages = {stage["name"]: stage for stage in report["stages"]} + assert set(stages) == {"sections", "deck_brief", "content_guard", "semantic_layout"} + assert stages["sections"]["calls"] == 1 + assert stages["deck_brief"]["calls"] == 0 + assert stages["deck_brief"]["local_cache_hits"] == 1 + assert stages["content_guard"]["input_tokens"] == 200 + assert stages["semantic_layout"]["provider"] == "qwen" + assert report["summary"]["calls"] == 4 + assert report["summary"]["input_tokens"] == 600 + + +def test_pricing_exchange_rates_merge_with_default_cny(tmp_path: Path): + out = tmp_path / "out" + out.mkdir() + pricing = tmp_path / "pricing.json" + pricing.write_text(json.dumps({"exchange_rates": {"EUR": 0.9}}), encoding="utf-8") + assert build_cost_report(out, pricing, currency="CNY")["exchange_rate_from_usd"] == 7.2 + assert build_cost_report(out, pricing, currency="EUR")["exchange_rate_from_usd"] == 0.9 + + pricing.write_text(json.dumps({"exchange_rates": {"CNY": 7.0}}), encoding="utf-8") + assert build_cost_report(out, pricing, currency="CNY")["exchange_rate_from_usd"] == 7.0 diff --git a/tests/test_exports.py b/tests/test_exports.py index c148ab1..cf21c5e 100644 --- a/tests/test_exports.py +++ b/tests/test_exports.py @@ -127,7 +127,8 @@ def fake_run(command, cwd=None, text=None, stdout=None, stderr=None, check=None) assert report["summary"]["blocking_failures"] == 1 assert report["results"][0]["format"] == "pdf" assert report["results"][0]["reason"] == "libreoffice_not_found" - assert (tmp_path / "notes.docx").exists() + # PDF-only exports build notes.docx in a temp dir; no unreported artifact is left behind. + assert not (tmp_path / "notes.docx").exists() assert not (tmp_path / "notes.pdf").exists() @@ -139,3 +140,77 @@ def test_pandoc_missing_marks_requested_formats_failed(tmp_path, monkeypatch): assert report["summary"]["failed"] == 2 assert report["summary"]["blocking_failures"] == 2 assert all(result["reason"] == "pandoc_not_found" for result in report["results"]) + + +def test_markdown_zip_uses_passed_markdown_not_stale_notes_file(tmp_path, monkeypatch): + monkeypatch.setattr("slidenote.exporting.shutil.which", lambda name: None) + (tmp_path / "notes.md").write_text("# Stale notes\n", encoding="utf-8") + + report = build_export_artifacts("# Fresh notes\n", tmp_path, ["markdown-zip"]) + + assert report["summary"]["succeeded"] == 1 + with zipfile.ZipFile(tmp_path / "notes.zip") as archive: + assert archive.read("notes.md").decode("utf-8") == "# Fresh notes\n" + assert (tmp_path / "notes.md").read_text(encoding="utf-8") == "# Stale notes\n" + + +def test_latex_only_export_does_not_build_docx_or_pass_pdf_engine(tmp_path, monkeypatch): + commands = [] + + def fake_run(command, cwd=None, text=None, stdout=None, stderr=None, check=None): + commands.append(command) + (tmp_path / command[command.index("-o") + 1]).write_bytes(b"tex") + return subprocess.CompletedProcess(command, 0, stdout="", stderr="") + + monkeypatch.setattr("slidenote.exporting.shutil.which", lambda name: "pandoc" if name == "pandoc" else None) + monkeypatch.setattr("slidenote.exporting.subprocess.run", fake_run) + + report = build_export_artifacts("# Lecture\n\n## Topic\n", tmp_path, ["latex"]) + + assert report["summary"]["succeeded"] == 1 + assert len(commands) == 1 + assert commands[0][commands[0].index("-o") + 1] == "notes.tex" + assert not any(token.startswith("--pdf-engine") for token in commands[0]) + assert not (tmp_path / "notes.docx").exists() + + +def test_pandoc_nonzero_return_is_reported_and_pdf_depends_on_it(tmp_path, monkeypatch): + def fake_run(command, cwd=None, text=None, stdout=None, stderr=None, check=None): + return subprocess.CompletedProcess(command, 3, stdout="", stderr="pandoc: boom") + + monkeypatch.setattr("slidenote.exporting.shutil.which", lambda name: {"pandoc": "pandoc", "soffice": "soffice"}.get(name)) + monkeypatch.setattr("slidenote.exporting.subprocess.run", fake_run) + + report = build_export_artifacts("# Lecture\n", tmp_path, ["docx", "pdf"]) + + by_format = {result["format"]: result for result in report["results"]} + assert by_format["docx"]["status"] == "failed" + assert by_format["docx"]["reason"] == "pandoc_failed" + assert by_format["docx"]["returncode"] == 3 + assert "boom" in by_format["docx"]["stderr"] + assert by_format["pdf"]["reason"] == "docx_required_failed" + assert report["summary"]["blocking_failures"] == 2 + assert any("docx export failed" in warning for warning in report["warnings"]) + + +def test_pdf_only_export_converts_temp_docx_with_libreoffice(tmp_path, monkeypatch): + converted = [] + + def fake_run(command, cwd=None, text=None, stdout=None, stderr=None, check=None): + if command[0] == "pandoc": + (tmp_path / command[command.index("-o") + 1]).write_bytes(b"docx") + return subprocess.CompletedProcess(command, 0, stdout="", stderr="") + converted.append(command[-1]) + (tmp_path / "notes.pdf").write_bytes(b"pdf") + return subprocess.CompletedProcess(command, 0, stdout="", stderr="") + + monkeypatch.setattr("slidenote.exporting.shutil.which", lambda name: {"pandoc": "pandoc", "soffice": "soffice"}.get(name)) + monkeypatch.setattr("slidenote.exporting.subprocess.run", fake_run) + + report = build_export_artifacts("# Lecture\n", tmp_path, ["pdf"]) + + assert report["summary"]["succeeded"] == 1 + assert [result["format"] for result in report["results"]] == ["pdf"] + assert converted and not converted[0].startswith(str(tmp_path)) + assert (tmp_path / "notes.pdf").exists() + assert not (tmp_path / "notes.docx").exists() diff --git a/tests/test_figure_grounding.py b/tests/test_figure_grounding.py index 179bfa8..5719458 100644 --- a/tests/test_figure_grounding.py +++ b/tests/test_figure_grounding.py @@ -201,3 +201,16 @@ class Result: image = deck.pages[0].images[0] assert image.anchor_element_ids == ["s3_t1"] assert report["summary"]["vision_fallback_images"] == 1 + + +def test_generic_auto_captions_are_not_used_as_explanations_or_anchors(): + from slidenote.figure_grounding import _figure_explanation, _is_generic_caption, _tokens + from slidenote.models import ImageAsset + + assert _is_generic_caption("第 3 页局部图 2") + assert _is_generic_caption("第 3 页组合图 1") + assert not _is_generic_caption("TCP 三次握手时序图") + crop = ImageAsset(id="s3_fig2", path="figures/a.png", caption="第 3 页局部图 2") + assert _figure_explanation(crop) == (None, "missing") + assert not _tokens("第三章概述").intersection(_tokens("这是章节")) + assert _tokens("三次握手流程").intersection(_tokens("握手阶段")) diff --git a/tests/test_figures.py b/tests/test_figures.py index 83c4874..28aade7 100644 --- a/tests/test_figures.py +++ b/tests/test_figures.py @@ -323,10 +323,14 @@ def test_figure_enrichment_uses_vision_model_and_cache(tmp_path, monkeypatch): screenshot = tmp_path / "screenshots" / "slide1.png" screenshot.parent.mkdir() Image.new("RGB", (1000, 600), "white").save(screenshot) + monkeypatch.setenv("SLIDENOTE_BASE_URL", "https://text-only.example/v1") + monkeypatch.delenv("SLIDENOTE_VISION_BASE_URL", raising=False) class FakeFigureClient: def __init__(self, **kwargs): - pass + from slidenote.llm import LLMClient + + assert LLMClient(**kwargs).base_url is None def generate_image_with_usage(self, image_path: Path, prompt: str, system_prompt: str, image_detail: str): class Result: @@ -421,3 +425,17 @@ def test_composite_figures_crop_cluster_and_absorb_children(tmp_path): children = [image for image in page.images if image.role == "composite_child"] assert len(children) == 4 assert all(image.ignored for image in children) + + +def test_foreground_mask_matches_per_pixel_rule(): + import random + + from slidenote.figures import _foreground_mask, _is_foreground_pixel + + rng = random.Random(7) + image = Image.new("RGB", (41, 19)) + pixels = [tuple(rng.choice([rng.randint(0, 255), rng.randint(215, 255)]) for _ in range(3)) for _ in range(41 * 19)] + image.putdata(pixels) + for background in [(255, 255, 255), (236, 236, 236), (20, 30, 40)]: + expected = bytes(255 if _is_foreground_pixel(pixel, background) else 0 for pixel in pixels) + assert _foreground_mask(image, background) == expected diff --git a/tests/test_geometry.py b/tests/test_geometry.py new file mode 100644 index 0000000..a0bf4f2 --- /dev/null +++ b/tests/test_geometry.py @@ -0,0 +1,31 @@ +from slidenote.geometry import asset_source_bbox, normalize_asset_bbox, normalize_bbox +from slidenote.models import ImageAsset, SlidePage +from slidenote.utils import file_sha256 + + +def test_normalize_bbox_handles_pptx_xywh_and_pdf_xyxy(): + assert normalize_bbox("pptx", [100, 50, 200, 100], 1000, 500) == [0.1, 0.1, 0.3, 0.3] + assert normalize_bbox("pdf", [100, 50, 200, 100], 1000, 500) == [0.1, 0.1, 0.2, 0.2] + assert normalize_bbox("pdf", [0.2, 0.1, 0.4, 0.3], None, None) == [0.2, 0.1, 0.4, 0.3] + assert normalize_bbox("pdf", [100, 50, 200, 100], None, 500) is None + + +def test_screenshot_crop_pixel_bbox_is_never_treated_as_page_coordinates(): + page = SlidePage(slide_id=1, page_width=720, page_height=540) + crop = ImageAsset( + id="s1_fig1", + path="figures/crop.png", + bbox=[288.0, 216.0, 1152.0, 864.0], + crop_source_path="screenshots/slide-1.png", + crop_bbox=[0.2, 0.2, 0.8, 0.8], + ) + assert normalize_asset_bbox("pdf", page, crop) == [0.2, 0.2, 0.8, 0.8] + + crop.crop_bbox = None + assert asset_source_bbox(crop) is None + + +def test_file_sha256_hashes_file_bytes(tmp_path): + path = tmp_path / "data.bin" + path.write_bytes(b"abc") + assert file_sha256(path) == "sha256:ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" diff --git a/tests/test_image_assets.py b/tests/test_image_assets.py index 55d79cc..978e62b 100644 --- a/tests/test_image_assets.py +++ b/tests/test_image_assets.py @@ -2,8 +2,7 @@ from slidenote.image_assets import classify_image_asset, refine_image_role_for_placement from slidenote.image_ranking import rank_deck_images, sorted_images_by_importance -from slidenote.extractors.pdf import _is_page_like_bbox -from slidenote.extractors.pptx import _is_page_like_shape +from slidenote.geometry import placement_metrics from slidenote.models import Deck, ImageAsset, SlidePage @@ -40,13 +39,16 @@ def test_refine_keeps_medium_edge_content_image(): def test_pdf_page_like_bbox_detection(): - assert _is_page_like_bbox([0, 0, 950, 950], (1000, 1000)) is True - assert _is_page_like_bbox([100, 100, 500, 500], (1000, 1000)) is False + assert placement_metrics("pdf", [0, 0, 950, 950], 1000, 1000)[2] is True + assert placement_metrics("pdf", [100, 100, 500, 500], 1000, 1000)[2] is False def test_pptx_page_like_shape_detection(): - assert _is_page_like_shape([0, 0, 950, 950], 1000, 1000) is True - assert _is_page_like_shape([0, 0, 400, 400], 1000, 1000) is False + assert placement_metrics("pptx", [0, 0, 950, 950], 1000, 1000)[2] is True + assert placement_metrics("pptx", [0, 0, 400, 400], 1000, 1000)[2] is False + area_ratio, near_edge, _ = placement_metrics("pptx", [450, 450, 50, 50], 1000, 1000) + assert round(area_ratio, 4) == 0.0025 + assert near_edge is False def test_image_importance_ranks_figure_crop_above_generic_image(tmp_path): diff --git a/tests/test_llm.py b/tests/test_llm.py index bb490f3..58fec2d 100644 --- a/tests/test_llm.py +++ b/tests/test_llm.py @@ -5,7 +5,28 @@ from slidenote.llm import LLMClient, LLMResult, get_provider_spec, resolve_provider_runtime from slidenote.models import ImageAsset, SlidePage -from slidenote.notes.prompts import _llm_page_prompt +from slidenote.notes.contexts import NoteContext +from slidenote.notes.prompt_templates import _llm_context_prompt + + +def _llm_page_prompt(page: SlidePage, supports_image_input: bool = False) -> str: + """Single-page note prompt with default options (test convenience).""" + context = NoteContext(id=f"p{page.slide_id}", kind="page", title=page.title or f"第 {page.slide_id} 页", pages=[page]) + return _llm_context_prompt( + context, + supports_image_input=supports_image_input, + asset_map={}, + source_display="hidden", + note_context="page", + note_style="article", + note_profile="auto", + note_depth="detailed", + note_language="zh", + term_policy="bilingual", + screenshot_policy="fallback", + figure_placement="inline", + source_type="pdf", + ) def test_provider_aliases(): diff --git a/tests/test_modality.py b/tests/test_modality.py index 197b6d4..4230bbe 100644 --- a/tests/test_modality.py +++ b/tests/test_modality.py @@ -1,5 +1,11 @@ -from slidenote.modality import classify_page_modality, enrich_deck_with_modalities +import hashlib +import json + +from slidenote.figures import select_figure_targets +from slidenote.modality import apply_modality_overrides, classify_page_modality, enrich_deck_with_modalities from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock +from slidenote.ocr import select_ocr_targets +from slidenote.vision import select_vision_targets def test_modality_detects_mixed_page_with_embedded_content_image(): @@ -63,3 +69,143 @@ def test_enrich_deck_with_modalities_writes_page_fields_and_report(): assert deck.pages[1].page_modality == "image_only" assert report["summary"]["pages_total"] == 2 assert report["summary"]["ocr_recommended_pages"] == 1 + + +def test_manual_image_only_override_updates_report_and_visual_routing(tmp_path): + original_source = tmp_path / "first-upload.pdf" + new_source = tmp_path / "second-upload.pdf" + original_source.write_bytes(b"same deck bytes") + new_source.write_bytes(original_source.read_bytes()) + page = SlidePage( + slide_id=1, + text_blocks=[TextBlock(id="s1_t1", type="paragraph", content="A" * 1000)], + page_screenshot="screenshots/slide1.png", + ) + deck = Deck(source_path=str(new_source), source_type="pdf", pages=[page]) + report = enrich_deck_with_modalities(deck) + assert page.page_modality == "native_text" + + manifest_path = tmp_path / "page_modalities.overrides.json" + manifest_path.write_text(json.dumps({ + "schema_version": 1, + "source_path": str(original_source), + "source_sha256": hashlib.sha256(original_source.read_bytes()).hexdigest(), + "pages": {"1": {"modality": "image_only", "note": "Scanned screenshot", "updated_at": "2026-09-23T10:00:00Z"}}, + }), encoding="utf-8") + + updated = apply_modality_overrides(deck, report, manifest_path) + + assert updated is report + assert page.page_modality == "image_only" + assert page.modality_reasons == ["manual_override"] + assert {"ocr_page_screenshot", "vision_page_screenshot", "crop_figures_from_screenshot"} <= set(page.processing_hints) + assert report["summary"]["modalities"] == {"image_only": 1} + assert report["summary"]["ocr_recommended_pages"] == 1 + assert report["summary"]["override_pages"] == 1 + assert report["pages"][0]["classifier_modality"] == "native_text" + assert report["pages"][0]["manual_override"]["note"] == "Scanned screenshot" + assert [target.slide_id for target in select_ocr_targets(deck, tmp_path, mode="auto")] == [1] + assert [target.slide_id for target in select_vision_targets(deck, tmp_path, mode="auto")] == [1] + assert [target.slide_id for target in select_figure_targets(deck)] == [1] + + +def test_manual_native_text_override_suppresses_auto_visual_routing(tmp_path): + page = SlidePage(slide_id=1, page_screenshot="screenshots/slide1.png") + deck = Deck(source_path=str(tmp_path / "lecture.pdf"), source_type="pdf", pages=[page]) + report = enrich_deck_with_modalities(deck) + assert page.page_modality == "image_only" + manifest_path = tmp_path / "page_modalities.overrides.json" + manifest_path.write_text(json.dumps({ + "schema_version": 1, + "pages": {"1": {"modality": "native_text", "note": "Reviewed page"}}, + }), encoding="utf-8") + + apply_modality_overrides(deck, report, manifest_path) + + assert page.page_modality == "native_text" + assert select_ocr_targets(deck, tmp_path, mode="auto") == [] + assert select_vision_targets(deck, tmp_path, mode="auto") == [] + assert select_figure_targets(deck) == [] + assert report["pages"][0]["classifier_modality"] == "image_only" + assert report["pages"][0]["manual_override"]["note"] == "Reviewed page" + + +def test_manual_image_only_override_uses_embedded_image_when_no_screenshot(tmp_path): + page = SlidePage( + slide_id=1, + text_blocks=[TextBlock(id="s1_t1", type="paragraph", content="A" * 1000)], + images=[ImageAsset(id="s1_img1", path="images/diagram.png")], + ) + deck = Deck(source_path=str(tmp_path / "lecture.pptx"), source_type="pptx", pages=[page]) + report = enrich_deck_with_modalities(deck) + manifest_path = tmp_path / "page_modalities.overrides.json" + manifest_path.write_text(json.dumps({ + "schema_version": 1, + "pages": {"1": {"modality": "image_only"}}, + }), encoding="utf-8") + + apply_modality_overrides(deck, report, manifest_path) + + assert "ocr_page_screenshot" in page.processing_hints + assert [target.image_id for target in select_ocr_targets(deck, tmp_path, mode="auto")] == ["s1_img1"] + assert [target.slide_id for target in select_vision_targets(deck, tmp_path, mode="auto")] == [1] + + +def test_manual_unknown_preserves_auto_visual_routing(tmp_path): + page = SlidePage(slide_id=1, page_screenshot="screenshots/slide1.png") + deck = Deck(source_path=str(tmp_path / "lecture.pdf"), source_type="pdf", pages=[page]) + report = enrich_deck_with_modalities(deck) + original_hints = list(page.processing_hints) + manifest_path = tmp_path / "page_modalities.overrides.json" + manifest_path.write_text(json.dumps({ + "schema_version": 1, + "pages": {"1": {"modality": "unknown"}}, + }), encoding="utf-8") + + apply_modality_overrides(deck, report, manifest_path) + + assert page.page_modality == "unknown" + assert page.modality_confidence == 0.0 + assert page.processing_hints == original_hints + assert [target.slide_id for target in select_ocr_targets(deck, tmp_path, mode="auto")] == [1] + assert [target.slide_id for target in select_vision_targets(deck, tmp_path, mode="auto")] == [1] + assert [target.slide_id for target in select_figure_targets(deck)] == [1] + + +def test_manual_override_skips_invalid_pages_and_modalities(tmp_path): + deck = Deck(source_path="lecture.pdf", source_type="pdf", pages=[SlidePage(slide_id=1, page_screenshot="s1.png")]) + report = enrich_deck_with_modalities(deck) + manifest_path = tmp_path / "page_modalities.overrides.json" + manifest_path.write_text(json.dumps({ + "schema_version": 1, + "pages": { + "0": {"modality": "decorative"}, + "2": {"modality": "decorative"}, + "1": {"modality": []}, + }, + }), encoding="utf-8") + + apply_modality_overrides(deck, report, manifest_path) + + assert deck.pages[0].page_modality == "image_only" + assert report["summary"]["override_pages"] == 0 + assert len(report["overrides"]["warnings"]) == 3 + + +def test_manual_override_rejects_mismatched_source_hash(tmp_path): + source = tmp_path / "deck.pdf" + source.write_bytes(b"current deck") + deck = Deck(source_path=str(source), source_type="pdf", pages=[SlidePage(slide_id=1)]) + report = enrich_deck_with_modalities(deck) + manifest_path = tmp_path / "page_modalities.overrides.json" + manifest_path.write_text(json.dumps({ + "schema_version": 1, + "source_sha256": hashlib.sha256(b"different deck").hexdigest(), + "pages": {"1": {"modality": "image_only"}}, + }), encoding="utf-8") + + apply_modality_overrides(deck, report, manifest_path) + + assert deck.pages[0].page_modality == "decorative" + assert report["summary"]["override_pages"] == 0 + assert "does not match" in report["overrides"]["warnings"][0] diff --git a/tests/test_note_repair.py b/tests/test_note_repair.py index 9073754..63df885 100644 --- a/tests/test_note_repair.py +++ b/tests/test_note_repair.py @@ -10,7 +10,7 @@ from slidenote.llm_cache import LLMCache from slidenote.models import Deck, SlidePage, TextBlock from slidenote.notes import NoteOptions, generate_notes_result -from slidenote.notes.assembly import NoteContext +from slidenote.notes.contexts import NoteContext from slidenote.notes.repair import _repair_required_markdown_once diff --git a/tests/test_notes.py b/tests/test_notes.py index c4d964f..6f8bd11 100644 --- a/tests/test_notes.py +++ b/tests/test_notes.py @@ -3,11 +3,47 @@ from slidenote.coverage import analyze_coverage from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock from slidenote.notes import generate_notes, generate_notes_result -from slidenote.notes.assembly import _postprocess_llm_markdown +from slidenote.notes.contexts import NoteContext +from slidenote.notes.postprocess import _postprocess_llm_markdown +from slidenote.notes.prompt_templates import _llm_context_prompt from slidenote.semantic_layout import enrich_deck_with_semantic_layout from slidenote.table_understanding import enrich_deck_with_table_understanding +def _llm_page_prompt(page: SlidePage, supports_image_input: bool = False) -> str: + """Single-page note prompt with default options (test convenience).""" + context = NoteContext(id=f"p{page.slide_id}", kind="page", title=page.title or f"第 {page.slide_id} 页", pages=[page]) + return _llm_context_prompt( + context, + supports_image_input=supports_image_input, + asset_map={}, + source_display="hidden", + note_context="page", + note_style="article", + note_profile="auto", + note_depth="detailed", + note_language="zh", + term_policy="bilingual", + screenshot_policy="fallback", + figure_placement="inline", + source_type="pdf", + ) + + +def test_legacy_generate_notes_returns_markdown(tmp_path): + deck = Deck( + source_path="lecture.pptx", + source_type="pptx", + pages=[SlidePage(slide_id=1, text_blocks=[TextBlock(id="s1_t1", type="paragraph", content="TCP")])], + ) + + markdown = generate_notes(deck, tmp_path, False, "openai", source_display="inline") + + assert isinstance(markdown, str) + assert "TCP" in markdown + assert "【对应 PPT" in markdown + + def test_local_notes_include_all_element_ids(): deck = Deck( source_path="lecture.pptx", @@ -26,7 +62,7 @@ def test_local_notes_include_all_element_ids(): ], ) - notes = generate_notes(deck, Path("out")) + notes = generate_notes_result(deck, Path("out")).markdown report = analyze_coverage(deck, notes) assert "本页主题是“Transport”。" in notes @@ -58,7 +94,7 @@ def test_local_notes_use_table_conclusion_before_raw_cells(): ) enrich_deck_with_table_understanding(deck) - notes = generate_notes(deck, Path("out")) + notes = generate_notes_result(deck, Path("out")).markdown assert "表格结论" in notes assert "关键行" in notes @@ -95,7 +131,7 @@ def test_composite_figure_source_marker_covers_child_images(tmp_path): ], ) - notes = generate_notes(deck, tmp_path) + notes = generate_notes_result(deck, tmp_path).markdown source_map_report = analyze_coverage(deck, notes) assert "" in notes @@ -125,7 +161,7 @@ def test_local_notes_include_ocr_and_visual_fields(): ], ) - notes = generate_notes(deck, Path("out")) + notes = generate_notes_result(deck, Path("out")).markdown assert "页截图视觉解析" in notes assert "截图展示了客户端与服务端之间的握手流程" in notes @@ -150,7 +186,7 @@ def test_local_notes_show_image_ocr_when_no_visual_explanation(): ], ) - notes = generate_notes(deck, Path("out")) + notes = generate_notes_result(deck, Path("out")).markdown assert "图片 OCR 文字" in notes assert "cout << value;" in notes @@ -171,7 +207,7 @@ def test_local_notes_skip_ignored_images_in_coverage(): ], ) - notes = generate_notes(deck, Path("out")) + notes = generate_notes_result(deck, Path("out")).markdown report = analyze_coverage(deck, notes) assert "s1_img1" not in notes @@ -465,7 +501,7 @@ def test_source_display_footnote_keeps_clean_page_reference(): pages=[SlidePage(slide_id=4, text_blocks=[TextBlock(id="s4_t1", type="paragraph", content="复制提高可靠性")])], ) - notes = generate_notes(deck, Path("out"), source_display="footnote") + notes = generate_notes_result(deck, Path("out"), source_display="footnote").markdown assert "（PPT 第 4 页）" in notes assert "" in notes @@ -511,7 +547,7 @@ class Result: cache_dir=tmp_path / "cache", note_strategy="direct", ) - assert first.llm_usage["pages"][0]["cache_status"] == "miss" + assert first.llm_usage["contexts"][0]["cache_status"] == "miss" assert first.llm_usage["summary"]["llm_calls"] == 1 assert "详细讲义式学习笔记" in prompts[0] assert "好的，这是" not in first.markdown @@ -534,12 +570,11 @@ def __init__(self, **kwargs): cache_dir=tmp_path / "cache", note_strategy="direct", ) - assert second.llm_usage["pages"][0]["cache_status"] == "local_hit" + assert second.llm_usage["contexts"][0]["cache_status"] == "local_hit" assert second.llm_usage["summary"]["llm_calls"] == 0 def test_llm_prompt_uses_page_visual_summary(): - from slidenote.notes.prompts import _llm_page_prompt page = SlidePage(slide_id=3, page_visual_summary="图中展示 TCP 三次握手流程。") @@ -550,7 +585,6 @@ def test_llm_prompt_uses_page_visual_summary(): def test_llm_prompt_includes_table_understanding_fields(): - from slidenote.notes.prompts import _llm_page_prompt table = TableBlock( id="s1_tbl1", @@ -568,7 +602,6 @@ def test_llm_prompt_includes_table_understanding_fields(): def test_llm_prompt_includes_semantic_layout_groups(): - from slidenote.notes.prompts import _llm_page_prompt page = SlidePage( slide_id=1, @@ -589,8 +622,8 @@ def test_llm_prompt_includes_semantic_layout_groups(): def test_article_prompt_prefers_study_notes_over_slide_translation(): - from slidenote.notes.assembly import NoteContext - from slidenote.notes.prompts import _llm_page_lecture_prompt + from slidenote.notes.contexts import NoteContext + from slidenote.notes.prompt_templates import _llm_page_lecture_prompt deck = Deck( source_path="lecture.pdf", @@ -640,8 +673,8 @@ def test_article_prompt_prefers_study_notes_over_slide_translation(): def test_lecture_notes_profile_prompt_requests_teaching_reconstruction(): - from slidenote.notes.assembly import NoteContext - from slidenote.notes.prompts import _llm_page_lecture_prompt + from slidenote.notes.contexts import NoteContext + from slidenote.notes.prompt_templates import _llm_page_lecture_prompt deck = Deck( source_path="lecture.pdf", @@ -997,6 +1030,58 @@ class Result: assert analyze_coverage(deck, result.markdown)["missing"] == 0 +def test_lecture_auto_skips_teaching_pass_when_weave_already_has_teaching_content(tmp_path, monkeypatch): + deck = Deck( + source_path="lecture.pdf", + source_type="pdf", + pages=[SlidePage(slide_id=1, title="Quorum", text_blocks=[TextBlock(id="s1_t1", type="paragraph", content="Read and write quorums overlap.")])], + ) + tasks = [] + + class FakeClient: + def __init__(self, **kwargs): + pass + + def generate_with_usage(self, prompt): + class Result: + usage = {"input_tokens": 3, "output_tokens": 4, "total_tokens": 7} + + result = Result() + if '"task": "page_lecture"' in prompt: + tasks.append("page") + result.text = "Read and write quorums overlap. " + elif '"task": "weave_page_lectures"' in prompt: + tasks.append("weave") + result.text = ( + "A read quorum intersects a write quorum, so a read can observe the latest write. " + "For example, three replicas can use overlapping sets to preserve visibility. " + "A common mistake is assuming that any two sets of replicas will intersect. " + "Review question: what happens when a read set misses the last write set? " + "" + ) + else: + raise AssertionError("The complete weave should not trigger another model call") + return result + + monkeypatch.setattr("slidenote.notes.llm_calls.LLMClient", FakeClient) + + result = generate_notes_result( + deck, + tmp_path, + use_llm=True, + provider="openai", + api_key="test", + note_strategy="lecture-weave", + note_profile="lecture-notes", + note_context="document", + ) + + assert tasks == ["page", "weave"] + assert result.teaching_report is None + assert result.llm_usage["summary"]["teaching_enrichment_calls"] == 0 + assert "Review question" in result.markdown + + def test_lecture_weave_prompt_uses_deck_brief_as_guarded_navigation(tmp_path, monkeypatch): deck = Deck( source_path="lecture.pdf", diff --git a/tests/test_ocr.py b/tests/test_ocr.py index e76213a..0d73c6f 100644 --- a/tests/test_ocr.py +++ b/tests/test_ocr.py @@ -146,3 +146,43 @@ def recognize(self, image_path: Path): enrich_deck_with_ocr(deck, tmp_path, mode="auto", provider="baidu", api_key="k", secret_key="s", cache_dir=tmp_path / "cache") assert seen["path"].exists() is False + + +def test_ocr_enrichment_records_single_target_failure_without_aborting(tmp_path, monkeypatch): + shots = tmp_path / "screenshots" + shots.mkdir() + for slide_id in (1, 2): + Image.new("RGB", (800, 450), "white").save(shots / f"slide{slide_id}.png") + deck = Deck( + source_path="lecture.pdf", + source_type="pdf", + pages=[SlidePage(slide_id=slide_id, page_screenshot=f"screenshots/slide{slide_id}.png") for slide_id in (1, 2)], + ) + created = {"count": 0} + + class PartlyFailingOCRClient: + def __init__(self, **kwargs): + created["count"] += 1 + + def recognize(self, image_path: Path): + if created.setdefault("calls", 0) == 0: + created["calls"] = 1 + raise ValueError("bad image") + + class Result: + text = "UDP" + usage = {} + raw = {} + + return Result() + + monkeypatch.setattr("slidenote.ocr.OCRClient", PartlyFailingOCRClient) + + report = enrich_deck_with_ocr(deck, tmp_path, mode="auto", provider="baidu", api_key="k", secret_key="s", cache_dir=tmp_path / "cache") + + assert created["count"] == 1 + assert report["summary"]["failed"] == 1 + assert report["targets"][0]["cache_status"] == "error" + assert "bad image" in report["targets"][0]["error"] + assert deck.pages[0].page_ocr_status == "failed" + assert deck.pages[1].page_ocr_text == "UDP" diff --git a/tests/test_parser_adapters.py b/tests/test_parser_adapters.py index 852f1f6..c8bda57 100644 --- a/tests/test_parser_adapters.py +++ b/tests/test_parser_adapters.py @@ -177,3 +177,37 @@ def test_external_cli_adapter_normalizes_generic_pages_stdout(tmp_path, monkeypa assert deck.pages[0].slide_id == 1 assert deck.pages[0].title == "Transport Layer" assert deck.pages[0].text_blocks[0].content == "Transport Layer" + + +def test_generic_json_does_not_duplicate_nested_text_or_scan_tables_as_text(tmp_path): + from slidenote.parser_adapters import _deck_from_generic_json + + data = { + "pages": [ + { + "page": 1, + "blocks": [ + {"type": "section", "text": "Parent", "children": [{"type": "paragraph", "text": "Parent"}]}, + {"type": "table", "rows": [["A", "B"], ["1", "2"]], "children": [{"type": "cell", "text": "A"}]}, + {"type": "picture", "path": "img.png", "text": "caption"}, + ], + } + ] + } + + deck = _deck_from_generic_json(data, tmp_path / "x.pdf", tmp_path, tmp_path) + page = deck.pages[0] + + assert [block.content for block in page.text_blocks] == ["Parent"] + assert len(page.tables) == 1 + assert len(page.images) == 1 + + +def test_command_template_keeps_spaced_paths_and_literal_braces(tmp_path): + from pathlib import Path + + from slidenote.parser_adapters import _command_from_template + + command = _command_from_template("tool --pages {page} {input} -o {out}", Path("my deck.pdf"), Path("out dir")) + + assert command == ["tool", "--pages", "{page}", "my deck.pdf", "-o", "out dir"] diff --git a/tests/test_pipeline.py b/tests/test_pipeline.py index bdc3e00..6ec8e78 100644 --- a/tests/test_pipeline.py +++ b/tests/test_pipeline.py @@ -1,45 +1,45 @@ -from argparse import Namespace - -from slidenote.models import Deck -from slidenote.pipeline import ArtifactRegistry, BuildContext, FunctionStage, StageResult, run_stage +from types import SimpleNamespace + +from slidenote.pipeline import ArtifactRegistry, BuildPhase, BuildStep, run_build_plan +from slidenote.progress import ProgressReporter + + +def test_build_plan_runs_enabled_steps_in_phase_order(tmp_path): + progress = ProgressReporter(tmp_path / "progress.json", quiet=True) + state = SimpleNamespace(progress=progress, calls=[]) + + def step(name, phase): + def run(current_state): + assert current_state.progress.current_phase == phase + current_state.progress.start_stage(name) + current_state.calls.append(name) + current_state.progress.finish_stage() + return run + + phases = ( + BuildPhase("ingest", (BuildStep("parse", step("parse", "ingest")),)), + BuildPhase("understand", ( + BuildStep("ocr", step("ocr", "understand"), enabled=lambda _: False), + BuildStep("layout", step("layout", "understand")), + )), + BuildPhase("write", ( + BuildStep("notes", step("notes", "write")), + BuildStep("summary", lambda current_state: current_state.calls.append("summary"), tracks_progress=False), + )), + ) + run_build_plan(state, phases) -def test_function_stage_records_result_and_registered_artifact(tmp_path): - deck = Deck(source_path="demo.pdf", source_type="pdf", pages=[]) - registry = ArtifactRegistry(tmp_path) - context = BuildContext( - args=Namespace(), - input_path=tmp_path / "demo.pdf", - output_root=tmp_path, - progress=None, - artifacts=registry, - ) + assert state.calls == ["parse", "layout", "notes", "summary"] + snapshot = progress.snapshot() + assert snapshot["planned_stages"] == state.calls[:-1] + assert [stage["name"] for stage in snapshot["stages"]] == state.calls[:-1] + assert snapshot["current_phase"] is None - def runner(stage_deck, stage_context): - stage_context.artifacts.write_json("demo", "demo.json", {"source_type": stage_deck.source_type}) - return StageResult(name="demo_stage", report={"ok": True}, artifacts={"demo": "demo.json"}) - result = run_stage(deck, context, FunctionStage(name="demo_stage", runner=runner, artifacts=["demo"])) +def test_artifact_registry_records_written_file(tmp_path): + registry = ArtifactRegistry(tmp_path) + registry.write_json("demo", "demo.json", {"ok": True}) - assert result.report == {"ok": True} - assert context.reports["demo_stage"] is result assert registry.as_summary()["demo"] == "demo.json" assert (tmp_path / "demo.json").exists() - - -def test_stage_dependencies_are_checked(tmp_path): - deck = Deck(source_path="demo.pdf", source_type="pdf", pages=[]) - context = BuildContext( - args=Namespace(), - input_path=tmp_path / "demo.pdf", - output_root=tmp_path, - progress=None, - ) - stage = FunctionStage(name="needs_parse", dependencies=["parse"], runner=lambda *_: {}) - - try: - run_stage(deck, context, stage) - except RuntimeError as exc: - assert "depends on missing stage" in str(exc) - else: - raise AssertionError("missing dependency should fail") diff --git a/tests/test_rendering.py b/tests/test_rendering.py new file mode 100644 index 0000000..51326bf --- /dev/null +++ b/tests/test_rendering.py @@ -0,0 +1,160 @@ +from __future__ import annotations + +import sys +import types +from pathlib import Path + +import pytest + +from slidenote import rendering + + +class FakeSlide: + def Export(self, path: str, fmt: str) -> None: + Path(path).write_bytes(b"png") + + +class FakeSlides: + def __init__(self, count: int) -> None: + self.Count = count + + def __call__(self, index: int) -> FakeSlide: + return FakeSlide() + + +class FakePresentation: + def __init__(self, app: "FakeApp", count: int) -> None: + self.app = app + self.Slides = FakeSlides(count) + self.closed = False + + def Close(self) -> None: + self.closed = True + self.app.open_presentations -= 1 + + +class FakePresentations: + def __init__(self, app: "FakeApp") -> None: + self.app = app + + @property + def Count(self) -> int: + return self.app.open_presentations + + def Open(self, path: str, WithWindow: bool = True) -> FakePresentation: + self.app.open_presentations += 1 + self.app.opened.append(FakePresentation(self.app, 2)) + return self.app.opened[-1] + + +class FakeApp: + def __init__(self, open_presentations: int = 0) -> None: + self.open_presentations = open_presentations + self.opened: list[FakePresentation] = [] + self.quit_called = False + self.Presentations = FakePresentations(self) + + def Quit(self) -> None: + self.quit_called = True + + +def _install_fake_win32com(monkeypatch, app: FakeApp, running: bool) -> None: + client = types.ModuleType("win32com.client") + client.Dispatch = lambda name: app + + def get_active_object(name): + if not running: + raise OSError("not running") + return app + + client.GetActiveObject = get_active_object + package = types.ModuleType("win32com") + package.client = client + monkeypatch.setitem(sys.modules, "win32com", package) + monkeypatch.setitem(sys.modules, "win32com.client", client) + + +def test_powerpoint_export_quits_only_instance_it_started(tmp_path, monkeypatch): + app = FakeApp() + _install_fake_win32com(monkeypatch, app, running=False) + shots = tmp_path / "screenshots" + shots.mkdir() + + result = rendering._render_with_powerpoint(tmp_path / "deck.pptx", shots, tmp_path) + + assert result == {1: "screenshots/slide1.png", 2: "screenshots/slide2.png"} + assert app.opened[0].closed + assert app.quit_called + + +def test_powerpoint_export_leaves_users_running_powerpoint_open(tmp_path, monkeypatch): + app = FakeApp(open_presentations=1) # the user's own deck + _install_fake_win32com(monkeypatch, app, running=True) + shots = tmp_path / "screenshots" + shots.mkdir() + + result = rendering._render_with_powerpoint(tmp_path / "deck.pptx", shots, tmp_path) + + assert len(result) == 2 + assert app.opened[0].closed + assert not app.quit_called + assert app.open_presentations == 1 + + +def test_pptx_screenshots_warn_when_no_renderer_available(tmp_path, monkeypatch): + monkeypatch.setattr(rendering, "_render_with_powerpoint", lambda *args: {}) + monkeypatch.setattr(rendering, "find_executable", lambda names: None) + + result, warnings = rendering.render_pptx_screenshots(tmp_path / "deck.pptx", tmp_path / "shots", tmp_path) + + assert result == {} + assert any("LibreOffice/PowerPoint not found" in warning for warning in warnings) + + +def test_pptx_screenshots_report_libreoffice_failure(tmp_path, monkeypatch): + monkeypatch.setattr(rendering, "_render_with_powerpoint", lambda *args: {}) + monkeypatch.setattr(rendering, "find_executable", lambda names: "soffice") + + def failing_run(args, cwd=None): + raise RuntimeError("convert crashed") + + monkeypatch.setattr(rendering, "run_command", failing_run) + + result, warnings = rendering.render_pptx_screenshots(tmp_path / "deck.pptx", tmp_path / "shots", tmp_path) + + assert result == {} + assert any("convert crashed" in warning for warning in warnings) + + +def test_pptx_screenshots_render_libreoffice_pdf_pages(tmp_path, monkeypatch): + fitz = pytest.importorskip("fitz") + monkeypatch.setattr(rendering, "_render_with_powerpoint", lambda *args: {}) + monkeypatch.setattr(rendering, "find_executable", lambda names: "soffice") + input_path = tmp_path / "deck.pptx" + + def fake_convert(args, cwd=None): + outdir = Path(args[args.index("--outdir") + 1]) + doc = fitz.open() + for _ in range(3): + doc.new_page(width=200, height=120) + doc.save(outdir / f"{input_path.stem}.pdf") + doc.close() + + monkeypatch.setattr(rendering, "run_command", fake_convert) + + result, warnings = rendering.render_pptx_screenshots(input_path, tmp_path / "shots", tmp_path) + + assert warnings == [] + assert result == {index: f"shots/slide{index}.png" for index in (1, 2, 3)} + assert all((tmp_path / "shots" / f"slide{index}.png").exists() for index in (1, 2, 3)) + + +def test_pptx_screenshots_warn_when_libreoffice_produces_no_pdf(tmp_path, monkeypatch): + monkeypatch.setattr(rendering, "_render_with_powerpoint", lambda *args: {}) + monkeypatch.setattr(rendering, "find_executable", lambda names: "soffice") + monkeypatch.setattr(rendering, "run_command", lambda args, cwd=None: None) + + result, warnings = rendering.render_pptx_screenshots(tmp_path / "deck.pptx", tmp_path / "shots", tmp_path) + + assert result == {} + assert any("did not produce a PDF" in warning for warning in warnings) diff --git a/tests/test_sections.py b/tests/test_sections.py index e5774cb..357907e 100644 --- a/tests/test_sections.py +++ b/tests/test_sections.py @@ -105,3 +105,50 @@ def __init__(self, **kwargs): assert second["summary"]["local_cache_hits"] == 1 assert second["summary"]["llm_call"] is False assert len(calls) == 1 + + +def _four_page_deck() -> Deck: + return Deck( + source_path="lecture.pdf", + source_type="pdf", + pages=[ + SlidePage(slide_id=index, title=title, text_blocks=[TextBlock(id=f"s{index}_t1", type="paragraph", content=title)]) + for index, title in enumerate(["Intro", "Replication", "Quorum", "Summary"], start=1) + ], + ) + + +def test_llm_section_plan_falls_back_to_local_plan_when_call_fails(tmp_path, monkeypatch): + class BrokenClient: + def __init__(self, **kwargs): + pass + + def generate_with_usage(self, prompt, system_prompt=None): + raise RuntimeError("HTTP 401 unauthorized") + + monkeypatch.setattr("slidenote.sections.LLMClient", BrokenClient) + plan = build_section_plan(_four_page_deck(), tmp_path, mode="llm", use_llm=True, provider="openai", api_key="bad", cache_dir=tmp_path / "cache") + + assert plan["method"] == "local_fallback" + assert "llm_section_call_failed" in plan["warnings"] + assert "401" in plan["llm"]["error"] + assert plan["sections"] + + +def test_llm_section_plan_accepts_numeric_string_slide_ids(tmp_path, monkeypatch): + class StringIdClient: + def __init__(self, **kwargs): + pass + + def generate_with_usage(self, prompt, system_prompt=None): + class Result: + text = '{"sections":[{"title":"Intro","start_slide_id":"1"},{"title":"Quorum","start_slide_id":"3"}]}' + usage = {} + + return Result() + + monkeypatch.setattr("slidenote.sections.LLMClient", StringIdClient) + plan = build_section_plan(_four_page_deck(), tmp_path, mode="llm", use_llm=True, provider="openai", api_key="k", cache_dir=tmp_path / "cache") + + assert plan["method"] == "llm" + assert [section["slide_ids"] for section in plan["sections"]] == [[1, 2], [3, 4]] diff --git a/tests/test_semantic_layout.py b/tests/test_semantic_layout.py index 3e0cb5d..41ed8bf 100644 --- a/tests/test_semantic_layout.py +++ b/tests/test_semantic_layout.py @@ -53,7 +53,7 @@ def test_semantic_layout_groups_code_output_cause_and_fix(): assert roles["s1_t4"] == "cause" assert roles["s1_t5"] == "fix" assert any(relation["relation"] == "fixes" for relation in page.semantic_relations) - assert "cin 提取运算符" in group["learning_goal"] + assert "换行符" in group["learning_goal"] assert semantic_layout_for_prompt(page)["groups"][0]["scene_type"] == "code_causal_explanation" assert "code_causal_explanation" in semantic_context_for_page(page) @@ -154,3 +154,18 @@ def __init__(self, **kwargs): assert second["summary"]["vision_cache_hits"] == 1 assert second["summary"]["vision_calls"] == 0 + + +def test_code_and_role_heuristics_do_not_fire_on_ordinary_prose(): + from slidenote.semantic_layout import _contains_cause_signal, _contains_fix_signal, _looks_like_code, _looks_like_output + + for prose in ["The data point shows a vaccine trial.", "A scout counts the print runs.", "所以这一节需要先回顾之前的定义。"]: + assert not _looks_like_code(prose) + assert not _looks_like_output(prose) + assert not _contains_cause_signal("所以这一节需要先回顾之前的定义。") + assert not _contains_fix_signal("必须掌握之后的内容，before class 预习。") + assert _looks_like_code("int count = 0;") + assert _looks_like_code("cout << value;") + assert _looks_like_output("Enter your name: Alice") + assert _contains_cause_signal("因为缓冲区里残留了换行符") + assert _contains_fix_signal("调用 ignore 清空缓冲区即可解决") diff --git a/tests/test_source_map.py b/tests/test_source_map.py index 019cbc2..5a8de46 100644 --- a/tests/test_source_map.py +++ b/tests/test_source_map.py @@ -1,5 +1,3 @@ -from pathlib import Path - from slidenote.models import Deck, ImageAsset, SlidePage, TableBlock, TextBlock from slidenote.source_map import build_source_map from slidenote.table_understanding import enrich_deck_with_table_understanding @@ -127,3 +125,37 @@ def test_source_map_includes_table_understanding_metadata(tmp_path): assert ref["element_id"] == "s1_tbl1" assert "TCP" in ref["table_conclusion"] assert ref["key_rows"][0]["label"] == "TCP" + + +def test_source_map_page_element_ir_uses_deck_source_type_for_pptx_bbox(tmp_path): + deck = Deck( + source_path="lecture.pptx", + source_type="pptx", + pages=[ + SlidePage( + slide_id=1, + page_width=1000, + page_height=500, + text_blocks=[TextBlock(id="s1_t1", type="paragraph", content="TCP", bbox=[100, 50, 200, 100])], + ) + ], + ) + + source_map = build_source_map(deck, "TCP ", tmp_path) + + element = source_map["pages"][0]["element_ir"]["elements"][0] + assert element["bbox_format"] == "source_xywh" + assert element["bbox_normalized"] == [0.1, 0.1, 0.3, 0.3] + + +def test_source_map_page_element_ir_includes_coverage_state(tmp_path): + deck = Deck( + source_path="lecture.pdf", + source_type="pdf", + pages=[SlidePage(slide_id=1, text_blocks=[TextBlock(id="s1_t1", type="paragraph", content="TCP")])], + ) + coverage = {"items": [{"id": "s1_t1", "trace_covered": True, "visible_covered": True}]} + + source_map = build_source_map(deck, "TCP", tmp_path, coverage_report=coverage) + + assert source_map["pages"][0]["element_ir"]["elements"][0]["coverage_state"] == "visible_covered" diff --git a/tests/test_studio_core.py b/tests/test_studio_core.py index e43f690..46ec378 100644 --- a/tests/test_studio_core.py +++ b/tests/test_studio_core.py @@ -1,9 +1,11 @@ from __future__ import annotations import ast -import os +import hashlib +import json import sys from pathlib import Path +from unittest.mock import MagicMock import pytest @@ -18,6 +20,7 @@ discover_outputs, discover_textbook_outputs, performance_tips, + progress_percent, safe_run_name, ) @@ -173,6 +176,15 @@ def test_env_and_speed_tips(tmp_path: Path): assert safe_run_name("我的 课件!!.pdf") +def test_progress_percent_uses_planned_stage_count(): + progress = { + "status": "running", + "planned_stages": ["parse", "understand", "notes", "export"], + "stages": [{"stage": "parse"}], + } + assert progress_percent(progress) == pytest.approx(0.25) + + def test_gui_api_status_accepts_provider_alias_env(monkeypatch): pytest.importorskip("streamlit") from gui.app import _api_status @@ -226,3 +238,188 @@ def test_gui_workbench_file_size_helper(): assert _format_file_size(512) == "512 B" assert _format_file_size(1536) == "1.5 KB" assert _format_file_size(None) == "unknown size" + + +def test_gui_carries_modality_corrections_for_identical_source_only(tmp_path: Path): + pytest.importorskip("streamlit") + from gui.app import _carry_modality_overrides + + previous_source = tmp_path / "previous.pdf" + previous_source.write_bytes(b"same input bytes") + current_source = tmp_path / "current.pdf" + current_source.write_bytes(previous_source.read_bytes()) + changed_source = tmp_path / "changed.pdf" + changed_source.write_bytes(b"same input byteX") + previous_output = tmp_path / "previous-output" + previous_output.mkdir() + (previous_output / "content.json").write_text(json.dumps({"source_path": str(previous_source)}), encoding="utf-8") + manifest = {"schema_version": 1, "pages": {"1": {"modality": "image_only", "note": "scan"}}} + (previous_output / "page_modalities.overrides.json").write_text(json.dumps(manifest), encoding="utf-8") + + matching_output = tmp_path / "matching-output" + matching_output.mkdir() + assert _carry_modality_overrides(previous_output, current_source, matching_output) + assert json.loads((matching_output / "page_modalities.overrides.json").read_text(encoding="utf-8")) == manifest + + changed_output = tmp_path / "changed-output" + changed_output.mkdir() + assert not _carry_modality_overrides(previous_output, changed_source, changed_output) + assert not (changed_output / "page_modalities.overrides.json").exists() + + +def test_gui_carries_hashed_corrections_after_original_upload_is_removed(tmp_path: Path): + pytest.importorskip("streamlit") + from gui.app import _carry_modality_overrides + + current_source = tmp_path / "current.pdf" + current_source.write_bytes(b"same input bytes") + previous_output = tmp_path / "previous-output" + previous_output.mkdir() + manifest = { + "schema_version": 1, + "source_sha256": hashlib.sha256(current_source.read_bytes()).hexdigest(), + "pages": {"1": {"modality": "image_only"}}, + } + (previous_output / "page_modalities.overrides.json").write_text(json.dumps(manifest), encoding="utf-8") + next_output = tmp_path / "next-output" + next_output.mkdir() + + assert _carry_modality_overrides(previous_output, current_source, next_output) + assert json.loads((next_output / "page_modalities.overrides.json").read_text(encoding="utf-8")) == manifest + + +def test_gui_preserves_stale_corrections_in_reused_output_dir(tmp_path: Path): + pytest.importorskip("streamlit") + from gui.app import _carry_modality_overrides + + old_source = tmp_path / "old.pdf" + old_source.write_bytes(b"old") + new_source = tmp_path / "new.pdf" + new_source.write_bytes(b"new") + output_dir = tmp_path / "output" + output_dir.mkdir() + (output_dir / "content.json").write_text(json.dumps({"source_path": str(old_source)}), encoding="utf-8") + manifest_path = output_dir / "page_modalities.overrides.json" + manifest_path.write_text('{"pages":{"1":{"modality":"image_only"}}}', encoding="utf-8") + + assert not _carry_modality_overrides(None, new_source, output_dir) + assert not manifest_path.exists() + backups = list(output_dir.glob("page_modalities.overrides.stale-*.json")) + assert len(backups) == 1 + assert 'image_only' in backups[0].read_text(encoding="utf-8") + + +def test_gui_saved_correction_records_source_hash(tmp_path: Path): + pytest.importorskip("streamlit") + from gui.app import _save_modality_override + + source = tmp_path / "source.pdf" + source.write_bytes(b"source bytes") + output_dir = tmp_path / "output" + output_dir.mkdir() + (output_dir / "content.json").write_text(json.dumps({"source_path": str(source)}), encoding="utf-8") + + _save_modality_override(output_dir, 2, "image_only", "scan") + manifest = json.loads((output_dir / "page_modalities.overrides.json").read_text(encoding="utf-8")) + assert manifest["source_sha256"] == hashlib.sha256(source.read_bytes()).hexdigest() + assert manifest["pages"]["2"]["modality"] == "image_only" + + +def test_gui_quiet_build_polls_progress_before_stdout(tmp_path: Path, monkeypatch): + pytest.importorskip("streamlit") + import gui.app as app + + output_dir = tmp_path / "output" + output_dir.mkdir() + marker = tmp_path / "finished.txt" + script = tmp_path / "quiet_build.py" + script.write_text( + "import json, sys, time\n" + "from pathlib import Path\n" + "progress = Path(sys.argv[1])\n" + "marker = Path(sys.argv[2])\n" + "progress.write_text(json.dumps({'status': 'running', 'message': 'working'}), encoding='utf-8')\n" + "time.sleep(1.0)\n" + "marker.write_text('done', encoding='utf-8')\n" + "print('finished', flush=True)\n", + encoding="utf-8", + ) + config = StudioConfig( + input_path=tmp_path / "source.pdf", + output_dir=output_dir, + progress_json=output_dir / "progress.json", + preset="local", + ) + monkeypatch.setattr(app, "build_slidenote_command", lambda cfg: [sys.executable, str(script), str(cfg.progress_json), str(marker)]) + monkeypatch.setattr(app, "_generate_cost_report", lambda _: None) + fake_st = MagicMock() + slots = [MagicMock() for _ in range(3)] + fake_st.empty.side_effect = slots + monkeypatch.setattr(app, "st", fake_st) + original_update = app._update_progress_ui + observed: list[tuple[dict | None, bool]] = [] + + def record_update(progress_path, progress_bar, status_box, stage_box): + original_update(progress_path, progress_bar, status_box, stage_box) + observed.append((app.read_json(progress_path), marker.exists())) + + monkeypatch.setattr(app, "_update_progress_ui", record_update) + app._run_build(config) + + assert sum(bool(progress and progress.get("status") == "running" and not finished) for progress, finished in observed) >= 2 + assert "finished" in slots[2].code.call_args.args[0] + fake_st.success.assert_called_once() + fake_st.error.assert_not_called() + + +def test_coverage_helpers_report_visible_coverage_and_unexplained_items(): + from gui.studio_core import coverage_missing_items, coverage_summary + + coverage = { + "total": 4, + "covered": 4, + "coverage_ratio": 1.0, + "visible_coverage": {"total": 3, "covered": 1, "missing": 2, "coverage_ratio": 0.3333}, + "required_visible_coverage": {"total": 1, "missing": 1}, + "items": [ + {"id": "s1_t1", "slide_id": 1, "kind": "text", "trace_covered": True, "visible_covered": True}, + {"id": "s2_t1", "slide_id": 2, "kind": "text", "trace_covered": True, "visible_covered": False, "marker_only": True}, + {"id": "s3_t1", "slide_id": 3, "kind": "text", "trace_covered": False, "visible_covered": False, "required": True}, + {"id": "s4_t1", "slide_id": 4, "kind": "text", "trace_covered": False, "visible_covered": False, "structural": True}, + ], + } + + summary = coverage_summary(coverage) + rows = coverage_missing_items(coverage) + + assert summary["visible"] is True + assert summary["missing"] == 2 + assert summary["required_missing"] == 1 + assert [row["element_id"] for row in rows] == ["s3_t1", "s2_t1"] + assert rows[0]["reason"] == "required, not explained in prose" + assert rows[1]["reason"] == "source marker only, no prose explanation" + + +def test_format_cost_tolerates_missing_or_invalid_values(): + from gui.studio_core import format_cost, format_count + + assert format_cost(0.0123, "CNY") == "0.012300 CNY" + assert format_cost(None) == "not recorded" + assert format_cost("n/a") == "not recorded" + assert format_count("12345") == "12,345" + assert format_count(None) == "—" + + +def test_output_zip_excludes_cache(tmp_path): + import io + import zipfile + + from gui.studio_core import zip_output_dir + + (tmp_path / "notes.md").write_text("notes", encoding="utf-8") + (tmp_path / ".cache" / "llm").mkdir(parents=True) + (tmp_path / ".cache" / "llm" / "entry.json").write_text("{}", encoding="utf-8") + + names = zipfile.ZipFile(io.BytesIO(zip_output_dir(tmp_path))).namelist() + + assert names == ["notes.md"] diff --git a/tests/test_study_pack.py b/tests/test_study_pack.py index 581e755..c97c405 100644 --- a/tests/test_study_pack.py +++ b/tests/test_study_pack.py @@ -149,3 +149,76 @@ def __init__(self, **kwargs): assert cached["summary"]["llm_call"] is False assert cached["summary"]["local_cache_hits"] == 1 assert cached["review"]["checklist"][0]["point"] == "Replica consistency" + + +def test_clean_inline_keeps_math_comparisons_but_strips_html_tags(): + from slidenote.study_pack.common import _clean_inline + + assert _clean_inline("if a < b and c > d then x") == "if a < b and c > d then x" + assert _clean_inline("ad") == "ad" + assert _clean_inline('TCP
三次握手') == "TCP 三次握手" + + +def test_normalize_exam_handles_malformed_choice_and_true_false_strings(): + from slidenote.study_pack import _normalize_exam + + raw = { + "questions": [ + {"type": "choice", "question": "只有一个选项", "options": ["A"], "answer": "见解析", "explanation": "解释"}, + {"type": "choice", "question": "越界答案", "options": ["A", "B"], "answer": 9}, + {"type": "true_false", "question": "判断", "answer": "正确"}, + {"type": "true_false", "question": "判断2", "answer": "false"}, + ] + } + + questions = _normalize_exam(raw, {"questions": []}, question_count=10)["questions"] + + assert questions[0]["type"] == "short" + assert questions[0]["points"] == 6 + assert questions[0]["answer"] == "见解析" + assert questions[1]["answer"] == 1 + assert questions[2]["answer"] is True + assert questions[3]["answer"] is False + + +def test_local_questions_vary_correct_choice_and_true_false_answers(): + from slidenote.study_pack.questions import _local_questions + + items = [ + {"point": f"概念{index}", "explanation": f"概念{index}的解释内容，用于区分不同知识点。", "source_refs": [f"P{index}"]} + for index in range(1, 9) + ] + + questions = _local_questions(items, question_count=32) + choice_answers = {question["answer"] for question in questions if question["type"] == "choice"} + tf_answers = {question["answer"] for question in questions if question["type"] == "true_false"} + + assert len(choice_answers) > 1 + assert tf_answers == {True, False} + for question in questions: + if question["type"] == "choice": + assert not any("只背" in option for option in question["options"]) + + +def test_collect_study_items_respects_limit(): + from slidenote.study_pack import _collect_study_items + + deck = Deck( + source_path="lecture.pdf", + source_type="pdf", + pages=[ + SlidePage( + slide_id=index, + title=f"Topic {index}", + text_blocks=[TextBlock(id=f"s{index}_t1", type="paragraph", content=f"Protocol{index} guarantees property number {index} for every message.")], + tables=[TableBlock(id=f"s{index}_tbl1", rows=[["k", "v"], ["a", str(index)]])], + ) + for index in range(1, 11) + ], + ) + guard = {"items": [{"element_id": f"s{index}_t1", "slide_id": index, "must_explain": True, "confidence": 0.9} for index in range(1, 11)]} + + items = _collect_study_items(deck, "", guard, limit=5) + + assert len(items) == 5 + assert [item["source_refs"] for item in items] == [["P1"], ["P2"], ["P3"], ["P4"], ["P5"]] diff --git a/tests/test_study_pack_runner.py b/tests/test_study_pack_runner.py new file mode 100644 index 0000000..95d8460 --- /dev/null +++ b/tests/test_study_pack_runner.py @@ -0,0 +1,138 @@ +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import pytest +import fitz + +from slidenote import study_pack_runner +from slidenote.build.errors import UserFacingConfigError +from slidenote.cli import main +from slidenote.llm import PROVIDERS +from slidenote.models import Deck, SlidePage, TextBlock + + +def _clear_provider_env(monkeypatch) -> None: + for spec in PROVIDERS.values(): + for name in (*spec.api_key_envs, *spec.model_envs): + monkeypatch.delenv(name, raising=False) + monkeypatch.delenv("SLIDENOTE_MODEL", raising=False) + + +def _args(out_dir: Path) -> argparse.Namespace: + return argparse.Namespace(build_out_dir=out_dir, question_count=4, quiet=True) + + +def _write_build(out_dir: Path, run_summary: dict | None) -> None: + out_dir.mkdir(parents=True, exist_ok=True) + deck = Deck( + source_path="lecture.pdf", + source_type="pdf", + pages=[SlidePage(slide_id=1, title="TCP", text_blocks=[TextBlock(id="s1_t1", type="paragraph", content="定义：TCP 提供可靠有序的字节流。")])], + ) + (out_dir / "content.json").write_text(json.dumps(deck.to_dict()), encoding="utf-8") + (out_dir / "notes.md").write_text("# TCP\n\n## 可靠传输\n\nTCP 提供可靠有序的字节流。\n", encoding="utf-8") + if run_summary is not None: + (out_dir / "run_summary.json").write_text(json.dumps(run_summary), encoding="utf-8") + + +def test_provider_can_run_requires_key_and_known_provider(monkeypatch): + _clear_provider_env(monkeypatch) + assert study_pack_runner._provider_can_run("not-a-provider") is False + assert study_pack_runner._provider_can_run("deepseek") is False + monkeypatch.setenv("DEEPSEEK_API_KEY", "key") + assert study_pack_runner._provider_can_run("deepseek") is True + + +def test_provider_can_run_accepts_explicit_model_for_providers_without_default(monkeypatch): + _clear_provider_env(monkeypatch) + no_default = next((name for name, spec in PROVIDERS.items() if not spec.default_model), None) + if no_default is None: + pytest.skip("every provider has a default model") + monkeypatch.setenv(PROVIDERS[no_default].api_key_envs[0], "key") + assert study_pack_runner._provider_can_run(no_default) is False + assert study_pack_runner._provider_can_run(no_default, model="custom-model") is True + + +def test_study_pack_requires_existing_build_dir(tmp_path): + with pytest.raises(UserFacingConfigError, match="does not exist"): + study_pack_runner.run_study_pack(_args(tmp_path / "missing")) + + +def test_study_pack_requires_content_and_notes(tmp_path): + (tmp_path / "notes.md").write_text("# Notes\n", encoding="utf-8") + with pytest.raises(UserFacingConfigError, match="content.json and notes.md"): + study_pack_runner.run_study_pack(_args(tmp_path)) + + +def test_study_pack_warns_when_run_summary_is_missing(tmp_path, monkeypatch, capsys): + _clear_provider_env(monkeypatch) + _write_build(tmp_path, run_summary=None) + + assert study_pack_runner.run_study_pack(_args(tmp_path)) == 0 + + report = json.loads((tmp_path / "study_pack.json").read_text(encoding="utf-8")) + assert any(warning.startswith("study_pack_provider_fallback:deepseek") for warning in report["warnings"]) + assert "study_pack_provider_fallback" in capsys.readouterr().err + assert (tmp_path / "exam.md").exists() + + +def test_study_pack_reuses_model_and_base_url_from_run_summary(tmp_path, monkeypatch): + _clear_provider_env(monkeypatch) + _write_build(tmp_path, run_summary={"run": {"provider": "openai", "model": "gpt-custom", "base_url": "https://proxy.example/v1"}}) + captured = {} + real_build = study_pack_runner.build_study_pack + + def spy(**kwargs): + captured.update(kwargs) + return real_build(**kwargs) + + monkeypatch.setattr(study_pack_runner, "build_study_pack", spy) + + assert study_pack_runner.run_study_pack(_args(tmp_path)) == 0 + + assert captured["provider"] == "openai" + assert captured["model"] == "gpt-custom" + assert captured["base_url"] == "https://proxy.example/v1" + assert captured["max_output_tokens"] == study_pack_runner.STUDY_PACK_MAX_OUTPUT_TOKENS + report = json.loads((tmp_path / "study_pack.json").read_text(encoding="utf-8")) + assert not any("provider_fallback" in warning for warning in report["warnings"]) + + +def test_study_pack_reuses_text_runtime_from_actual_build(tmp_path, monkeypatch): + _clear_provider_env(monkeypatch) + monkeypatch.setenv("SLIDENOTE_MODEL", "ep-custom") + monkeypatch.setenv("SLIDENOTE_BASE_URL", "https://gateway.example/v3") + source = tmp_path / "lecture.pdf" + doc = fitz.open() + page = doc.new_page() + page.insert_text((72, 72), "Reliable transport") + doc.save(source) + doc.close() + out = tmp_path / "out" + assert main([ + "build", str(source), "--out", str(out), "--quiet", "--preset", "local", "--provider", "doubao", + ]) == 0 + + run = json.loads((out / "run_summary.json").read_text(encoding="utf-8"))["run"] + assert run["provider"] == "doubao" + assert run["model"] == "ep-custom" + assert run["base_url"] == "https://gateway.example/v3" + + monkeypatch.delenv("SLIDENOTE_MODEL") + monkeypatch.delenv("SLIDENOTE_BASE_URL") + monkeypatch.setenv("DOUBAO_API_KEY", "dummy") + captured = {} + real_build = study_pack_runner.build_study_pack + + def spy(**kwargs): + captured.update(kwargs) + return real_build(**{**kwargs, "use_llm": False}) + + monkeypatch.setattr(study_pack_runner, "build_study_pack", spy) + assert study_pack_runner.run_study_pack(_args(out)) == 0 + assert captured["use_llm"] is True + assert captured["model"] == "ep-custom" + assert captured["base_url"] == "https://gateway.example/v3" diff --git a/tests/test_textbook_index.py b/tests/test_textbook_index.py index 21df9f0..52f127e 100644 --- a/tests/test_textbook_index.py +++ b/tests/test_textbook_index.py @@ -4,7 +4,6 @@ from pathlib import Path import fitz -import pytest from slidenote.cli import main from slidenote.textbook import (