diff --git a/AGENTS.md b/AGENTS.md index 0b46dda..2347522 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,144 +1,157 @@ -# AGENTS.md — Guía para desarrolladores y agentes de IA - -Este fichero es la referencia operativa para cualquier persona o agente de IA que trabaje -en el repositorio `docsai`. Léelo completo antes de tocar código. - -## 1. Qué es este proyecto - -`docsai` es un binario Rust multiplataforma (Windows/Linux/macOS) que convierte documentos -Office (`.doc`, `.docx`, `.xls`, `.xlsx`) y LibreOffice (`.odt`, `.ods`) a un Markdown -extendido llamado **DocMark**, y de vuelta, con pérdida mínima de formato. Se invoca como -CLI o como servidor **MCP por stdio**. - -**Estado actual**: **Fases 0 y 1 cerradas**. Existe el workspace con los siete crates, el IR -(`docsai-model`), el lector `.docx` (`docsai-office`), el serializador DocMark -(`docsai-docmark`), la orquestación (`docsai-convert`) y la CLI con `convert` y `formats`. -`docsai-odf` y `docsai-mcp` son esqueletos que sólo fijan las reglas de dependencia. - -Lo siguiente es la **Fase 2**: parser DocMark, writer `.docx` y el comando `roundtrip`. - -## 2. Documentos que debes leer antes de implementar - -Orden de lectura obligatorio: - -1. `docs/plan-desarrollo.md` — el plan por fases. **Identifica en qué fase está el proyecto - antes de escribir nada.** No implementes elementos de fases futuras. -2. `docs/arquitectura.md` — estructura del workspace, modelo IR, contratos entre crates. -3. `docs/especificacion-docmark.md` — el formato Markdown extendido. Es un contrato: - cualquier cambio requiere subir la versión del campo `docmark` del front matter y - documentar la migración. -4. `docs/analisis-tecnico.md` — por qué se eligió cada librería. No sustituyas una - dependencia clave (calamine, docx-rs, rmcp, comrak…) sin dejar constancia escrita del - motivo en ese documento. -5. `kb/` — base de conocimiento de lo **ya construido**: resumen de las fases cerradas, - estructura real del código, decisiones técnicas con su motivo, y las consideraciones para - la fase que vayas a abordar. Empieza por `kb/README.md`. Si tu cambio altera la estructura - o toma una decisión técnica no obvia, actualízala en el mismo PR. - -## 3. Estructura del repositorio (objetivo) +# AGENTS.md — Guide for developers and AI agents + +This file is the operational reference for anyone or any AI agent working +on the `docsai` repository. Read it fully before touching code. + +## 1. What this project is + +`docsai` is a cross-platform Rust binary (Windows/Linux/macOS) that converts +Office documents (`.doc`, `.docx`, `.xls`, `.xlsx`) and LibreOffice documents +(`.odt`, `.ods`) to an extended Markdown called **DocMark**, and back, with +minimal format loss. It is invoked as a CLI or as an **MCP server over stdio**. + +**Current status**: **Phases 0–2 are closed** for the core path. The workspace +has the seven crates, the IR (`docsai-model`), the `.docx` reader **and writer** +(`docsai-office`), DocMark serialize **and parse** (`docsai-docmark`), +orchestration (`docsai-convert`), and the CLI with `convert`, `formats` and +`roundtrip`. `docsai-odf` and `docsai-mcp` remain skeletons that only fix the +dependency rules. + +Next up is **Phase 3**: spreadsheets (`.xlsx` / `.xls` ⇄ DocMark). + +## 2. Documents you must read before implementing + +Mandatory reading order: + +1. `docs/development-plan.md` — the phased plan. **Identify which phase the + project is in before writing anything.** Do not implement items from future + phases. +2. `docs/architecture.md` — workspace structure, IR model, contracts between + crates. +3. `docs/docmark-specification.md` — the extended Markdown format. It is a + contract: any change requires bumping the `docmark` field version in the + front matter and documenting the migration. +4. `docs/technical-analysis.md` — why each library was chosen. Do not replace a + key dependency (calamine, docx-rs, rmcp, comrak…) without leaving a written + record of the reason in that document. +5. `kb/` — knowledge base of what is **already built**: summary of closed + phases, real code structure, technical decisions with their rationale, and + considerations for the phase you are about to tackle. Start with + `kb/README.md`. If your change alters the structure or takes a non-obvious + technical decision, update it in the same PR. + +## 3. Repository structure (target) ``` docsai/ -├── Cargo.toml # workspace raíz +├── Cargo.toml # root workspace ├── crates/ -│ ├── docsai-model/ # IR: modelo de documento intermedio (sin I/O, sin deps pesadas) -│ ├── docsai-docmark/ # serializador + parser de DocMark (IR ⇄ Markdown extendido) -│ ├── docsai-office/ # lectores/escritores OOXML: docx, xlsx (+ xls, doc lectura) -│ ├── docsai-odf/ # lectores/escritores ODF: odt, ods -│ ├── docsai-convert/ # orquestación: pipelines, detección de formato, assets, informes de fidelidad -│ ├── docsai-cli/ # binario CLI (clap) -│ └── docsai-mcp/ # servidor MCP stdio (rmcp) -├── docs/ # documentación de diseño (este conjunto) -├── corpus/ # documentos de prueba versionados (ver §6) -└── tests/ # tests de integración y round-trip +│ ├── docsai-model/ # IR: intermediate document model (no I/O, no heavy deps) +│ ├── docsai-docmark/ # DocMark serializer + parser (IR ⇄ extended Markdown) +│ ├── docsai-office/ # OOXML readers/writers: docx, xlsx (+ xls, doc read) +│ ├── docsai-odf/ # ODF readers/writers: odt, ods +│ ├── docsai-convert/ # orchestration: pipelines, format detection, assets, fidelity reports +│ ├── docsai-cli/ # CLI binary (clap) +│ └── docsai-mcp/ # MCP stdio server (rmcp) +├── docs/ # design documentation (this set) +├── corpus/ # versioned test documents (see §6) +└── tests/ # integration and round-trip tests ``` -Reglas de dependencia entre crates (violarlas es un error de arquitectura): +Dependency rules between crates (violating them is an architecture error): -- `docsai-model` no depende de ningún otro crate del workspace. -- `docsai-docmark`, `docsai-office`, `docsai-odf` dependen **solo** de `docsai-model`. -- `docsai-convert` depende de los cuatro anteriores. `docsai-cli` y `docsai-mcp` - dependen solo de `docsai-convert` (y `docsai-model` para tipos). -- Ningún crate de formato importa a otro crate de formato. +- `docsai-model` depends on no other crate in the workspace. +- `docsai-docmark`, `docsai-office`, `docsai-odf` depend **only** on `docsai-model`. +- `docsai-convert` depends on the four above. `docsai-cli` and `docsai-mcp` + depend only on `docsai-convert` (and `docsai-model` for types). +- No format crate imports another format crate. -## 4. Comandos de trabajo +## 4. Working commands -Cuando exista el workspace, estos son los comandos canónicos (deben mantenerse verdes): +When the workspace exists, these are the canonical commands (they must stay green): ```bash cargo build --workspace -cargo test --workspace # unit + integración + round-trip +cargo test --workspace # unit + integration + round-trip cargo clippy --workspace --all-targets -- -D warnings cargo fmt --all -- --check cargo run -p docsai-cli -- convert corpus/docx/basic-styles.docx -o /tmp/out.dmk.md -python3 corpus/generate.py --check # el corpus es generado; CI comprueba que está al día +python3 corpus/generate.py --check # the corpus is generated; CI checks it is up to date ``` -Actualizar un golden es un acto deliberado, y su diff se revisa a mano: +Updating a golden is a deliberate act, and its diff is reviewed by hand: ```bash DOCSAI_UPDATE_GOLDENS=1 cargo test -p docsai-convert --test goldens ``` -CI (GitHub Actions) ejecuta la matriz `{ubuntu-latest, windows-latest, macos-latest}` × -`{stable}`. Un PR no se fusiona con CI en rojo. - -## 5. Convenciones de código - -- **Edición Rust 2021+ / toolchain stable**. Nada de `nightly` en el árbol principal. -- `rustfmt` por defecto y `clippy -D warnings`. Sin excepciones sin comentario `#[allow]` justificado. -- Errores: `thiserror` en las bibliotecas, `anyhow` solo en los binarios (`docsai-cli`, `docsai-mcp`). -- Logging: `tracing`. En el servidor MCP **jamás escribir en stdout** salvo el protocolo - (stdout es el canal JSON-RPC); logs siempre a stderr. -- Nombres de código, mensajes de commit y comentarios de código en **inglés**; - la documentación de `docs/` se mantiene en español. -- Commits: Conventional Commits (`feat:`, `fix:`, `docs:`, `test:`, `refactor:`, `chore:`), - con ámbito de crate cuando aplique: `feat(office): read numbering.xml`. -- `unsafe` prohibido salvo justificación documentada en el PR (no debería ser necesario). -- Toda función pública de los crates de biblioteca lleva doc-comment con ejemplo cuando sea razonable. - -## 6. Estrategia de pruebas (resumen; detalle en el plan §Fase 0 y §Fase 8) - -- **Corpus versionado** en `corpus/`: documentos pequeños creados a propósito, un rasgo - por fichero (`basic-styles.docx`, `nested-lists.docx`, `formulas-basic.xlsx`…). Nunca - añadir documentos con datos reales o privados. -- **Golden files**: cada documento del corpus tiene su DocMark esperado al lado - (`.expected.dmk.md`). Los tests comparan la salida con el golden; actualizar un golden - requiere revisar el diff a mano. -- **Round-trip tests**: `Office → DocMark → Office → DocMark` debe producir DocMark - idéntico en la segunda pasada (idempotencia). La comparación primera-vs-segunda pasada - del fichero Office se hace a nivel de IR normalizado, no de bytes. -- **Fuzzing** (`cargo-fuzz`) sobre los parsers de entrada a partir de la Fase 8; los - parsers nunca deben entrar en pánico con entrada corrupta: siempre `Err`, nunca `panic!`. - -## 7. Reglas específicas para agentes de IA - -1. **No amplíes el alcance.** Si la tarea pide la Fase N, no adelantes trabajo de la fase - N+1 "ya que estás". El plan define el orden por dependencias reales. -2. **No cambies la especificación DocMark para hacer pasar un test.** Si el formato no - puede representar algo, documenta la limitación y usa el mecanismo `raw-block` descrito - en la especificación. -3. **Nunca degrades la fidelidad en silencio.** Toda pérdida de información en una - conversión debe emitir una advertencia estructurada (ver `ConversionReport` en - `docs/arquitectura.md`). -4. **No añadas dependencias pesadas sin justificación.** El objetivo es un binario único y - razonablemente pequeño. Antes de añadir un crate: ¿está mantenido?, ¿es pure-Rust?, - ¿qué añade al tamaño del binario? Deja la justificación en el PR. -5. **Documenta al terminar.** Si tu cambio altera comportamiento visible (CLI, formato, - tools MCP), actualiza README.md y el documento de `docs/` correspondiente en el mismo PR. -6. **Verifica en las tres plataformas mentales.** Rutas con `std::path` (nunca concatenar - con `/`), finales de línea (el serializador DocMark emite siempre `\n`; el parser acepta - `\r\n`), y nada de dependencias de herramientas POSIX en el código de producción. -7. Trabaja en ramas y push a la rama que se te indique; no hagas push a `main`. - -## 8. Definición de "hecho" (Definition of Done) - -Una tarea/fase se considera terminada cuando: - -- [ ] Compila y pasa `cargo test --workspace` en las tres plataformas de CI. -- [ ] `clippy` y `fmt` limpios. -- [ ] Tests nuevos cubren el comportamiento añadido (incluidos casos de error). -- [ ] Golden files y corpus actualizados si aplica. -- [ ] Documentación actualizada (README, docs/, `--help` de la CLI). -- [ ] Los criterios de aceptación de la fase correspondiente en `docs/plan-desarrollo.md` - están marcados y verificados. +CI (GitHub Actions) runs the matrix `{ubuntu-latest, windows-latest, macos-latest}` × +`{stable}`. A PR is not merged with red CI. + +## 5. Code conventions + +- **Rust edition 2021+ / stable toolchain**. No `nightly` in the main tree. +- Default `rustfmt` and `clippy -D warnings`. No exceptions without a justified + `#[allow]` comment. +- Errors: `thiserror` in libraries, `anyhow` only in binaries (`docsai-cli`, + `docsai-mcp`). +- Logging: `tracing`. In the MCP server **never write to stdout** except for the + protocol (stdout is the JSON-RPC channel); logs always go to stderr. +- **Hard language rule: everything is English.** All documentation (`docs/`, + `kb/`, `README.md`, `AGENTS.md`), code identifiers, commit messages, and code + comments **must** be in English. Do not write Spanish (or any other language) + in those places. +- Commits: Conventional Commits (`feat:`, `fix:`, `docs:`, `test:`, `refactor:`, + `chore:`), with crate scope when applicable: `feat(office): read numbering.xml`. +- `unsafe` is forbidden unless justified in the PR documentation (it should not + be necessary). +- Every public function in library crates has a doc-comment with an example when + reasonable. + +## 6. Testing strategy (summary; detail in the plan §Phase 0 and §Phase 8) + +- **Versioned corpus** in `corpus/`: small documents created on purpose, one + trait per file (`basic-styles.docx`, `nested-lists.docx`, + `formulas-basic.xlsx`…). Never add documents with real or private data. +- **Golden files**: each corpus document has its expected DocMark beside it + (`.expected.dmk.md`). Tests compare output against the golden; updating a + golden requires reviewing the diff by hand. +- **Round-trip tests**: `Office → DocMark → Office → DocMark` must produce + identical DocMark on the second pass (idempotence). First-vs-second pass + comparison of the Office file is done at the normalized IR level, not bytes. +- **Fuzzing** (`cargo-fuzz`) on input parsers from Phase 8 onward; parsers must + never panic on corrupt input: always `Err`, never `panic!`. + +## 7. Specific rules for AI agents + +1. **Do not expand scope.** If the task asks for Phase N, do not advance Phase + N+1 work "while you are at it". The plan defines order by real dependencies. +2. **Do not change the DocMark specification to make a test pass.** If the + format cannot represent something, document the limitation and use the + `raw-block` mechanism described in the specification. +3. **Never degrade fidelity silently.** Every information loss in a conversion + must emit a structured warning (see `ConversionReport` in + `docs/architecture.md`). +4. **Do not add heavy dependencies without justification.** The goal is a single, + reasonably small binary. Before adding a crate: is it maintained? is it + pure-Rust? what does it add to binary size? Leave the justification in the PR. +5. **Document when you finish.** If your change alters visible behavior (CLI, + format, MCP tools), update README.md and the corresponding document under + `docs/` in the same PR. +6. **Verify against the three mental platforms.** Paths with `std::path` (never + concatenate with `/`), line endings (the DocMark serializer always emits + `\n`; the parser accepts `\r\n`), and no POSIX tool dependencies in production + code. +7. Work on branches and push to the branch you are told; do not push to `main`. + +## 8. Definition of Done + +A task/phase is considered finished when: + +- [ ] It compiles and passes `cargo test --workspace` on all three CI platforms. +- [ ] `clippy` and `fmt` are clean. +- [ ] New tests cover the added behavior (including error cases). +- [ ] Golden files and corpus are updated if applicable. +- [ ] Documentation is updated (README, docs/, CLI `--help`). +- [ ] The acceptance criteria for the corresponding phase in + `docs/development-plan.md` are marked and verified. diff --git a/README.md b/README.md index 72ab3cc..908d4ae 100644 --- a/README.md +++ b/README.md @@ -1,103 +1,104 @@ # docsai -**Conversor bidireccional de documentos Office / LibreOffice ⇄ Markdown extendido, escrito en Rust.** +**Bidirectional converter between Office / LibreOffice documents and extended Markdown, written in Rust.** -`docsai` es un binario único y multiplataforma (Windows, Linux, macOS) que convierte documentos ofimáticos a un perfil de Markdown extendido — **DocMark** — diseñado para conservar la máxima información posible (estilos, imágenes, propiedades, fórmulas) y permitir la **conversión inversa con pérdida mínima de formato**. Puede usarse como herramienta CLI o como **servidor MCP (Model Context Protocol) por stdio**, para integrarse con asistentes de IA como Claude. +`docsai` is a single cross-platform binary (Windows, Linux, macOS) that converts office documents to an extended Markdown profile — **DocMark** — designed to keep as much information as possible (styles, images, properties, formulas) and to allow the **reverse conversion with minimal format loss**. It can be used as a CLI tool or as an **MCP (Model Context Protocol) server over stdio**, for integration with AI assistants such as Claude. -> **Estado del proyecto: Fases 0 y 1 completadas.** -> Ya se convierte `.docx` → DocMark con estilos, listas, tablas, imágenes (con toda su -> geometría), cabeceras, pies, notas al pie, campos y propiedades. La conversión inversa -> (DocMark → `.docx`) es la Fase 2 y todavía no existe; las hojas de cálculo llegan en la -> Fase 3. Ver [`docs/plan-desarrollo.md`](docs/plan-desarrollo.md). +> **Project status: Phases 0–2 completed.** +> `.docx` ⇄ DocMark works with styles, lists, tables, images, headers/footers, +> footnotes, fields and properties. `docsai roundtrip` checks DocMark idempotence +> after a write/read cycle. Spreadsheets arrive in Phase 3. See +> [`docs/development-plan.md`](docs/development-plan.md). ```bash -cargo run -p docsai-cli -- convert informe.docx -o informe.dmk.md +cargo run -p docsai-cli -- convert report.docx -o report.dmk.md cargo run -p docsai-cli -- formats ``` --- -## Formatos soportados (objetivo) +## Supported formats (target) -✅ = funciona hoy · 🕓 = planificado, con la fase que lo trae · ➖ = fuera de alcance +✅ = works today · 🕓 = planned, with the phase that lands it · ➖ = out of scope -| Formato | Extensión | Lectura | Escritura | Notas | +| Format | Extension | Read | Write | Notes | |---|---|---|---|---| -| Word OOXML | `.docx` | ✅ | 🕓 Fase 2 | Estilos, imágenes, tablas, listas, cabeceras/pies, notas al pie, campos, propiedades | -| Word binario | `.doc` | 🕓 Fase 5 | ➖ | Solo lectura (vía parser nativo o fallback LibreOffice headless) | -| Excel OOXML | `.xlsx` | 🕓 Fase 3 | 🕓 Fase 3 | Valores **y fórmulas**, formatos de número, celdas combinadas, imágenes ancladas | -| Excel binario | `.xls` | 🕓 Fase 3 | ➖ | Solo lectura (calamine) | -| OpenDocument Text | `.odt` | 🕓 Fase 4 | 🕓 Fase 4 | Equivalente libre de `.docx` | -| OpenDocument Spreadsheet | `.ods` | 🕓 Fase 4 | 🕓 Fase 4 | Equivalente libre de `.xlsx` | -| Markdown extendido | `.dmk.md` | 🕓 Fase 2 | ✅ | Formato pivote **DocMark** (superconjunto de CommonMark + GFM) | +| Word OOXML | `.docx` | ✅ | ✅ | Styles, images, tables, lists, headers/footers, footnotes, fields, properties | +| Word binary | `.doc` | 🕓 Phase 5 | ➖ | Read only (native parser or LibreOffice headless fallback) | +| Excel OOXML | `.xlsx` | 🕓 Phase 3 | 🕓 Phase 3 | Values **and formulas**, number formats, merged cells, anchored images | +| Excel binary | `.xls` | 🕓 Phase 3 | ➖ | Read only (calamine) | +| OpenDocument Text | `.odt` | 🕓 Phase 4 | 🕓 Phase 4 | Free equivalent of `.docx` | +| OpenDocument Spreadsheet | `.ods` | 🕓 Phase 4 | 🕓 Phase 4 | Free equivalent of `.xlsx` | +| Extended Markdown | `.dmk.md` | ✅ | ✅ | Pivot format **DocMark** (superset of CommonMark + GFM) | -`docsai formats` imprime esta misma matriz según lo que el binario realmente sabe hacer. +`docsai formats` prints this same matrix for what the binary can actually do. -La escritura de `.doc` y `.xls` (formatos binarios legados) queda fuera de alcance de forma deliberada: la ruta de salida recomendada hacia el ecosistema Microsoft es siempre OOXML (`.docx` / `.xlsx`). +Writing `.doc` and `.xls` (legacy binary formats) is deliberately out of scope: the recommended output path into the Microsoft ecosystem is always OOXML (`.docx` / `.xlsx`). -## ¿Qué es DocMark? +## What is DocMark? -DocMark es un perfil de Markdown extendido definido en este proyecto (ver [`docs/especificacion-docmark.md`](docs/especificacion-docmark.md)). Es **Markdown legible y editable a mano**, que se renderiza de forma razonable en GitHub o cualquier visor CommonMark, pero que añade capas de metadatos para no perder información: +DocMark is an extended Markdown profile defined in this project (see [`docs/docmark-specification.md`](docs/docmark-specification.md)). It is **human-readable and hand-editable Markdown** that renders reasonably on GitHub or any CommonMark viewer, but adds metadata layers so information is not lost: -- **Front matter YAML** con las propiedades del documento (título, autor, idioma…) y el **catálogo de estilos** original. -- **Atributos inline y de bloque** `{#id .clase clave="valor"}` (sintaxis compatible con Pandoc) para anclar estilos, dimensiones de imagen, propiedades de celda, etc. -- **Contenedores fenced** `::: {...}` para secciones, cuadros de texto, cabeceras y pies. -- **Tablas extendidas** con metadatos por celda (fórmulas, tipos, formatos de número, combinaciones) para hojas de cálculo. -- **Activos externos**: las imágenes se extraen a un directorio `assets/` (deduplicadas por hash de contenido) y se referencian con un modelo completo de atributos de geometría: tamaño mostrado y nativo, posición y anclaje (en línea, flotante con wrap y z-order, o anclada a celdas en hojas de cálculo), rotación, recorte, volteo, texto alternativo e hipervínculo. -- **Escotilla de fidelidad** (`raw-block`) para fragmentos sin representación Markdown posible, que se conservan de forma opaca y se restauran en la conversión inversa. +- **YAML front matter** with document properties (title, author, language…) and the original **style catalogue**. +- **Inline and block attributes** `{#id .class key="value"}` (Pandoc-compatible syntax) to attach styles, image dimensions, cell properties, and more. +- **Fenced containers** `::: {...}` for sections, text boxes, headers and footers. +- **Extended tables** with per-cell metadata (formulas, types, number formats, merges) for spreadsheets. +- **External assets**: images are extracted to an `assets/` directory (deduplicated by content hash) and referenced with a full geometry attribute model: display and native size, position and anchoring (inline, floating with wrap and z-order, or cell-anchored in spreadsheets), rotation, crop, flip, alternative text and hyperlink. +- **Fidelity hatch** (`raw-block`) for fragments with no Markdown representation, kept opaque and restored on the reverse conversion. -Ejemplo mínimo: +Minimal example: ```markdown --- docmark: "1.0" source-format: docx -title: "Informe Anual" +title: "Annual Report" styles: Heading1: { font: "Calibri Light", size: 16pt, color: "#2E74B5" } --- -# Informe Anual {.Heading1} +# Annual Report {.Heading1} -Texto con **negrita** y [color]{color="#FF0000"} personalizado. +Text with **bold** and [color]{color="#FF0000"} custom colour. -![Diagrama de ventas](assets/img-001.png){width=450px height=300px anchor=inline} +![Sales chart](assets/img-001.png){width=450px height=300px anchor=inline} ``` -## Uso (CLI) +## Usage (CLI) -Lo que funciona hoy: +What works today: ```bash -docsai convert informe.docx -o informe.dmk.md # extrae assets/ junto al .md -docsai convert informe.docx # a stdout, sin escribir medios al lado -docsai convert informe.docx --fidelity plain # CommonMark limpio, para LLM/RAG -docsai convert informe.docx -o out.md --json # informe de conversión en JSON -docsai formats # matriz de soporte de este binario +docsai convert report.docx -o report.dmk.md # extracts assets/ next to the .md +docsai convert report.docx # to stdout, without writing media beside it +docsai convert report.docx --fidelity plain # clean CommonMark, for LLM/RAG +docsai convert report.docx -o out.md --json # conversion report as JSON +docsai formats # support matrix for this binary ``` -Grados de fidelidad (`--fidelity`, spec §6): `full` (defecto, apto para ida y vuelta), -`standard` (Markdown rico sin catálogos ni raw-blocks) y `plain` (CommonMark+GFM puro). +Fidelity levels (`--fidelity`, spec §6): `full` (default, round-trip grade), +`standard` (rich Markdown without catalogues or raw-blocks) and `plain` (pure +CommonMark+GFM). -Códigos de salida: `0` correcto, `1` conversión con pérdidas (`--strict` incluye las advertencias -menores), `2` error de entrada, `3` formato no soportado. +Exit codes: `0` success, `1` conversion with losses (`--strict` also treats minor +warnings as failures), `2` input error, `3` unsupported format. -Planificado: +Planned: ```bash -docsai convert informe.dmk.md -o informe.docx # Fase 2 -docsai convert ventas.xlsx -o ventas.dmk.md # Fase 3 -docsai inspect informe.docx # Fase 6 -docsai roundtrip informe.docx # Fase 2 +docsai convert report.dmk.md -o report.docx +docsai convert sales.xlsx -o sales.dmk.md # Phase 3 +docsai inspect report.docx # Phase 6 +docsai roundtrip report.docx ``` -## Uso previsto (servidor MCP, Fase 7) +## Intended usage (MCP server, Phase 7) ```bash -docsai mcp # arranca el servidor MCP por stdio +docsai mcp # starts the MCP server over stdio ``` -Registro en un cliente MCP (p. ej. Claude Desktop / Claude Code): +Registration in an MCP client (e.g. Claude Desktop / Claude Code): ```json { @@ -107,41 +108,41 @@ Registro en un cliente MCP (p. ej. Claude Desktop / Claude Code): } ``` -Tools MCP previstas: `convert_to_markdown`, `convert_from_markdown`, `inspect_document`, `list_supported_formats`. Ver detalle en [`docs/arquitectura.md`](docs/arquitectura.md). +Planned MCP tools: `convert_to_markdown`, `convert_from_markdown`, `inspect_document`, `list_supported_formats`. Details in [`docs/architecture.md`](docs/architecture.md). -## Documentación del proyecto +## Project documentation -| Documento | Contenido | +| Document | Contents | |---|---| -| [`docs/analisis-tecnico.md`](docs/analisis-tecnico.md) | Análisis de formatos, librerías Rust evaluadas, proyectos open source previos (Pandoc, MarkItDown, Docling, mammoth…), decisiones y riesgos | -| [`docs/especificacion-docmark.md`](docs/especificacion-docmark.md) | Especificación del formato DocMark (Markdown extendido) v1.0-draft | -| [`docs/arquitectura.md`](docs/arquitectura.md) | Arquitectura del software: workspace de crates, modelo de documento intermedio (IR), CLI, servidor MCP | -| [`docs/plan-desarrollo.md`](docs/plan-desarrollo.md) | Plan de desarrollo detallado en 9 fases, con entregables, criterios de aceptación, estimaciones y estrategia de pruebas | -| [`AGENTS.md`](AGENTS.md) | Guía operativa para desarrolladores y agentes de IA que trabajen en este repositorio | -| [`corpus/README.md`](corpus/README.md) | El corpus de pruebas: qué aísla cada documento y cómo se regenera | -| [`docs/spikes/`](docs/spikes/) | Informes de los spikes de riesgo, con la decisión que cerró cada uno | -| [`kb/`](kb/) | Base de conocimiento: qué hay construido, cómo está estructurado, las decisiones técnicas y lo que espera a las fases siguientes | +| [`docs/technical-analysis.md`](docs/technical-analysis.md) | Format analysis, evaluated Rust libraries, prior open-source projects (Pandoc, MarkItDown, Docling, mammoth…), decisions and risks | +| [`docs/docmark-specification.md`](docs/docmark-specification.md) | DocMark format specification (extended Markdown) v1.0-draft | +| [`docs/architecture.md`](docs/architecture.md) | Software architecture: crate workspace, intermediate document model (IR), CLI, MCP server | +| [`docs/development-plan.md`](docs/development-plan.md) | Detailed development plan in 9 phases, with deliverables, acceptance criteria, estimates and testing strategy | +| [`AGENTS.md`](AGENTS.md) | Operational guide for developers and AI agents working in this repository | +| [`corpus/README.md`](corpus/README.md) | The test corpus: what each document isolates and how it is regenerated | +| [`docs/spikes/`](docs/spikes/) | Risk-spike reports, with the decision that closed each one | +| [`kb/`](kb/) | Knowledge base: what is built, how it is structured, technical decisions and what later phases will face | -## Desarrollo +## Development ```bash cargo test --workspace cargo clippy --workspace --all-targets -- -D warnings cargo fmt --all -- --check -python3 corpus/generate.py --check # el corpus es generado, no dibujado a mano +python3 corpus/generate.py --check # the corpus is generated, not hand-drawn ``` -Los golden files viven junto al corpus (`corpus/docx/*.expected.dmk.md`). Para actualizarlos: -`DOCSAI_UPDATE_GOLDENS=1 cargo test -p docsai-convert --test goldens`, y **se revisa el diff**. +Golden files live next to the corpus (`corpus/docx/*.expected.dmk.md`). To update them: +`DOCSAI_UPDATE_GOLDENS=1 cargo test -p docsai-convert --test goldens`, and **review the diff**. -## Principios de diseño +## Design principles -1. **Binario único, sin runtime externo**: pura biblioteca Rust siempre que sea posible; los fallbacks externos (LibreOffice headless para `.doc`) son opcionales y detectados en tiempo de ejecución, nunca requeridos. -2. **IR pivote**: todos los formatos convergen en un modelo de documento intermedio (inspirado en el AST de Pandoc y en DoclingDocument); los conversores nunca se hablan entre sí directamente. -3. **Fidelidad medible**: la pérdida de formato no se estima, se mide — el comando `roundtrip` y la suite de tests de ida y vuelta forman parte del producto. -4. **Markdown primero legible, después completo**: los metadatos extendidos degradan con elegancia; un visor Markdown normal muestra un documento útil aunque ignore los atributos. -5. **Los datos del usuario nunca se pierden en silencio**: lo que no se pueda representar se conserva en bloques raw o se reporta como advertencia explícita. +1. **Single binary, no external runtime**: pure Rust libraries whenever possible; external fallbacks (LibreOffice headless for `.doc`) are optional and detected at runtime, never required. +2. **Pivot IR**: every format converges on an intermediate document model (inspired by Pandoc's AST and DoclingDocument); converters never talk to each other directly. +3. **Measurable fidelity**: format loss is not estimated, it is measured — the `roundtrip` command and the round-trip test suite are part of the product. +4. **Markdown readable first, complete second**: extended metadata degrades gracefully; a normal Markdown viewer shows a useful document even if it ignores the attributes. +5. **User data is never lost silently**: what cannot be represented is kept in raw blocks or reported as an explicit warning. -## Licencia +## License -Ver [LICENSE](LICENSE). +See [LICENSE](LICENSE). diff --git a/corpus/README.md b/corpus/README.md index 7bd3bdb..a558e16 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -1,83 +1,87 @@ -# Corpus de pruebas +# Test corpus -Documentos de prueba versionados. **Un rasgo por fichero**: cuando un golden falla, el fichero -que falla señala el área del lector que se ha roto. +Versioned test documents. **One trait per file**: when a golden fails, the failing +file points at the reader area that broke. -Ningún documento contiene datos reales ni privados (`AGENTS.md` §6). +No document contains real or private data (`AGENTS.md` §6). -## Cómo se generan +## How they are generated -Todos los ficheros los produce `corpus/generate.py`, sin dependencias más allá de la biblioteca -estándar de Python 3: +Every file is produced by `corpus/generate.py`, with no dependencies beyond the +Python 3 standard library: ```bash -python3 corpus/generate.py # regenera todo -python3 corpus/generate.py --check # falla si el árbol está desfasado (lo ejecuta CI) +python3 corpus/generate.py # regenerate everything +python3 corpus/generate.py --check # fails if the tree is out of date (run by CI) ``` -Que el corpus sea **generado y no dibujado a mano** es deliberado: +That the corpus is **generated rather than hand-drawn** is deliberate: -- El XML de cada documento vive en el generador, donde se revisa en un `git diff` normal; un - `.docx` hecho con Word es una caja opaca en la revisión. -- Los paquetes se escriben con marca de tiempo y orden de miembros fijos, así que regenerar - produce archivos byte a byte idénticos y el repositorio no acumula ruido binario. -- Los medios (PNG, GIF, EMF) se sintetizan en Python puro, sin Pillow, para que el generador - funcione igual en las tres plataformas de CI. +- The XML of each document lives in the generator, where it is reviewed in a + normal `git diff`; a `.docx` made with Word is an opaque box in review. +- Packages are written with a fixed timestamp and member order, so regenerating + produces byte-identical files and the repository does not accumulate binary + noise. +- Media (PNG, GIF, EMF) are synthesised in pure Python, without Pillow, so the + generator works the same on the three CI platforms. -La contrapartida: son documentos *mínimos*, no documentos de Word reales. Los documentos reales -anonimizados que pide la Fase 1 (tarea 10) y los corpus de rendimiento y adversarios de la Fase 8 -se añadirán aparte; el test de rendimiento de 50 páginas sintetiza su propio documento en tiempo -de ejecución (`crates/docsai-convert/tests/goldens.rs`). +The trade-off: these are *minimal* documents, not real Word documents. The +anonymised real-world documents required by Phase 1 (task 10) and the +performance and adversarial corpora of Phase 8 will be added separately; the +50-page performance test synthesises its own document at runtime +(`crates/docsai-convert/tests/goldens.rs`). ## Golden files -Cada `docx/.docx` tiene al lado su DocMark esperado, `docx/.expected.dmk.md`. -Los comparan los tests de `crates/docsai-convert/tests/goldens.rs`. Para actualizarlos: +Each `docx/.docx` has its expected DocMark beside it, +`docx/.expected.dmk.md`. They are compared by the tests in +`crates/docsai-convert/tests/goldens.rs`. To update them: ```bash DOCSAI_UPDATE_GOLDENS=1 cargo test -p docsai-convert --test goldens ``` -El diff resultante **se revisa a mano** antes de confirmarlo: un golden actualizado sin mirar es -un test que ha dejado de comprobar nada. +The resulting diff **is reviewed by hand** before confirming: a golden updated +without looking is a test that has stopped checking anything. -## Documentos de texto (`docx/`) +## Text documents (`docx/`) -| Fichero | Rasgo que aísla | +| File | Trait it isolates | |---|---| -| `basic-text.docx` | Párrafos, salto de línea manual, párrafo vacío y los caracteres que Markdown escapa | -| `basic-styles.docx` | Negrita, cursiva, tachado, subrayado, color, resaltado, fuente y tamaño, sub/superíndice, hipervínculo, alineación y sangrías | -| `nested-lists.docx` | `numbering.xml` con tres niveles numerados y dos de viñetas; reconstrucción del árbol desde pares `(numId, ilvl)` | -| `table-simple.docx` | Tabla regular con estilo de tabla y rejilla | -| `table-merged.docx` | `gridSpan` y `vMerge` → `colspan`/`rowspan` y celdas absorbidas | -| `images-inline.docx` | Imágenes `wp:inline`: PNG con alt/título/nombre, GIF entre texto, EMF vectorial | -| `images-floating.docx` | `wp:anchor`: offsets relativos al margen con `wrapSquare`, alineación simbólica relativa a la página con `wrapTopAndBottom`, y marca de agua con `behindDoc` | -| `images-transformed.docx` | Rotación 45°, recorte `a:srcRect` con borde `a:ln`, volteo H+V y escala ≠ 100 % | -| `images-duplicated.docx` | El mismo mapa de bits en tres partes distintas del paquete con geometrías distintas: prueba la deduplicación del `AssetStore` | -| `images-vml.docx` | `w:pict` con VML heredado (documentos convertidos desde `.doc`) | -| `headers-footers.docx` | `sectPr` con cabecera por defecto y de primera página, pie con campos `PAGE`/`NUMPAGES`, dos columnas y `titlePg` | -| `footnotes.docx` | `footnotes.xml` con dos notas, una con formato dentro | -| `custom-styles.docx` | Estilo personalizado, estilo heredado con delta directo, estilo de carácter y propiedades personalizadas del documento | -| `fields-raw.docx` | Control de contenido `w:sdt`, campo complejo `TOC` y campo simple `DATE` | - -## Hojas de cálculo (`xlsx/`) - -Generadas en la Fase 0 para que el corpus esté completo; las consume la **Fase 3**, que es cuando -existirá el lector de `xlsx`. - -| Fichero | Rasgo que aísla | +| `basic-text.docx` | Paragraphs, manual line break, empty paragraph and characters that Markdown escapes | +| `basic-styles.docx` | Bold, italic, strike, underline, colour, highlight, font and size, sub/superscript, hyperlink, alignment and indents | +| `nested-lists.docx` | `numbering.xml` with three numbered levels and two bullet levels; tree rebuild from `(numId, ilvl)` pairs | +| `table-simple.docx` | Regular table with table style and grid | +| `table-merged.docx` | `gridSpan` and `vMerge` → `colspan`/`rowspan` and absorbed cells | +| `images-inline.docx` | `wp:inline` images: PNG with alt/title/name, GIF between text, EMF vector | +| `images-floating.docx` | `wp:anchor`: margin-relative offsets with `wrapSquare`, symbolic page-relative alignment with `wrapTopAndBottom`, and watermark with `behindDoc` | +| `images-transformed.docx` | 45° rotation, `a:srcRect` crop with `a:ln` border, H+V flip and scale ≠ 100 % | +| `images-duplicated.docx` | The same bitmap in three distinct package parts with different geometries: tests `AssetStore` deduplication | +| `images-vml.docx` | `w:pict` with legacy VML (documents converted from `.doc`) | +| `headers-footers.docx` | `sectPr` with default and first-page header, footer with `PAGE`/`NUMPAGES` fields, two columns and `titlePg` | +| `footnotes.docx` | `footnotes.xml` with two notes, one with inner formatting | +| `custom-styles.docx` | Custom style, inherited style with direct delta, character style and custom document properties | +| `fields-raw.docx` | `w:sdt` content control, complex `TOC` field and simple `DATE` field | + +## Spreadsheets (`xlsx/`) + +Generated in Phase 0 so the corpus is complete; consumed by **Phase 3**, when +the `xlsx` reader exists. + +| File | Trait it isolates | |---|---| -| `values-types.xlsx` | Los seis tipos de celda: entero, decimal, booleano, error, fecha (serial + `numFmt`) y cadena en línea | -| `formulas-basic.xlsx` | Fórmulas con valor cacheado, referencia entre celdas y un nombre definido | -| `formulas-shared.xlsx` | Fórmulas compartidas (`t="shared"`) y de matriz (`t="array"`) | -| `number-formats.xlsx` | Moneda, fecha, porcentaje y millares por `numFmtId` | -| `merged-cells.xlsx` | `mergeCells` horizontal y vertical, y ancho de columna personalizado | -| `images-anchored.xlsx` | Los tres anclajes de hoja: `twoCellAnchor`, `oneCellAnchor` y `absoluteAnchor` | - -## Añadir un documento - -1. Escribe una función `docx_()` en `generate.py` y añádela a `GENERATORS`. -2. Regenera (`python3 corpus/generate.py`) y añade la fila a la tabla de arriba. -3. Genera su golden y **revisa el diff**. -4. Si el rasgo aún no está implementado, el golden documentará la degradación actual (un - raw-block, por ejemplo). Eso es correcto: hace visible lo que falta. +| `values-types.xlsx` | The six cell types: integer, decimal, boolean, error, date (serial + `numFmt`) and inline string | +| `formulas-basic.xlsx` | Formulas with cached value, cell reference and a defined name | +| `formulas-shared.xlsx` | Shared formulas (`t="shared"`) and array formulas (`t="array"`) | +| `number-formats.xlsx` | Currency, date, percentage and thousands via `numFmtId` | +| `merged-cells.xlsx` | Horizontal and vertical `mergeCells`, and custom column width | +| `images-anchored.xlsx` | The three sheet anchors: `twoCellAnchor`, `oneCellAnchor` and `absoluteAnchor` | + +## Adding a document + +1. Write a `docx_()` function in `generate.py` and add it to `GENERATORS`. +2. Regenerate (`python3 corpus/generate.py`) and add the row to the table above. +3. Generate its golden and **review the diff**. +4. If the trait is not implemented yet, the golden will document the current + degradation (a raw-block, for example). That is correct: it makes the gap + visible. diff --git a/crates/docsai-cli/src/main.rs b/crates/docsai-cli/src/main.rs index 9fb8677..3c6faa5 100644 --- a/crates/docsai-cli/src/main.rs +++ b/crates/docsai-cli/src/main.rs @@ -8,7 +8,7 @@ use clap::{Parser, Subcommand}; use docsai_convert::{ConvertOptions, Fidelity, SUPPORT}; use docsai_model::Format; -/// Exit codes (arquitectura §5). +/// Exit codes (architecture §5). const EXIT_OK: u8 = 0; const EXIT_WARNINGS: u8 = 1; const EXIT_INPUT: u8 = 2; @@ -61,6 +61,20 @@ enum Command { #[arg(long)] json: bool, }, + /// Round-trip a document (Office → DocMark → Office → DocMark) and compare. + Roundtrip { + /// Input Office document. + input: PathBuf, + /// Optional path for the regenerated .docx. + #[arg(short, long)] + output: Option, + /// How much of the source survives: full, standard or plain. + #[arg(long, default_value = "full")] + fidelity: String, + /// Print the fidelity report as JSON on stdout. + #[arg(long)] + json: bool, + }, } fn main() -> ExitCode { @@ -89,6 +103,60 @@ fn run(cli: &Cli) -> anyhow::Result { print_formats(*json); Ok(EXIT_OK) } + Command::Roundtrip { + input, + output, + fidelity, + json, + } => { + let fidelity = docsai_convert::Fidelity::parse(fidelity).ok_or_else(|| { + anyhow::anyhow!("unknown --fidelity `{fidelity}`; use full, standard or plain") + })?; + let options = docsai_convert::ConvertOptions { + fidelity, + ..Default::default() + }; + let outcome = docsai_convert::roundtrip_file(input, output.as_deref(), &options)?; + if *json { + let payload = serde_json::json!({ + "source_format": outcome.source_format.as_str(), + "identical": outcome.identical, + "report": outcome.report, + }); + println!("{}", serde_json::to_string_pretty(&payload)?); + } else { + eprintln!( + "docsai: roundtrip {} — second DocMark {} first", + outcome.source_format.as_str(), + if outcome.identical { + "matches" + } else { + "differs from" + } + ); + if !outcome.identical { + eprintln!( + " first: {} bytes, second: {} bytes", + outcome.first_markdown.len(), + outcome.second_markdown.len() + ); + } + if !outcome.report.warnings.is_empty() { + eprintln!( + " {} warning(s), {} raw-block(s)", + outcome.report.warnings.len(), + outcome.report.raw_blocks_emitted + ); + if cli.verbose { + for warning in &outcome.report.warnings { + eprintln!(" [{:?}] {}", warning.severity(), warning.message()); + } + } + } + } + let lost = outcome.report.has_severe() || !outcome.identical; + Ok(if lost { EXIT_WARNINGS } else { EXIT_OK }) + } Command::Convert { input, output, diff --git a/crates/docsai-convert/src/assets.rs b/crates/docsai-convert/src/assets.rs index f4361b2..0ca4e45 100644 --- a/crates/docsai-convert/src/assets.rs +++ b/crates/docsai-convert/src/assets.rs @@ -1,4 +1,4 @@ -//! [`AssetStore`] implementations (arquitectura §3.2). +//! [`AssetStore`] implementations (architecture §3.2). use std::collections::BTreeMap; use std::path::{Path, PathBuf}; diff --git a/crates/docsai-convert/src/lib.rs b/crates/docsai-convert/src/lib.rs index d61fc38..b64f445 100644 --- a/crates/docsai-convert/src/lib.rs +++ b/crates/docsai-convert/src/lib.rs @@ -1,7 +1,7 @@ //! Conversion orchestration: format detection, pipelines and asset management. //! //! This is the only crate that knows about more than one format; the readers -//! and writers never talk to each other (arquitectura §1). +//! and writers never talk to each other (architecture §1). #![forbid(unsafe_code)] @@ -10,7 +10,10 @@ mod pipeline; pub use assets::DirAssetStore; pub use docsai_docmark::{Fidelity, Options as DocMarkOptions}; -pub use pipeline::{convert_file, read_document, ConvertOptions, Outcome}; +pub use pipeline::{ + convert_file, read_document, read_path, roundtrip_file, ConvertOptions, Outcome, + RoundtripOutcome, +}; use docsai_model::Format; @@ -20,6 +23,12 @@ pub enum ConvertError { #[error("{0}")] Read(#[from] docsai_office::ReadError), + #[error("{0}")] + Write(#[from] docsai_office::WriteError), + + #[error("{0}")] + Parse(#[from] docsai_docmark::ParseError), + #[error("i/o error on `{path}`: {source}")] Io { path: String, @@ -43,7 +52,7 @@ pub struct FormatSupport { pub format: Format, pub read: bool, pub write: bool, - /// Phase of `docs/plan-desarrollo.md` that lands the missing direction. + /// Phase of `docs/development-plan.md` that lands the missing direction. pub note: &'static str, } @@ -52,44 +61,44 @@ pub const SUPPORT: &[FormatSupport] = &[ FormatSupport { format: Format::Docx, read: true, - write: false, - note: "writing arrives in Fase 2", + write: true, + note: "Phase 2", }, FormatSupport { format: Format::Doc, read: false, write: false, - note: "reading arrives in Fase 5; writing is out of scope", + note: "reading arrives in Phase 5; writing is out of scope", }, FormatSupport { format: Format::Xlsx, read: false, write: false, - note: "arrives in Fase 3", + note: "arrives in Phase 3", }, FormatSupport { format: Format::Xls, read: false, write: false, - note: "reading arrives in Fase 3; writing is out of scope", + note: "reading arrives in Phase 3; writing is out of scope", }, FormatSupport { format: Format::Odt, read: false, write: false, - note: "arrives in Fase 4", + note: "arrives in Phase 4", }, FormatSupport { format: Format::Ods, read: false, write: false, - note: "arrives in Fase 4", + note: "arrives in Phase 4", }, FormatSupport { format: Format::DocMark, - read: false, + read: true, write: true, - note: "reading arrives in Fase 2", + note: "Phase 2", }, ]; @@ -120,8 +129,13 @@ mod tests { ); } assert!(can_read(Format::Docx)); - assert!(!can_write(Format::Docx), "docx writing is Fase 2"); + assert!(can_write(Format::Docx)); + assert!(can_read(Format::DocMark)); assert!(can_write(Format::DocMark)); + assert_eq!( + can_write(Format::Docx), + docsai_office::WRITABLE.contains(&Format::Docx) + ); } #[test] diff --git a/crates/docsai-convert/src/pipeline.rs b/crates/docsai-convert/src/pipeline.rs index b3f4ee2..a810256 100644 --- a/crates/docsai-convert/src/pipeline.rs +++ b/crates/docsai-convert/src/pipeline.rs @@ -1,11 +1,12 @@ //! The conversion pipelines. -use std::io::{Read, Seek}; +use std::fs::File; +use std::io::{Cursor, Read, Seek}; use std::path::{Path, PathBuf}; use docsai_docmark::{Fidelity, Options as DocMarkOptions}; use docsai_model::assets::AssetStore; -use docsai_model::{ConversionReport, Document, Format}; +use docsai_model::{ConversionReport, Document, Format, MemoryAssetStore}; use crate::assets::DirAssetStore; use crate::ConvertError; @@ -35,13 +36,24 @@ impl Default for ConvertOptions { pub struct Outcome { pub source_format: Format, pub target_format: Format, - /// The DocMark text, also written to `output_path` when one was given. + /// The DocMark text when the target (or intermediate) is DocMark. pub markdown: String, pub output_path: Option, pub assets_written: Vec, pub report: ConversionReport, } +/// Result of a round-trip fidelity check. +#[derive(Debug)] +pub struct RoundtripOutcome { + pub source_format: Format, + pub first_markdown: String, + pub second_markdown: String, + pub identical: bool, + pub report: ConversionReport, + pub output_path: Option, +} + /// Reads any supported document into the IR, detecting its format by content. pub fn read_document( reader: R, @@ -59,7 +71,88 @@ pub fn read_document( _ => ConvertError::UnknownFormat(hint.unwrap_or("").to_string()), }); } - let (document, report) = docsai_office::read(format, reader, assets)?; + let (document, report) = match format { + Format::DocMark => { + let mut bytes = Vec::new(); + reader + .read_to_end(&mut bytes) + .map_err(|source| ConvertError::Io { + path: hint.unwrap_or("").to_string(), + source, + })?; + let text = String::from_utf8_lossy(&bytes); + let base = hint.and_then(|h| { + let p = Path::new(h); + if p.is_absolute() || p.components().count() > 1 { + p.parent() + } else { + None + } + }); + let (document, report) = docsai_docmark::parse_with_base(&text, base, assets) + .map_err(ConvertError::Parse)?; + (document, report) + } + other => docsai_office::read(other, reader, assets)?, + }; + docsai_model::validate::validate(&document).map_err(|errors| { + ConvertError::Invalid( + errors + .iter() + .map(|e| e.to_string()) + .collect::>() + .join("; "), + ) + })?; + Ok((document, format, report)) +} + +/// Reads a document from a path on disk. +pub fn read_path( + input: &Path, + assets: &mut dyn AssetStore, +) -> Result<(Document, Format, ConversionReport), ConvertError> { + let hint = input.file_name().and_then(|n| n.to_str()); + let (format, score) = { + let mut probe = File::open(input).map_err(|source| ConvertError::Io { + path: input.display().to_string(), + source, + })?; + docsai_office::detect(&mut probe, hint) + }; + if score == docsai_office::DetectScore::No || !crate::can_read(format) { + return Err(match crate::SUPPORT.iter().find(|s| s.format == format) { + Some(support) if score != docsai_office::DetectScore::No => ConvertError::Unsupported { + from: support.format, + to: Format::DocMark, + }, + _ => ConvertError::UnknownFormat(input.display().to_string()), + }); + } + if format == Format::DocMark { + let text = std::fs::read_to_string(input).map_err(|source| ConvertError::Io { + path: input.display().to_string(), + source, + })?; + let base = input.parent(); + let (document, report) = + docsai_docmark::parse_with_base(&text, base, assets).map_err(ConvertError::Parse)?; + docsai_model::validate::validate(&document).map_err(|errors| { + ConvertError::Invalid( + errors + .iter() + .map(|e| e.to_string()) + .collect::>() + .join("; "), + ) + })?; + return Ok((document, format, report)); + } + let file = File::open(input).map_err(|source| ConvertError::Io { + path: input.display().to_string(), + source, + })?; + let (document, report) = docsai_office::read(format, file, assets)?; docsai_model::validate::validate(&document).map_err(|errors| { ConvertError::Invalid( errors @@ -74,8 +167,7 @@ pub fn read_document( /// Converts a file on disk. /// -/// With `output` set, the DocMark is written there and media go to the assets -/// directory; without it the text is returned for the caller to print. +/// Supports Office → DocMark, DocMark → Office, and DocMark → DocMark. pub fn convert_file( input: &Path, output: Option<&Path>, @@ -89,61 +181,208 @@ pub fn convert_file( }); } - let file = std::fs::File::open(input).map_err(|source| ConvertError::Io { - path: input.display().to_string(), - source, - })?; - let hint = input.file_name().and_then(|n| n.to_str()); - let assets_dir = options .assets_dir .clone() .unwrap_or_else(|| default_assets_dir(input, output)); let mut store = DirAssetStore::new(assets_dir.clone()); - let (document, source_format, mut report) = read_document(file, hint, &mut store)?; + let (document, source_format, mut report) = read_path(input, &mut store)?; + if !crate::can_read(source_format) { + return Err(ConvertError::Unsupported { + from: source_format, + to: target, + }); + } - let docmark_options = DocMarkOptions { - fidelity: options.fidelity, - assets_dir: relative_assets_dir(&assets_dir, output), - source_format, - }; - let (markdown, write_report) = docsai_docmark::serialize(&document, &store, &docmark_options); - report.merge(write_report); + match target { + Format::DocMark => { + let docmark_options = DocMarkOptions { + fidelity: options.fidelity, + assets_dir: relative_assets_dir(&assets_dir, output), + source_format, + }; + let (markdown, write_report) = + docsai_docmark::serialize(&document, &store, &docmark_options); + report.merge(write_report); - if let Some(output) = output { - if let Some(parent) = output.parent().filter(|p| !p.as_os_str().is_empty()) { - std::fs::create_dir_all(parent).map_err(|source| ConvertError::Io { - path: parent.display().to_string(), + if let Some(output) = output { + ensure_parent(output)?; + std::fs::write(output, markdown.as_bytes()).map_err(|source| ConvertError::Io { + path: output.display().to_string(), + source, + })?; + } + + Ok(Outcome { + source_format, + target_format: target, + markdown, + output_path: output.map(Path::to_path_buf), + assets_written: store.written().to_vec(), + report, + }) + } + Format::Docx => { + let output = output.ok_or_else(|| { + ConvertError::Invalid("writing .docx requires an --output path".into()) + })?; + ensure_parent(output)?; + let file = File::create(output).map_err(|source| ConvertError::Io { + path: output.display().to_string(), source, })?; + let write_report = docsai_office::write_docx(&document, &store, file)?; + report.merge(write_report); + + let (markdown, _) = docsai_docmark::serialize( + &document, + &store, + &DocMarkOptions { + fidelity: options.fidelity, + assets_dir: relative_assets_dir(&assets_dir, Some(output)), + source_format, + }, + ); + + Ok(Outcome { + source_format, + target_format: target, + markdown, + output_path: Some(output.to_path_buf()), + assets_written: store.written().to_vec(), + report, + }) } - std::fs::write(output, markdown.as_bytes()).map_err(|source| ConvertError::Io { + other => Err(ConvertError::Unsupported { + from: source_format, + to: other, + }), + } +} + +/// Office → DocMark → Office → DocMark idempotence check. +/// +/// Writes the regenerated Office file to `output` when given, and returns the +/// second DocMark pass in `markdown`. +pub fn roundtrip_file( + input: &Path, + output: Option<&Path>, + options: &ConvertOptions, +) -> Result { + let assets_dir = options + .assets_dir + .clone() + .unwrap_or_else(|| default_assets_dir(input, output)); + + let mut store1 = MemoryAssetStore::new(); + let (document1, source_format, mut report) = read_path(input, &mut store1)?; + if source_format != Format::Docx && source_format != Format::DocMark { + return Err(ConvertError::Unsupported { + from: source_format, + to: Format::Docx, + }); + } + + let (md1, r1) = docsai_docmark::serialize( + &document1, + &store1, + &DocMarkOptions { + fidelity: Fidelity::Full, + assets_dir: "assets".into(), + source_format, + }, + ); + report.merge(r1); + + let mut store2 = MemoryAssetStore::new(); + for id in store1.ids() { + if let Some(bytes) = store1.get(&id) { + let _ = store2.put(bytes); + } + } + let (document2, r2) = docsai_docmark::parse(&md1, &mut store2).map_err(ConvertError::Parse)?; + report.merge(r2); + + let mut docx_buf = Cursor::new(Vec::new()); + let r3 = docsai_office::write_docx(&document2, &store2, &mut docx_buf)?; + report.merge(r3); + + if let Some(output) = output { + ensure_parent(output)?; + std::fs::write(output, docx_buf.get_ref()).map_err(|source| ConvertError::Io { path: output.display().to_string(), source, })?; } - Ok(Outcome { + docx_buf.set_position(0); + let mut store3 = MemoryAssetStore::new(); + let (document3, r4) = docsai_office::read_docx(&mut docx_buf, &mut store3)?; + report.merge(r4); + let (md2, r5) = docsai_docmark::serialize( + &document3, + &store3, + &DocMarkOptions { + fidelity: Fidelity::Full, + assets_dir: "assets".into(), + source_format: Format::Docx, + }, + ); + report.merge(r5); + + let mut written = Vec::new(); + if output.is_some() { + let dir = assets_dir; + for id in store3.ids() { + if let (Some(bytes), Some(info)) = (store3.get(&id), store3.info(&id)) { + let _ = std::fs::create_dir_all(&dir); + let path = dir.join(&info.file_name); + if std::fs::write(&path, bytes).is_ok() { + written.push(path); + } + } + } + } + + let identical = md1 == md2; + if !identical { + report.warn(docsai_model::Warning::Degraded { + what: "roundtrip".into(), + why: "second DocMark pass differs from the first".into(), + }); + } + Ok(RoundtripOutcome { source_format, - target_format: target, - markdown, - output_path: output.map(Path::to_path_buf), - assets_written: store.written().to_vec(), + first_markdown: md1, + second_markdown: md2, + identical, report, + output_path: output.map(Path::to_path_buf), }) } +fn ensure_parent(output: &Path) -> Result<(), ConvertError> { + if let Some(parent) = output.parent().filter(|p| !p.as_os_str().is_empty()) { + std::fs::create_dir_all(parent).map_err(|source| ConvertError::Io { + path: parent.display().to_string(), + source, + })?; + } + Ok(()) +} + fn resolve_target(output: Option<&Path>, options: &ConvertOptions) -> Result { if let Some(target) = options.target { return Ok(target); } let Some(output) = output else { - // Writing to stdout: DocMark is the only text format there is. return Ok(Format::DocMark); }; let name = output.file_name().and_then(|n| n.to_str()).unwrap_or(""); - // `.dmk.md` and `.md` both mean DocMark. + if name.ends_with(".dmk.md") || name.ends_with(".md") { + return Ok(Format::DocMark); + } match name.rsplit('.').next().and_then(Format::parse) { Some(format) => Ok(format), None => Err(ConvertError::UnknownFormat(output.display().to_string())), @@ -159,12 +398,9 @@ fn default_assets_dir(input: &Path, output: Option<&Path>) -> PathBuf { base.join("assets") } -/// The path DocMark links should use: relative to the output file when -/// possible, so the `.dmk.md` stays portable. fn relative_assets_dir(assets_dir: &Path, output: Option<&Path>) -> String { let parent = output.and_then(|o| o.parent()).unwrap_or(Path::new("")); let relative = assets_dir.strip_prefix(parent).unwrap_or(assets_dir); - // DocMark links always use `/`, on every platform. relative.to_string_lossy().replace('\\', "/") } @@ -220,9 +456,45 @@ mod tests { } #[test] - fn an_unsupported_target_is_rejected_before_reading() { + fn docx_target_writes_a_package() { + let dir = temp_dir("to-docx"); + let dmk = dir.join("in.dmk.md"); + let md = convert_file(&corpus("basic-text.docx"), None, &ConvertOptions::default()) + .unwrap() + .markdown; + std::fs::write(&dmk, md).unwrap(); + let out = dir.join("out.docx"); + let outcome = convert_file( + &dmk, + Some(&out), + &ConvertOptions { + target: Some(Format::Docx), + ..Default::default() + }, + ) + .expect("docx write"); + assert_eq!(outcome.target_format, Format::Docx); + assert!(out.exists()); + let bytes = std::fs::read(&out).unwrap(); + assert!(bytes.starts_with(b"PK")); + let _ = std::fs::remove_dir_all(&dir); + } + + #[test] + fn roundtrip_basic_text_is_stable() { + let outcome = roundtrip_file(&corpus("basic-text.docx"), None, &ConvertOptions::default()) + .expect("roundtrip"); + assert!( + outcome.identical, + "basic-text should round-trip cleanly\n--- first ---\n{}\n--- second ---\n{}", + outcome.first_markdown, outcome.second_markdown + ); + } + + #[test] + fn an_unsupported_target_is_rejected() { let options = ConvertOptions { - target: Some(Format::Docx), + target: Some(Format::Odt), ..Default::default() }; let error = convert_file(&corpus("basic-text.docx"), None, &options).unwrap_err(); diff --git a/crates/docsai-convert/tests/goldens.rs b/crates/docsai-convert/tests/goldens.rs index 2f100af..7334357 100644 --- a/crates/docsai-convert/tests/goldens.rs +++ b/crates/docsai-convert/tests/goldens.rs @@ -109,7 +109,7 @@ fn serialisation_is_deterministic() { assert_eq!( first, second, - "{} serialises differently on a second run", + "{} serializes differently on a second run", document.display() ); } @@ -137,8 +137,8 @@ fn every_document_uses_unix_line_endings_and_no_bom() { } } -/// Fase 1 acceptance criterion: *«La salida en `--fidelity plain` es CommonMark -/// limpio verificado con comrak»*. +/// Phase 1 acceptance criterion: *"`--fidelity plain` output is clean CommonMark +/// verified with comrak"*. #[test] fn plain_fidelity_is_clean_commonmark() { let mut options = comrak::Options::default(); @@ -204,8 +204,8 @@ fn standard_fidelity_drops_raw_blocks_and_says_so() { assert_eq!(dropped as u32, full_report.raw_blocks_emitted); } -/// Fase 1 acceptance criterion: *«Un docx real de 50+ páginas convierte en -/// < 1 s con < 10 raw-blocks»*. +/// Phase 1 acceptance criterion: *"A real 50+ page docx converts in +/// < 1 s with < 10 raw-blocks"*. #[test] fn a_fifty_page_document_converts_quickly_with_few_raw_blocks() { let path = std::env::temp_dir().join(format!("docsai-perf-{}.docx", std::process::id())); @@ -230,7 +230,7 @@ fn a_fifty_page_document_converts_quickly_with_few_raw_blocks() { report.raw_blocks_emitted ); - // The budget of arquitectura §8 is for an optimised build; a debug build + // The budget of architecture §8 is for an optimised build; a debug build // runs roughly an order of magnitude slower, so it gets a looser bound // rather than no bound at all. let budget = if cfg!(debug_assertions) { diff --git a/crates/docsai-docmark/src/attrs.rs b/crates/docsai-docmark/src/attrs.rs index 8823361..efc1287 100644 --- a/crates/docsai-docmark/src/attrs.rs +++ b/crates/docsai-docmark/src/attrs.rs @@ -2,7 +2,7 @@ //! //! The canonical order of spec §8 is enforced here, in one place: id first, //! then classes alphabetically, then keys alphabetically. That ordering is -//! what makes the serialiser deterministic and the golden diffs stable. +//! what makes the serializer deterministic and the golden diffs stable. use std::collections::BTreeMap; @@ -99,6 +99,150 @@ impl Attrs { format!(" {}", self.render()) } } + + /// The id, if any (`#id`). + pub fn id_ref(&self) -> Option<&str> { + self.id.as_deref() + } + + /// Classes in insertion order. + pub fn classes(&self) -> &[String] { + &self.classes + } + + /// True when `class` is present. + pub fn has_class(&self, class: &str) -> bool { + self.classes.iter().any(|c| c == class) + } + + /// First class, commonly the style id. + pub fn first_class(&self) -> Option<&str> { + self.classes.first().map(String::as_str) + } + + /// Value of `key`, if present. + pub fn get(&self, key: &str) -> Option<&str> { + self.pairs.get(key).map(String::as_str) + } + + /// Removes and returns the value of `key`. + pub fn take(&mut self, key: &str) -> Option { + self.pairs.remove(key) + } + + /// All key/value pairs in sorted order. + pub fn pairs(&self) -> &BTreeMap { + &self.pairs + } + + /// Parses a Pandoc-style attribute block, with or without surrounding `{}`. + /// + /// ``` + /// use docsai_docmark::attrs::Attrs; + /// let a = Attrs::parse(r#"{#img-1 .Quote title="Figura 1" width=450px}"#).unwrap(); + /// assert_eq!(a.id_ref(), Some("img-1")); + /// assert!(a.has_class("Quote")); + /// assert_eq!(a.get("width"), Some("450px")); + /// assert_eq!(a.get("title"), Some("Figura 1")); + /// ``` + pub fn parse(input: &str) -> Option { + let input = input.trim(); + let inner = if input.starts_with('{') && input.ends_with('}') && input.len() >= 2 { + &input[1..input.len() - 1] + } else { + input + }; + let mut attrs = Attrs::new(); + let mut rest = inner.trim(); + while !rest.is_empty() { + rest = rest.trim_start(); + if rest.is_empty() { + break; + } + if let Some(stripped) = rest.strip_prefix('#') { + let (id, next) = take_token(stripped); + if id.is_empty() { + return None; + } + attrs.id = Some(id.to_string()); + rest = next; + continue; + } + if let Some(stripped) = rest.strip_prefix('.') { + let (class, next) = take_token(stripped); + if class.is_empty() { + return None; + } + attrs.class(class); + rest = next; + continue; + } + let (key, after_key) = take_ident(rest); + if key.is_empty() { + return None; + } + let after_key = after_key.trim_start(); + let after_eq = after_key.strip_prefix('=')?; + let after_eq = after_eq.trim_start(); + let (value, next) = take_value(after_eq)?; + attrs.set(key, value); + rest = next; + } + Some(attrs) + } +} + +/// Identifier or class/id token: letters, digits, `_`, `-`, `.` (for class names). +fn take_token(input: &str) -> (&str, &str) { + let end = input + .char_indices() + .find(|(_, c)| !(c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.'))) + .map(|(i, _)| i) + .unwrap_or(input.len()); + (&input[..end], &input[end..]) +} + +fn take_ident(input: &str) -> (&str, &str) { + let end = input + .char_indices() + .find(|(_, c)| !(c.is_ascii_alphanumeric() || matches!(c, '-' | '_'))) + .map(|(i, _)| i) + .unwrap_or(input.len()); + (&input[..end], &input[end..]) +} + +fn take_value(input: &str) -> Option<(String, &str)> { + if let Some(rest) = input.strip_prefix('"') { + let mut out = String::new(); + let mut chars = rest.char_indices(); + while let Some((i, c)) = chars.next() { + match c { + '\\' => match chars.next() { + Some((_, '"')) => out.push('"'), + Some((_, '\\')) => out.push('\\'), + Some((_, 'n')) => out.push('\n'), + Some((_, other)) => { + out.push('\\'); + out.push(other); + } + None => out.push('\\'), + }, + '"' => return Some((out, &rest[i + 1..])), + other => out.push(other), + } + } + None + } else { + let end = input + .char_indices() + .find(|(_, c)| c.is_whitespace()) + .map(|(i, _)| i) + .unwrap_or(input.len()); + if end == 0 { + return None; + } + Some((input[..end].to_string(), &input[end..])) + } } #[cfg(test)] @@ -152,4 +296,30 @@ mod tests { a.set("k", "1").set("k", "2"); assert_eq!(a.render(), "{k=2}"); } + + #[test] + fn parse_round_trips_with_render() { + let samples = [ + r#"{#img-1 .Quote title="Figura 1" width=450px}"#, + "{#x .Alpha .Beta a=2 z=1}", + r#"{color=#FF0000 link="https://example.com/a b" name="Logo corporativo" width=450px}"#, + "{on=true}", + "{.Heading1}", + "{align=center indent-first-line=24px}", + ]; + for sample in samples { + let parsed = Attrs::parse(sample).unwrap_or_else(|| panic!("parse {sample}")); + assert_eq!(parsed.render(), sample, "render mismatch for {sample}"); + } + } + + #[test] + fn parse_getters_and_take() { + let mut a = Attrs::parse(r#"{#id .A .B k=v}"#).unwrap(); + assert_eq!(a.id_ref(), Some("id")); + assert_eq!(a.classes(), &["A".to_string(), "B".to_string()]); + assert_eq!(a.get("k"), Some("v")); + assert_eq!(a.take("k"), Some("v".into())); + assert_eq!(a.get("k"), None); + } } diff --git a/crates/docsai-docmark/src/error.rs b/crates/docsai-docmark/src/error.rs new file mode 100644 index 0000000..0ac82a5 --- /dev/null +++ b/crates/docsai-docmark/src/error.rs @@ -0,0 +1,54 @@ +//! Parser errors for DocMark → IR. + +use std::io; +use std::path::PathBuf; + +/// Something went wrong parsing a DocMark document. +#[derive(Debug, thiserror::Error)] +pub enum ParseError { + #[error("invalid front matter at line {line}: {message}")] + InvalidFrontMatter { line: usize, message: String }, + + #[error("unexpected content at line {line}: {message}")] + Unexpected { line: usize, message: String }, + + #[error("unsupported: {what}")] + Unsupported { what: String }, + + #[error("i/o error{path}: {source}", path = path_suffix(.path))] + Io { + path: Option, + #[source] + source: io::Error, + }, + + #[error("asset error: {0}")] + Asset(#[from] docsai_model::assets::AssetError), +} + +fn path_suffix(path: &Option) -> String { + match path { + Some(p) => format!(" on `{}`", p.display()), + None => String::new(), + } +} + +impl ParseError { + pub fn front_matter(line: usize, message: impl Into) -> Self { + ParseError::InvalidFrontMatter { + line, + message: message.into(), + } + } + + pub fn unexpected(line: usize, message: impl Into) -> Self { + ParseError::Unexpected { + line, + message: message.into(), + } + } + + pub fn io(path: Option, source: io::Error) -> Self { + ParseError::Io { path, source } + } +} diff --git a/crates/docsai-docmark/src/escape.rs b/crates/docsai-docmark/src/escape.rs index b0a6dde..a3774ff 100644 --- a/crates/docsai-docmark/src/escape.rs +++ b/crates/docsai-docmark/src/escape.rs @@ -71,6 +71,32 @@ pub fn escape_attr_value(value: &str) -> String { out } +/// Inverse of [`escape`] for block (and table-cell) content: each backslash +/// escapes the following character, and is dropped. +/// +/// ``` +/// use docsai_docmark::escape::{escape, unescape, TextContext}; +/// let raw = r"a*b_c[d]"; +/// let once = escape(raw, TextContext::Block); +/// assert_eq!(unescape(&once), raw); +/// ``` +pub fn unescape(text: &str) -> String { + let mut out = String::with_capacity(text.len()); + let mut chars = text.chars(); + while let Some(c) = chars.next() { + if c == '\\' { + if let Some(next) = chars.next() { + out.push(next); + } else { + out.push('\\'); + } + } else { + out.push(c); + } + } + out +} + /// True when a value can be written without quotes: a number or a simple /// identifier (spec §8). pub fn is_bare_value(value: &str) -> bool { @@ -139,4 +165,24 @@ mod tests { let twice = escape(&once, TextContext::Block); assert_eq!(twice, r"a\\\*b"); } + + #[test] + fn unescape_inverts_escape() { + let samples = [ + "*asterisco* _guion_ `code` [x] \\z", + "# titulo", + "- item", + "1. uno", + "a|b", + "a&b", + "coma, punto. dos: puntos; ¿eñe?", + r"\*asterisco\* \_guion\_", + ]; + for sample in samples { + let escaped = escape(sample, TextContext::Block); + assert_eq!(unescape(&escaped), sample, "failed on {sample:?}"); + } + assert_eq!(unescape(r"a\|b"), "a|b"); + assert_eq!(unescape("trailing\\"), "trailing\\"); + } } diff --git a/crates/docsai-docmark/src/frontmatter.rs b/crates/docsai-docmark/src/frontmatter.rs index 8063548..5b6a3c8 100644 --- a/crates/docsai-docmark/src/frontmatter.rs +++ b/crates/docsai-docmark/src/frontmatter.rs @@ -2,7 +2,7 @@ //! //! Written by hand rather than through a YAML library: the spec demands byte //! determinism, and the schema is small and fully known. Doing it here also -//! keeps a YAML serialiser out of the dependency tree. +//! keeps a YAML serializer out of the dependency tree. use docsai_model::list::{ListCatalog, NumFormat}; use docsai_model::sheet::Workbook; diff --git a/crates/docsai-docmark/src/frontmatter_parse.rs b/crates/docsai-docmark/src/frontmatter_parse.rs new file mode 100644 index 0000000..4be4b1b --- /dev/null +++ b/crates/docsai-docmark/src/frontmatter_parse.rs @@ -0,0 +1,396 @@ +//! Front matter → IR catalogues and metadata. + +use std::collections::BTreeMap; + +use docsai_model::list::{ListCatalog, ListDef, ListId, ListLevel, NumFormat}; +use docsai_model::style::{ + Align, DocDefaults, FontProps, LineHeight, ParaProps, Style, StyleCatalog, StyleId, StyleType, + Underline, VertAlign, +}; +use docsai_model::text::{DocumentMeta, Margins, Orientation, PageGeometry}; +use docsai_model::units::{Length, Size}; +use docsai_model::{Format, DOCMARK_VERSION}; + +use crate::error::ParseError; +use crate::yaml::{self, Value}; + +#[derive(Debug)] +pub struct FrontMatter { + pub source_format: Format, + pub meta: DocumentMeta, + pub page: Option, + pub styles: StyleCatalog, + pub list_defs: ListCatalog, +} + +impl Default for FrontMatter { + fn default() -> Self { + FrontMatter { + source_format: Format::DocMark, + meta: DocumentMeta::default(), + page: None, + styles: StyleCatalog::default(), + list_defs: ListCatalog::default(), + } + } +} + +/// Parses the text between the opening and closing `---` fences. +pub fn parse(text: &str, start_line: usize) -> Result { + let map = yaml::parse_document(text).map_err(|message| { + // yaml errors already include line numbers relative to the body; + // shift them if needed. + let line = message + .strip_prefix("line ") + .and_then(|s| s.split(':').next()) + .and_then(|n| n.parse::().ok()) + .map(|n| start_line + n.saturating_sub(1)) + .unwrap_or(start_line); + ParseError::front_matter(line, message) + })?; + + let mut fm = FrontMatter { + source_format: Format::Docx, + ..Default::default() + }; + + if let Some(v) = map.get("docmark") { + let version = v.as_str().unwrap_or("").trim(); + if !version.is_empty() && version != DOCMARK_VERSION { + // Accept with a soft approach: still parse; future versions may + // need migration, but v1.0 is the only supported contract today. + if !version.starts_with('1') { + return Err(ParseError::front_matter( + start_line, + format!("unsupported docmark version `{version}`"), + )); + } + } + } + + if let Some(v) = map.get("source-format").and_then(|v| v.as_str()) { + fm.source_format = Format::parse(v).unwrap_or(Format::Docx); + } + + fm.meta = read_meta(&map); + if let Some(page) = map.get("page") { + fm.page = Some(read_page(page, start_line)?); + } + if let Some(defaults) = map.get("style-defaults") { + fm.styles.defaults = read_defaults(defaults); + } + if let Some(styles) = map.get("styles").and_then(|v| v.as_map()) { + for (id, value) in styles { + if let Some(style) = read_style(id, value) { + fm.styles.insert(style); + } + } + } + if let Some(lists) = map.get("list-definitions").and_then(|v| v.as_map()) { + for (id, value) in lists { + if let Some(def) = read_list_def(value) { + fm.list_defs.insert(ListId::new(id.clone()), def); + } + } + } + + Ok(fm) +} + +fn read_meta(map: &BTreeMap) -> DocumentMeta { + let text = |key: &str| map.get(key).and_then(|v| v.as_str()).map(str::to_string); + let mut meta = DocumentMeta { + title: text("title"), + author: text("author"), + last_modified_by: text("last-modified-by"), + subject: text("subject"), + keywords: text("keywords"), + description: text("description"), + created: map + .get("created") + .map(|v| v.string_or_empty()) + .filter(|s| !s.is_empty()), + modified: map + .get("modified") + .map(|v| v.string_or_empty()) + .filter(|s| !s.is_empty()), + language: text("language"), + application: text("application"), + custom: BTreeMap::new(), + }; + if let Some(custom) = map.get("custom-properties").and_then(|v| v.as_map()) { + for (k, v) in custom { + meta.custom.insert(k.clone(), v.string_or_empty()); + } + } + meta +} + +fn read_page(value: &Value, line: usize) -> Result { + let map = value + .as_map() + .ok_or_else(|| ParseError::front_matter(line, "page must be a mapping"))?; + let mut page = PageGeometry { + columns: 1, + ..Default::default() + }; + + if let Some(size) = map.get("size") { + if let Some(name) = size.as_str() { + page.size = paper_size(name).ok_or_else(|| { + ParseError::front_matter(line, format!("unknown paper size `{name}`")) + })?; + } else if let Some(m) = size.as_map() { + page.size = Size::new( + len_value(m.get("width")).unwrap_or(Length::ZERO), + len_value(m.get("height")).unwrap_or(Length::ZERO), + ); + } + } + + if let Some(margins) = map.get("margins").and_then(|v| v.as_map()) { + page.margins = Margins { + top: len_value(margins.get("top")).unwrap_or(Length::ZERO), + bottom: len_value(margins.get("bottom")).unwrap_or(Length::ZERO), + left: len_value(margins.get("left")).unwrap_or(Length::ZERO), + right: len_value(margins.get("right")).unwrap_or(Length::ZERO), + header: len_value(margins.get("header")).unwrap_or(Length::ZERO), + footer: len_value(margins.get("footer")).unwrap_or(Length::ZERO), + }; + } + + if let Some(orient) = map.get("orientation").and_then(|v| v.as_str()) { + page.orientation = match orient { + "landscape" => Orientation::Landscape, + _ => Orientation::Portrait, + }; + } + if let Some(cols) = map.get("columns").and_then(|v| v.as_u16()) { + page.columns = cols.max(1); + } + if map.get("title-page").and_then(|v| v.as_bool()) == Some(true) { + page.title_page = true; + } + Ok(page) +} + +fn paper_size(name: &str) -> Option { + // Portrait short × long in mm, matching PageGeometry::paper_name. + let (w_mm, h_mm) = match name { + "A3" => (297.0, 420.0), + "A4" => (210.0, 297.0), + "A5" => (148.0, 210.0), + "Letter" => (215.9, 279.4), + "Legal" => (215.9, 355.6), + "Tabloid" => (279.4, 431.8), + _ => return None, + }; + Some(Size::new(Length::from_mm(w_mm), Length::from_mm(h_mm))) +} + +fn read_defaults(value: &Value) -> DocDefaults { + let mut defaults = DocDefaults::default(); + if let Some(font) = value.get("font") { + defaults.font = read_font(font); + } + if let Some(para) = value.get("paragraph") { + defaults.paragraph = read_para(para); + } + defaults +} + +fn read_style(id: &str, value: &Value) -> Option