Repository navigation
Fases 0 y 1: fundamentos, IR y lectura DOCX → DocMark - #2
Merged
Merged
Conversation
Sets up the Cargo workspace with the seven crates of arquitectura.md §2 and lands the first Fase 0 deliverables: - docsai-model v1: EMU-based Length newtype, StyleCatalog with the four-level cascade, ListCatalog, TextDocument/Workbook trees, the normalised image model (ImageRef/ImageGeometry/Anchor), AssetStore with content-addressed deduplication, ConversionReport and an IR invariant validator. - corpus/generate.py: reproducible generator for 14 docx and 6 xlsx fixtures, one feature per file, with no external dependencies. - docs/spikes/R1-estrategia-docx.md: measured evaluation of docx-rs against the corpus. Decision: own parser over zip + quick-xml. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SgqYJYCmrfL21w585SQFkR
Implements the reading half of Fase 1 with the parser chosen by spike R1: - XML tree with byte spans, so unrecognised elements become raw-blocks holding their original bytes rather than a re-serialisation. - OPC package: relationship resolution, ZIP-bomb caps and member-name sanitising (no path traversal reaches the AssetStore). - styles.xml with docDefaults and the based-on chain; numbering.xml with the numId -> abstractNum indirection flattened per list. - Body: paragraphs and runs with the full inline table of spec 3.2, hyperlinks, breaks, simple and complex fields, footnotes inlined at their reference, tracked changes taken as accepted, content controls flattened. - Lists rebuilt into a tree from the flat (numId, ilvl) pairs. - Tables with gridSpan/vMerge resolved into colspan/rowspan plus covered cells. - Images: DrawingML and legacy VML into one normalised model, with wrap, position, z-order, crop, rotation, flips, alt, title, name, hyperlink and linked-not-embedded sources. Media deduplicated by content hash. - Sections split at every sectPr, with headers and footers resolved. - Format detection by content, not by extension. Acceptance: no panics over 900+ corrupt inputs (truncations and byte flips), every corpus document satisfies the IR invariants. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SgqYJYCmrfL21w585SQFkR
…e 1) - docsai-docmark: IR -> DocMark per spec 3/7/8. Deterministic attribute order, a fixed escaping table, hand-written YAML front matter, image attributes covering the whole normalised geometry, fenced-div containers for sections, headers, footers, complex tables and raw-blocks, and the three --fidelity modes. - docsai-convert: format detection, DirAssetStore, the docx -> DocMark pipeline, the support matrix and IR validation on every conversion. - docsai-cli: `docsai convert` and `docsai formats`, --fidelity, --assets-dir, --json, --strict, --verbose and the exit codes of arquitectura 5. Logs go to stderr so stdout can carry the document. - Goldens for all 14 corpus documents as reviewable .expected.dmk.md files, plus determinism, line-ending, plain-CommonMark (checked with comrak) and 50-page performance tests. Three defects the corpus caught and this fixes: - Length display was lossy (a 1417-twip margin printed as 2.499cm); it now picks the first unit that is exact, so round-trips cannot drift. - Empty paragraphs disappeared; full fidelity now records them. - Hyperlink character styles were emitted twice, inside and outside the label. Drops the unused serde_yaml and sha2 dependencies and swaps insta for comrak, the goldens being plain files beside the corpus as AGENTS.md 6 prescribes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SgqYJYCmrfL21w585SQFkR
- especificacion-docmark.md goes from draft to frozen v1.0. Annex A (complex tables) is resolved in 3.4, and a new section 10 lists every construct the Fase 1 implementation had to make explicit: the exact-unit rule, style defaults, the four indent attributes, empty paragraphs, page and column breaks, field syntax, header-row=false and the extra image attributes. - corpus/README.md documents what each fixture isolates, why the corpus is generated rather than drawn by hand, and how to update a golden. - analisis-tecnico.md records the dependency decisions AGENTS.md 2 requires: docx-rs dropped from the read path, and insta, image and serde_yaml not used at all, each with its reason. - plan-desarrollo.md marks the Fase 0 and Fase 1 acceptance criteria and notes what is deliberately left for later (real anonymised documents, text boxes). - GitHub Actions: build, test and a corpus-freshness check on the three-OS matrix, plus fmt, clippy and rustdoc, all warnings-as-errors. - README.md tells the truth about what works today versus what is planned. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SgqYJYCmrfL21w585SQFkR
`docs/` says what the project wants to be; `kb/` says what it is today. - 01: what phases 0 and 1 delivered, task by task and acceptance criterion by acceptance criterion, the three fidelity defects the corpus exposed, and what was deliberately left out with the reason. - 02: the real shape of the code — the seven crates, every module and what it owns, the dependency rules, the flow of one conversion, and where the tests live. - 03: the fourteen decisions a newcomer could undo by accident, each with its motive and its cost: the own docx parser, byte spans for raw-blocks, lossless length formatting, the serde tagging trap, the generated corpus, and the rest. - 04: what phases 2 to 9 will find already solved, what they still have to build, the traps already identified, and the known technical debt. README.md and AGENTS.md point at it; AGENTS.md makes reading it part of the required order before implementing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SgqYJYCmrfL21w585SQFkR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implementa las dos primeras fases de
docs/plan-desarrollo.md. El repositorio pasa de sólo documentación a un binario que convierte.docxa DocMark.Fase 0 — Fundamentos
arquitectura.md§2.docsai-odfydocsai-mcpvan vacíos a propósito: así el compilador impone las reglas de dependencia desde el primer día en vez de descubrirlas rotas en la Fase 4.docsai-model:Lengthen EMU, cascada de estilos de cuatro niveles con semántica referencia + delta,ListCatalog, el modelo normalizado de imágenes,AssetStorecon deduplicación por contenido,ConversionReporttipado y el validador de invariantes del IR.docs/spikes/R1-estrategia-docx.md.corpus/generate.py, reproducible byte a byte.El spike R1, que cambió el plan
El plan contemplaba
docx-rscon un complemento propio. Lo medí sobre el corpus en vez de suponerlo, y el resultado fue concluyente:behindDoc, recorte, volteos, texto alternativo, título, nombre del objeto, hipervínculo e imágenes enlazadas. La rotación tiene campo pero devuelve0pararot="2700000".w:footnoteReferencey elw:instrdew:fldSimple.r:id, sin estilo y sinalt.Err.Decisión: parser propio sobre
zip+quick-xml. Sigue siendo candidato para el writer de la Fase 2; eso es una decisión aparte y aún abierta.Fase 1 — Lectura DOCX → DocMark
Lector completo: estilos con
docDefaultsy cadenabasedOn, listas reconstruidas en árbol desde los pares planos(numId, ilvl), tablas congridSpan/vMergeresueltos acolspan/rowspan, imágenes DrawingML y VML heredado con la geometría entera, secciones partidas en cadasectPrcon cabeceras y pies por ámbito, notas al pie insertadas en su referencia, campos simples y complejos con su instrucción, y raw-blocks con los bytes exactos de todo lo no reconocido.Serializador DocMark determinista con los tres modos de
--fidelity, orquestación endocsai-converty CLI conconvertyformats.Un detalle que sostiene la fidelidad: cada nodo del árbol XML recuerda su rango de bytes en el origen, así que un raw-block cita los bytes originales en lugar de una re-serialización.
Tres defectos que el corpus destapó
Ninguno se habría visto sin pasar documentos por el pipeline completo:
2.499cmy al releerlo daba otro valor: habría movido márgenes e imágenes en cada ida y vuelta, poniendo un suelo artificial a la métrica de fidelidad de la Fase 2. Ahora se elige la primera unidad exacta (px→cm→pt→emu), con test que lo verifica caso a caso.El propio spike destapó además un error en el generador del corpus (
w:drawingfuera de unw:r, OOXML inválido) antes de que contaminara ningún golden.Verificación
164 tests,
clippy -D warningsyfmtlimpios. Los cuatro criterios de aceptación de cada fase están cubiertos por tests, no por inspección:.expected.dmk.mdrevisables junto al corpus.Err.--fidelity plainverificado parseándolo con comrak.proptest, y determinismo del serializador.Documentación
Nueva carpeta
kb/con cuatro documentos: resumen de lo entregado, estructura real del código, las decisiones técnicas no obvias con su motivo y su coste, y lo que espera a las fases siguientes.docs/dice qué se quiere construir;kb/dice qué hay hoy.También quedan registradas en
docs/analisis-tecnico.mdlas decisiones de dependencias que exigeAGENTS.md§2:docx-rsfuera del camino de lectura, yinsta,imageyserde_yamlsin usar, cada una con su razón.Alcance deliberado
No hay nada de la Fase 2: ni parser DocMark, ni writer docx, ni
roundtrip.Queda fuera, anotado en el plan:
Revisión sugerida
Los goldens (
corpus/docx/*.expected.dmk.md) son la salida real del conversor y el mejor sitio para juzgar el resultado.kb/03-decisiones-tecnicas.mdexplica lo que no se deduce del código.🤖 Generated with Claude Code
https://claude.ai/code/session_01SgqYJYCmrfL21w585SQFkR
Generated by Claude Code