feat(pdf-codec): leave document-level surfaces out of a read

packages/documents.js/README.md

@@ -387,7 +387,7 @@ It carries the ten `pdfTo*` conversions, `PdfToDocumentOptions`, `readDocumentMe
 
 ### Converting a selection of a PDF's pages
 
-The PDF-sourced entry points that produce content take a `pages` option: the `pdfTo*` conversions (`PdfToDocumentOptions.pages`), `convertDocument` with a pdf source (`UnifiedConversionOptions.pages`), and `readNativeDocumentTree`/`readContentDocument` (`ReadNativeDocumentTreeOptions.pages`). It is pdf-codec's `ReadPdfOptions.pages` threaded through: zero-based page-tree indices, a negative index counted from the end (`-1` is the last page), duplicates collapsed, the result in document order, and `PdfPageSelectionError` for an index naming no page. Two entry points never take it: `readDocumentMetadata` always reads with an empty selection, since metadata never depends on page content, so identifying a document reads no page at all; and `openPdf` always reads every page, since an editor holds a document to write back. `convertDocument` applies `pages` only when the source itself is pdf: a conversion from another format ignores it, including one whose route renders an intermediate pdf internally (docx to ods), because that pdf is not the caller's document.
+The PDF-sourced entry points that produce content take a `pages` option: the `pdfTo*` conversions (`PdfToDocumentOptions.pages`), `convertDocument` with a pdf source (`UnifiedConversionOptions.pages`), and `readNativeDocumentTree`/`readContentDocument` (`ReadNativeDocumentTreeOptions.pages`). It is pdf-codec's `ReadPdfOptions.pages` threaded through: zero-based page-tree indices, a negative index counted from the end (`-1` is the last page), duplicates collapsed, the result in document order, and `PdfPageSelectionError` for an index naming no page. Two entry points never take it: `readDocumentMetadata` always reads with an empty selection (and every document-level surface skipped, below), since metadata never depends on page content, so identifying a document reads no page at all; and `openPdf` always reads every page, since an editor holds a document to write back. `convertDocument` applies `pages` only when the source itself is pdf: a conversion from another format ignores it, including one whose route renders an intermediate pdf internally (docx to ods), because that pdf is not the caller's document.
 
 ```ts
 import { pdfToMarkdown } from "documents.js/read";
@@ -403,6 +403,30 @@ Markdown output labels each selected page with its original one-based page numbe
 
 Every reconstructed node's frames name the page's index in the source PDF (the same index every destination and form widget names), which is why a page-selected conversion's reported `DocumentTree` carries no `pages` envelope: that array is indexed by frame page index, and a selection cannot fill it. Such a tree cannot be rebuilt to pdf either: `buildDocumentBytes(tree, "pdf")` needs that envelope and throws without it. A later stage's own page-addressing option, such as `page` for an svg target, counts within the selection. A page-selected `LayoutDocument` cannot be written back: `writePdf` refuses one with `PdfPartialDocumentError`. `PdfPageSelectionError`, `PdfPartialDocumentError` and `sourcePageIndex` are exported from the root barrel; the `documents.js/read` entry re-exports nothing (this package allows re-exports only in its root barrel), so a caller of that entry recognises a bad selection by the thrown error's `code` (`pdf/page-out-of-range` or `pdf/page-index-not-integer`), and resolves a page's source index as `page.index ?? position`.
 
+### Leaving a PDF's document-level surfaces out
+
+The same entry points take a `surfaces` option (`PdfToDocumentOptions.surfaces`, `UnifiedConversionOptions.surfaces`, `ReadNativeDocumentTreeOptions.surfaces`): pdf-codec's `ReadPdfOptions.surfaces` threaded through, one boolean per document-level surface (`outline`, `destinations`, `form`, `layers`, `structure`, `residue`), each read unless set to `false`, so omitting it converts exactly as before. It is independent of `pages`, and `convertDocument` hands it to the same hop `pages` goes to, so an intermediate pdf a route renders internally is always read whole. Combined with a small page selection it is what keeps identifying a document from a few pages cheap on a long tagged PDF, whose structure tree and `/ParentTree` a page selection alone still parses whole.
+
+```ts
+import { pdfToMarkdown } from "documents.js/read";
+
+const firstPages = pdfToMarkdown(pdfBytes, {
+  pages: [0, 1, 2, -1],
+  surfaces: {
+    outline: false,
+    destinations: false,
+    form: false,
+    layers: false,
+    structure: false,
+    residue: false,
+  },
+});
+```
+
+A skipped surface is never parsed, and the conversion carries nothing built from it. The `LayoutDocument` such a read produces lists the skipped surfaces in `skippedSurfaces`, so an absent field there means "not read" rather than "none", and `writePdf` refuses that `LayoutDocument` with `PdfSkippedSurfacesError`. The reported `DocumentTree` keeps its outline as `outline-N` entries inside the `destinations` table: without `outline` those entries are absent (and so are the `destN` entries only the outline minted, below), and without `destinations` the table holds none of the document's destinations, while a read outline's `outline-N` entries stay in it, each without a `destination`. There are no layers or residue tables for a skipped surface, and no structure entries in `definitions` without `structure`; internal link constructs are left out without `destinations` (links to a URI are kept), and the form's `contentControl` constructs are absent without `form`. Skipping the outline also renames internal link anchors: a destination stated as a direct array gets a reader-minted `destN` name, numbered in the order the reader meets them, and the outline is read before any page, so without it the names minted for page links (and the anchors their link constructs carry) are renumbered, while each still names the same target, and the tree's `destinations` table loses the `destN` entries only the outline minted. Two skips change reconstruction itself. Without `structure`, a tagged PDF's headings, table cells and divisions are inferred from geometry exactly as for an untagged file, so they can come out differently than in a full conversion. Without `layers`, nothing marks content as belonging to a hidden layer, so content the default view hides is kept as if it were visible. Those are also the two differences that reach a rebuild: a `DocumentTree` from a conversion without `pages` still carries its `pages` envelope, so `buildDocumentBytes(tree, "pdf")` rebuilds it, and the rebuilt PDF draws hidden-layer text visibly when `layers` was skipped and holds the geometry-inferred blocks when `structure` was skipped. The rebuild never re-emits the outline, destinations, form, layers, structure or residue whether or not a surface was skipped, since it builds the PDF from the tree's metadata, page sizes, embedded fonts and the positioned content alone: text runs, images, vectors, formulas, table-cell background rectangles and border lines, sheet-cell rectangles and border lines, and URI links attached to a recovered run. An external link whose rectangle covers no recovered run (a link over an image, say) stays a link construct in the tree and is not re-emitted. Its `LayoutDocument` never carries `skippedSurfaces`, so `writePdf` never refuses a rebuild.
+
+`readDocumentMetadata` always reads a pdf with an empty page selection and every surface skipped, since metadata comes from the trailer's `/Info`, the catalog's `/Lang` and the XMP packet alone, so identifying a document parses neither its pages nor its outline, destinations, form, layers, structure tree or residue, and returns the same metadata a full read does. `openPdf` takes neither option, since an editor holds a whole document to write back. `ReadPdfSurfaces`, `LayoutDocumentSurface` and `PdfSkippedSurfacesError` are exported from the root barrel.
+
 ### Reading and building xlsx content directly
 
 Every other content format has its own standalone `readXContent`-shaped entry point (`readDocxContent`, `readPptxContent`, `readOdtContent`, `readOdpContent`, `readOdsContent`, `readOdgContent`) — xlsx is no longer the exception. `readXlsxContent`/`buildXlsxPackage` are this package's names for `ooxml.js`'s own spreadsheet `ContentDocument` read/build pair — the same one the `ods⇄xlsx` bridge and every xlsx metadata-rebuild path already use internally — re-exported here directly rather than wrapped, since `readXlsxContent` already produces the right shape on its own. (Since `ooxml.js` 4.0.0 the upstream flat builder is named `buildXlsxPackageFromContent` — the bare `buildXlsxPackage` name moved to that package's tree-form `DocumentTree` builder — so this package re-exports the flat builder under its own long-standing `buildXlsxPackage` name and the `ContentDocument`-in/`Package`-out contract is unchanged.) csv's `readCsvContent`/`buildCsvText` are the same kind of directly-exported stage pair, one level further in: they operate on RFC 4180 text rather than a decoded package (see `src/csv/` under Architecture). svg's `readSvgContent`/`buildSvgText` are the drawing-variant counterpart of csv's pair, operating on SVG text rather than a decoded package (see `src/svg/` under Architecture).
@@ -840,7 +864,7 @@ The package is layered from generic primitives outward to the two conversion dir
 - **`src/odb/`** — decoder-selection and pivot-mapping: `read.ts` routes to the right tier, `spreadsheet.ts`/`csv.ts` map to output formats. `odb/sql/` is the bounded SQL engine, `odb/formula/` is the rpt formula engine, `odb/report/` is the renderer, `odb/values.ts` is shared comparison/aggregation semantics.
 - **`src/convert/`** — the composition layer: `convert.ts` (the named conversion functions and their option types), `composition.ts` (the pathfinder, the primitive registry, the bridge/fromPdf executors, and `convertDocumentFromPdf` — the read half of the engine), `composition-to-pdf.ts` (`executeToPdf`, the layout-engine registry, and the full `convertDocument` binding — the write half, split out so a read-only consumer never statically imports a renderer), `from-pdf.ts` (the `pdfTo*` family, `readDocumentMetadata`, `readNativeDocumentTree`, `readContentDocument`, and the `documents.js/read` entry), `codec.ts` (`z.codec()` pairs), `port.ts`/`local.ts` (the `DocumentConverter` port), `variant-bridges.ts` (cross-variant semantic transforms), and `from-package.ts` (`buildDocumentBytes`, which flattens once at the boundary). The tree ⇄ flat transform those construction sites call — `assembleTree`, `decompose`, `flattenTree`, `factorStyles` — is not implemented here: it lives in `document-schema.js`, which owns both encodings. `bijection.test.ts` is this package's own gate on it, re-running the three laws over the real corpus every reader, editor, and conversion here produces.
 - **`src/codecs/`** — `DOCUMENT_FORMAT_CODECS`: every format's read/build capability as data, so `setDocumentMetadata`/`buildDocumentBytes` dispatch through one registry; its read halves live in `read.ts`, whose `CONTENT_READERS` `readDocumentMetadata` uses for content formats (it reads pdf through pdf-codec's `readPdf` directly, with no page selected).
-- **`src/metadata/`**: cross-format metadata write via `DOCUMENT_FORMAT_CODECS`; the read side is `readDocumentMetadata` in `src/convert/from-pdf.ts`, which reads a content format through `CONTENT_READERS` and a pdf through `readPdf` with an empty page selection.
+- **`src/metadata/`**: cross-format metadata write via `DOCUMENT_FORMAT_CODECS`; the read side is `readDocumentMetadata` in `src/convert/from-pdf.ts`, which reads a content format through `CONTENT_READERS` and a pdf through `readPdf` with an empty page selection and every document-level surface skipped.
 - **`src/package-codec.ts`** — `decodeDocumentPackage`/`encodeDocumentPackage`/`decodeOdbPackage`.
 
 Dependency direction is downward and checkable. Twelve external dependencies each own a distinct concern: `ooxml.js` (docx/pptx/xlsx), `odf.js` (odt/ods/odp/odg), `document-schema.js` (shared schemas + port contracts), `pdf-codec` (PDF codec + text-layout/font primitives), `byte-codec` (byte/image utilities), `markdown-codec` (markdown), `rtf-codec` (rtf), `wpd-codec` (wpd, read-only), `doc-codec` (doc), `xls-codec` (xls), `ppt-codec` (ppt, through this package's own `src/ppt/` envelope adapter), `archive-codec` (the `[MS-CFB]` compound-file detection all three legacy binary codecs' own bytes schemas build on). No `PdfObject`/`PdfDict`/`PdfStream` type appears anywhere in this package.

packages/documents.js/src/convert/composition.ts

@@ -24,6 +24,7 @@ import {
 } from "odf.js";
 import { sourcePageIndex } from "pdf-codec/layout";
 import { readPdf } from "pdf-codec/read";
+import type { ReadPdfSurfaces } from "pdf-codec/read";
 import type {
   LayoutDocument,
   PdfDiagnosticSink,
@@ -324,6 +325,10 @@ export interface UnifiedConversionOptions {
    * The output holds only those pages. A markdown target labels each with its original one-based page number (`<!-- page 7 -->` at the start of the page, beside the usual `<!-- page break -->` between pages), so the text says which pages it holds. A page-addressing option of a later stage, such as `page` for an svg target, counts within the selection. Reconstruction infers from the selected pages alone, so a page can reconstruct differently than it would in a full conversion: the body text size heading detection compares against is the most common size across the selection, which on a cover page dominated by large title text is the title's own size, leaving the title a paragraph rather than a heading. The DocumentTree `onDocument` reports carries no `pages` envelope for a page-selected conversion: its frames name source page indices, which an envelope holding only the selected pages could not be indexed by.
    */
   readonly pages?: readonly number[];
+  /**
+   * The document-level surfaces of a pdf source to read, as pdf-codec's `ReadPdfOptions.surfaces` takes them: one boolean per surface (`outline`, `destinations`, `form`, `layers`, `structure`, `residue`), each read unless set to `false`, so omitting it reads everything as before. Independent of `pages`, and handed to the same hop: only the plan's first hop, and only when the conversion's source is pdf, so an intermediate pdf a route renders internally is always read whole. A skipped surface is never parsed and is absent from the read, so what the conversion builds from it is absent too: no `outline-N` entries in the tree's `destinations` table without `outline`, no destinations and no internal link constructs without `destinations` (a link to a URI is kept, and a read outline's `outline-N` entries stay, each without a `destination`), no form `contentControl` constructs without `form`, no layers table without `layers`, no structure entries in `definitions` without `structure`, no residue without `residue`; skipping the outline also renumbers the reader-minted `destN` names internal link anchors carry, since the outline mints its direct destinations first. Two skips also change reconstruction: without `structure` a tagged PDF's headings, table cells and divisions are inferred from geometry as for an untagged file, and without `layers` content the default view hides cannot be told apart and is kept as if visible. See `ReadPdfOptions.surfaces` for exactly what each covers.
+   */
+  readonly surfaces?: ReadPdfSurfaces;
 }
 
 // What the fromPdf executor knows about a conversion's pages that a text build needs and the caller's options do not carry, passed to a text node's build alongside those options: the original one-based page number of each page the read returned, set only for a page-selected read (see UnifiedConversionOptions.pages), where a page's position in the output no longer says which page it is. Internal to the executors, never caller-supplied.
@@ -651,6 +656,7 @@ export function executeFromPdf(
     signal: options?.signal,
     sink: options?.sink,
     pages: options?.pages,
+    surfaces: options?.surfaces,
   });
   const content = RECONSTRUCTORS[node.variant](layout, {
     signal: options?.signal,
@@ -720,7 +726,7 @@ export function runCompositionPlan(
     }
     // onDocument fires on the last hop only: null out onDocument for every hop before the last, then pass the caller's options through unchanged on the final hop so the callback receives the package that actually produced the output bytes. This mirrors the convention the original hand-written composed functions (xlsxToPdf/pdfToXlsx/xlsxToMarkdown/markdownToXlsx) already followed — each forwarded onDocument to its own last hop, never the first — so a caller of xlsxToPdf still receives the odsToPdf hop's content+layout package rather than the intermediate xlsx->ods bridge's content-only one.
     const isLastHop = i === plan.hops.length - 1;
-    // pages selects pages of the caller's own pdf source, so only the hop that reads that source receives it: the first hop of a plan whose source is pdf. Every other hop's pdf (an intermediate rendering, as on a docx -> ods route through toPdf then fromPdf) is not the caller's document, and selecting its pages would silently drop content or throw for an index the caller never meant against it.
+    // pages and surfaces select what to read of the caller's own pdf source, so only the hop that reads that source receives them: the first hop of a plan whose source is pdf. Every other hop's pdf (an intermediate rendering, as on a docx -> ods route through toPdf then fromPdf) is not the caller's document, and selecting its pages would silently drop content or throw for an index the caller never meant against it, and skipping its surfaces would drop what the caller's own source carried through it.
     const readsCallerPdf = i === 0 && hop.from === "pdf";
     const hopOptions: UnifiedConversionOptions | undefined =
       options === undefined
@@ -729,6 +735,7 @@ export function runCompositionPlan(
             ...options,
             onDocument: isLastHop ? options.onDocument : undefined,
             pages: readsCallerPdf ? options.pages : undefined,
+            surfaces: readsCallerPdf ? options.surfaces : undefined,
           };
     if (hop.executor === "toPdf") {
       if (!isSourceContentFormat(hop.from)) {