From c29ca2fd7af9d4c96f2e1b314b21027931d9d202 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:19:23 +0100 Subject: [PATCH 1/9] test(document-operations): tell a pdf_inspect summary apart by formatVersion A LayoutDocument from a page-selected read carries pageCount, so the pageCount property no longer narrows pdf_inspect's result to its summary shape. formatVersion is on every LayoutDocument and on no summary. --- .../document-operations/src/operations/pdf-inspect.test.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/document-operations/src/operations/pdf-inspect.test.ts b/packages/document-operations/src/operations/pdf-inspect.test.ts index c4885a7d41..b0b231f6f5 100644 --- a/packages/document-operations/src/operations/pdf-inspect.test.ts +++ b/packages/document-operations/src/operations/pdf-inspect.test.ts @@ -52,7 +52,8 @@ describe("pdfInspectOperation", () => { source: { bytesBase64, format: "pdf" }, }); - if (!("pageCount" in result)) { + // formatVersion, not pageCount, tells the two shapes apart: a LayoutDocument carries pageCount too when it comes from a page-selected read, while formatVersion is on every LayoutDocument and on no summary. + if ("formatVersion" in result) { throw new Error("expected a summary, not a full LayoutDocument"); } expect(result.pageCount).toBe(1); From 5c0dbc4346b05c4e1c057497f35613f81cfaf37c Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:19:00 +0100 Subject: [PATCH 2/9] feat(pdf-codec): read a selection of pages with ReadPdfOptions.pages readPdf takes a pages option of zero-based page-tree indices, negative indices counting from the end. An unselected page's content stream is never fetched, decoded or interpreted, so the fonts and images only unselected pages use are never resolved, and a page-selected read decodes no embedded file, leaving attachments out of its result. Structure ownership and its parent-tree diagnostics cover selected pages only; the structure element tree stays document-level. The selected pages come back in document order, each carrying its source position as index, with the page tree's length as pageCount; a full read sets neither. Document-level page references keep naming source indices, and sourcePageIndex resolves either shape. An empty selection reads no page. An entry naming no page throws PdfPageSelectionError before any page is read. writePdf refuses a document carrying pageCount with PdfPartialDocumentError, since it places pages by position and the selection's references name source pages. Refs #1688 --- packages/pdf-codec/README.md | 4 +- packages/pdf-codec/src/diagnostics.ts | 23 ++ packages/pdf-codec/src/index.ts | 3 +- packages/pdf-codec/src/layout.ts | 14 + .../pdf-codec/src/read-page-selection.test.ts | 307 ++++++++++++++++++ packages/pdf-codec/src/read.ts | 59 +++- packages/pdf-codec/src/structure.ts | 23 +- packages/pdf-codec/src/test-support/pdf.ts | 94 ++++++ packages/pdf-codec/src/write.ts | 20 ++ 9 files changed, 539 insertions(+), 8 deletions(-) create mode 100644 packages/pdf-codec/src/read-page-selection.test.ts diff --git a/packages/pdf-codec/README.md b/packages/pdf-codec/README.md index 28997c048e..d75595096b 100644 --- a/packages/pdf-codec/README.md +++ b/packages/pdf-codec/README.md @@ -93,6 +93,8 @@ An encrypted PDF that opens without a password decrypts transparently — no ext Both accept an optional `signal` (`AbortSignal`); `readPdf` additionally takes a `sink` (`PdfDiagnosticSink`, called once per recoverable parse diagnostic — see the three-tier failure policy under [Conventions](#conventions)), and `writePdf` an `onSubstitution` callback (called once per character not representable in a standard-14 font — see [Fidelity](#fidelity)). +**Reading a subset of pages.** `readPdf(bytes, { pages: [0, 1, 2, -1] })` reads only the listed pages: zero-based indices into the page tree, a negative index counted from the end (`-1` is the last page). What a selected read skips: an unselected page's content stream is never fetched, decoded or interpreted, so the fonts and images only unselected pages use are never resolved, and no embedded file is decoded, so `attachments` is absent from a page-selected result (an attachment is document-level payload whose size is independent of the pages asked for). The case this exists for is identifying a document from its first pages, or reading a code from its last page, inside a memory-capped runtime such as a Cloudflare Worker isolate, without paying for every page's content. What it still reads whole, because page references resolve against it or because it is document-level and holds no page content stream: the page tree, the outline, named destinations, optional-content layers, the AcroForm, the structure element tree (whose size scales with the document), metadata and package residue. Two surfaces are built from page content and so cover selected pages only: the structure-ownership association an item's `structure` field names (with its parent-tree diagnostics) and the `page-boxes` residue row. A destination a link states as a direct array is minted into `destinations` (as `dest1`, `dest2`, ...) only when its page is interpreted, so an unselected page's direct link destinations are absent and a minted name depends on which pages were read; named destinations are unaffected. Duplicates collapse and the result's `pages` holds the selected pages in ascending document order, each carrying its source position as `index` (read it through `sourcePageIndex(page, position)`, which falls back to the position on a full read), with the page tree's length as the document's `pageCount`; every document-level page reference keeps naming source indices, so a reference to a selected page resolves through `index`, and one to an unselected page names a page the result does not hold. A full read sets neither field, because there `pages.length` and each page's position already state them. An empty selection is valid and reads no page, which makes `{ pages: [] }` the cheapest way to learn a file's page count and metadata. An entry naming no page (outside `-pageCount` to `pageCount - 1`, or not an integer, `NaN` and the infinities included; `-0` is page 0) throws `PdfPageSelectionError` (a `PdfParseError` with code `pdf/page-out-of-range` or `pdf/page-index-not-integer`, carrying `pageIndex` and `pageCount`) before any page is read. A page-selected `LayoutDocument` is a reading of part of a file, not a document to re-emit: `writePdf` refuses one (any document carrying `pageCount`) with `PdfPartialDocumentError`, since it places pages by position and the references would point at the wrong pages. + **Page boundaries: the crop box is the visible region** (ISO 32000-1 14.11.2). A page's reported `widthPt`/`heightPt` and coordinate frame come from its effective `/CropBox` — the rectangle a viewer displays and prints, inherited through the page tree like `/MediaBox` and defaulting to it — not from the media box: content an author placed outside the crop box (printer's marks, bleed, off-page slugs) is not visible and does not extract. Content wholly outside the crop box is dropped; content straddling the boundary keeps its original unclipped geometry (the item layer records what the file states — a viewer's clipping is a rendering fact, not source data); link annotations are anchored constructs rather than painted content and are never filtered, the same line the optional-content filter draws. The declared boundary rectangles a distinct crop box hides — plus `/BleedBox`/`/TrimBox`/`/ArtBox`, print-production facts with no field in the layout model — are quarantined verbatim as the `page-boxes` package-level residue row rather than silently dropped. A degenerate `/CropBox` (zero width or height) falls back to the media box with a `pdf/invalid-crop-box` warning. **Cancellation granularity and cost, for CPU-metered runtimes.** Both pipelines are synchronous end to end — there is no `await` point for cancellation to hook into implicitly — so the `signal` is checked explicitly, once per page-loop iteration (and once before `readPdf`'s document-open phase begins). A signal aborted mid-parse therefore takes effect at the next page boundary, not instantly: `readPdf`'s document-open phase (cross-reference resolution, object parsing) and a single page's content-stream interpretation are the two spans that cannot be interrupted, and a document consisting of one enormous page is effectively uninterruptible however many pages it claims. Cost is roughly linear in decompressed content length, so budget for the worst single page, not the page count. On Cloudflare Workers this is the honest shape of the trade: the parse holds the isolate for its whole duration with no opportunity to yield or report progress, and an `AbortSignal` shared with whatever can abort concurrently (a binding, another context) makes a deadline enforceable at page granularity — but it cannot convert a synchronous parse into a resumable one. An async page-at-a-time API is a deliberate non-goal of this package's current surface. @@ -275,7 +277,7 @@ Dependency direction is strictly downward and checkable: `layout` imports only ` - **`PdfObject` has no Zod schema at all**, deliberately: it never crosses a public boundary or round-trips through JSON, and is constructed exclusively by this package's own parser. It narrows natively on its own `kind` discriminant. - **The `MathBox`/`MathFontMetrics` family is structurally typed on purpose, not validated by Zod** — the mechanism that lets a caller (`documents.js`) hand this package a real value produced by a completely independent module, with zero cast, zero wrapper, and no shared class or branded type. - **No type assertions anywhere.** Every third-party or loosely-typed value is narrowed through a type guard or a Zod parse at the boundary. -- **A three-tier PDF-read failure policy**: throw a typed `PdfParseError`/`PdfEncryptedError`/`PdfPasswordRequiredError` for a file that cannot be meaningfully processed at all; recover with a `PdfDiagnostic` (`severity: 'warning'`) for something malformed but salvageable; degrade with a diagnostic for an individual unsupported feature while the rest of the document still reads. +- **A three-tier PDF-read failure policy**: throw a typed `PdfParseError`/`PdfEncryptedError`/`PdfPasswordRequiredError` for a file that cannot be meaningfully processed at all (or `PdfPageSelectionError` for a page selection naming no page of it); recover with a `PdfDiagnostic` (`severity: 'warning'`) for something malformed but salvageable; degrade with a diagnostic for an individual unsupported feature while the rest of the document still reads. - **Conventional commits**, enforced via commitlint + husky. ## Gotchas and quirks diff --git a/packages/pdf-codec/src/diagnostics.ts b/packages/pdf-codec/src/diagnostics.ts index ca6fc0403a..a2cee8d0cf 100644 --- a/packages/pdf-codec/src/diagnostics.ts +++ b/packages/pdf-codec/src/diagnostics.ts @@ -28,6 +28,29 @@ export class PdfParseError extends Error { } } +/** + * A `ReadPdfOptions.pages` entry that names no page of the document: an integer outside `[-pageCount, pageCount - 1]` (code `pdf/page-out-of-range`) or a value that is not an integer at all (code `pdf/page-index-not-integer`). Thrown rather than skipped, because a caller asking for a page it believes exists has a wrong belief about the file, and a result quietly missing that page would let the wrong belief stand. + * + * `pageIndex` is the entry exactly as the caller wrote it (negative indices included); `pageCount` is the number of pages the document's page tree holds, so the caller can correct the selection without a second read. + */ +export class PdfPageSelectionError extends PdfParseError { + readonly pageIndex: number; + readonly pageCount: number; + + constructor(pageIndex: number, pageCount: number) { + const isInteger = Number.isInteger(pageIndex); + super( + isInteger ? "pdf/page-out-of-range" : "pdf/page-index-not-integer", + isInteger + ? `page index ${pageIndex} names no page of this ${pageCount}-page document; a selection takes zero-based indices from 0 to pageCount - 1, or negative indices from -pageCount to -1 counted from the end` + : `page index ${pageIndex} is not an integer; a selection takes zero-based integer page indices`, + ); + this.name = "PdfPageSelectionError"; + this.pageIndex = pageIndex; + this.pageCount = pageCount; + } +} + // /Encrypt present, and encrypted in a way this codec cannot read at all: a non-standard security handler (/Filter other than /Standard, e.g. a public-key one), an unpublished or unrecognised /V, or a crypt filter naming a /CFM this codec does not implement. Distinct from PdfPasswordRequiredError below, which means the encryption itself IS supported and the file simply needs a password we do not have. export class PdfEncryptedError extends PdfParseError { constructor(message = "this PDF is encrypted and unsupported") { diff --git a/packages/pdf-codec/src/index.ts b/packages/pdf-codec/src/index.ts index 6f3de043de..05953787d6 100644 --- a/packages/pdf-codec/src/index.ts +++ b/packages/pdf-codec/src/index.ts @@ -4,7 +4,7 @@ export type { ReadPdfOptions } from "./read"; export { readPdf } from "./read"; export type { WritePdfOptions } from "./write"; -export { writePdf } from "./write"; +export { PdfPartialDocumentError, writePdf } from "./write"; // Write-side standard security handler: encrypts a document written by writePdf under WritePdfOptions.encryption. See encrypt-write.ts's own module comment for the four schemes and their scope. export type { PdfEncryptionOptions, @@ -20,6 +20,7 @@ export type { export { NOOP_DIAGNOSTIC_SINK, PdfEncryptedError, + PdfPageSelectionError, PdfParseError, PdfPasswordRequiredError, } from "./diagnostics"; diff --git a/packages/pdf-codec/src/layout.ts b/packages/pdf-codec/src/layout.ts index 976b88d3b8..7b579500a0 100644 --- a/packages/pdf-codec/src/layout.ts +++ b/packages/pdf-codec/src/layout.ts @@ -204,9 +204,21 @@ export const LayoutPageSchema = z.object({ annotations: z.array(LayoutAnnotationSchema).optional(), // pptx speaker notes for the slide this page came from, if any — carried as a private, non-visible entry on the PDF page's own dictionary, never painted into the page content. PDF has no native concept of hidden presenter notes, so this is a round-trip mechanism specific to a writer/reader pair that both honour it, not a real PDF feature — a PDF produced by anything else will never have it, and a PDF consumer that doesn't specifically know this convention will never see it either. notes: z.string().optional(), + // The page's zero-based position in the source PDF's page tree, set by readPdf on every page of a page-selected read (ReadPdfOptions.pages), where the page's position in `pages` no longer says which page it is. Absent on a full read and on a page any other producer builds: there the position in `pages` IS the index, and stating it twice would be a second copy to keep in agreement. Every document-level reference to a page (a destination's or form widget's pageIndex, a page-boxes residue row, a structure association) names this source index, never the position. Read it through sourcePageIndex rather than branching on its presence. + index: z.number().int().nonnegative().optional(), }); export type LayoutPage = z.infer; +/** + * The zero-based index in the source PDF's page tree of the page at `position` in a `LayoutDocument`'s `pages`: the page's own `index` when a page-selected read set one, otherwise `position` itself, which is the source index whenever `pages` holds every page in document order. The one place that rule is spelled, so every consumer resolving a page against a document-level reference (a destination, a form widget, a frame) agrees on it. + * @param page - the page as it appears in `pages` + * @param position - the page's position in `pages` + * @returns the page's index in the source document's page tree + */ +export function sourcePageIndex(page: LayoutPage, position: number): number { + return page.index ?? position; +} + // An entry in the top-level image registry: bytes live here once, keyed by imageId, so a repeated logo across many pages/slides embeds (or extracts) exactly once. Bytes are the original file bytes for the given format — PNG bytes are re-encoded from decoded pixels where needed; JPEG bytes are the original encoded stream, verbatim, in both directions. export const LayoutImageAssetSchema = z.object({ format: z.enum(["png", "jpeg"]), @@ -373,6 +385,8 @@ export const LayoutDocumentSchema = z.object({ formatVersion: z.literal(LAYOUT_FORMAT_VERSION), metadata: LayoutMetadataSchema, pages: z.array(LayoutPageSchema), + // The number of pages in the source PDF's page tree, set by readPdf on a page-selected read (ReadPdfOptions.pages), where `pages` holds only the selection and so no longer says how long the document is: a caller that asked for the last page, or wants to know which pages it has not read, needs the total from the same read rather than from a second pass over the file. Absent on a full read, where it would restate pages.length. A field rather than a separate page-count function because the page tree is walked by readPdf anyway (destinations and the selection both resolve against it), so reporting its length costs nothing, while a separate call would parse the cross-reference table and walk the tree a second time. + pageCount: z.number().int().nonnegative().optional(), images: z.record(z.string(), LayoutImageAssetSchema), destinations: z.array(LayoutDestinationSchema).optional(), outline: z.array(LayoutOutlineItemSchema).optional(), diff --git a/packages/pdf-codec/src/read-page-selection.test.ts b/packages/pdf-codec/src/read-page-selection.test.ts new file mode 100644 index 0000000000..fb5b91812e --- /dev/null +++ b/packages/pdf-codec/src/read-page-selection.test.ts @@ -0,0 +1,307 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; +import type * as Filters from "./filters"; +import type * as Interpret from "./interpret"; +import type { PdfDiagnostic } from "./diagnostics"; +import { PdfPageSelectionError, PdfParseError } from "./diagnostics"; +import { decodeStream } from "./filters"; +import { interpretContentStream } from "./interpret"; +import { sourcePageIndex } from "./layout"; +import { asName, dictGet } from "./objects"; +import { readPdf } from "./read"; +import { PdfPartialDocumentError, writePdf } from "./write"; +import { + PAGE_SELECTION_ATTACHMENT_NAME, + PAGE_SELECTION_IMAGE_PAGE_INDEX, + PAGE_SELECTION_STREAM, + PAGE_SELECTION_TRAP_PAGE_INDEX, + navigationClusterPdf, + pagelessPdf, + pageSelectionPdf, +} from "./test-support/pdf"; +import { parentTreeMissingEntryPdf } from "./test-support/pdf-structures"; + +// The interpreter entry readPage calls, wrapped so a test can count how many content streams a read interpreted. The real implementation still runs: this observes, it does not stub. +vi.mock("./interpret", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + interpretContentStream: vi.fn(actual.interpretContentStream), + }; +}); + +// The stream decoder every reader goes through, wrapped the same way, so a test can see which of the fixture's marked streams a read decoded. +vi.mock("./filters", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, decodeStream: vi.fn(actual.decodeStream) }; +}); + +// The /FixtureStream names of the streams the last read decoded. +function decodedFixtureStreams(): string[] { + return vi + .mocked(decodeStream) + .mock.calls.flatMap(([, dict]) => { + const marker = asName(dictGet(dict, "FixtureStream")); + return marker === undefined ? [] : [marker]; + }) + .sort(); +} + +const PAGE_COUNT = 4; +const LAST_PAGE_INDEX = PAGE_COUNT - 1; + +function pageTexts(doc: ReturnType): string[] { + return doc.pages.map((page) => + page.items + .flatMap((item) => (item.kind === "text" ? [item.text] : [])) + .join(""), + ); +} + +function collectingSink(): { + readonly diagnostics: PdfDiagnostic[]; + readonly sink: (diagnostic: PdfDiagnostic) => void; +} { + const diagnostics: PdfDiagnostic[] = []; + return { + diagnostics, + sink: (diagnostic) => { + diagnostics.push(diagnostic); + }, + }; +} + +beforeEach(() => { + vi.mocked(interpretContentStream).mockClear(); + vi.mocked(decodeStream).mockClear(); +}); + +describe("readPdf: page selection never touches an unselected page's content", () => { + it("interprets one content stream per selected page, and none for the rest", () => { + readPdf(pageSelectionPdf()); + expect(interpretContentStream).toHaveBeenCalledTimes(PAGE_COUNT); + + vi.mocked(interpretContentStream).mockClear(); + readPdf(pageSelectionPdf(), { pages: [0, -1] }); + expect(interpretContentStream).toHaveBeenCalledTimes(2); + }); + + it("never decodes an unselected page's content stream: the trap page's undecodable filter is reported only when that page is read", () => { + const full = collectingSink(); + readPdf(pageSelectionPdf(), { sink: full.sink }); + expect(full.diagnostics.map((d) => d.code)).toContain( + "pdf/unsupported-filter", + ); + + const selected = collectingSink(); + readPdf(pageSelectionPdf(), { + sink: selected.sink, + pages: [...Array(PAGE_COUNT).keys()].filter( + (index) => index !== PAGE_SELECTION_TRAP_PAGE_INDEX, + ), + }); + expect(selected.diagnostics).toEqual([]); + }); + + it("never decodes a font program only an unselected page uses, nor the content stream of that page", () => { + readPdf(pageSelectionPdf()); + expect(decodedFixtureStreams()).toEqual( + expect.arrayContaining([ + PAGE_SELECTION_STREAM.trapContent, + PAGE_SELECTION_STREAM.trapFontProgram, + ]), + ); + + vi.mocked(decodeStream).mockClear(); + readPdf(pageSelectionPdf(), { pages: [0, -1] }); + expect(decodedFixtureStreams()).toEqual([]); + }); + + it("decodes no embedded file on a page-selected read, and leaves attachments out of its result", () => { + const full = readPdf(pageSelectionPdf()); + expect(decodedFixtureStreams()).toContain(PAGE_SELECTION_STREAM.attachment); + expect(full.attachments?.map((attachment) => attachment.name)).toEqual([ + PAGE_SELECTION_ATTACHMENT_NAME, + ]); + + for (const pages of [[], [0], [PAGE_SELECTION_TRAP_PAGE_INDEX]]) { + vi.mocked(decodeStream).mockClear(); + const selected = readPdf(pageSelectionPdf(), { pages }); + expect(decodedFixtureStreams()).not.toContain( + PAGE_SELECTION_STREAM.attachment, + ); + expect(selected.attachments).toBeUndefined(); + } + }); + + it("builds structure ownership only for selected pages: an unselected page's broken /StructParents reports nothing", () => { + const full = collectingSink(); + readPdf(parentTreeMissingEntryPdf(), { sink: full.sink }); + expect(full.diagnostics.map((d) => d.code)).toContain( + "pdf/parent-tree-missing-entry", + ); + + const selected = collectingSink(); + const doc = readPdf(parentTreeMissingEntryPdf(), { + sink: selected.sink, + pages: [], + }); + expect(selected.diagnostics).toEqual([]); + expect(doc.structure).toEqual( + readPdf(parentTreeMissingEntryPdf()).structure, + ); + }); + + it("never resolves an image only an unselected page draws", () => { + expect(Object.keys(readPdf(pageSelectionPdf()).images)).toHaveLength(1); + expect(readPdf(pageSelectionPdf(), { pages: [0, -1] }).images).toEqual({}); + expect( + Object.keys( + readPdf(pageSelectionPdf(), { + pages: [PAGE_SELECTION_IMAGE_PAGE_INDEX], + }).images, + ), + ).toHaveLength(1); + }); +}); + +describe("readPdf: page selection result", () => { + it("returns exactly the selected pages, page-for-page equal to the same pages of a full read, each carrying its source index", () => { + const full = readPdf(pageSelectionPdf()); + const selection = [0, PAGE_SELECTION_IMAGE_PAGE_INDEX, LAST_PAGE_INDEX]; + const selected = readPdf(pageSelectionPdf(), { pages: selection }); + expect(selected.pages).toEqual( + selection.map((index) => ({ ...full.pages[index], index })), + ); + expect(selected.images).toEqual(full.images); + }); + + it("reports the page tree's length as pageCount on a selected read, and leaves both pageCount and index off a full read", () => { + const full = readPdf(pageSelectionPdf()); + expect(full.pageCount).toBeUndefined(); + expect(full.pages.every((page) => page.index === undefined)).toBe(true); + expect(readPdf(pageSelectionPdf(), { pages: [0] }).pageCount).toBe( + PAGE_COUNT, + ); + }); + + it("counts a negative index from the end", () => { + const doc = readPdf(pageSelectionPdf(), { pages: [-1, -PAGE_COUNT] }); + expect(doc.pages.map((page) => page.index)).toEqual([0, LAST_PAGE_INDEX]); + expect(pageTexts(doc)).toEqual(["Page 1", `Page ${PAGE_COUNT}`]); + }); + + it("collapses duplicates, including a negative and a positive index naming the same page, and orders the result by document order", () => { + const doc = readPdf(pageSelectionPdf(), { + pages: [LAST_PAGE_INDEX, 0, -1, 0], + }); + expect(doc.pages.map((page) => page.index)).toEqual([0, LAST_PAGE_INDEX]); + }); + + it("reads no page for an empty selection, while still reporting pageCount and the document-level surfaces", () => { + const doc = readPdf(navigationClusterPdf(), { pages: [] }); + expect(interpretContentStream).not.toHaveBeenCalled(); + const full = readPdf(navigationClusterPdf()); + expect(doc.pages).toEqual([]); + expect(doc.pageCount).toBe(full.pages.length); + expect(doc.outline).toEqual(full.outline); + }); + + it("keeps document-level page references on source indices", () => { + const full = readPdf(navigationClusterPdf()); + const secondPageOnly = readPdf(navigationClusterPdf(), { pages: [1] }); + expect(secondPageOnly.outline).toEqual(full.outline); + const named = (doc: ReturnType, name: string) => + doc.destinations?.find((destination) => destination.name === name); + expect(named(secondPageOnly, "second")).toEqual(named(full, "second")); + expect(named(secondPageOnly, "second")?.pageIndex).toBe(1); + expect(secondPageOnly.pages[0]?.index).toBe(1); + }); + + it("reads every page when pages is omitted", () => { + expect(pageTexts(readPdf(pageSelectionPdf()))).toEqual([ + "Page 1", + "Page 2", + "Page 3", + "Page 4", + ]); + }); +}); + +describe("readPdf: an invalid page selection throws", () => { + it.each([PAGE_COUNT, -PAGE_COUNT - 1])( + "refuses index %i, naming the index and the page count", + (requested) => { + const read = () => readPdf(pageSelectionPdf(), { pages: [0, requested] }); + expect(read).toThrow(PdfPageSelectionError); + expect(read).toThrow(PdfParseError); + try { + read(); + } catch (error) { + expect(error).toMatchObject({ + code: "pdf/page-out-of-range", + pageIndex: requested, + pageCount: PAGE_COUNT, + }); + expect(String(error)).toContain(String(requested)); + expect(String(error)).toContain(String(PAGE_COUNT)); + } + }, + ); + + it.each([ + 0.5, + Number.NaN, + Number.POSITIVE_INFINITY, + Number.NEGATIVE_INFINITY, + ])("refuses the non-integer index %d", (requested) => { + expect(() => readPdf(pageSelectionPdf(), { pages: [requested] })).toThrow( + expect.objectContaining({ + code: "pdf/page-index-not-integer", + pageIndex: requested, + }), + ); + }); + + it("reads -0 as page 0", () => { + expect( + readPdf(pageSelectionPdf(), { pages: [-0] }).pages.map( + (page) => page.index, + ), + ).toEqual([0]); + }); + + it("refuses any index on a document with no pages", () => { + expect(() => readPdf(pagelessPdf(), { pages: [0] })).toThrow( + expect.objectContaining({ code: "pdf/page-out-of-range", pageCount: 0 }), + ); + expect(readPdf(pagelessPdf(), { pages: [] }).pageCount).toBe(0); + }); + + it("throws before interpreting any page", () => { + expect(() => + readPdf(pageSelectionPdf(), { pages: [0, PAGE_COUNT] }), + ).toThrow(PdfPageSelectionError); + expect(interpretContentStream).not.toHaveBeenCalled(); + }); +}); + +describe("sourcePageIndex", () => { + it("prefers a page's own index over its position, and falls back to the position", () => { + const page = { widthPt: 1, heightPt: 1, items: [] }; + expect(sourcePageIndex({ ...page, index: LAST_PAGE_INDEX }, 0)).toBe( + LAST_PAGE_INDEX, + ); + expect(sourcePageIndex(page, 2)).toBe(2); + }); +}); + +describe("writePdf refuses a page-selected read", () => { + it("throws PdfPartialDocumentError naming both counts, and still writes a full read", () => { + const selected = readPdf(pageSelectionPdf(), { pages: [0, -1] }); + expect(() => writePdf(selected)).toThrow(PdfPartialDocumentError); + expect(() => writePdf(selected)).toThrow( + expect.objectContaining({ pageCount: PAGE_COUNT, selectedPageCount: 2 }), + ); + expect(writePdf(readPdf(pageSelectionPdf())).length).toBeGreaterThan(0); + }); +}); diff --git a/packages/pdf-codec/src/read.ts b/packages/pdf-codec/src/read.ts index 5ef85dbd0d..db11dc281b 100644 --- a/packages/pdf-codec/src/read.ts +++ b/packages/pdf-codec/src/read.ts @@ -21,7 +21,11 @@ import { LAYOUT_FORMAT_VERSION } from "./layout"; import { openPdfDocument } from "./document"; import type { PdfDocument } from "./document"; import type { PdfDiagnosticSink } from "./diagnostics"; -import { NOOP_DIAGNOSTIC_SINK, PdfParseError } from "./diagnostics"; +import { + NOOP_DIAGNOSTIC_SINK, + PdfPageSelectionError, + PdfParseError, +} from "./diagnostics"; import { decodeStream } from "./filters"; import { throwIfAborted } from "./util/abort"; import type { FontResolverService } from "./font-read"; @@ -57,6 +61,32 @@ import { decodePdfString, parsePdfDate } from "./pdf-text"; export interface ReadPdfOptions { readonly sink?: PdfDiagnosticSink; readonly signal?: AbortSignal; + /** + * The pages to read, as zero-based indices into the document's page tree; a negative index counts from the end (`-1` is the last page). Omitted, every page is read. + * + * What a selected read skips: an unselected page's content stream is never fetched, decoded or interpreted, so the fonts and images only unselected pages use are never resolved; embedded-file payloads are never decoded, so `attachments` is absent from a page-selected result (an attachment is document-level payload whose size is independent of the pages asked for, and decoding it would cost the same whichever pages were selected). + * + * What it still reads whole, because a negative index and every document-level page reference resolve against it or because it is document-level and holds no page content stream: the page tree, the outline, named destinations, optional-content layers, the AcroForm, the structure element tree (whose size scales with the document), metadata and package residue. Two surfaces are built from page content and so cover selected pages only: the structure-ownership association an item's `structure` field names (and its parent-tree diagnostics), and the `page-boxes` residue row. Destinations a link states as a direct array are minted into `destinations` (as `dest1`, `dest2`, ...) only when their page is interpreted, so an unselected page's direct link destinations are absent and a minted name depends on which pages were read; named destinations are unaffected. + * + * Duplicates collapse, and the result's `pages` holds the selected pages in ascending document order whatever order they were asked for in, each carrying its source position as `index`, with the document's total as `pageCount`; every document-level reference keeps naming source indices. An empty array is a valid selection: it reads no page and returns the document-level surfaces with `pageCount`, the cheapest way to learn a file's length and metadata. An entry naming no page throws `PdfPageSelectionError` rather than being skipped. + */ + readonly pages?: readonly number[]; +} + +// Resolves a page selection against the page tree's length: negative indices counted from the end, duplicates collapsed, and the result as a set, since readPdf walks the tree in document order and asks it per page. Throws on the first entry that names no page, before any page is read. +function resolvePageSelection( + selection: readonly number[], + pageCount: number, +): ReadonlySet { + const resolved = new Set(); + for (const requested of selection) { + const index = requested < 0 ? pageCount + requested : requested; + if (!Number.isInteger(requested) || index < 0 || index >= pageCount) { + throw new PdfPageSelectionError(requested, pageCount); + } + resolved.add(index); + } + return resolved; } const PDF_HEADER_BYTES = new TextEncoder().encode("%PDF-"); @@ -103,6 +133,10 @@ export function readPdf( // The page tree is walked first (materialising the page-index map a destination's page reference resolves against), then the navigation surfaces: page interpretation interns direct destination arrays into the registry as it meets them, so links on page N can name a destination minted by nothing but themselves. const pageDicts = doc.pages(); + const selection = + options?.pages === undefined + ? undefined + : resolvePageSelection(options.pages, pageDicts.length); const destinationRegistry = createDestinationRegistry( doc.catalog, doc, @@ -110,7 +144,11 @@ export function readPdf( sink, ); const outline = readOutline(doc.catalog, destinationRegistry, doc, sink); - const attachments = readAttachments(doc.catalog, pageDicts, doc, sink); + // A page-selected read decodes no embedded file: see ReadPdfOptions.pages. + const attachments = + selection === undefined + ? readAttachments(doc.catalog, pageDicts, doc, sink) + : []; const optionalContent = readOptionalContent(doc.catalog, doc, sink); const form = readAcroForm( doc.catalog, @@ -118,11 +156,21 @@ export function readPdf( (obj) => doc.pageIndex(obj), sink, ); - const structure = readStructure(doc.catalog, pageDicts, doc, sink); + const structure = readStructure( + doc.catalog, + pageDicts, + (index) => selection === undefined || selection.has(index), + doc, + sink, + ); const source = readDocumentResidue(doc, sink); const pageBoxRows: PdfObject[] = []; - const pages = pageDicts.map((pageDict, index) => { + // An unselected page is skipped before readPage, the only path that fetches a page's /Contents, interprets it, and resolves the fonts and images it names: those costs are paid for selected pages alone. Each page keeps its source position as the page index readPage stamps (diagnostics, the structure-ownership lookup, the page-boxes residue row), and a selected page also carries it as `index`, since its position in `pages` no longer says which page it is. + const pages = pageDicts.flatMap((pageDict, index) => { + if (selection !== undefined && !selection.has(index)) { + return []; + } throwIfAborted(signal); const read = readPage( index, @@ -139,7 +187,7 @@ export function readPdf( if (read.boxRow !== undefined) { pageBoxRows.push(read.boxRow); } - return read.page; + return [selection === undefined ? read.page : { ...read.page, index }]; }); // The per-page boundary declarations a distinct crop box or a print-production box left behind, quarantined with the other package-level residue rows. @@ -154,6 +202,7 @@ export function readPdf( formatVersion: LAYOUT_FORMAT_VERSION, metadata: readMetadata(doc.trailer, doc, doc.catalog, sink), pages, + ...(selection !== undefined ? { pageCount: pageDicts.length } : {}), images, ...(destinationRegistry.entries.length > 0 ? { destinations: [...destinationRegistry.entries] } diff --git a/packages/pdf-codec/src/structure.ts b/packages/pdf-codec/src/structure.ts index fa1c26b816..6ebaf295bf 100644 --- a/packages/pdf-codec/src/structure.ts +++ b/packages/pdf-codec/src/structure.ts @@ -16,9 +16,19 @@ export interface StructureContext { readonly ownerOf: (pageIndex: number, mcid: number) => string | undefined; } +/** + * Reads the tagged-structure element tree and the (page, MCID) ownership map. + * @param catalog - the document catalog + * @param pages - every page of the page tree, in document order, so a position is a source page index + * @param includePage - whether the page at a source index is being read: ownership entries and parent-tree diagnostics are built for those pages only, while the element tree itself is document-level and always read whole + * @param resolver - the object resolver + * @param sink - where recoverable parse diagnostics go + * @returns the element tree and an owner lookup answering for included pages + */ export function readStructure( catalog: PdfDict, pages: readonly PdfDict[], + includePage: (pageIndex: number) => boolean, resolver: Readonly, sink: PdfDiagnosticSink, ): StructureContext { @@ -97,7 +107,14 @@ export function readStructure( return { tree, - ownerOf: parentTreeOwners(root, pages, resolver, idByDict, sink), + ownerOf: parentTreeOwners( + root, + pages, + includePage, + resolver, + idByDict, + sink, + ), }; } @@ -229,6 +246,7 @@ function numberTreeEntries( function parentTreeOwners( root: PdfDict, pages: readonly PdfDict[], + includePage: (pageIndex: number) => boolean, resolver: Readonly, idByDict: Map, sink: PdfDiagnosticSink, @@ -248,6 +266,9 @@ function parentTreeOwners( const entries = new Map(numberTreeEntries(documentTree, resolver, new Set())); const owners = new Map(); pages.forEach((page, pageIndex) => { + if (!includePage(pageIndex)) { + return; // a page not being read has no items to associate, so its ownership and its parent-tree diagnostics are skipped + } const structParents = asNumber(dictGet(page, "StructParents")); if (structParents === undefined) { return; // a page declaring no /StructParents states no MCID numbering of its own: nothing it paints can be associated diff --git a/packages/pdf-codec/src/test-support/pdf.ts b/packages/pdf-codec/src/test-support/pdf.ts index ffb10ea260..1e72abea8d 100644 --- a/packages/pdf-codec/src/test-support/pdf.ts +++ b/packages/pdf-codec/src/test-support/pdf.ts @@ -547,3 +547,97 @@ export function navigationClusterPdf(): Uint8Array { b.classicXrefAndTrailer(LINK_TO_GOTO_ACTION_OBJ, "/Root 1 0 R"); return b.bytes(); } + +// A four-page document built to prove a page-selected read never touches what only unselected pages, or the document's embedded files, hold. Page 1 (index 1) is the trap: its content stream declares a filter no reader implements, so decoding it reports pdf/unsupported-filter (the bytes then pass through undecoded, which here are plain content), and it is the only page using font F2, an embedded TrueType whose font program is its own stream. Page 2 (index 2) is the only page drawing the image XObject, so the image registry is non-empty exactly when that page's content was interpreted. The catalog carries one embedded file. Every stream a test needs to recognise carries a /FixtureStream name (PAGE_SELECTION_STREAM), a key no reader interprets, so a spy on the stream decoder can tell which of them a read decoded. Every page shows its own one-based page number as text ("Page 1" to "Page 4"), so a test can tell from the text alone which pages it got. +export const PAGE_SELECTION_TRAP_PAGE_INDEX = 1; +export const PAGE_SELECTION_IMAGE_PAGE_INDEX = 2; +export const PAGE_SELECTION_STREAM = { + trapContent: "TrapContent", + trapFontProgram: "TrapFontProgram", + attachment: "Attachment", +} as const; +export const PAGE_SELECTION_ATTACHMENT_NAME = "notes.txt"; +export function pageSelectionPdf(): Uint8Array { + const b = new FixtureBuilder().header(PDF_1_4); + const PAGE_OBJS = [3, 4, 5, 6] as const; + const SHARED_FONT_OBJ = 7; + const CONTENT_OBJS = [8, 9, 10, 11] as const; + const IMAGE_OBJ = 12; + const TRAP_FONT_OBJ = 13; + const TRAP_FONT_DESCRIPTOR_OBJ = 14; + const TRAP_FONT_PROGRAM_OBJ = 15; + const FILESPEC_OBJ = 16; + const EMBEDDED_FILE_OBJ = 17; + b.object( + 1, + `<< /Type /Catalog /Pages 2 0 R /Names << /EmbeddedFiles << /Names [(${PAGE_SELECTION_ATTACHMENT_NAME}) ${FILESPEC_OBJ} 0 R] >> >> >>`, + ); + b.object( + 2, + `<< /Type /Pages /Kids [${PAGE_OBJS.map((num) => `${num} 0 R`).join(" ")}] /Count ${PAGE_OBJS.length} /MediaBox [0 0 200 100] >>`, + ); + PAGE_OBJS.forEach((pageObj, index) => { + const xObjects = + index === PAGE_SELECTION_IMAGE_PAGE_INDEX + ? ` /XObject << /Im1 ${IMAGE_OBJ} 0 R >>` + : ""; + const fonts = + index === PAGE_SELECTION_TRAP_PAGE_INDEX + ? `/F2 ${TRAP_FONT_OBJ} 0 R` + : `/F1 ${SHARED_FONT_OBJ} 0 R`; + b.object( + pageObj, + `<< /Type /Page /Parent 2 0 R /Resources << /Font << ${fonts} >>${xObjects} >> /Contents ${CONTENT_OBJS[index]} 0 R >>`, + ); + }); + b.object(SHARED_FONT_OBJ, HELVETICA_FONT_DICT); + CONTENT_OBJS.forEach((contentObj, index) => { + if (index === PAGE_SELECTION_TRAP_PAGE_INDEX) { + b.stream( + contentObj, + `<< /Filter /NoSuchDecode /FixtureStream /${PAGE_SELECTION_STREAM.trapContent} >>`, + enc(`BT /F2 12 Tf 10 50 Td (Page ${index + 1}) Tj ET`), + ); + return; + } + const text = `BT /F1 12 Tf 10 50 Td (Page ${index + 1}) Tj ET`; + b.stream( + contentObj, + EMPTY_DICT, + enc( + index === PAGE_SELECTION_IMAGE_PAGE_INDEX + ? `${text} q 10 0 0 10 150 10 cm /Im1 Do Q` + : text, + ), + ); + }); + b.stream( + IMAGE_OBJ, + "<< /Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 >>", + new Uint8Array([0x80]), + ); + b.object( + TRAP_FONT_OBJ, + `<< /Type /Font /Subtype /TrueType /BaseFont /FixtureTrapFont /FontDescriptor ${TRAP_FONT_DESCRIPTOR_OBJ} 0 R >>`, + ); + b.object( + TRAP_FONT_DESCRIPTOR_OBJ, + `<< /Type /FontDescriptor /FontName /FixtureTrapFont /Flags 32 /FontBBox [0 0 1000 1000] /ItalicAngle 0 /Ascent 800 /Descent -200 /CapHeight 700 /StemV 80 /FontFile2 ${TRAP_FONT_PROGRAM_OBJ} 0 R >>`, + ); + b.stream( + TRAP_FONT_PROGRAM_OBJ, + `<< /FixtureStream /${PAGE_SELECTION_STREAM.trapFontProgram} >>`, + enc("not a font program"), + ); + b.object( + FILESPEC_OBJ, + `<< /Type /Filespec /F (${PAGE_SELECTION_ATTACHMENT_NAME}) /EF << /F ${EMBEDDED_FILE_OBJ} 0 R >> >>`, + ); + b.stream( + EMBEDDED_FILE_OBJ, + `<< /Type /EmbeddedFile /FixtureStream /${PAGE_SELECTION_STREAM.attachment} >>`, + enc("Attached file body"), + ); + b.classicXrefAndTrailer(EMBEDDED_FILE_OBJ, "/Root 1 0 R"); + return b.bytes(); +} diff --git a/packages/pdf-codec/src/write.ts b/packages/pdf-codec/src/write.ts index a4c0e0925e..1314bcaad4 100644 --- a/packages/pdf-codec/src/write.ts +++ b/packages/pdf-codec/src/write.ts @@ -162,6 +162,23 @@ function codePointsOf(texts: readonly string[]): Set { ); } +/** + * writePdf was handed a LayoutDocument from a page-selected read (one carrying `pageCount`). Such a document is a reading of part of a file, not a document to re-emit: its pages are a selection while its destinations, form widgets and outline still name source page indices, and the writer places pages by position, so writing it would point those references at the wrong pages or past the end. Thrown rather than writing a file whose links silently lie. + */ +export class PdfPartialDocumentError extends Error { + readonly pageCount: number; + readonly selectedPageCount: number; + + constructor(pageCount: number, selectedPageCount: number) { + super( + `this LayoutDocument holds ${String(selectedPageCount)} pages selected from a ${String(pageCount)}-page file (it carries pageCount, which only a page-selected read sets); its page references name pages of the source file, so it cannot be written as a document of its own`, + ); + this.name = "PdfPartialDocumentError"; + this.pageCount = pageCount; + this.selectedPageCount = selectedPageCount; + } +} + // Assembles a LayoutDocument into a complete PDF file: the object graph (Catalog, Pages, Info, one Font+FontDescriptor pair per standard-14 face actually used, one Image XObject (+SMask) per image asset actually referenced, one embedded math composite font group when options.formulas is non-empty (Type0/CIDFontType0/FontDescriptor/FontFile3/ToUnicode — see math-font-write.ts), one embedded text font group per subsetted face when options.fonts resolved any (Type0/CIDFontType2/FontDescriptor/FontFile2/ToUnicode — see embedded-font-write.ts), then each page's own Page dict, Contents stream (ordinary LayoutItem bytes followed by that page's own formula bytes, if any — see math-content-write.ts), and optional Annots), a classic cross-reference table, and a trailer. Objects are allocated in this fixed order — never derived from Map/object iteration order — so identical input always produces byte-identical output (see the determinism tests). // // Without options.fonts, no embedded text face can exist, so that group consumes no object numbers and every other object is numbered exactly as it was before embedded-font support: output is byte-identical to a build with none of it (proved by the golden digests in write-embedded-font.test.ts). @@ -169,6 +186,9 @@ export function writePdf( doc: LayoutDocument, options: WritePdfOptions = {}, ): Uint8Array { + if (doc.pageCount !== undefined) { + throw new PdfPartialDocumentError(doc.pageCount, doc.pages.length); + } const compress = options.compress ?? true; const registry = options.fonts; const measurer = createFontMeasurer(registry); From b062e1b068bdb321ab931c8006804bd2a3b011c6 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:19:16 +0100 Subject: [PATCH 3/9] feat(documents.js): convert a selection of a PDF's pages The pdfTo* conversions, convertDocument with a pdf source and readNativeDocumentTree/readContentDocument take a pages option, passed to readPdf. The plan runner hands it only to the first hop of a plan whose source is pdf, so a route that renders an intermediate pdf (docx to ods) never selects pages of a document the caller did not supply. readDocumentMetadata reads a pdf with an empty selection, and openPdf always reads every page. Every reconstruction now stamps frames, comment anchor keys and sheet names with the page's source index (sourcePageIndex) rather than its position, so they agree with the destinations and form widgets the layout already names by source index. A page-selected conversion's DocumentTree carries no pages envelope, since that array is indexed by frame page index and a selection cannot fill it. Markdown output of a page-selected conversion opens each page with its original one-based page number, written as beside the existing page-break marker. buildMarkdownText and renderContentDocumentToMarkdown take the same labels as pageNumbers for callers composing the stages themselves. The label is output-only: reading markdown keeps it as ordinary content. Refs #1688 --- packages/documents.js/README.md | 18 + .../documents.js/src/convert/composition.ts | 56 ++- .../src/convert/from-pdf-pages.test.ts | 441 ++++++++++++++++++ packages/documents.js/src/convert/from-pdf.ts | 30 +- .../src/convert/pdf-package-tables.ts | 4 +- .../read-document-metadata-pages.test.ts | 26 ++ packages/documents.js/src/edit/pdf/editor.ts | 8 +- packages/documents.js/src/index.ts | 17 +- .../layout/reconstruct-presentations-prod.ts | 18 +- .../documents.js/src/layout/reconstruct.ts | 23 +- packages/documents.js/src/markdown/render.ts | 5 +- packages/documents.js/src/markdown/write.ts | 87 +++- 12 files changed, 691 insertions(+), 42 deletions(-) create mode 100644 packages/documents.js/src/convert/from-pdf-pages.test.ts create mode 100644 packages/documents.js/src/convert/read-document-metadata-pages.test.ts diff --git a/packages/documents.js/README.md b/packages/documents.js/README.md index 79c5e0847e..890a80f0f5 100644 --- a/packages/documents.js/README.md +++ b/packages/documents.js/README.md @@ -385,6 +385,24 @@ import { pdfToMarkdown } from "documents.js/read"; It carries the ten `pdfTo*` conversions, `PdfToDocumentOptions`, `readDocumentMetadata` (with `ReadDocumentMetadataOptions`), `readNativeDocumentTree` (with `ReadNativeDocumentTreeOptions`), and `readContentDocument` (with `ReadContentDocumentOptions`) — identical functions to the root barrel's (the same forwarders, run through the composition engine's read half, and the same dispatch through the read-only codec half `src/codecs/read.ts`), never a forked behaviour; `convertDocument` and every X-to-PDF direction stay on the root barrel. `src/read-graph.test.ts` walks the entry's static import graph, follows `pdf-codec` specifiers through that package's real `exports` map into its source, and fails the build if the write path or any font asset becomes reachable. +### Converting a selection of a PDF's pages + +The PDF-sourced entry points that produce content take a `pages` option: the `pdfTo*` conversions (`PdfToDocumentOptions.pages`), `convertDocument` with a pdf source (`UnifiedConversionOptions.pages`), and `readNativeDocumentTree`/`readContentDocument` (`ReadNativeDocumentTreeOptions.pages`). It is pdf-codec's `ReadPdfOptions.pages` threaded through: zero-based page-tree indices, a negative index counted from the end (`-1` is the last page), duplicates collapsed, the result in document order, and `PdfPageSelectionError` for an index naming no page. Two entry points never take it: `readDocumentMetadata` always reads with an empty selection, since metadata never depends on page content, so identifying a document reads no page at all; and `openPdf` always reads every page, since an editor holds a document to write back. `convertDocument` applies `pages` only when the source itself is pdf: a conversion from another format ignores it, including one whose route renders an intermediate pdf internally (docx to ods), because that pdf is not the caller's document. + +```ts +import { pdfToMarkdown } from "documents.js/read"; + +const coverAndLastPage = pdfToMarkdown(pdfBytes, { pages: [0, 1, 2, -1] }); +``` + +What a selection skips: the unselected pages' content streams are never fetched, decoded or interpreted, the fonts and images only those pages use are never resolved, and the source's embedded files are never decoded (see pdf-codec's README for exactly what is and is not read). That is what lets a caller identify a document from its first pages, or read a code from its last page, inside a memory-capped runtime such as a Cloudflare Worker isolate, without paying for every page's content. + +The output holds only the selected pages, and reconstruction sees only them, so a page can come out differently than in a full conversion. Heading detection measures text against the body size, taken as the most common text size across the pages read: a cover page read on its own, dominated by large title text, takes the title's size as its body, so the title becomes a paragraph rather than a heading. Reading the cover together with a few ordinary pages restores the document's real body size. + +Markdown output labels each selected page with its original one-based page number, written as `` on its own line at the start of the page (`pageNumberMarker`), alongside the `` marker between pages that share a section, so a caller can tell which pages it got without re-deriving the selection. The label is output-only: reading markdown back keeps it as an ordinary HTML-comment paragraph, exactly as any other comment in a markdown source is kept. A full conversion writes no labels, so its output is unchanged. A caller composing the stages itself (`readPdf` with `pages`, `reconstructWordprocessing`, then `buildMarkdownText` or `renderContentDocumentToMarkdown`) gets the same labels by passing `pageNumbers: layout.pages.map((page, position) => sourcePageIndex(page, position) + 1)`; `buildMarkdownText` throws `MarkdownPageNumberCountError` when the count does not match the document's pages, and a non-wordprocessing document flattened by `renderContentDocumentToMarkdown` counts as a single page. + +Every reconstructed node's frames name the page's index in the source PDF (the same index every destination and form widget names), which is why a page-selected conversion's reported `DocumentTree` carries no `pages` envelope: that array is indexed by frame page index, and a selection cannot fill it. A later stage's own page-addressing option, such as `page` for an svg target, counts within the selection. A page-selected `LayoutDocument` cannot be written back: `writePdf` refuses one with `PdfPartialDocumentError`. `PdfPageSelectionError`, `PdfPartialDocumentError` and `sourcePageIndex` are exported from the root barrel; the `documents.js/read` entry re-exports nothing (this package allows re-exports only in its root barrel), so a caller of that entry recognises a bad selection by the thrown error's `code` (`pdf/page-out-of-range` or `pdf/page-index-not-integer`), and resolves a page's source index as `page.index ?? position`. + ### Reading and building xlsx content directly Every other content format has its own standalone `readXContent`-shaped entry point (`readDocxContent`, `readPptxContent`, `readOdtContent`, `readOdpContent`, `readOdsContent`, `readOdgContent`) — xlsx is no longer the exception. `readXlsxContent`/`buildXlsxPackage` are this package's names for `ooxml.js`'s own spreadsheet `ContentDocument` read/build pair — the same one the `ods⇄xlsx` bridge and every xlsx metadata-rebuild path already use internally — re-exported here directly rather than wrapped, since `readXlsxContent` already produces the right shape on its own. (Since `ooxml.js` 4.0.0 the upstream flat builder is named `buildXlsxPackageFromContent` — the bare `buildXlsxPackage` name moved to that package's tree-form `DocumentTree` builder — so this package re-exports the flat builder under its own long-standing `buildXlsxPackage` name and the `ContentDocument`-in/`Package`-out contract is unchanged.) csv's `readCsvContent`/`buildCsvText` are the same kind of directly-exported stage pair, one level further in: they operate on RFC 4180 text rather than a decoded package (see `src/csv/` under Architecture). svg's `readSvgContent`/`buildSvgText` are the drawing-variant counterpart of csv's pair, operating on SVG text rather than a decoded package (see `src/svg/` under Architecture). diff --git a/packages/documents.js/src/convert/composition.ts b/packages/documents.js/src/convert/composition.ts index 9da298bdc6..937b62bf43 100644 --- a/packages/documents.js/src/convert/composition.ts +++ b/packages/documents.js/src/convert/composition.ts @@ -22,6 +22,7 @@ import { decodePackage as decodeOdfPackage, encodePackage as encodeOdfPackage, } from "odf.js"; +import { sourcePageIndex } from "pdf-codec/layout"; import { readPdf } from "pdf-codec/read"; import type { LayoutDocument, @@ -196,7 +197,8 @@ export const FORMAT_NODES: Readonly> = { signal: options?.signal, images: options?.images, }), - build: (content) => buildMarkdownText(content), + build: (content, options) => + buildMarkdownText(content, { pageNumbers: options?.pageNumbers }), encode: (text) => encodeMarkdownText(text), hasSourcePackage: false, }, @@ -316,6 +318,17 @@ export interface UnifiedConversionOptions { readonly page?: number; readonly onSvgDiagnostic?: SvgDiagnosticSink; readonly clock?: ClockPort; + /** + * The pages of a pdf source to convert, as pdf-codec's `ReadPdfOptions.pages` reads them: zero-based page-tree indices, a negative index counted from the end, duplicates collapsed, the result in document order, and an index naming no page throwing `PdfPageSelectionError`. Only the plan's first hop receives it, and only when the conversion's source is pdf: a conversion from any other format ignores it, including one whose route renders an intermediate pdf internally (docx to ods goes through toPdf then fromPdf), since that pdf is not the caller's document. The unselected pages' content streams, the fonts and images only they use, and the source's embedded files are never read (see `ReadPdfOptions.pages` for exactly what is and is not skipped). + * + * The output holds only those pages. A markdown target labels each with its original one-based page number (`` at the start of the page, beside the usual `` between pages), so the text says which pages it holds. A page-addressing option of a later stage, such as `page` for an svg target, counts within the selection. Reconstruction infers from the selected pages alone, so a page can reconstruct differently than it would in a full conversion: the body text size heading detection compares against is the most common size across the selection, which on a cover page dominated by large title text is the title's own size, leaving the title a paragraph rather than a heading. The DocumentTree `onDocument` reports carries no `pages` envelope for a page-selected conversion: its frames name source page indices, which an envelope holding only the selected pages could not be indexed by. + */ + readonly pages?: readonly number[]; +} + +// What the fromPdf executor knows about a conversion's pages that a text build needs and the caller's options do not carry, passed to a text node's build alongside those options: the original one-based page number of each page the read returned, set only for a page-selected read (see UnifiedConversionOptions.pages), where a page's position in the output no longer says which page it is. Internal to the executors, never caller-supplied. +interface TextBuildContext { + readonly pageNumbers?: readonly number[]; } // --- Registry: declarative per-format primitive wiring ----------------------------------------- @@ -391,7 +404,7 @@ interface TextFormatNode { ) => ContentDocument; readonly build: ( content: ContentDocument, - options?: UnifiedConversionOptions, + options?: UnifiedConversionOptions & TextBuildContext, ) => string; readonly encode: (text: string) => Uint8Array; readonly hasSourcePackage: false; @@ -620,6 +633,13 @@ export function executeBridge( return out; } +// The original one-based page number of each page a read returned, in order: what a page-selected markdown output labels its pages with. +function layoutPageNumbers(layout: LayoutDocument): number[] { + return layout.pages.map( + (page, position) => sourcePageIndex(page, position) + 1, + ); +} + // readPdf -> reconstruct(target variant) -> build(target) -> encode(target), reproducing the exact sequence and option-threading of convert.ts's pdfTo* functions (pdfToDocx/pdfToOdt/pdfToOdp/pdfToOds/pdfToOdg/pdfToMarkdown). sink reaches readPdf; signal reaches both readPdf and the reconstructor. onCellTypeInference reaches the reconstructor (reconstructSpreadsheet's audit channel) AND csv's read — the two places a cell re-typing decision can happen, threaded on the one options object every hop receives; the ergonomic pdfTo* functions do not declare it, so a caller wanting that audit channel on a pdf -> spreadsheet route passes it to convertDocument directly (the first-class entry point every named function forwards to). The package build half is called with no options, matching pdfTo*'s own `buildXPackage(content)` calls (no clock, no onMathDiagnostic threaded on this direction); the text build half receives options because csv's build consumes { delimiter, sheet }. export function executeFromPdf( target: ContentFormat, @@ -630,21 +650,28 @@ export function executeFromPdf( const layout = readPdf(bytes, { signal: options?.signal, sink: options?.sink, + pages: options?.pages, }); const content = RECONSTRUCTORS[node.variant](layout, { signal: options?.signal, onCellTypeInference: options?.onCellTypeInference, }); - // The pages half derives from the read LayoutDocument's own pages — every rendered page's size, indexed to match the frames the reconstructor attached to the content it built. - const pages = layout.pages.map((page) => ({ - widthPt: page.widthPt, - heightPt: page.heightPt, - })); + const selected = options?.pages !== undefined; + // The pages half derives from the read LayoutDocument's own pages: every rendered page's size, indexed to match the frames the reconstructor attached to the content it built. A page-selected read has no such array to offer, since its frames name source page indices and its pages are only the selection, so the envelope is left off rather than indexed wrongly. + const pages = selected + ? undefined + : layout.pages.map((page) => ({ + widthPt: page.widthPt, + heightPt: page.heightPt, + })); // Build + encode the target first, then report the package (the ownership rule every construction site follows), with assembleTree decomposing the reconstructed content + page sizes into the tree-form DocumentTree. stampPdfPackageTables then lands the layout's document-level surfaces (destinations, outline, attachments, layers, residue, comment bodies) on the tree — the tables the flat ContentDocument has no root for. let out: Uint8Array; if (isTextFormatNode(node)) { - const text = node.build(content, options); + const text = node.build(content, { + ...options, + pageNumbers: selected ? layoutPageNumbers(layout) : undefined, + }); out = node.encode(text); } else { const pkg = node.build(content); @@ -693,11 +720,16 @@ export function runCompositionPlan( } // onDocument fires on the last hop only: null out onDocument for every hop before the last, then pass the caller's options through unchanged on the final hop so the callback receives the package that actually produced the output bytes. This mirrors the convention the original hand-written composed functions (xlsxToPdf/pdfToXlsx/xlsxToMarkdown/markdownToXlsx) already followed — each forwarded onDocument to its own last hop, never the first — so a caller of xlsxToPdf still receives the odsToPdf hop's content+layout package rather than the intermediate xlsx->ods bridge's content-only one. const isLastHop = i === plan.hops.length - 1; - const hopOptions: UnifiedConversionOptions | undefined = isLastHop - ? options - : options === undefined + // pages selects pages of the caller's own pdf source, so only the hop that reads that source receives it: the first hop of a plan whose source is pdf. Every other hop's pdf (an intermediate rendering, as on a docx -> ods route through toPdf then fromPdf) is not the caller's document, and selecting its pages would silently drop content or throw for an index the caller never meant against it. + const readsCallerPdf = i === 0 && hop.from === "pdf"; + const hopOptions: UnifiedConversionOptions | undefined = + options === undefined ? undefined - : { ...options, onDocument: undefined }; + : { + ...options, + onDocument: isLastHop ? options.onDocument : undefined, + pages: readsCallerPdf ? options.pages : undefined, + }; if (hop.executor === "toPdf") { if (!isSourceContentFormat(hop.from)) { throw new Error( diff --git a/packages/documents.js/src/convert/from-pdf-pages.test.ts b/packages/documents.js/src/convert/from-pdf-pages.test.ts new file mode 100644 index 0000000000..9172153bc1 --- /dev/null +++ b/packages/documents.js/src/convert/from-pdf-pages.test.ts @@ -0,0 +1,441 @@ +import { + assembleTree, + type ContentBlock, + type ContentDocument, + DEFAULT_LAYOUT_FONT, + type DocumentTree, +} from "document-schema.js"; +import { + LAYOUT_FORMAT_VERSION, + type LayoutDocument, + PdfPageSelectionError, + readPdf, + sourcePageIndex, +} from "pdf-codec"; +import { describe, expect, it } from "vitest"; +import { decodePackage } from "odf.js"; +import { createDocx } from "../edit/docx/editor"; +import { readOdsContent } from "../odf/ods/read"; +import { createPdf, openPdf } from "../edit/pdf/editor"; +import { fixedClock } from "../ports/clock"; +import { convertDocument } from "./composition-to-pdf"; +import { + reconstructSpreadsheet, + reconstructWordprocessing, +} from "../layout/reconstruct"; +import { + reconstructDrawing, + reconstructPresentation, +} from "../layout/reconstruct-presentations-prod"; +import { stampPdfPackageTables } from "./pdf-package-tables"; +import { readMarkdownContent } from "../markdown/read"; +import { decodeMarkdownText } from "../markdown/text"; +import { + buildMarkdownText, + MarkdownPageNumberCountError, + PAGE_BREAK_MARKER, + pageNumberMarker, +} from "../markdown/write"; +import { renderContentDocumentToMarkdown } from "../markdown/render"; +import { + pdfToMarkdown, + pdfToSvg, + readContentDocument, + readNativeDocumentTree, +} from "./from-pdf"; + +const BLACK = { r: 0, g: 0, b: 0 }; +const LETTER = { widthPt: 612, heightPt: 792 }; +// A page size differing from LETTER, so the page carrying it starts a section of its own in the wordprocessing reconstruction: a page selection spanning it must still label every page, not only those separated by page breaks. +const WIDE = { widthPt: 792, heightPt: 612 }; +const PAGE_SIZES = [LETTER, LETTER, WIDE, LETTER, LETTER] as const; +const LAST_PAGE_INDEX = PAGE_SIZES.length - 1; +const TEXT_SIZE_PT = 12; +const TEXT_X_PT = 72; +const TEXT_Y_PT = 500; + +// "Body of page N" on page N (one-based), so the text says which page it came from. +function bodyOf(pageNumber: number): string { + return `Body of page ${String(pageNumber)}`; +} + +function fivePagePdf(): Uint8Array { + const [first, ...rest] = PAGE_SIZES; + const editor = createPdf(first); + for (const size of rest) { + editor.appendPage(size); + } + editor.pages().forEach((page, index) => { + page.appendText({ + xPt: TEXT_X_PT, + yPt: TEXT_Y_PT, + text: bodyOf(index + 1), + font: DEFAULT_LAYOUT_FONT, + sizePt: TEXT_SIZE_PT, + color: BLACK, + }); + }); + return editor.toBytes(); +} + +function markdownOf(pages?: readonly number[]): string { + return decodeMarkdownText(pdfToMarkdown(fivePagePdf(), { pages })); +} + +function frameIndicesOf(value: unknown): number[] { + const found: number[] = []; + const walk = (node: unknown): void => { + if (Array.isArray(node)) { + node.forEach(walk); + return; + } + if (typeof node !== "object" || node === null) { + return; + } + if ("frames" in node && Array.isArray(node.frames)) { + const frames: readonly unknown[] = node.frames; + for (const frame of frames) { + if ( + typeof frame === "object" && + frame !== null && + "pageIndex" in frame && + typeof frame.pageIndex === "number" + ) { + found.push(frame.pageIndex); + } + } + } + Object.values(node).forEach(walk); + }; + walk(value); + return [...new Set(found)].sort((a, b) => a - b); +} + +describe("pdfToMarkdown with a page selection", () => { + it("writes only the selected pages, each opened by its original one-based page number", () => { + const text = markdownOf([0, -1]); + expect(text).toContain(bodyOf(1)); + expect(text).toContain(bodyOf(PAGE_SIZES.length)); + for (const pageNumber of [2, 3, 4]) { + expect(text).not.toContain(bodyOf(pageNumber)); + } + const labels = text.match(//gu); + expect(labels).toEqual([ + pageNumberMarker(1), + pageNumberMarker(PAGE_SIZES.length), + ]); + expect(text.indexOf(pageNumberMarker(1))).toBeLessThan( + text.indexOf(bodyOf(1)), + ); + expect(text.indexOf(pageNumberMarker(PAGE_SIZES.length))).toBeLessThan( + text.indexOf(bodyOf(PAGE_SIZES.length)), + ); + expect(text.indexOf(bodyOf(1))).toBeLessThan( + text.indexOf(pageNumberMarker(PAGE_SIZES.length)), + ); + }); + + it("keeps the page-break marker between two selected pages that share a section", () => { + const text = markdownOf([0, -1]); + expect(text.split(PAGE_BREAK_MARKER)).toHaveLength(2); + }); + + it("labels a page that starts a section of its own as well as pages after a page break", () => { + const text = markdownOf([1, 2, 3]); + expect(text.match(//gu)).toEqual([ + pageNumberMarker(2), + pageNumberMarker(3), + pageNumberMarker(4), + ]); + }); + + it("labels nothing on a full conversion, whose output is unchanged by the option's existence", () => { + const text = markdownOf(); + expect(text).not.toMatch(//u); + for (const [index] of PAGE_SIZES.entries()) { + expect(text).toContain(bodyOf(index + 1)); + } + }); + + it("reports a DocumentTree with no pages envelope, its frames naming source page indices", () => { + let reported: DocumentTree | undefined; + pdfToMarkdown(fivePagePdf(), { + pages: [-1], + onDocument: (tree) => { + reported = tree; + }, + }); + expect(reported?.pages).toBeUndefined(); + expect(frameIndicesOf(reported)).toEqual([LAST_PAGE_INDEX]); + }); + + it("throws PdfPageSelectionError for an index naming no page", () => { + expect(() => markdownOf([PAGE_SIZES.length])).toThrow( + PdfPageSelectionError, + ); + }); +}); + +describe("readNativeDocumentTree and readContentDocument with a page selection", () => { + it("stamps every frame with the page's index in the source PDF and leaves the pages envelope off", () => { + const tree = readNativeDocumentTree("pdf", fivePagePdf(), { + pages: [1, LAST_PAGE_INDEX], + }); + expect(tree.pages).toBeUndefined(); + expect(frameIndicesOf(tree)).toEqual([1, LAST_PAGE_INDEX]); + expect(readNativeDocumentTree("pdf", fivePagePdf()).pages).toHaveLength( + PAGE_SIZES.length, + ); + }); + + it("reconstructs only the selected pages' content", () => { + const content = readContentDocument("pdf", fivePagePdf(), { + pages: [-1], + }); + const text = JSON.stringify(content); + expect(text).toContain(bodyOf(PAGE_SIZES.length)); + expect(text).not.toContain(bodyOf(1)); + }); +}); + +describe("pdfToSvg with a page selection", () => { + // The svg writer renders vectors only, so the page is identified by its size: the selected page is the one WIDE page. + const WIDE_PAGE_INDEX = PAGE_SIZES.indexOf(WIDE); + + it("needs no page option once the selection holds a single page, and writes that page", () => { + expect(() => pdfToSvg(fivePagePdf())).toThrow(); + const svg = new TextDecoder().decode( + pdfToSvg(fivePagePdf(), { pages: [WIDE_PAGE_INDEX] }), + ); + expect(svg).toContain( + `width="${String(WIDE.widthPt)}pt" height="${String(WIDE.heightPt)}pt"`, + ); + }); +}); + +describe("buildMarkdownText pageNumbers", () => { + it("is what a caller composing readPdf, reconstructWordprocessing and the writer passes to get the same labels", () => { + const layout = readPdf(fivePagePdf(), { pages: [0, -1] }); + const composed = renderContentDocumentToMarkdown( + reconstructWordprocessing(layout), + { + pageNumbers: layout.pages.map( + (page, position) => sourcePageIndex(page, position) + 1, + ), + }, + ); + expect(composed.trim()).toBe(markdownOf([0, -1]).trim()); + }); + + it("refuses a page-number count that differs from the document's page count", () => { + const content = reconstructWordprocessing( + readPdf(fivePagePdf(), { pages: [0, -1] }), + ); + expect(() => buildMarkdownText(content, { pageNumbers: [1] })).toThrow( + MarkdownPageNumberCountError, + ); + expect(() => + buildMarkdownText(content, { pageNumbers: [1, 2, 3] }), + ).toThrow(MarkdownPageNumberCountError); + }); + + it("is output-only: reading markdown keeps a page label, hand-written or generated, as an ordinary paragraph", () => { + const handWritten = "Intro\n\n\n\nBody\n"; + const read = readMarkdownContent(handWritten); + if (read.kind !== "wordprocessing") { + throw new Error("expected a wordprocessing document"); + } + const texts = read.sections + .flatMap((section) => section.blocks) + .flatMap((block) => + block.kind === "paragraph" + ? [block.runs.map((run) => run.text).join("")] + : [], + ); + expect(texts).toContain(pageNumberMarker(12)); + expect(JSON.stringify(readMarkdownContent(markdownOf([0, -1])))).toContain( + pageNumberMarker(PAGE_SIZES.length), + ); + }); +}); + +describe("reconstruction of a page-selected layout", () => { + // One page, the last of a five-page source: index 4 at position 0, so any reconstruction reading the position instead of the source index reports page 0. + + const selectedLayout: LayoutDocument = { + formatVersion: LAYOUT_FORMAT_VERSION, + metadata: {}, + pageCount: PAGE_SIZES.length, + pages: [ + { + ...LETTER, + index: LAST_PAGE_INDEX, + items: [ + { + kind: "text", + text: bodyOf(LAST_PAGE_INDEX + 1), + xPt: TEXT_X_PT, + yPt: TEXT_Y_PT, + font: DEFAULT_LAYOUT_FONT, + sizePt: TEXT_SIZE_PT, + color: BLACK, + }, + ], + annotations: [ + { + subtype: "Text", + xPt: TEXT_X_PT, + yPt: TEXT_Y_PT, + widthPt: TEXT_SIZE_PT, + heightPt: TEXT_SIZE_PT, + contents: "A note", + }, + ], + }, + ], + images: {}, + }; + + it.each<{ + readonly kind: string; + readonly reconstruct: (layout: LayoutDocument) => ContentDocument; + }>([ + { kind: "wordprocessing", reconstruct: reconstructWordprocessing }, + { kind: "presentation", reconstruct: reconstructPresentation }, + { kind: "drawing", reconstruct: reconstructDrawing }, + { kind: "spreadsheet", reconstruct: reconstructSpreadsheet }, + ])( + "the $kind reconstruction stamps frames with the source page index", + ({ reconstruct }) => { + expect(frameIndicesOf(reconstruct(selectedLayout))).toEqual([ + LAST_PAGE_INDEX, + ]); + }, + ); + + it("names a reconstructed sheet after the source page", () => { + const content = reconstructSpreadsheet(selectedLayout); + if (content.kind !== "spreadsheet") { + throw new Error("expected a spreadsheet"); + } + expect(content.sheets.map((sheet) => sheet.name)).toEqual([ + `Sheet${String(LAST_PAGE_INDEX + 1)}`, + ]); + }); + + it("keys a comment's anchor and its definition by the source page index, so the two still meet", () => { + const key = `pdf-annot-${String(LAST_PAGE_INDEX)}-0`; + const content = reconstructWordprocessing(selectedLayout); + expect(JSON.stringify(content)).toContain(`"name":"${key}"`); + const tree = assembleTree(content); + stampPdfPackageTables(tree, selectedLayout); + expect(Object.keys(tree.definitions ?? {})).toContain(key); + }); +}); + +describe("buildMarkdownText pageNumbers page-start rule", () => { + it("does not start a page at a section that continues its predecessor's page", () => { + const section = { + pageSize: LETTER, + margins: { topPt: 0, rightPt: 0, bottomPt: 0, leftPt: 0 }, + }; + const paragraph = (text: string): ContentBlock => ({ + kind: "paragraph", + runs: [{ text }], + }); + const content: ContentDocument = { + kind: "wordprocessing", + metadata: {}, + sections: [ + { ...section, blocks: [paragraph("first")] }, + { + ...section, + breakType: "continuous", + blocks: [paragraph("same page")], + }, + { ...section, blocks: [paragraph("second")] }, + ], + }; + const text = buildMarkdownText(content, { pageNumbers: [7, 9] }); + expect(text.match(//gu)).toEqual([ + pageNumberMarker(7), + pageNumberMarker(9), + ]); + expect(text.indexOf("same page")).toBeLessThan( + text.indexOf(pageNumberMarker(9)), + ); + }); +}); + +describe("pages reaches only the caller's own pdf", () => { + const CLOCK = fixedClock(new Date("2026-01-02T03:04:05Z")); + // A docx long enough to render several pages: docx to ods routes through toPdf then fromPdf, so an intermediate pdf exists whose pages a leaked selection would cut. + function multiPageDocx(): Uint8Array { + const editor = createDocx(); + PAGE_SIZES.forEach((_size, index) => { + if (index > 0) { + editor.body.appendPageBreak(); + } + editor.body.appendParagraph({ text: bodyOf(index + 1) }); + }); + return editor.toBytes(); + } + + // Compared by the spreadsheet content read back rather than by bytes: the package records its own write moment, so two conversions a second apart differ in bytes while converting the same content. + function odsSheetsOf(bytes: Uint8Array): unknown { + const content = readOdsContent(decodePackage(bytes)); + return content.kind === "spreadsheet" ? content.sheets : content; + } + + it("ignores pages on a docx to ods conversion: the whole document converts, and an index past its rendering never throws", () => { + const whole = odsSheetsOf( + convertDocument("docx", "ods", multiPageDocx(), { clock: CLOCK }), + ); + expect(JSON.stringify(whole)).toContain(bodyOf(PAGE_SIZES.length)); + expect( + odsSheetsOf( + convertDocument("docx", "ods", multiPageDocx(), { + clock: CLOCK, + pages: [0], + }), + ), + ).toEqual(whole); + expect( + odsSheetsOf( + convertDocument("docx", "ods", multiPageDocx(), { + clock: CLOCK, + pages: [PAGE_SIZES.length * PAGE_SIZES.length], + }), + ), + ).toEqual(whole); + }); + + it("still reports onDocument exactly once, from the last hop, on a multi-hop route", () => { + let calls = 0; + convertDocument("docx", "ods", multiPageDocx(), { + clock: CLOCK, + pages: [0], + onDocument: () => { + calls++; + }, + }); + expect(calls).toBe(1); + }); + + it("still selects pages on a conversion whose source is pdf", () => { + const text = new TextDecoder().decode( + convertDocument("pdf", "markdown", fivePagePdf(), { pages: [-1] }), + ); + expect(text).toContain(bodyOf(PAGE_SIZES.length)); + expect(text).not.toContain(bodyOf(1)); + }); +}); + +describe("openPdf reads every page", () => { + it("opens the whole document even when handed a selection, so its editor can be written back", () => { + const options: Parameters[1] = { pages: [0] }; + const editor = openPdf(fivePagePdf(), options); + expect(editor.pages()).toHaveLength(PAGE_SIZES.length); + expect(editor.toBytes().length).toBeGreaterThan(0); + }); +}); diff --git a/packages/documents.js/src/convert/from-pdf.ts b/packages/documents.js/src/convert/from-pdf.ts index 01a8307865..c044914e2c 100644 --- a/packages/documents.js/src/convert/from-pdf.ts +++ b/packages/documents.js/src/convert/from-pdf.ts @@ -1,6 +1,6 @@ // The pdf -> X conversion family and this package's read-only entry point: every pdfTo* function lives here rather than in convert.ts so that a consumer importing documents.js/read (package.json's explicit ./read export onto this module) gets a module graph that excludes every X-to-PDF renderer and therefore every vendored font asset — the same split pdf-codec made with its own ./read entry. convert.ts holds both directions in one module, so its graph statically contains the write path; each pdfTo* function below forwards to convertDocumentFromPdf (src/convert/composition.ts, the composition engine's read half), which resolves and runs exactly the plan convertDocument('pdf', target, ...) would — identical executors, identical routes (a route out of pdf never re-enters pdf), no forked behaviour. src/read-graph.test.ts walks this module's static import graph across the workspace boundary into pdf-codec's own source and fails the build if the write path or a font asset becomes reachable. // -// readDocumentMetadata is defined here too (#744), owning its dispatch through the read-only codec half (src/codecs/read.ts: CONTENT_READERS for the eleven content formats, readDocumentLayout for pdf) rather than the both-directions registry — whose write halves import writePdf from the pdf-codec root barrel and would drag the write path, and pdf-codec's vendored font assets, back into this entry's graph. The root barrel re-exports it from here unchanged (one definition, one identity, the same arrangement the pdfTo* family already has). Its xlsx branch no longer renders a preview PDF: the old xlsxToPdf-then-readPdf path reported facts about the render rather than the workbook — createdIso/modifiedIso stamped at the render moment for a file carrying none of its own (buildOdsPackage's default systemClock, which is why the old test for that case needed fake timers), and a producer naming the preview PDF's writer. A workbook that does declare its own dcterms:created/dcterms:modified keeps reporting them exactly as before (the render path never overwrote a source timestamp either), app.xml's Application still arrives as creator, and producer stays unset per the schema's own rule that it is a PDF-only concept no semantic reader ever sets. setDocumentMetadata — the write-side sibling — stays in src/metadata/write.ts, which has no graph constraint to satisfy. +// readDocumentMetadata is defined here too (#744), owning its dispatch through the read-only codec half (src/codecs/read.ts: CONTENT_READERS for the eleven content formats; pdf reads through pdf-codec/read's readPdf with an empty page selection, since metadata never depends on page content) rather than the both-directions registry — whose write halves import writePdf from the pdf-codec root barrel and would drag the write path, and pdf-codec's vendored font assets, back into this entry's graph. The root barrel re-exports it from here unchanged (one definition, one identity, the same arrangement the pdfTo* family already has). Its xlsx branch no longer renders a preview PDF: the old xlsxToPdf-then-readPdf path reported facts about the render rather than the workbook — createdIso/modifiedIso stamped at the render moment for a file carrying none of its own (buildOdsPackage's default systemClock, which is why the old test for that case needed fake timers), and a producer naming the preview PDF's writer. A workbook that does declare its own dcterms:created/dcterms:modified keeps reporting them exactly as before (the render path never overwrote a source timestamp either), app.xml's Application still arrives as creator, and producer stays unset per the schema's own rule that it is a PDF-only concept no semantic reader ever sets. setDocumentMetadata — the write-side sibling — stays in src/metadata/write.ts, which has no graph constraint to satisfy. // // readNativeDocumentTree (#823 Ask 1) is the third dispatcher in this same read-only family: it reports the SOURCE format's own native DocumentTree, with no cross-format bridging or conversion target involved at all — unlike ConversionResult.package/onDocument, which report whatever hop actually produced a REQUESTED conversion's output (a lossy cross-variant bridge's shape for a target sharing no ContentDocument variant with the source, e.g. xlsx -> markdown composing through a pdf pivot and reporting that pivot's wordprocessing-shaped ContentDocument, with no sheet/cell/formula/A1 data at all — the exact bug #823 reports). Every DocumentFormat member reads through CONTENT_READERS directly (no reconstruction, no pages, a bridge-free ContentDocument straight from that format's own reader — odf's readOdfFormulaContent included, since CONTENT_READERS already covers it) except pdf, which has no ContentDocument reader of its own: its native representation is readPdf's LayoutDocument reconstructed into a wordprocessing ContentDocument via reconstructWordprocessing (the identical reconstruction executeFromPdf's own pdf-to-docx hop runs, and the one OUTLINE_CONVERSION_TARGET/OUTLINE_PROBE_TARGETS already treat as pdf's "native" tree), with pages/frames and the document-level PDF tables (destinations/outline/attachments/layers/structure/comment bodies) stamped on exactly as that hop's own onDocument report already carries. readPdf is called directly (not through readDocumentLayout, which carries no sink) so a caller of readNativeDocumentTree gets the identical PDF-parse diagnostics a real pdf-sourced conversion would report. @@ -17,7 +17,7 @@ import { convertDocumentFromPdf } from "./composition"; import type { CsvWriteOptions, SvgWriteOptions } from "./convert"; import { stampPdfPackageTables } from "./pdf-package-tables"; import type { DocumentFormat } from "./port"; -import { CONTENT_READERS, readDocumentLayout } from "../codecs/read"; +import { CONTENT_READERS } from "../codecs/read"; import { reconstructWordprocessing } from "../layout/reconstruct"; export interface ReadDocumentMetadataOptions { @@ -31,7 +31,8 @@ export function readDocumentMetadata( options?: ReadDocumentMetadataOptions, ): LayoutMetadata { if (format === "pdf") { - return readDocumentLayout(bytes, { signal: options?.signal }).metadata; + // Metadata is document-level, so no page is read: an empty selection skips every page's content stream (and the source's embedded files), which keeps identifying a document as cheap as its catalog and /Info allow. + return readPdf(bytes, { signal: options?.signal, pages: [] }).metadata; } return CONTENT_READERS[format](bytes, { signal: options?.signal }).metadata; } @@ -42,6 +43,10 @@ export interface ReadNativeDocumentTreeOptions { readonly images?: MarkdownImageResolver; // Consulted only for a pdf source — every other format's reader has nothing to report through it. readonly sink?: PdfDiagnosticSink; + /** + * The pages of a pdf source to read, exactly as UnifiedConversionOptions.pages describes them (zero-based, negative from the end, document order, `PdfPageSelectionError` for an index naming no page). Consulted only for a pdf source, since no other format's reader has pages to select. The unselected pages' content streams, the fonts and images only they use, and the source's embedded files are never read, and the selected pages are reconstructed from themselves alone (see PdfToDocumentOptions.pages for how that can change a cover page's headings). Every frame in the result names the page's index in the source PDF, so a DocumentTree read this way carries no `pages` envelope: that array is indexed by those source indices and a selection cannot fill it. + */ + readonly pages?: readonly number[]; } // readContentDocument's own options are the identical shape — one options type for both, so a caller forwarding {signal, images, sink} through either entry point never needs to know which fields the OTHER one ignores. @@ -57,6 +62,7 @@ function readNativeContent( const layout = readPdf(bytes, { signal: options?.signal, sink: options?.sink, + pages: options?.pages, }); const content = reconstructWordprocessing(layout, { signal: options?.signal, @@ -78,10 +84,14 @@ export function readNativeDocumentTree( ): DocumentTree { const { content, layout } = readNativeContent(format, bytes, options); if (layout !== undefined) { - const pages = layout.pages.map((page) => ({ - widthPt: page.widthPt, - heightPt: page.heightPt, - })); + // A page-selected read's frames name source page indices, which an array of only the selected pages' sizes could not be indexed by, so its tree carries no pages envelope (see ReadNativeDocumentTreeOptions.pages). + const pages = + options?.pages === undefined + ? layout.pages.map((page) => ({ + widthPt: page.widthPt, + heightPt: page.heightPt, + })) + : undefined; const reported = assembleTree(content, pages); stampPdfPackageTables(reported, layout); return reported; @@ -101,6 +111,12 @@ export function readContentDocument( export interface PdfToDocumentOptions { readonly signal?: AbortSignal; readonly sink?: PdfDiagnosticSink; + /** + * The pages to convert, exactly as UnifiedConversionOptions.pages describes them: zero-based page-tree indices, a negative index counted from the end, duplicates collapsed, document order, and `PdfPageSelectionError` for an index naming no page. The unselected pages' content streams, the fonts and images only they use, and the source's embedded files are never read. pdfToMarkdown labels each page it writes with its original one-based page number (``), so the text says which pages it holds. + * + * Reconstruction sees only the selected pages, so a page can come out differently than it would in a full conversion. Heading detection measures text against the body size, taken as the most common size across the pages read: a cover page read on its own, dominated by large title text, takes the title's size as its body, and the title is then a paragraph rather than a heading. Reading the cover together with a few ordinary pages restores the document's real body size. + */ + readonly pages?: readonly number[]; // Called exactly once, synchronously, with the DocumentTree (the readPdf-produced layout, and the reconstructed content built from it) this conversion built internally, before the function returns its bytes — see convert.ts's DocumentToPdfOptions.onDocument for the mirrored X-to-PDF side of this same side channel. readonly onDocument?: (pkg: DocumentTree) => void; } diff --git a/packages/documents.js/src/convert/pdf-package-tables.ts b/packages/documents.js/src/convert/pdf-package-tables.ts index 1468f7569e..f546ee424b 100644 --- a/packages/documents.js/src/convert/pdf-package-tables.ts +++ b/packages/documents.js/src/convert/pdf-package-tables.ts @@ -5,6 +5,7 @@ import type { LayoutOutlineItem, LayoutStructureElement, } from "pdf-codec"; +import { sourcePageIndex } from "pdf-codec/layout"; // The package-table half of PDF-side construct surfacing (#721): stamping the LayoutDocument's document-level surfaces onto the tree the fromPdf executor assembles. The flat ContentDocument a reconstructor returns has no root for these tables (they are tree-only by design), so this runs immediately after assembleTree — the one place both the layout (the facts) and the package (the home) are in hand. Entry vocabularies are this package's own tenants inside the generic definitions-table facility: a destinations entry names a page and a view, an outline entry names its title/destination and parent, an attachment carries decoded bytes, a layer its visibility, and (#760) a structure entry states one tagged-PDF element's type and attributes, parent stated as a reference. @@ -100,7 +101,8 @@ export function stampPdfPackageTables( // Comment bodies: the same deterministic pdf-annot-{page}-{index} keys reconstruct.ts's anchor constructs reference, so the marker and its definition can never drift apart — both derive from one walk of the same annotations array. const definitions: Record = {}; - layout.pages.forEach((page, pageIndex) => { + layout.pages.forEach((page, position) => { + const pageIndex = sourcePageIndex(page, position); (page.annotations ?? []).forEach( (annotation: LayoutAnnotation, annotIndex) => { definitions[`pdf-annot-${String(pageIndex)}-${String(annotIndex)}`] = { diff --git a/packages/documents.js/src/convert/read-document-metadata-pages.test.ts b/packages/documents.js/src/convert/read-document-metadata-pages.test.ts new file mode 100644 index 0000000000..10adde7b55 --- /dev/null +++ b/packages/documents.js/src/convert/read-document-metadata-pages.test.ts @@ -0,0 +1,26 @@ +import { describe, expect, it, vi } from "vitest"; +import type * as PdfCodecRead from "pdf-codec/read"; +import { readPdf } from "pdf-codec/read"; +import { markdownToPdf } from "./convert"; +import { readDocumentMetadata } from "./from-pdf"; +import { encodeMarkdownText } from "../markdown/text"; + +// readPdf wrapped so the test can see which pages a metadata read asked for. The real reader still runs. +vi.mock("pdf-codec/read", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, readPdf: vi.fn(actual.readPdf) }; +}); + +describe("readDocumentMetadata for a pdf", () => { + it("reads no page, and reports the same metadata a full read does", () => { + const pdf = markdownToPdf( + encodeMarkdownText("# Title\n\nBody.\n\n\n\nMore.\n"), + ); + vi.mocked(readPdf).mockClear(); + const metadata = readDocumentMetadata("pdf", pdf); + expect( + vi.mocked(readPdf).mock.calls.map(([, options]) => options?.pages), + ).toEqual([[]]); + expect(metadata).toEqual(vi.mocked(readPdf)(pdf).metadata); + }); +}); diff --git a/packages/documents.js/src/edit/pdf/editor.ts b/packages/documents.js/src/edit/pdf/editor.ts index 1c38c3783c..215f665471 100644 --- a/packages/documents.js/src/edit/pdf/editor.ts +++ b/packages/documents.js/src/edit/pdf/editor.ts @@ -68,11 +68,15 @@ export class PdfEditor { // Parses `bytes` via pdf-codec's own readPdf and wraps the result as the one live, mutable LayoutDocument this editor and every PdfPage/PdfItem it hands out share. // // SECURITY NOTE: if `bytes` is encrypted with the PDF standard security handler and an empty user password (the common "restrict editing, not reading" case), readPdf decrypts it transparently and this call succeeds with a perfectly ordinary-looking PdfEditor — there is no field on the resulting document recording that this happened. toBytes() on the returned editor writes a plain, UNENCRYPTED PDF regardless, even though the source was encrypted. A caller that needs to preserve or re-apply the source's encryption must do so itself; this editor has no re-encryption path at all. A source requiring a real (non-empty) password throws PdfPasswordRequiredError, and one using an unsupported security handler throws PdfEncryptedError, both from readPdf itself before this function returns. +// +// Always reads every page, which is why `pages` is not among the options: an editor holds a document to change and write back, and a page-selected read is only part of one (its references still name source pages, and writePdf refuses it with PdfPartialDocumentError). The two options it does take are forwarded field by field, so a wider-typed options value carrying `pages` cannot slip a selection through either. export function openPdf( bytes: Uint8Array, - options?: ReadPdfOptions, + options?: Omit, ): PdfEditor { - return new PdfEditor(readPdf(bytes, options)); + return new PdfEditor( + readPdf(bytes, { sink: options?.sink, signal: options?.signal }), + ); } export interface CreatePdfOptions { diff --git a/packages/documents.js/src/index.ts b/packages/documents.js/src/index.ts index 4fe3ed7835..d86ba66cc3 100644 --- a/packages/documents.js/src/index.ts +++ b/packages/documents.js/src/index.ts @@ -162,7 +162,11 @@ export type { LayoutSubpath, LayoutText, } from "pdf-codec"; -export { LAYOUT_FORMAT_VERSION, LayoutDocumentSchema } from "pdf-codec"; +export { + LAYOUT_FORMAT_VERSION, + LayoutDocumentSchema, + sourcePageIndex, +} from "pdf-codec"; export type { LayoutMetadata } from "document-schema.js"; // --- document-schema.js's own tree-form DocumentTree (structure, layout, and content fused in one tree since 4.0.0) and its self-describing-JSON helpers. contentDocumentWithSchema / documentSchemaKindOf / documentFromJson's ContentDocument branch / ContentDocumentJson all operate on the identical ContentDocument type exported above. The package's own layoutDocumentWithSchema/LayoutDocumentJson exports retired with the demotion: a LayoutDocument has no schema-stamped JSON envelope any more, it is a pdf-codec value. --- @@ -389,7 +393,7 @@ export { export type { ReadPdfOptions } from "pdf-codec"; export { readPdf } from "pdf-codec"; export type { WritePdfOptions } from "pdf-codec"; -export { writePdf } from "pdf-codec"; +export { PdfPartialDocumentError, writePdf } from "pdf-codec"; export type { PdfDiagnostic, PdfDiagnosticSeverity, @@ -398,6 +402,7 @@ export type { export { NOOP_DIAGNOSTIC_SINK, PdfEncryptedError, + PdfPageSelectionError, PdfParseError, } from "pdf-codec"; export type { WinAnsiSubstitution } from "pdf-codec"; @@ -555,7 +560,13 @@ export { // markdown <-> ContentDocument — readMarkdownContent/buildMarkdownText are thin adapters over markdown-codec's own readMarkdownContent/writeMarkdownContent (see src/markdown/read.ts's own module comment), never re-exported here directly for the same reason ooxml.js's flat docx/pptx readers aren't (see this section's own top-of-file note): markdown-codec's own readMarkdownContent/writeMarkdownContent operate on document-schema.js's ContentDocument shape directly, a nominally different type from this package's own local ContentDocument above, so exposing both would invite a caller to reach for the wrong one. decodeMarkdownText/encodeMarkdownText are the byte<->text boundary markdown-codec itself has no opinion on (it operates on strings, not bytes), decodeMarkdownText working the character encoding out from the bytes rather than assuming UTF-8 and encodeMarkdownText always writing UTF-8 — exported for a caller composing readMarkdownContent/buildMarkdownText directly, matching every other independently-exported pipeline stage in this section. Construct markers pass straight through to markdown-codec's own writer, which resolves them as balanced brackets and renders each construct it has a markdown spelling for (a footnote, a blockquote division, a titled image) and renders the rest transparently. export { decodeMarkdownText, encodeMarkdownText } from "./markdown/text"; export { readMarkdownContent } from "./markdown/read"; -export { buildMarkdownText } from "./markdown/write"; +export type { BuildMarkdownTextOptions } from "./markdown/write"; +// pageNumbers labels each page of the written markdown with pageNumberMarker; the pdf conversions supply it for a page-selected read so the text says which original pages it holds, and a caller composing readPdf with a page selection, reconstructWordprocessing and this writer (or renderContentDocumentToMarkdown) supplies it the same way, from sourcePageIndex(page, position) + 1 per page. +export { + buildMarkdownText, + MarkdownPageNumberCountError, + pageNumberMarker, +} from "./markdown/write"; // csv <-> ContentDocument — the same independently-usable pipeline stages the other adapter families above expose: decodeCsvText/encodeCsvText are the byte<->text boundary, with decodeCsvText working the character encoding out from the bytes (byte-codec's decodeText) rather than assuming UTF-8, so Excel's own windows-1252 export and a UTF-16 save both read, and encodeCsvText always writing UTF-8, readCsvContent parses RFC 4180 text into a one-sheet spreadsheet ContentDocument (first record as the header row, data cells heuristically re-typed by inferCellValue — the identical heuristic pdfToOds applies, exported standalone further above), buildCsvText writes a sheet back out emitting each cell's displayText. TSV is a delimiter option on these same stages ({ delimiter: '\t' }), not a separate format. The errors are exported beside them so a caller composing the stages directly can branch on them by class: CsvUndecodableTextError (bytes that are not text in any supported character encoding), CsvParseError (RFC 4180 violations — an unterminated quoted field, a multi-character delimiter), CsvUnsupportedDocumentKindError (a non-spreadsheet ContentDocument), and the multi-sheet selection pair CsvSheetNotSpecifiedError/CsvSheetNotFoundError — the identical contract odbToCsv's own table selection follows. export { decodeCsvText, encodeCsvText } from "./csv/text"; export { CsvUndecodableTextError } from "./csv/text"; diff --git a/packages/documents.js/src/layout/reconstruct-presentations-prod.ts b/packages/documents.js/src/layout/reconstruct-presentations-prod.ts index 4a2d8c1fa9..6edf020b32 100644 --- a/packages/documents.js/src/layout/reconstruct-presentations-prod.ts +++ b/packages/documents.js/src/layout/reconstruct-presentations-prod.ts @@ -28,6 +28,7 @@ import type { LayoutRect, LayoutSubpath, } from "pdf-codec"; +import { sourcePageIndex } from "pdf-codec/layout"; import { flipY } from "../model/geometry"; import { buildDrawingBlock } from "../model/embedded-drawing"; import { stampFrame } from "./shared"; @@ -64,9 +65,14 @@ export function reconstructPresentation( ): ContentDocument { const signal = options?.signal; const structure = indexStructure(doc); - const slides = doc.pages.map((page, pageIndex) => { + const slides = doc.pages.map((page, position) => { throwIfAborted(signal); - return reconstructSlide(page, pageIndex, doc.images, structure); + return reconstructSlide( + page, + sourcePageIndex(page, position), + doc.images, + structure, + ); }); return { kind: "presentation", metadata: doc.metadata, slides }; } @@ -320,9 +326,13 @@ export function reconstructDrawing( options?: ReconstructOptions, ): ContentDocument { const signal = options?.signal; - const pages: ContentDrawPage[] = doc.pages.map((page, pageIndex) => { + const pages: ContentDrawPage[] = doc.pages.map((page, position) => { throwIfAborted(signal); - return reconstructDrawPage(page, pageIndex, doc.images); + return reconstructDrawPage( + page, + sourcePageIndex(page, position), + doc.images, + ); }); return { kind: "drawing", metadata: doc.metadata, pages }; } diff --git a/packages/documents.js/src/layout/reconstruct.ts b/packages/documents.js/src/layout/reconstruct.ts index 6e26a0c0d6..e9fc81e03e 100644 --- a/packages/documents.js/src/layout/reconstruct.ts +++ b/packages/documents.js/src/layout/reconstruct.ts @@ -35,6 +35,7 @@ import type { LayoutText, LayoutDocument, } from "pdf-codec"; +import { sourcePageIndex } from "pdf-codec/layout"; import type { Margins } from "document-schema.js"; import { throwIfAborted } from "../ports/abort"; import { stampFrame } from "./shared"; @@ -76,21 +77,21 @@ export function reconstructWordprocessing( const structure = indexStructure(doc); const sections: ContentSection[] = []; let currentGroup: LayoutPage[] = []; - let groupStartPageIndex = 0; + let groupStartPosition = 0; for (const page of doc.pages) { throwIfAborted(signal); if (currentGroup.length > 0 && !samePageSize(currentGroup[0]!, page)) { sections.push( buildSection( currentGroup, - groupStartPageIndex, + groupStartPosition, doc.images, headingLevels, doc.form, structure, ), ); - groupStartPageIndex += currentGroup.length; + groupStartPosition += currentGroup.length; currentGroup = []; } currentGroup.push(page); @@ -99,7 +100,7 @@ export function reconstructWordprocessing( sections.push( buildSection( currentGroup, - groupStartPageIndex, + groupStartPosition, doc.images, headingLevels, doc.form, @@ -323,10 +324,10 @@ function samePageSize(a: LayoutPage, b: LayoutPage): boolean { } // Margins have no PDF equivalent to recover — there is no principled way to distinguish "intentional margin" from "wherever the content happened to start" from geometry alone, so this deliberately reports zero rather than fabricating a plausible-looking value (ZERO_MARGINS, defined above). -// startPageIndex is this section's own first page's absolute index in the whole LayoutDocument — a frame's pageIndex names a page of the SOURCE document, not a page within one section, so every block this section builds stamps absolute indices derived from it. +// startPosition is this section's own first page's position in the whole LayoutDocument's pages. A frame's pageIndex names a page of the SOURCE document, not a page within one section, so every block this section builds stamps the source index sourcePageIndex resolves from the page and its position: the position itself on a full read, the page's own index on a page-selected one, where it is also the index every form widget and destination already names. function buildSection( pages: readonly LayoutPage[], - startPageIndex: number, + startPosition: number, images: Record, headingLevels: ReadonlyMap, form: readonly LayoutFormField[] | undefined, @@ -342,7 +343,7 @@ function buildSection( blocks.push( ...reconstructPageBlocks( page, - startPageIndex + i, + sourcePageIndex(page, startPosition + i), images, headingLevels, form, @@ -731,9 +732,13 @@ export function reconstructSpreadsheet( options?: ReconstructOptions, ): ContentDocument { const signal = options?.signal; - const sheets: ContentSheet[] = doc.pages.map((page, index) => { + const sheets: ContentSheet[] = doc.pages.map((page, position) => { throwIfAborted(signal); - return reconstructSheet(page, index, options?.onCellTypeInference); + return reconstructSheet( + page, + sourcePageIndex(page, position), + options?.onCellTypeInference, + ); }); return { kind: "spreadsheet", metadata: doc.metadata, sheets }; } diff --git a/packages/documents.js/src/markdown/render.ts b/packages/documents.js/src/markdown/render.ts index d1152e5601..1be3d4c7d0 100644 --- a/packages/documents.js/src/markdown/render.ts +++ b/packages/documents.js/src/markdown/render.ts @@ -13,9 +13,9 @@ import type { LayoutMetadata, } from "document-schema.js"; import { PAGE_SIZE_A4 } from "document-schema.js"; -import type { WriteMarkdownOptions } from "markdown-codec"; import { headingStyleId } from "markdown-codec"; import { formulaPlaceholderText } from "../model/formula"; +import type { BuildMarkdownTextOptions } from "./write"; import { buildMarkdownText } from "./write"; // documents.js has plenty of ways to get INTO a ContentDocument from any of its eight content formats, and one way out to real bytes per format via buildXPackage — but no way to get plain Markdown text out of an arbitrary ContentDocument, regardless of which of the five kinds it happens to be. buildMarkdownText (src/markdown/write.ts) already covers the one kind markdown-codec itself understands ('wordprocessing'); this module is what makes every OTHER kind reachable too, by flattening its own shape-specific structure (slides, sheets, drawing pages, a bare formula) into the same ContentBlock vocabulary markdown-codec's writeMarkdownContent already knows how to render, then delegating to buildMarkdownText. @@ -56,7 +56,8 @@ const NOOP_RENDER_DIAGNOSTIC_SINK: MarkdownRenderDiagnosticSink = () => { // Deliberately empty — the default when a caller supplies no onDiagnostic, mirroring markdown-codec's own NOOP_MARKDOWN_DIAGNOSTIC_SINK. }; -export interface RenderMarkdownOptions extends WriteMarkdownOptions { +// pageNumbers (inherited from BuildMarkdownTextOptions) is meant for a wordprocessing document, whose pages it labels. Any other kind is flattened into one section with no page breaks before it is written, so it counts as a single page: one number labels the whole output, and more than one throws MarkdownPageNumberCountError. +export interface RenderMarkdownOptions extends BuildMarkdownTextOptions { // Called once per degrade decision this module makes while flattening a non-wordprocessing ContentDocument. Never called for a 'wordprocessing' document — there is nothing to degrade on that path, since it goes straight to writeMarkdownContent. readonly onDiagnostic?: MarkdownRenderDiagnosticSink; } diff --git a/packages/documents.js/src/markdown/write.ts b/packages/documents.js/src/markdown/write.ts index 9841246f44..641c42726d 100644 --- a/packages/documents.js/src/markdown/write.ts +++ b/packages/documents.js/src/markdown/write.ts @@ -24,6 +24,84 @@ const MATH_INLINE_SOURCE = "markdown:math-inline"; // The read-side inverse lives in src/markdown/read.ts and turns this exact literal back into a pageBreak block, so the marker MEANS a page break rather than decorating one: markdownToPdf re-renders a real page boundary, and a pdfToMarkdown -> markdownToPdf round trip regenerates markers from real boundaries instead of accumulating the previous round's markers as visible literal text. export const PAGE_BREAK_MARKER = ""; +/** + * The marker opening a page that `BuildMarkdownTextOptions.pageNumbers` labels: an HTML comment naming the page's number, `` for page 7, on a line of its own. An output-only annotation for readers of the text rather than a directive: unlike {@link PAGE_BREAK_MARKER} it states which page follows, not that a page ends, and nothing reads it back as one. Reading markdown that carries it keeps it as an ordinary HTML-comment paragraph, exactly as any other comment in a markdown source is kept, so a hand-written comment of the same shape is never mistaken for a label and lost. + * @param pageNumber - the number the page is labelled with, as the caller supplied it + * @returns the marker text + */ +export function pageNumberMarker(pageNumber: number): string { + return ``; +} + +/** + * `BuildMarkdownTextOptions.pageNumbers` did not supply exactly one number per page of the document. Thrown rather than labelling a prefix or padding, since a label on the wrong page is worse than none. + */ +export class MarkdownPageNumberCountError extends Error { + readonly pageCount: number; + readonly pageNumberCount: number; + + constructor(pageCount: number, pageNumberCount: number) { + super( + `pageNumbers supplies ${String(pageNumberCount)} page numbers for a document of ${String(pageCount)} pages; it must supply exactly one per page`, + ); + this.name = "MarkdownPageNumberCountError"; + this.pageCount = pageCount; + this.pageNumberCount = pageNumberCount; + } +} + +export interface BuildMarkdownTextOptions extends WriteMarkdownOptions { + /** + * One number per page of the document, in page order, written as {@link pageNumberMarker} at the start of each page so a reader of the text can tell which page each part came from. A page starts at the beginning of the document, after every top-level page break, and at the start of every section after the first that does not continue its predecessor's page (`breakType` other than `continuous`). Omitted, no page is labelled. The pdf conversions supply it for a page-selected read (`pages`), with each page's original one-based page number, since there the position of a part no longer says which page it is. + * @throws MarkdownPageNumberCountError when the count differs from the document's page count + */ + readonly pageNumbers?: readonly number[]; +} + +function pageNumberParagraph(pageNumber: number): ContentBlock { + return { + kind: "paragraph", + runs: [{ text: pageNumberMarker(pageNumber) }], + styleId: HTML_PREFORMATTED_STYLE_ID, + }; +} + +type WordprocessingSection = Extract< + ContentDocument, + { kind: "wordprocessing" } +>["sections"][number]; + +// Inserts pageNumberMarker at the start of every page, by the page-start rule BuildMarkdownTextOptions.pageNumbers states. Runs on the content before markdownBlock lowers page breaks, so a page break is still a pageBreak block here. +function labelPages( + sections: readonly WordprocessingSection[], + pageNumbers: readonly number[], +): WordprocessingSection[] { + let pageCount = 0; + // A page beyond the supplied numbers is counted but left unlabelled; the count check below then refuses the whole call, so no partially labelled text is ever returned. + const label = (): ContentBlock[] => { + const pageNumber = pageNumbers[pageCount]; + pageCount++; + return pageNumber === undefined ? [] : [pageNumberParagraph(pageNumber)]; + }; + const labelled = sections.map((section, sectionIndex) => { + const blocks: ContentBlock[] = []; + if (sectionIndex === 0 || section.breakType !== "continuous") { + blocks.push(...label()); + } + for (const block of section.blocks) { + blocks.push(block); + if (block.kind === "pageBreak") { + blocks.push(...label()); + } + } + return { ...section, blocks }; + }); + if (pageCount !== pageNumbers.length) { + throw new MarkdownPageNumberCountError(pageCount, pageNumbers.length); + } + return labelled; +} + function formulaParagraph( formula: NonNullable>, ): ContentBlock { @@ -89,12 +167,17 @@ function markdownTableCell(cell: ContentTableCell): ContentTableCell { // writeMarkdownContent throws MarkdownUnsupportedDocumentKindError on its own for a non-'wordprocessing' input — this adapter's own guard mirrors that exact check, now for a genuinely different reason than it used to: TypeScript needs `document` narrowed to the wordprocessing variant before this function can map over its own `sections` field at all (a presentation/spreadsheet/drawing/formula-kind ContentDocument has no `sections` field), not to work around any version-skew mismatch. export function buildMarkdownText( document: ContentDocument, - options?: WriteMarkdownOptions, + options?: BuildMarkdownTextOptions, ): string { if (document.kind !== "wordprocessing") { throw new MarkdownUnsupportedDocumentKindError(document.kind); } - const sections = document.sections.map((section) => ({ + const pageNumbers = options?.pageNumbers; + const labelled = + pageNumbers === undefined + ? document.sections + : labelPages(document.sections, pageNumbers); + const sections = labelled.map((section) => ({ ...section, blocks: section.blocks.map(markdownBlock), })); From ff9a8a45db0216247ae5e11dfa53524531391d83 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:35:21 +0100 Subject: [PATCH 4/9] docs(pdf-codec): state what a page-selected read still reads whole The /ParentTree number tree joins the structure element tree in the list of what a selected read still reads whole, and the README tells a caller to check pageCount before taking an absent attachments list to mean the document has none. The LAYOUT_FORMAT_VERSION comment records why index and pageCount do not bump the version. Tests cover a selected non-first page keeping its structure ownership and the page-boxes residue holding only the selected pages' rows. Refs #1688 --- packages/pdf-codec/README.md | 2 +- packages/pdf-codec/src/layout.ts | 2 +- .../pdf-codec/src/read-page-selection.test.ts | 31 ++++++++++++++++++- packages/pdf-codec/src/read.ts | 2 +- packages/pdf-codec/src/test-support/pdf.ts | 9 ++++-- 5 files changed, 40 insertions(+), 6 deletions(-) diff --git a/packages/pdf-codec/README.md b/packages/pdf-codec/README.md index d75595096b..1a8f588342 100644 --- a/packages/pdf-codec/README.md +++ b/packages/pdf-codec/README.md @@ -93,7 +93,7 @@ An encrypted PDF that opens without a password decrypts transparently — no ext Both accept an optional `signal` (`AbortSignal`); `readPdf` additionally takes a `sink` (`PdfDiagnosticSink`, called once per recoverable parse diagnostic — see the three-tier failure policy under [Conventions](#conventions)), and `writePdf` an `onSubstitution` callback (called once per character not representable in a standard-14 font — see [Fidelity](#fidelity)). -**Reading a subset of pages.** `readPdf(bytes, { pages: [0, 1, 2, -1] })` reads only the listed pages: zero-based indices into the page tree, a negative index counted from the end (`-1` is the last page). What a selected read skips: an unselected page's content stream is never fetched, decoded or interpreted, so the fonts and images only unselected pages use are never resolved, and no embedded file is decoded, so `attachments` is absent from a page-selected result (an attachment is document-level payload whose size is independent of the pages asked for). The case this exists for is identifying a document from its first pages, or reading a code from its last page, inside a memory-capped runtime such as a Cloudflare Worker isolate, without paying for every page's content. What it still reads whole, because page references resolve against it or because it is document-level and holds no page content stream: the page tree, the outline, named destinations, optional-content layers, the AcroForm, the structure element tree (whose size scales with the document), metadata and package residue. Two surfaces are built from page content and so cover selected pages only: the structure-ownership association an item's `structure` field names (with its parent-tree diagnostics) and the `page-boxes` residue row. A destination a link states as a direct array is minted into `destinations` (as `dest1`, `dest2`, ...) only when its page is interpreted, so an unselected page's direct link destinations are absent and a minted name depends on which pages were read; named destinations are unaffected. Duplicates collapse and the result's `pages` holds the selected pages in ascending document order, each carrying its source position as `index` (read it through `sourcePageIndex(page, position)`, which falls back to the position on a full read), with the page tree's length as the document's `pageCount`; every document-level page reference keeps naming source indices, so a reference to a selected page resolves through `index`, and one to an unselected page names a page the result does not hold. A full read sets neither field, because there `pages.length` and each page's position already state them. An empty selection is valid and reads no page, which makes `{ pages: [] }` the cheapest way to learn a file's page count and metadata. An entry naming no page (outside `-pageCount` to `pageCount - 1`, or not an integer, `NaN` and the infinities included; `-0` is page 0) throws `PdfPageSelectionError` (a `PdfParseError` with code `pdf/page-out-of-range` or `pdf/page-index-not-integer`, carrying `pageIndex` and `pageCount`) before any page is read. A page-selected `LayoutDocument` is a reading of part of a file, not a document to re-emit: `writePdf` refuses one (any document carrying `pageCount`) with `PdfPartialDocumentError`, since it places pages by position and the references would point at the wrong pages. +**Reading a subset of pages.** `readPdf(bytes, { pages: [0, 1, 2, -1] })` reads only the listed pages: zero-based indices into the page tree, a negative index counted from the end (`-1` is the last page). What a selected read skips: an unselected page's content stream is never fetched, decoded or interpreted, so the fonts and images only unselected pages use are never resolved, and no embedded file is decoded, so `attachments` is absent from a page-selected result (an attachment is document-level payload whose size is independent of the pages asked for); an absent `attachments` therefore means "not read" whenever `pageCount` is set, which only a page-selected read does, so check `pageCount` before concluding a document has no attachments. The case this exists for is identifying a document from its first pages, or reading a code from its last page, inside a memory-capped runtime such as a Cloudflare Worker isolate, without paying for every page's content. What it still reads whole, because page references resolve against it or because it is document-level and holds no page content stream: the page tree, the outline, named destinations, optional-content layers, the AcroForm, the structure element tree and the `/ParentTree` number tree (both of which scale with the document), metadata and package residue. Two surfaces are built from page content and so cover selected pages only: the structure-ownership association an item's `structure` field names (with its parent-tree diagnostics) and the `page-boxes` residue row. A destination a link states as a direct array is minted into `destinations` (as `dest1`, `dest2`, ...) only when its page is interpreted, so an unselected page's direct link destinations are absent and a minted name depends on which pages were read; named destinations are unaffected. Duplicates collapse and the result's `pages` holds the selected pages in ascending document order, each carrying its source position as `index` (read it through `sourcePageIndex(page, position)`, which falls back to the position on a full read), with the page tree's length as the document's `pageCount`; every document-level page reference keeps naming source indices, so a reference to a selected page resolves through `index`, and one to an unselected page names a page the result does not hold. A full read sets neither field, because there `pages.length` and each page's position already state them. An empty selection is valid and reads no page, which makes `{ pages: [] }` the cheapest way to learn a file's page count and metadata. An entry naming no page (outside `-pageCount` to `pageCount - 1`, or not an integer, `NaN` and the infinities included; `-0` is page 0) throws `PdfPageSelectionError` (a `PdfParseError` with code `pdf/page-out-of-range` or `pdf/page-index-not-integer`, carrying `pageIndex` and `pageCount`) before any page is read. A page-selected `LayoutDocument` is a reading of part of a file, not a document to re-emit: `writePdf` refuses one (any document carrying `pageCount`) with `PdfPartialDocumentError`, since it places pages by position and the references would point at the wrong pages. **Page boundaries: the crop box is the visible region** (ISO 32000-1 14.11.2). A page's reported `widthPt`/`heightPt` and coordinate frame come from its effective `/CropBox` — the rectangle a viewer displays and prints, inherited through the page tree like `/MediaBox` and defaulting to it — not from the media box: content an author placed outside the crop box (printer's marks, bleed, off-page slugs) is not visible and does not extract. Content wholly outside the crop box is dropped; content straddling the boundary keeps its original unclipped geometry (the item layer records what the file states — a viewer's clipping is a rendering fact, not source data); link annotations are anchored constructs rather than painted content and are never filtered, the same line the optional-content filter draws. The declared boundary rectangles a distinct crop box hides — plus `/BleedBox`/`/TrimBox`/`/ArtBox`, print-production facts with no field in the layout model — are quarantined verbatim as the `page-boxes` package-level residue row rather than silently dropped. A degenerate `/CropBox` (zero width or height) falls back to the media box with a `pdf/invalid-crop-box` warning. diff --git a/packages/pdf-codec/src/layout.ts b/packages/pdf-codec/src/layout.ts index 7b579500a0..032cf2eb68 100644 --- a/packages/pdf-codec/src/layout.ts +++ b/packages/pdf-codec/src/layout.ts @@ -9,7 +9,7 @@ import { // pdf-codec's own native document model: LayoutDocument, the positioned item layer readPdf assembles from a PDF's bytes and writePdf draws into new ones. Ported verbatim from document-schema.js's own src/layout.ts (its home from the content pivot until that package's 4.0.0 promoted DocumentTree and dropped it) because a codec's native model belongs in the codec — the same family pattern as ooxml.js's Package/XmlElement and markdown-codec's AST; only PDF's native model was ever a public shared-schema export, an accident of this package predating the content pivot. The item layer remains the honest boundary between what the format says (positions) and what we think it means (structure): when documents.js's reconstruction misjudges a wrapped paragraph, these items stay inspectable as the PDF's actual testimony. Reconstruction heuristics are semantic policy and stay in documents.js, not here. The shared leaf shapes the family composes from (Color, ContentStrokeStyleSchema, LayoutFont, LayoutMetadata) stay in document-schema.js and are imported above, so content and layout keep one definition of each. -// Bumped whenever LayoutDocumentSchema's shape changes incompatibly, so a value serialized by one version of a consumer can be recognised (and rejected, rather than silently misread) by another. Still 1: moving the schemas between packages changed where the family lives, not what it accepts. +// Bumped whenever LayoutDocumentSchema's shape changes incompatibly, so a value serialized by one version of a consumer can be recognised (and rejected, rather than silently misread) by another. Still 1: moving the schemas between packages changed where the family lives, not what it accepts. Additive optional fields do not bump it either, following every field added since (structure, layers, attachments, writingMode among them); that holds for LayoutPage.index and LayoutDocument.pageCount too, which only a page-selected read writes, because the package has no serialised form of a page-selected layout and writePdf refuses one, so an older consumer never receives one from this package with those fields stripped. export const LAYOUT_FORMAT_VERSION = 1; // sourcePath is assigned by each format's reader at read time and copied onto emitted LayoutItems by the layout engine; this module only defines the field, it doesn't generate values. Known limitation: sourcePath values are stable within one read+layout pass over a single document, not across edits — inserting content earlier in a document shifts every later path. This is not a stable identity scheme for incremental re-layout; it exists for tagged/accessible-PDF-style traceability and debugging, not edit-tracking. diff --git a/packages/pdf-codec/src/read-page-selection.test.ts b/packages/pdf-codec/src/read-page-selection.test.ts index fb5b91812e..0dbc3976fd 100644 --- a/packages/pdf-codec/src/read-page-selection.test.ts +++ b/packages/pdf-codec/src/read-page-selection.test.ts @@ -14,11 +14,15 @@ import { PAGE_SELECTION_IMAGE_PAGE_INDEX, PAGE_SELECTION_STREAM, PAGE_SELECTION_TRAP_PAGE_INDEX, + PAGE_SELECTION_TRIM_BOX_PAGE_INDICES, navigationClusterPdf, pagelessPdf, pageSelectionPdf, } from "./test-support/pdf"; -import { parentTreeMissingEntryPdf } from "./test-support/pdf-structures"; +import { + parentTreeMissingEntryPdf, + taggedStructurePdf, +} from "./test-support/pdf-structures"; // The interpreter entry readPage calls, wrapped so a test can count how many content streams a read interpreted. The real implementation still runs: this observes, it does not stub. vi.mock("./interpret", async (importOriginal) => { @@ -151,6 +155,31 @@ describe("readPdf: page selection never touches an unselected page's content", ( ); }); + it("still associates a selected non-first page's marked content with its owning structure element, exactly as a full read does", () => { + const full = readPdf(taggedStructurePdf()); + const secondPage = full.pages[1]; + const owned = secondPage?.items.filter( + (item) => "structure" in item && item.structure !== undefined, + ); + expect(owned?.length).toBeGreaterThan(0); + expect(readPdf(taggedStructurePdf(), { pages: [1] }).pages).toEqual([ + { ...secondPage, index: 1 }, + ]); + }); + + it("keeps only the selected pages' rows in the page-boxes residue", () => { + const rowsOf = (pages?: readonly number[]): string[] => + [ + ...( + readPdf(pageSelectionPdf(), { pages }).source?.["page-boxes"]?.xml ?? + "" + ).matchAll(/\/Page (\d+)/gu), + ].map((match) => match[1] ?? ""); + expect(rowsOf()).toEqual(PAGE_SELECTION_TRIM_BOX_PAGE_INDICES.map(String)); + expect(rowsOf([LAST_PAGE_INDEX])).toEqual([String(LAST_PAGE_INDEX)]); + expect(rowsOf([1, 2])).toEqual([]); + }); + it("never resolves an image only an unselected page draws", () => { expect(Object.keys(readPdf(pageSelectionPdf()).images)).toHaveLength(1); expect(readPdf(pageSelectionPdf(), { pages: [0, -1] }).images).toEqual({}); diff --git a/packages/pdf-codec/src/read.ts b/packages/pdf-codec/src/read.ts index db11dc281b..d1ab1bfcaa 100644 --- a/packages/pdf-codec/src/read.ts +++ b/packages/pdf-codec/src/read.ts @@ -66,7 +66,7 @@ export interface ReadPdfOptions { * * What a selected read skips: an unselected page's content stream is never fetched, decoded or interpreted, so the fonts and images only unselected pages use are never resolved; embedded-file payloads are never decoded, so `attachments` is absent from a page-selected result (an attachment is document-level payload whose size is independent of the pages asked for, and decoding it would cost the same whichever pages were selected). * - * What it still reads whole, because a negative index and every document-level page reference resolve against it or because it is document-level and holds no page content stream: the page tree, the outline, named destinations, optional-content layers, the AcroForm, the structure element tree (whose size scales with the document), metadata and package residue. Two surfaces are built from page content and so cover selected pages only: the structure-ownership association an item's `structure` field names (and its parent-tree diagnostics), and the `page-boxes` residue row. Destinations a link states as a direct array are minted into `destinations` (as `dest1`, `dest2`, ...) only when their page is interpreted, so an unselected page's direct link destinations are absent and a minted name depends on which pages were read; named destinations are unaffected. + * What it still reads whole, because a negative index and every document-level page reference resolve against it or because it is document-level and holds no page content stream: the page tree, the outline, named destinations, optional-content layers, the AcroForm, the structure element tree and the /ParentTree number tree (both of which scale with the document), metadata and package residue. Two surfaces are built from page content and so cover selected pages only: the structure-ownership association an item's `structure` field names (and its parent-tree diagnostics), and the `page-boxes` residue row. Destinations a link states as a direct array are minted into `destinations` (as `dest1`, `dest2`, ...) only when their page is interpreted, so an unselected page's direct link destinations are absent and a minted name depends on which pages were read; named destinations are unaffected. * * Duplicates collapse, and the result's `pages` holds the selected pages in ascending document order whatever order they were asked for in, each carrying its source position as `index`, with the document's total as `pageCount`; every document-level reference keeps naming source indices. An empty array is a valid selection: it reads no page and returns the document-level surfaces with `pageCount`, the cheapest way to learn a file's length and metadata. An entry naming no page throws `PdfPageSelectionError` rather than being skipped. */ diff --git a/packages/pdf-codec/src/test-support/pdf.ts b/packages/pdf-codec/src/test-support/pdf.ts index 1e72abea8d..34d110569b 100644 --- a/packages/pdf-codec/src/test-support/pdf.ts +++ b/packages/pdf-codec/src/test-support/pdf.ts @@ -548,7 +548,7 @@ export function navigationClusterPdf(): Uint8Array { return b.bytes(); } -// A four-page document built to prove a page-selected read never touches what only unselected pages, or the document's embedded files, hold. Page 1 (index 1) is the trap: its content stream declares a filter no reader implements, so decoding it reports pdf/unsupported-filter (the bytes then pass through undecoded, which here are plain content), and it is the only page using font F2, an embedded TrueType whose font program is its own stream. Page 2 (index 2) is the only page drawing the image XObject, so the image registry is non-empty exactly when that page's content was interpreted. The catalog carries one embedded file. Every stream a test needs to recognise carries a /FixtureStream name (PAGE_SELECTION_STREAM), a key no reader interprets, so a spy on the stream decoder can tell which of them a read decoded. Every page shows its own one-based page number as text ("Page 1" to "Page 4"), so a test can tell from the text alone which pages it got. +// A four-page document built to prove a page-selected read never touches what only unselected pages, or the document's embedded files, hold. Page 1 (index 1) is the trap: its content stream declares a filter no reader implements, so decoding it reports pdf/unsupported-filter (the bytes then pass through undecoded, which here are plain content), and it is the only page using font F2, an embedded TrueType whose font program is its own stream. Page 2 (index 2) is the only page drawing the image XObject, so the image registry is non-empty exactly when that page's content was interpreted. The catalog carries one embedded file, and pages 0 and 3 declare a /TrimBox, so each contributes a page-boxes residue row naming its page. Every stream a test needs to recognise carries a /FixtureStream name (PAGE_SELECTION_STREAM), a key no reader interprets, so a spy on the stream decoder can tell which of them a read decoded. Every page shows its own one-based page number as text ("Page 1" to "Page 4"), so a test can tell from the text alone which pages it got. export const PAGE_SELECTION_TRAP_PAGE_INDEX = 1; export const PAGE_SELECTION_IMAGE_PAGE_INDEX = 2; export const PAGE_SELECTION_STREAM = { @@ -557,6 +557,8 @@ export const PAGE_SELECTION_STREAM = { attachment: "Attachment", } as const; export const PAGE_SELECTION_ATTACHMENT_NAME = "notes.txt"; +// The pages declaring a /TrimBox, each therefore contributing one row to the page-boxes residue. +export const PAGE_SELECTION_TRIM_BOX_PAGE_INDICES: readonly number[] = [0, 3]; export function pageSelectionPdf(): Uint8Array { const b = new FixtureBuilder().header(PDF_1_4); const PAGE_OBJS = [3, 4, 5, 6] as const; @@ -585,9 +587,12 @@ export function pageSelectionPdf(): Uint8Array { index === PAGE_SELECTION_TRAP_PAGE_INDEX ? `/F2 ${TRAP_FONT_OBJ} 0 R` : `/F1 ${SHARED_FONT_OBJ} 0 R`; + const boxes = PAGE_SELECTION_TRIM_BOX_PAGE_INDICES.includes(index) + ? " /TrimBox [10 10 190 90]" + : ""; b.object( pageObj, - `<< /Type /Page /Parent 2 0 R /Resources << /Font << ${fonts} >>${xObjects} >> /Contents ${CONTENT_OBJS[index]} 0 R >>`, + `<< /Type /Page /Parent 2 0 R /Resources << /Font << ${fonts} >>${xObjects} >> /Contents ${CONTENT_OBJS[index]} 0 R${boxes} >>`, ); }); b.object(SHARED_FONT_OBJ, HELVETICA_FONT_DICT); From 5591aede12a131963804a5a5c5b195767e094d01 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:35:27 +0100 Subject: [PATCH 5/9] fix(documents.js): name the page-selected case when a tree cannot become pdf buildDocumentBytes(tree, "pdf") said every pdf-to-format dump carries pages, which a page-selected conversion's dump does not. The message and the README now say so, and readDocumentMetadata's comment states what an empty selection skips rather than implying a catalog-only read. Refs #1688 --- packages/documents.js/README.md | 4 ++-- .../documents.js/src/convert/from-package.ts | 2 +- .../src/convert/from-pdf-pages.test.ts | 24 ++++++++++++++++++- packages/documents.js/src/convert/from-pdf.ts | 2 +- 4 files changed, 27 insertions(+), 5 deletions(-) diff --git a/packages/documents.js/README.md b/packages/documents.js/README.md index 890a80f0f5..18e142a937 100644 --- a/packages/documents.js/README.md +++ b/packages/documents.js/README.md @@ -395,13 +395,13 @@ import { pdfToMarkdown } from "documents.js/read"; const coverAndLastPage = pdfToMarkdown(pdfBytes, { pages: [0, 1, 2, -1] }); ``` -What a selection skips: the unselected pages' content streams are never fetched, decoded or interpreted, the fonts and images only those pages use are never resolved, and the source's embedded files are never decoded (see pdf-codec's README for exactly what is and is not read). That is what lets a caller identify a document from its first pages, or read a code from its last page, inside a memory-capped runtime such as a Cloudflare Worker isolate, without paying for every page's content. +What a selection skips: the unselected pages' content streams are never fetched, decoded or interpreted, the fonts and images only those pages use are never resolved, and the source's embedded files are never decoded, so the result carries no attachments; a caller holding the `LayoutDocument` checks `pageCount` (set only on a page-selected read) before concluding the document has none (see pdf-codec's README for exactly what is and is not read). That is what lets a caller identify a document from its first pages, or read a code from its last page, inside a memory-capped runtime such as a Cloudflare Worker isolate, without paying for every page's content. The output holds only the selected pages, and reconstruction sees only them, so a page can come out differently than in a full conversion. Heading detection measures text against the body size, taken as the most common text size across the pages read: a cover page read on its own, dominated by large title text, takes the title's size as its body, so the title becomes a paragraph rather than a heading. Reading the cover together with a few ordinary pages restores the document's real body size. Markdown output labels each selected page with its original one-based page number, written as `` on its own line at the start of the page (`pageNumberMarker`), alongside the `` marker between pages that share a section, so a caller can tell which pages it got without re-deriving the selection. The label is output-only: reading markdown back keeps it as an ordinary HTML-comment paragraph, exactly as any other comment in a markdown source is kept. A full conversion writes no labels, so its output is unchanged. A caller composing the stages itself (`readPdf` with `pages`, `reconstructWordprocessing`, then `buildMarkdownText` or `renderContentDocumentToMarkdown`) gets the same labels by passing `pageNumbers: layout.pages.map((page, position) => sourcePageIndex(page, position) + 1)`; `buildMarkdownText` throws `MarkdownPageNumberCountError` when the count does not match the document's pages, and a non-wordprocessing document flattened by `renderContentDocumentToMarkdown` counts as a single page. -Every reconstructed node's frames name the page's index in the source PDF (the same index every destination and form widget names), which is why a page-selected conversion's reported `DocumentTree` carries no `pages` envelope: that array is indexed by frame page index, and a selection cannot fill it. A later stage's own page-addressing option, such as `page` for an svg target, counts within the selection. A page-selected `LayoutDocument` cannot be written back: `writePdf` refuses one with `PdfPartialDocumentError`. `PdfPageSelectionError`, `PdfPartialDocumentError` and `sourcePageIndex` are exported from the root barrel; the `documents.js/read` entry re-exports nothing (this package allows re-exports only in its root barrel), so a caller of that entry recognises a bad selection by the thrown error's `code` (`pdf/page-out-of-range` or `pdf/page-index-not-integer`), and resolves a page's source index as `page.index ?? position`. +Every reconstructed node's frames name the page's index in the source PDF (the same index every destination and form widget names), which is why a page-selected conversion's reported `DocumentTree` carries no `pages` envelope: that array is indexed by frame page index, and a selection cannot fill it. Such a tree cannot be rebuilt to pdf either: `buildDocumentBytes(tree, "pdf")` needs that envelope and throws without it. A later stage's own page-addressing option, such as `page` for an svg target, counts within the selection. A page-selected `LayoutDocument` cannot be written back: `writePdf` refuses one with `PdfPartialDocumentError`. `PdfPageSelectionError`, `PdfPartialDocumentError` and `sourcePageIndex` are exported from the root barrel; the `documents.js/read` entry re-exports nothing (this package allows re-exports only in its root barrel), so a caller of that entry recognises a bad selection by the thrown error's `code` (`pdf/page-out-of-range` or `pdf/page-index-not-integer`), and resolves a page's source index as `page.index ?? position`. ### Reading and building xlsx content directly diff --git a/packages/documents.js/src/convert/from-package.ts b/packages/documents.js/src/convert/from-package.ts index e3769f2a47..21c65f9178 100644 --- a/packages/documents.js/src/convert/from-package.ts +++ b/packages/documents.js/src/convert/from-package.ts @@ -66,7 +66,7 @@ export function buildDocumentBytes( if (target === "pdf") { if (pkg.pages === undefined) { throw new Error( - "this DocumentTree has no pages — only a package dumped from a -to-pdf or pdf-to- conversion carries them; a bridge conversion's own dump (e.g. odt-to-docx) never does, so 'pdf' is not a reachable target from it", + "this DocumentTree has no pages, so 'pdf' is not a reachable target from it: only a package dumped from a -to-pdf conversion or a full pdf-to- conversion carries them. A bridge conversion's own dump (e.g. odt-to-docx) never does, and neither does a page-selected pdf-to- conversion's, whose frames name source page indices a pages array of only the selection could not be indexed by", ); } const { document: layout, formulas, fonts } = packageToLayout(pkg); diff --git a/packages/documents.js/src/convert/from-pdf-pages.test.ts b/packages/documents.js/src/convert/from-pdf-pages.test.ts index 9172153bc1..aa858df542 100644 --- a/packages/documents.js/src/convert/from-pdf-pages.test.ts +++ b/packages/documents.js/src/convert/from-pdf-pages.test.ts @@ -15,6 +15,8 @@ import { import { describe, expect, it } from "vitest"; import { decodePackage } from "odf.js"; import { createDocx } from "../edit/docx/editor"; +import { SvgMultiPageNotSpecifiedError } from "../svg/write"; +import { buildDocumentBytes } from "./from-package"; import { readOdsContent } from "../odf/ods/read"; import { createPdf, openPdf } from "../edit/pdf/editor"; import { fixedClock } from "../ports/clock"; @@ -38,6 +40,7 @@ import { } from "../markdown/write"; import { renderContentDocumentToMarkdown } from "../markdown/render"; import { + pdfToDocx, pdfToMarkdown, pdfToSvg, readContentDocument, @@ -169,6 +172,23 @@ describe("pdfToMarkdown with a page selection", () => { expect(frameIndicesOf(reported)).toEqual([LAST_PAGE_INDEX]); }); + it("reports a tree that cannot be rebuilt to pdf, with an error naming the page-selected case", () => { + let reported: DocumentTree | undefined; + pdfToDocx(fivePagePdf(), { + pages: [0], + onDocument: (tree) => { + reported = tree; + }, + }); + if (reported === undefined) { + throw new Error("expected onDocument to report a tree"); + } + const tree = reported; + expect(() => buildDocumentBytes(tree, "pdf")).toThrow( + /page-selected pdf-to- conversion/u, + ); + }); + it("throws PdfPageSelectionError for an index naming no page", () => { expect(() => markdownOf([PAGE_SIZES.length])).toThrow( PdfPageSelectionError, @@ -203,7 +223,9 @@ describe("pdfToSvg with a page selection", () => { const WIDE_PAGE_INDEX = PAGE_SIZES.indexOf(WIDE); it("needs no page option once the selection holds a single page, and writes that page", () => { - expect(() => pdfToSvg(fivePagePdf())).toThrow(); + expect(() => pdfToSvg(fivePagePdf())).toThrow( + SvgMultiPageNotSpecifiedError, + ); const svg = new TextDecoder().decode( pdfToSvg(fivePagePdf(), { pages: [WIDE_PAGE_INDEX] }), ); diff --git a/packages/documents.js/src/convert/from-pdf.ts b/packages/documents.js/src/convert/from-pdf.ts index c044914e2c..ea1313697a 100644 --- a/packages/documents.js/src/convert/from-pdf.ts +++ b/packages/documents.js/src/convert/from-pdf.ts @@ -31,7 +31,7 @@ export function readDocumentMetadata( options?: ReadDocumentMetadataOptions, ): LayoutMetadata { if (format === "pdf") { - // Metadata is document-level, so no page is read: an empty selection skips every page's content stream (and the source's embedded files), which keeps identifying a document as cheap as its catalog and /Info allow. + // Metadata is document-level, so no page is read: an empty selection skips every page's content streams and the fonts and images they use, and the source's embedded files. It does not reduce the read to the catalog and /Info: the outline, destinations, AcroForm, structure tree and /ParentTree, and package residue are still read whole (see ReadPdfOptions.pages). return readPdf(bytes, { signal: options?.signal, pages: [] }).metadata; } return CONTENT_READERS[format](bytes, { signal: options?.signal }).metadata; From 3683a104ed2973c2ef1751a667df75a9c4e4d000 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:47:12 +0100 Subject: [PATCH 6/9] docs(pdf-codec): spell serialized consistently in the format version comment --- packages/pdf-codec/src/layout.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/pdf-codec/src/layout.ts b/packages/pdf-codec/src/layout.ts index 032cf2eb68..72f4be218f 100644 --- a/packages/pdf-codec/src/layout.ts +++ b/packages/pdf-codec/src/layout.ts @@ -9,7 +9,7 @@ import { // pdf-codec's own native document model: LayoutDocument, the positioned item layer readPdf assembles from a PDF's bytes and writePdf draws into new ones. Ported verbatim from document-schema.js's own src/layout.ts (its home from the content pivot until that package's 4.0.0 promoted DocumentTree and dropped it) because a codec's native model belongs in the codec — the same family pattern as ooxml.js's Package/XmlElement and markdown-codec's AST; only PDF's native model was ever a public shared-schema export, an accident of this package predating the content pivot. The item layer remains the honest boundary between what the format says (positions) and what we think it means (structure): when documents.js's reconstruction misjudges a wrapped paragraph, these items stay inspectable as the PDF's actual testimony. Reconstruction heuristics are semantic policy and stay in documents.js, not here. The shared leaf shapes the family composes from (Color, ContentStrokeStyleSchema, LayoutFont, LayoutMetadata) stay in document-schema.js and are imported above, so content and layout keep one definition of each. -// Bumped whenever LayoutDocumentSchema's shape changes incompatibly, so a value serialized by one version of a consumer can be recognised (and rejected, rather than silently misread) by another. Still 1: moving the schemas between packages changed where the family lives, not what it accepts. Additive optional fields do not bump it either, following every field added since (structure, layers, attachments, writingMode among them); that holds for LayoutPage.index and LayoutDocument.pageCount too, which only a page-selected read writes, because the package has no serialised form of a page-selected layout and writePdf refuses one, so an older consumer never receives one from this package with those fields stripped. +// Bumped whenever LayoutDocumentSchema's shape changes incompatibly, so a value serialized by one version of a consumer can be recognised (and rejected, rather than silently misread) by another. Still 1: moving the schemas between packages changed where the family lives, not what it accepts. Additive optional fields do not bump it either, following every field added since (structure, layers, attachments, writingMode among them); that holds for LayoutPage.index and LayoutDocument.pageCount too, which only a page-selected read writes, because the package has no serialized form of a page-selected layout and writePdf refuses one, so an older consumer never receives one from this package with those fields stripped. export const LAYOUT_FORMAT_VERSION = 1; // sourcePath is assigned by each format's reader at read time and copied onto emitted LayoutItems by the layout engine; this module only defines the field, it doesn't generate values. Known limitation: sourcePath values are stable within one read+layout pass over a single document, not across edits — inserting content earlier in a document shifts every later path. This is not a stable identity scheme for incremental re-layout; it exists for tagged/accessible-PDF-style traceability and debugging, not edit-tracking. From feae1ef67dfac0b1ced469ca1def0a7fcce46efd Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:47:15 +0100 Subject: [PATCH 7/9] docs(documents.js): say readDocumentMetadata reads pdf through readPdf readDocumentMetadata dispatches a content format through CONTENT_READERS and reads pdf with readPdf directly, not through readDocumentLayout or the registry. The codec module comments, the README and the abort-signal test name now say so. Refs #1688 --- packages/documents.js/README.md | 4 ++-- packages/documents.js/src/codecs/read.ts | 2 +- packages/documents.js/src/codecs/registry.ts | 2 +- packages/documents.js/src/convert/from-pdf.test.ts | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/packages/documents.js/README.md b/packages/documents.js/README.md index 18e142a937..83346e6a3a 100644 --- a/packages/documents.js/README.md +++ b/packages/documents.js/README.md @@ -383,7 +383,7 @@ One subpath is a declared entry point in its own right: **`documents.js/read`** import { pdfToMarkdown } from "documents.js/read"; ``` -It carries the ten `pdfTo*` conversions, `PdfToDocumentOptions`, `readDocumentMetadata` (with `ReadDocumentMetadataOptions`), `readNativeDocumentTree` (with `ReadNativeDocumentTreeOptions`), and `readContentDocument` (with `ReadContentDocumentOptions`) — identical functions to the root barrel's (the same forwarders, run through the composition engine's read half, and the same dispatch through the read-only codec half `src/codecs/read.ts`), never a forked behaviour; `convertDocument` and every X-to-PDF direction stay on the root barrel. `src/read-graph.test.ts` walks the entry's static import graph, follows `pdf-codec` specifiers through that package's real `exports` map into its source, and fails the build if the write path or any font asset becomes reachable. +It carries the ten `pdfTo*` conversions, `PdfToDocumentOptions`, `readDocumentMetadata` (with `ReadDocumentMetadataOptions`), `readNativeDocumentTree` (with `ReadNativeDocumentTreeOptions`), and `readContentDocument` (with `ReadContentDocumentOptions`) — identical functions to the root barrel's (the same forwarders, run through the composition engine's read half, and the same read-only dispatch: `src/codecs/read.ts`'s `CONTENT_READERS` for a content format, and pdf-codec's `readPdf` directly for pdf), never a forked behaviour; `convertDocument` and every X-to-PDF direction stay on the root barrel. `src/read-graph.test.ts` walks the entry's static import graph, follows `pdf-codec` specifiers through that package's real `exports` map into its source, and fails the build if the write path or any font asset becomes reachable. ### Converting a selection of a PDF's pages @@ -839,7 +839,7 @@ The package is layered from generic primitives outward to the two conversion dir - **`src/firebird/`** — Tier 3: gbak logical-backup reader. `reader.ts` (attribute framing + RLE decompression + XDR decoding), `schema.ts`/`data.ts` (table/row walking). No ratified spec — built against Firebird's own engine source. - **`src/odb/`** — decoder-selection and pivot-mapping: `read.ts` routes to the right tier, `spreadsheet.ts`/`csv.ts` map to output formats. `odb/sql/` is the bounded SQL engine, `odb/formula/` is the rpt formula engine, `odb/report/` is the renderer, `odb/values.ts` is shared comparison/aggregation semantics. - **`src/convert/`** — the composition layer: `convert.ts` (the named conversion functions and their option types), `composition.ts` (the pathfinder, the primitive registry, the bridge/fromPdf executors, and `convertDocumentFromPdf` — the read half of the engine), `composition-to-pdf.ts` (`executeToPdf`, the layout-engine registry, and the full `convertDocument` binding — the write half, split out so a read-only consumer never statically imports a renderer), `from-pdf.ts` (the `pdfTo*` family, `readDocumentMetadata`, `readNativeDocumentTree`, `readContentDocument`, and the `documents.js/read` entry), `codec.ts` (`z.codec()` pairs), `port.ts`/`local.ts` (the `DocumentConverter` port), `variant-bridges.ts` (cross-variant semantic transforms), and `from-package.ts` (`buildDocumentBytes`, which flattens once at the boundary). The tree ⇄ flat transform those construction sites call — `assembleTree`, `decompose`, `flattenTree`, `factorStyles` — is not implemented here: it lives in `document-schema.js`, which owns both encodings. `bijection.test.ts` is this package's own gate on it, re-running the three laws over the real corpus every reader, editor, and conversion here produces. -- **`src/codecs/`** — `DOCUMENT_FORMAT_CODECS`: every format's read/build capability as data, so `readDocumentMetadata`/`setDocumentMetadata`/`buildDocumentBytes` dispatch through one registry. +- **`src/codecs/`** — `DOCUMENT_FORMAT_CODECS`: every format's read/build capability as data, so `setDocumentMetadata`/`buildDocumentBytes` dispatch through one registry; its read halves live in `read.ts`, whose `CONTENT_READERS` `readDocumentMetadata` uses for content formats (it reads pdf through pdf-codec's `readPdf` directly, with no page selected). - **`src/metadata/`** — cross-format metadata read/write via `DOCUMENT_FORMAT_CODECS`. - **`src/package-codec.ts`** — `decodeDocumentPackage`/`encodeDocumentPackage`/`decodeOdbPackage`. diff --git a/packages/documents.js/src/codecs/read.ts b/packages/documents.js/src/codecs/read.ts index 866ef35f2f..137f2653ee 100644 --- a/packages/documents.js/src/codecs/read.ts +++ b/packages/documents.js/src/codecs/read.ts @@ -29,7 +29,7 @@ import { throwIfAborted } from "../ports/abort"; // The read-only half of DOCUMENT_FORMAT_CODECS (src/codecs/registry.ts), split from it by direction for the identical reason the composition engine split (src/convert/composition.ts versus composition-to-pdf.ts): the registry's write halves legitimately import writePdf from the pdf-codec ROOT barrel, whose own font-registry/math-font imports put every vendored font asset one import away — so any module that values the registry cannot sit on the read-only documents.js/read entry (src/convert/from-pdf.ts), whose graph src/read-graph.test.ts holds free of exactly those modules. Everything here is the same per-format read wiring the registry wraps, in a module a read-only consumer may safely reach: readPdf arrives from 'pdf-codec/read' (the read pipeline's own owning module) rather than the root barrel, and no buildXPackage/encode/write import appears. // -// readDocumentMetadata (defined on the read entry itself, src/convert/from-pdf.ts) dispatches through THIS module rather than the registry for that reason (#744), which is what lets metadata reads join the read entry without dragging the write path back into its graph. +// readDocumentMetadata (defined on the read entry itself, src/convert/from-pdf.ts) never goes through the registry for that reason (#744): a content format dispatches through this module's CONTENT_READERS, and pdf reads pdf-codec/read's readPdf directly with an empty page selection (readDocumentLayout below serves the registry, not metadata). That is what lets metadata reads join the read entry without dragging the write path back into its graph. // The one option every read here genuinely needs to know about — an AbortSignal — plus an optional MarkdownImageResolver consulted only by the markdown read (every other format ignores it). Identical to the shape the registry declares (moved here when the read halves were, so both modules state one options type rather than two that could drift); matches the parameterisation every registry consumer already threads through this exact call path, so a caller can forward its own signal straight through without a cast at either end. export interface DocumentCodecOptions { diff --git a/packages/documents.js/src/codecs/registry.ts b/packages/documents.js/src/codecs/registry.ts index cb43494d3e..67fe29e263 100644 --- a/packages/documents.js/src/codecs/registry.ts +++ b/packages/documents.js/src/codecs/registry.ts @@ -35,7 +35,7 @@ export interface LayoutEntryCodec { // Every DocumentFormat's own capability, expressed as data rather than as three independent switch statements re-deriving the same "given a format, which read/build function do I call" dispatch. A format's `content` entry wraps the identical readXContent/buildXPackage pair every ergonomic conversion in this package already uses for it (via decodeDocumentPackage/encodeDocumentPackage for the raw-package half); a format's `layout` entry wraps a LayoutDocument codec (pdf only, so far). // -// Direction split (#744): every `read` closure lives in src/codecs/read.ts, imported here as CONTENT_READERS/readDocumentLayout, so this module keeps only the write halves — and the pdf-codec ROOT barrel import (writePdf) those write halves genuinely need. That barrel's own font-registry/math-font imports put every vendored font asset one import away, which is exactly what the read-only documents.js/read entry (src/convert/from-pdf.ts) must never reach, so a read-only consumer (readDocumentMetadata, defined on the read entry) dispatches through src/codecs/read.ts directly rather than through this both-directions registry — the identical split the composition engine already made (src/convert/composition.ts versus composition-to-pdf.ts), and the one src/read-graph.test.ts holds by walking the read entry's graph. +// Direction split (#744): every `read` closure lives in src/codecs/read.ts, imported here as CONTENT_READERS/readDocumentLayout, so this module keeps only the write halves — and the pdf-codec ROOT barrel import (writePdf) those write halves genuinely need. That barrel's own font-registry/math-font imports put every vendored font asset one import away, which is exactly what the read-only documents.js/read entry (src/convert/from-pdf.ts) must never reach, so a read-only consumer (readDocumentMetadata, defined on the read entry) never goes through this both-directions registry: a content format dispatches through src/codecs/read.ts's CONTENT_READERS, and pdf reads pdf-codec/read's readPdf directly — the identical split the composition engine already made (src/convert/composition.ts versus composition-to-pdf.ts), and the one src/read-graph.test.ts holds by walking the read entry's graph. export interface DocumentFormatCodecs { readonly content?: ContentCodec; readonly layout?: LayoutEntryCodec; diff --git a/packages/documents.js/src/convert/from-pdf.test.ts b/packages/documents.js/src/convert/from-pdf.test.ts index 0cad396fc5..ed9a10d4ce 100644 --- a/packages/documents.js/src/convert/from-pdf.test.ts +++ b/packages/documents.js/src/convert/from-pdf.test.ts @@ -152,7 +152,7 @@ describe("readDocumentMetadata", () => { expect(metadata.producer).toBeUndefined(); }); - it("pdf: forwards the abort signal to readDocumentLayout, which checks it before parsing", () => { + it("pdf: forwards the abort signal to readPdf, which checks it before parsing", () => { const controller = new AbortController(); controller.abort(); const bytes = docxToPdf(minimalDocxBytes()); From 45226d4f7c7da70d2b63b0274d8d97bceb612801 Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:47:21 +0100 Subject: [PATCH 8/9] docs(document-operations): tell pdf_inspect results apart by formatVersion --- packages/document-operations/src/operations/pdf-inspect.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/packages/document-operations/src/operations/pdf-inspect.ts b/packages/document-operations/src/operations/pdf-inspect.ts index 30a4864c91..54b582b3dd 100644 --- a/packages/document-operations/src/operations/pdf-inspect.ts +++ b/packages/document-operations/src/operations/pdf-inspect.ts @@ -43,6 +43,7 @@ const PdfInspectInputSchema = z.object({ ), }); +// Both shapes carry pageCount (a LayoutDocument does on a page-selected read), so a caller telling them apart discriminates on formatVersion, which every LayoutDocument has and no summary does; pdf_inspect itself never page-selects. interface PdfInspectSummary { readonly pageCount: number; readonly pages: readonly { From 1ed8f86e10f7229464425dbfdae69bd77caee58b Mon Sep 17 00:00:00 2001 From: Joseph Mearman Date: Sat, 17 Oct 2026 03:53:42 +0100 Subject: [PATCH 9/9] docs(documents.js): correct the metadata architecture bullet and drop stale format counts The metadata directory holds only the write side. readDocumentMetadata lives with the pdf conversions and never goes through the codec registry. Comments no longer state how many content formats there are. --- packages/documents.js/README.md | 2 +- packages/documents.js/src/codecs/read.ts | 4 ++-- packages/documents.js/src/convert/from-pdf.ts | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/packages/documents.js/README.md b/packages/documents.js/README.md index 83346e6a3a..374a4bb36b 100644 --- a/packages/documents.js/README.md +++ b/packages/documents.js/README.md @@ -840,7 +840,7 @@ The package is layered from generic primitives outward to the two conversion dir - **`src/odb/`** — decoder-selection and pivot-mapping: `read.ts` routes to the right tier, `spreadsheet.ts`/`csv.ts` map to output formats. `odb/sql/` is the bounded SQL engine, `odb/formula/` is the rpt formula engine, `odb/report/` is the renderer, `odb/values.ts` is shared comparison/aggregation semantics. - **`src/convert/`** — the composition layer: `convert.ts` (the named conversion functions and their option types), `composition.ts` (the pathfinder, the primitive registry, the bridge/fromPdf executors, and `convertDocumentFromPdf` — the read half of the engine), `composition-to-pdf.ts` (`executeToPdf`, the layout-engine registry, and the full `convertDocument` binding — the write half, split out so a read-only consumer never statically imports a renderer), `from-pdf.ts` (the `pdfTo*` family, `readDocumentMetadata`, `readNativeDocumentTree`, `readContentDocument`, and the `documents.js/read` entry), `codec.ts` (`z.codec()` pairs), `port.ts`/`local.ts` (the `DocumentConverter` port), `variant-bridges.ts` (cross-variant semantic transforms), and `from-package.ts` (`buildDocumentBytes`, which flattens once at the boundary). The tree ⇄ flat transform those construction sites call — `assembleTree`, `decompose`, `flattenTree`, `factorStyles` — is not implemented here: it lives in `document-schema.js`, which owns both encodings. `bijection.test.ts` is this package's own gate on it, re-running the three laws over the real corpus every reader, editor, and conversion here produces. - **`src/codecs/`** — `DOCUMENT_FORMAT_CODECS`: every format's read/build capability as data, so `setDocumentMetadata`/`buildDocumentBytes` dispatch through one registry; its read halves live in `read.ts`, whose `CONTENT_READERS` `readDocumentMetadata` uses for content formats (it reads pdf through pdf-codec's `readPdf` directly, with no page selected). -- **`src/metadata/`** — cross-format metadata read/write via `DOCUMENT_FORMAT_CODECS`. +- **`src/metadata/`**: cross-format metadata write via `DOCUMENT_FORMAT_CODECS`; the read side is `readDocumentMetadata` in `src/convert/from-pdf.ts`, which reads a content format through `CONTENT_READERS` and a pdf through `readPdf` with an empty page selection. - **`src/package-codec.ts`** — `decodeDocumentPackage`/`encodeDocumentPackage`/`decodeOdbPackage`. Dependency direction is downward and checkable. Twelve external dependencies each own a distinct concern: `ooxml.js` (docx/pptx/xlsx), `odf.js` (odt/ods/odp/odg), `document-schema.js` (shared schemas + port contracts), `pdf-codec` (PDF codec + text-layout/font primitives), `byte-codec` (byte/image utilities), `markdown-codec` (markdown), `rtf-codec` (rtf), `wpd-codec` (wpd, read-only), `doc-codec` (doc), `xls-codec` (xls), `ppt-codec` (ppt, through this package's own `src/ppt/` envelope adapter), `archive-codec` (the `[MS-CFB]` compound-file detection all three legacy binary codecs' own bytes schemas build on). No `PdfObject`/`PdfDict`/`PdfStream` type appears anywhere in this package. diff --git a/packages/documents.js/src/codecs/read.ts b/packages/documents.js/src/codecs/read.ts index 137f2653ee..41920a7713 100644 --- a/packages/documents.js/src/codecs/read.ts +++ b/packages/documents.js/src/codecs/read.ts @@ -37,7 +37,7 @@ export interface DocumentCodecOptions { readonly images?: MarkdownImageResolver; } -// Every format whose bytes decode into a ContentDocument — all sixteen DocumentFormat members except pdf, the one layout format (its read produces a LayoutDocument through readDocumentLayout below instead, mirroring the registry's content/layout entry split). +// Every format whose bytes decode into a ContentDocument — every DocumentFormat member except pdf, the one layout format (its read produces a LayoutDocument through readDocumentLayout below instead, mirroring the registry's content/layout entry split). export type ReadContentFormat = Exclude; export type ContentReader = ( @@ -45,7 +45,7 @@ export type ContentReader = ( options?: DocumentCodecOptions, ) => ContentDocument; -// The fifteen per-format read closures, moved verbatim from the registry: each wraps the identical decode/read pair every ergonomic conversion in this package already uses for that format, with each format's own cancellation policy preserved exactly (docx/pptx/odt/odp/ods/odg/odf/xlsx have no loop of their own to hook a signal into, so their read checks it once via throwIfAborted before decoding; markdown does have one, so its read forwards the signal straight into the reader instead of checking it separately; csv/svg decoders are fatal one-pass functions with no loop, needing no separate check; rtf's own readRtfContent takes bytes directly and checks the signal itself internally, once, before tokenizing — matching markdown's own single-check-inside-the-reader shape, so rtf's closure forwards the signal straight through rather than checking it twice; doc-codec's readDocContent, xls-codec's readXlsContent, and this package's own src/ppt/read.ts wrapper over ppt-codec's readPptContent take no options at all — none of the three legacy binary codecs has a loop or a diagnostic sink of its own yet — so each of their closures checks the signal once via throwIfAborted before decoding, the identical no-loop-format shape docx/pptx/odt/odp/ods/odg/odf/xlsx already get). The registry composes its content entries from these, so there is exactly one place the "which reader for which format" dispatch lives. +// The per-format read closures, moved verbatim from the registry: each wraps the identical decode/read pair every ergonomic conversion in this package already uses for that format, with each format's own cancellation policy preserved exactly (docx/pptx/odt/odp/ods/odg/odf/xlsx have no loop of their own to hook a signal into, so their read checks it once via throwIfAborted before decoding; markdown does have one, so its read forwards the signal straight into the reader instead of checking it separately; csv/svg decoders are fatal one-pass functions with no loop, needing no separate check; rtf's own readRtfContent takes bytes directly and checks the signal itself internally, once, before tokenizing — matching markdown's own single-check-inside-the-reader shape, so rtf's closure forwards the signal straight through rather than checking it twice; doc-codec's readDocContent, xls-codec's readXlsContent, and this package's own src/ppt/read.ts wrapper over ppt-codec's readPptContent take no options at all — none of the three legacy binary codecs has a loop or a diagnostic sink of its own yet — so each of their closures checks the signal once via throwIfAborted before decoding, the identical no-loop-format shape docx/pptx/odt/odp/ods/odg/odf/xlsx already get). The registry composes its content entries from these, so there is exactly one place the "which reader for which format" dispatch lives. export const CONTENT_READERS: Readonly< Record > = { diff --git a/packages/documents.js/src/convert/from-pdf.ts b/packages/documents.js/src/convert/from-pdf.ts index ea1313697a..a58837e695 100644 --- a/packages/documents.js/src/convert/from-pdf.ts +++ b/packages/documents.js/src/convert/from-pdf.ts @@ -1,6 +1,6 @@ // The pdf -> X conversion family and this package's read-only entry point: every pdfTo* function lives here rather than in convert.ts so that a consumer importing documents.js/read (package.json's explicit ./read export onto this module) gets a module graph that excludes every X-to-PDF renderer and therefore every vendored font asset — the same split pdf-codec made with its own ./read entry. convert.ts holds both directions in one module, so its graph statically contains the write path; each pdfTo* function below forwards to convertDocumentFromPdf (src/convert/composition.ts, the composition engine's read half), which resolves and runs exactly the plan convertDocument('pdf', target, ...) would — identical executors, identical routes (a route out of pdf never re-enters pdf), no forked behaviour. src/read-graph.test.ts walks this module's static import graph across the workspace boundary into pdf-codec's own source and fails the build if the write path or a font asset becomes reachable. // -// readDocumentMetadata is defined here too (#744), owning its dispatch through the read-only codec half (src/codecs/read.ts: CONTENT_READERS for the eleven content formats; pdf reads through pdf-codec/read's readPdf with an empty page selection, since metadata never depends on page content) rather than the both-directions registry — whose write halves import writePdf from the pdf-codec root barrel and would drag the write path, and pdf-codec's vendored font assets, back into this entry's graph. The root barrel re-exports it from here unchanged (one definition, one identity, the same arrangement the pdfTo* family already has). Its xlsx branch no longer renders a preview PDF: the old xlsxToPdf-then-readPdf path reported facts about the render rather than the workbook — createdIso/modifiedIso stamped at the render moment for a file carrying none of its own (buildOdsPackage's default systemClock, which is why the old test for that case needed fake timers), and a producer naming the preview PDF's writer. A workbook that does declare its own dcterms:created/dcterms:modified keeps reporting them exactly as before (the render path never overwrote a source timestamp either), app.xml's Application still arrives as creator, and producer stays unset per the schema's own rule that it is a PDF-only concept no semantic reader ever sets. setDocumentMetadata — the write-side sibling — stays in src/metadata/write.ts, which has no graph constraint to satisfy. +// readDocumentMetadata is defined here too (#744), owning its dispatch through the read-only codec half (src/codecs/read.ts: CONTENT_READERS for every content format; pdf reads through pdf-codec/read's readPdf with an empty page selection, since metadata never depends on page content) rather than the both-directions registry — whose write halves import writePdf from the pdf-codec root barrel and would drag the write path, and pdf-codec's vendored font assets, back into this entry's graph. The root barrel re-exports it from here unchanged (one definition, one identity, the same arrangement the pdfTo* family already has). Its xlsx branch no longer renders a preview PDF: the old xlsxToPdf-then-readPdf path reported facts about the render rather than the workbook — createdIso/modifiedIso stamped at the render moment for a file carrying none of its own (buildOdsPackage's default systemClock, which is why the old test for that case needed fake timers), and a producer naming the preview PDF's writer. A workbook that does declare its own dcterms:created/dcterms:modified keeps reporting them exactly as before (the render path never overwrote a source timestamp either), app.xml's Application still arrives as creator, and producer stays unset per the schema's own rule that it is a PDF-only concept no semantic reader ever sets. setDocumentMetadata — the write-side sibling — stays in src/metadata/write.ts, which has no graph constraint to satisfy. // // readNativeDocumentTree (#823 Ask 1) is the third dispatcher in this same read-only family: it reports the SOURCE format's own native DocumentTree, with no cross-format bridging or conversion target involved at all — unlike ConversionResult.package/onDocument, which report whatever hop actually produced a REQUESTED conversion's output (a lossy cross-variant bridge's shape for a target sharing no ContentDocument variant with the source, e.g. xlsx -> markdown composing through a pdf pivot and reporting that pivot's wordprocessing-shaped ContentDocument, with no sheet/cell/formula/A1 data at all — the exact bug #823 reports). Every DocumentFormat member reads through CONTENT_READERS directly (no reconstruction, no pages, a bridge-free ContentDocument straight from that format's own reader — odf's readOdfFormulaContent included, since CONTENT_READERS already covers it) except pdf, which has no ContentDocument reader of its own: its native representation is readPdf's LayoutDocument reconstructed into a wordprocessing ContentDocument via reconstructWordprocessing (the identical reconstruction executeFromPdf's own pdf-to-docx hop runs, and the one OUTLINE_CONVERSION_TARGET/OUTLINE_PROBE_TARGETS already treat as pdf's "native" tree), with pages/frames and the document-level PDF tables (destinations/outline/attachments/layers/structure/comment bodies) stamped on exactly as that hop's own onDocument report already carries. readPdf is called directly (not through readDocumentLayout, which carries no sink) so a caller of readNativeDocumentTree gets the identical PDF-parse diagnostics a real pdf-sourced conversion would report.