Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ To add a tool: manifest, run, panel, three registry entries, and tests. Reuse th

## Platform seams

- Image codec: `codec()` / `setCodec()` in `image-core`. The app's worker sets `appCodec` (`image-core/browser`): our own decoders and encoders for PNG, JPEG, GIF and BMP, and the browser (OffscreenCanvas) only for WebP and for files ours refuse with `UnsupportedImage`. Tests call `setCodec(ownCodec)`; WebP can't be tested in Node. Decoders and encoders are pure and synchronous (`decodeJpeg`, `encodeJpeg`, `decodeGif`, `encodeGif`, `decodeBmp`, `encodeBmp`, `decodePng`, `encodePng`) and are used directly by tools that need more than one picture out (GIF frames, JPEG metadata segments).
- Image codec: `codec()` / `setCodec()` in `image-core`. The app's worker sets `appCodec` (`image-core/browser`): the browser's native codec (OffscreenCanvas) where ours is many times slower (JPEG read and written, opaque PNG written, WebP), ours for PNG reading, see-through PNG writing (a canvas premultiplies alpha, so lossless would stop being lossless), GIF and BMP, and ours again whenever the browser fails. Native JPEG reading keeps samples as stored (`colorSpaceConversion: 'none'`) like ours; the browser's JPEG adds an sRGB profile, so Compress image keeps only its JFIF header and puts the file's own segments back. Tests call `setCodec(ownCodec)`; the native paths can't be tested in Node and are checked in Chrome. Decoders and encoders are pure and synchronous (`decodeJpeg`, `encodeJpeg`, `decodeGif`, `encodeGif`, `decodeBmp`, `encodeBmp`, `decodePng`, `encodePng`) and are used directly where the codec can't do the job (GIF frames, palette PNGs (PDF to image keeps ours for its pages: small files for text), grey JPEGs). Compress PDF reads colour JPEGs through `Platform.decodeJpeg` (the browser's, as the renderer draws them) and the rest with ours.
- Text on pictures: the engine has no font code. Panels draw text to a PNG with the browser canvas (`bitmaps.ts`: `textMark`, `pictureMark`, `trimmedPng`; `meme/caption.ts`) and put it in the options; `run.ts` only decodes and places it with `overlay`. Keep it that way; a Node test passes a ready-made PNG. `blur-image` is the one panel that imports engine functions (`apply.ts`): it shows the same operations on a 1400 px copy so the preview matches the saved file.
- Flate: `deflate` / `inflate` in `core` use `CompressionStream` (async); `inflate` keeps its output when bytes follow the end of the data (a stream Length that counts a line break), which the platform stream rejects.
- Renderer `Platform`: `imageFromRGBA` and `decodeJpeg`. Tests pass fakes.
Expand All @@ -137,7 +137,7 @@ To add a tool: manifest, run, panel, three registry entries, and tests. Reuse th
- `ToolPage.tsx`: files | preview | options at `xl`; at `lg` the options stay on the right with the files above the preview; stacked below that, options before the preview. Options are always the right-hand region. Tools with their own surface (crop, sign) set `placesActions` and render the `actions` prop (run button, progress, errors) beside their controls. A finished run opens a modal `<dialog>` (download, keep editing, start over, use in another tool), not a card under the options. Locked PDFs open a password dialog and are replaced by a decrypted in-memory copy; the password is never stored.
- `previewController.ts` (pure, fake-timer tested) runs the same engine as the Run button: **600 ms debounce**, cancels stale runs, keeps the last result on screen. `ready` states have `stale` (outdated; Run never reuses a stale result) and `computing` (dim and "Updating..."). Do not make the UI react per keystroke.
- `previewWorker.ts` / `previewClient.ts`: one worker for inspect, render, unlock and metadata. The main thread never parses a PDF. Page canvases keep the old picture until the new one is ready (no flashing).
- Tool runs go through `runtime.ts` and `worker.ts` (progress, cancel, ZIP for several outputs). Limits: 50 files, 200 MB per file, 500 MB total.
- Tool runs go through `runtime.ts` and `worker.ts` (progress, cancel, ZIP for several outputs). Limits: 50 files, 200 MB per file, 500 MB total. One tool worker is kept warm between runs (`warmTool` loads the engine when a tool page opens; a run that starts while it's busy gets its own worker; cancelling terminates it). Each input carries a `key` (the same `File`, the same key), and `decodeFile` keeps decoded pictures under it (up to 512 MB, only for the current inputs), so a preview after an option change doesn't decode again. Run functions get a copy: they may change their pixels.
- Office files preview as PDFs: `previewClient` sends the kind with the bytes (`officeKind` in `usePdfInfo.ts`, kept free of engine code), `previewWorker` converts once with `officeToPdf` from `@ream/tools/office-preview` and then inspects and draws it like a PDF, adding an "approximate" warning. `isPreviewable` (PDF or Office) decides what `Preview` and `Thumb` show.
- Zoom: `ZoomBar`, `useZoom`, `zoomedWidth`, `renderWidthFor` and `useWidth` live in `@ream/ui` (`Zoom.tsx`). The Original/Result pane starts at fit width, with its `ZoomBar` (Fit width and Fit page) on the tabs row: `Preview` owns the zoom and `PdfView` reads it from `ZoomContext` and reports the size shown (no bar for pictures or text); panels that work on one page (Crop PDF, Sign PDF, Redact PDF, Fill PDF form) use `usePageZoom` (`tools/src/PageZoom.tsx`), which starts with the whole page in view and also offers Fit width. A zoom is a share of the actual size (a point is 4/3 CSS pixels), the page box is sized in CSS pixels inside a scrolling area, and everything drawn over it is positioned in fractions, so overlays scale with it. Pages are re-rendered at the shown size (in 200 px steps, at most 4000).
- Edit PDF (`tools/src/edit-pdf`):
Expand Down Expand Up @@ -195,6 +195,7 @@ To add a tool: manifest, run, panel, three registry entries, and tests. Reuse th
- Match the surrounding code: comment density, naming and idiom. Comments explain *why* (limits, gotchas, reasons), not what.
- Mark deliberate shortcuts that have a known ceiling with a `ponytail:` comment naming the ceiling and the upgrade path.
- Small files, one folder per tool, no abstractions without a second user, no config for values that never change.
- Pixel loops are hot (a phone photo is 12 to 50 million pixels): no allocation per pixel (`data.set(src.subarray(i, i + 4))`, array literals, destructuring), write the four bytes; walk memory in row order (a column at a time misses the cache); compare against the old code's output when speeding something up. Measure with a 12 MP photo and a few hundred pages before and after.
- Prefer the platform (CSS, native elements) over code, and the standard library over dependencies. Do not add runtime dependencies to engine packages. UI-only asset packages (fonts) are fine. Ream is MIT: anything shipped must be MIT-compatible (no GPL, AGPL or LGPL), and adding, removing or updating a runtime dependency means running `node scripts/notices.ts`. Code copied in by hand (like Lucide's shapes) keeps its notice in the file and gets an entry in `scripts/notices.ts`.
- Do not put personal data in URLs, logs or tests.

Expand Down
3 changes: 2 additions & 1 deletion apps/web/src/ToolPage.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { Button, Card, Dropzone, Icon } from '@ream/ui'
import { UserError, acceptsFile, formatBytes, type FileData, type Tool } from '@ream/core'
import { tools } from '@ream/tools'
import { panels, type PanelModule } from '@ream/tools/panels'
import { runTool, type Running } from './runtime'
import { runTool, warmTool, type Running } from './runtime'
import { download, toFile } from './download'
import { clearHandoff, peekHandoff, setHandoff } from './handoff'
import { FileMeta, Thumb } from './Thumb'
Expand Down Expand Up @@ -77,6 +77,7 @@ export function ToolPage({ tool }: { tool?: Tool }) {
useEffect(() => {
clearHandoff()
if (!tool || tool.status === 'soon' || !panels[tool.id]) return
warmTool(tool.id)
let live = true
panels[tool.id]().then((m) => {
if (!live) return
Expand Down
37 changes: 31 additions & 6 deletions apps/web/src/runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,32 +2,57 @@ import { UserError, checkLimits, type FileData } from '@ream/core'

export type Running = { promise: Promise<FileData[]>; cancel: () => void }

/** Run a tool's engine in a fresh Web Worker. Cancelling terminates the worker and rejects with an AbortError. */
const spawn = () => new Worker(new URL('./worker.ts', import.meta.url), { type: 'module' })

// One worker is kept between runs, so a preview after an option change doesn't load the engine again and finds the
// files it decoded last time (see `keyed` in worker.ts). A run that starts while it is busy gets a worker of its own.
let idle: Worker | undefined

/** start the worker and load a tool's engine before the first run needs them (a tool page calls it on open) */
export function warmTool(id: string) {
idle ??= spawn()
idle.postMessage({ warm: id })
}

// the same File keeps its key across runs, so the worker can tell it already has it
const keys = new WeakMap<Blob, string>()
let nextKey = 0
const keyOf = (f: Blob) => keys.get(f) ?? (keys.set(f, String(++nextKey)), keys.get(f)!)

/** Run a tool's engine in a Web Worker. Cancelling terminates the worker (the only way to stop work in progress) and rejects with an AbortError. */
export function runTool(id: string, files: File[], options: unknown, onProgress: (fraction: number) => void): Running {
const worker = new Worker(new URL('./worker.ts', import.meta.url), { type: 'module' })
const worker = idle ?? spawn()
idle = undefined
let finished = false
const finish = (reuse: boolean) => {
finished = true
if (reuse && !idle) idle = worker
else worker.terminate()
}
let reject: (e: unknown) => void = () => {}
const promise = new Promise<FileData[]>((resolve, rej) => {
reject = rej
worker.onmessage = (e) => {
const m = e.data
if (m.type === 'progress') return onProgress(m.value)
worker.terminate()
finish(true)
if (m.type === 'done') resolve(m.outputs)
else rej(m.user ? new UserError(m.message) : new Error(m.message))
}
worker.onerror = (e) => {
worker.terminate()
finish(false)
rej(new Error(e.message || "Couldn't start the background worker"))
}
checkLimits(files) // throws inside the executor, which rejects the promise
Promise.all(files.map(async (f) => ({ name: f.name, type: f.type, bytes: new Uint8Array(await f.arrayBuffer()) })))
Promise.all(files.map(async (f) => ({ name: f.name, type: f.type, key: keyOf(f), bytes: new Uint8Array(await f.arrayBuffer()) })))
.then((inputs) => worker.postMessage({ id, files: inputs, options }, inputs.map((i) => i.bytes.buffer)))
.catch(rej)
})
return {
promise,
cancel() {
worker.terminate()
if (finished) return
finish(false)
reject(new DOMException('Cancelled', 'AbortError'))
},
}
Expand Down
6 changes: 5 additions & 1 deletion apps/web/src/worker.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,22 @@
import { UserError, type FileData } from '@ream/core'
import { setCodec } from '@ream/image-core'
import { appCodec } from '@ream/image-core/browser'
import { keepDecoded } from '@ream/tools/image-common'
import { runners } from '@ream/tools/runners'
import { installStandardFonts } from '@ream/tools/standard-fonts'

setCodec(appCodec)
installStandardFonts()

type Request = { id: string; files: FileData[]; options: unknown }
type Request = { id: string; files: FileData[]; options: unknown } | { warm: string }

const post = (message: unknown, transfer: Transferable[] = []) => (self as unknown as Worker).postMessage(message, transfer)

self.onmessage = async (e: MessageEvent<Request>) => {
// a tool page opened: load its engine now, quietly (runtime.ts expects no answer)
if ('warm' in e.data) return void runners[e.data.warm]?.().catch(() => {})
const { id, files, options } = e.data
keepDecoded(files.map((f) => f.key))
try {
const load = runners[id]
if (!load) throw new UserError("That tool isn't available yet.")
Expand Down
38 changes: 37 additions & 1 deletion packages/core/src/core.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { describe, expect, test } from 'vitest'
import { UserError, acceptsFile, checkLimits, crc32, formatBytes, unzip, zip, zipPacked } from './index.ts'
import { UserError, acceptsFile, checkLimits, crc32, formatBytes, pngPredictor, unzip, zip, zipPacked } from './index.ts'

const enc = (s: string) => new TextEncoder().encode(s)

Expand Down Expand Up @@ -85,3 +85,39 @@ describe('unzip', () => {
await expect(unzip(packed)).rejects.toBeInstanceOf(UserError)
})
})

test('pngPredictor matches the PNG definition for every filter, pixel width and bit depth, the first row included', () => {
// the definition, byte by byte (PNG spec section 9)
const reference = (data: Uint8Array, columns: number, colors: number, bpc: number) => {
const bpp = Math.max(1, (colors * bpc + 7) >> 3)
const row = (colors * bpc * columns + 7) >> 3
const rows = Math.floor(data.length / (row + 1))
const out = new Uint8Array(rows * row)
for (let r = 0; r < rows; r++) {
const ft = data[r * (row + 1)]
for (let i = 0; i < row; i++) {
const x = data[r * (row + 1) + 1 + i]
const a = i >= bpp ? out[r * row + i - bpp] : 0
const b = r > 0 ? out[(r - 1) * row + i] : 0
const c = i >= bpp && r > 0 ? out[(r - 1) * row + i - bpp] : 0
const p = a + b - c
const paeth = Math.abs(p - a) <= Math.abs(p - b) && Math.abs(p - a) <= Math.abs(p - c) ? a : Math.abs(p - b) <= Math.abs(p - c) ? b : c
out[r * row + i] = x + [0, a, b, (a + b) >> 1, paeth][ft > 4 ? 0 : ft]
}
}
return out
}
let seed = 7
const rand = () => (seed = (seed * 1103515245 + 12345) & 0x7fffffff) >> 16
for (const [colors, bpc] of [[1, 8], [3, 8], [4, 8], [1, 1], [2, 16], [4, 16]]) {
const columns = 13
const row = (colors * bpc * columns + 7) >> 3
const data = new Uint8Array(12 * (row + 1)).map(() => rand() & 255)
// every filter type on the first row and below, and one unknown type
for (let r = 0; r < 12; r++) data[r * (row + 1)] = r % 6 === 5 ? 9 : r % 6
for (const first of [0, 1, 2, 3, 4, 9]) {
data[0] = first
expect(Array.from(pngPredictor(data, columns, colors, bpc))).toEqual(Array.from(reference(data, columns, colors, bpc)))
}
}
})
34 changes: 22 additions & 12 deletions packages/core/src/flate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,27 +58,37 @@ export function pngPredictor(data: Uint8Array, columns: number, colors = 1, bpc
const row = (colors * bpc * columns + 7) >> 3
const rows = Math.floor(data.length / (row + 1))
const out = new Uint8Array(rows * row)
// one loop per filter type: deciding per byte was most of the cost of reading a big PNG or a predicted PDF stream
for (let r = 0; r < rows; r++) {
const ft = data[r * (row + 1)]
const src = r * (row + 1) + 1
const dst = r * row
for (let i = 0; i < row; i++) {
const x = data[src + i]
const a = i >= bpp ? out[dst + i - bpp] : 0
const b = r > 0 ? out[dst - row + i] : 0
const c = i >= bpp && r > 0 ? out[dst - row + i - bpp] : 0
let v = x
if (ft === 1) v = x + a
else if (ft === 2) v = x + b
else if (ft === 3) v = x + ((a + b) >> 1)
else if (ft === 4) {
const up = dst - row // the row above, when there is one (r > 0)
if (ft === 1) {
for (let i = 0; i < row; i++) out[dst + i] = data[src + i] + (i >= bpp ? out[dst + i - bpp] : 0)
} else if (ft === 2 && r > 0) {
for (let i = 0; i < row; i++) out[dst + i] = data[src + i] + out[up + i]
} else if (ft === 3) {
for (let i = 0; i < row; i++) out[dst + i] = data[src + i] + (((i >= bpp ? out[dst + i - bpp] : 0) + (r > 0 ? out[up + i] : 0)) >> 1)
} else if (ft === 4 && r === 0) {
// Paeth with nothing above always picks the left neighbour (Sub)
for (let i = 0; i < row; i++) out[dst + i] = data[src + i] + (i >= bpp ? out[dst + i - bpp] : 0)
} else if (ft === 4) {
// with nothing to the left it picks the byte above (Up)
for (let i = 0; i < bpp && i < row; i++) out[dst + i] = data[src + i] + out[up + i]
for (let i = bpp; i < row; i++) {
const a = out[dst + i - bpp]
const b = out[up + i]
const c = out[up + i - bpp]
const p = a + b - c
const pa = Math.abs(p - a)
const pb = Math.abs(p - b)
const pc = Math.abs(p - c)
v = x + (pa <= pb && pa <= pc ? a : pb <= pc ? b : c)
out[dst + i] = data[src + i] + (pa <= pb && pa <= pc ? a : pb <= pc ? b : c)
}
out[dst + i] = v & 255
} else {
// none, "up" on the first row, and unknown types: the bytes as they are
out.set(data.subarray(src, src + row), dst)
}
}
return out
Expand Down
3 changes: 2 additions & 1 deletion packages/core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ export type Tool = {
status: 'ready' | 'soon'
}

export type FileData = { name: string; type: string; bytes: Uint8Array }
/** `key` names the same input across runs (the app's worker keeps decoded pictures under it); absent in tests and outputs */
export type FileData = { name: string; type: string; bytes: Uint8Array; key?: string }

export type RunContext = { progress: (fraction: number) => void }

Expand Down
36 changes: 31 additions & 5 deletions packages/image-core/src/browser.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { UserError } from '@ream/core'
import { UnsupportedImage, checkPixels } from './buffer.ts'
import { UnsupportedImage, checkPixels, type ImageBuffer } from './buffer.ts'
import type { Codec, OutputMime } from './codec.ts'
import { ownCodec } from './own.ts'

Expand All @@ -10,7 +10,8 @@ export const browserCodec: Codec = {
async decode(bytes, type) {
let bitmap: ImageBitmap
try {
bitmap = await createImageBitmap(new Blob([bytes as BlobPart], { type }))
// samples as stored, like our decoder (a colour profile is carried over by the tools that keep it, not applied); turned upright by EXIF
bitmap = await createImageBitmap(new Blob([bytes as BlobPart], { type }), { colorSpaceConversion: 'none', imageOrientation: 'from-image' })
} catch {
throw new UserError("Couldn't read this image. It may be damaged, or this browser can't open that kind of file.")
}
Expand All @@ -35,12 +36,27 @@ export const browserCodec: Codec = {
},
}

const isJpeg = (b: Uint8Array) => b[0] === 0xff && b[1] === 0xd8
const opaque = (img: ImageBuffer) => {
for (let i = 3; i < img.data.length; i += 4) if (img.data[i] !== 255) return false
return true
}

/**
* What the app uses: our own codec for everything it can do, the browser only for WebP and for files that use something
* ours doesn't (when the browser can't read them either, the more precise message of ours is shown).
* What the app uses: the browser's native codec where it is many times faster than ours (JPEG both ways, PNG writing),
* ours for everything else and whenever the browser fails. A canvas stores premultiplied alpha, so it can't keep
* see-through pixels exact: PNGs with transparency, and every PNG read, stay with ours (lossless stays lossless).
* GIF and BMP are ours only: browsers can't write them.
*/
export const appCodec: Codec = {
async decode(bytes, type) {
if (isJpeg(bytes)) {
try {
return await browserCodec.decode(bytes, 'image/jpeg')
} catch {
// ours gives the precise reason, or reads what the browser couldn't
}
}
try {
return await ownCodec.decode(bytes, type)
} catch (e) {
Expand All @@ -52,5 +68,15 @@ export const appCodec: Codec = {
}
}
},
encode: (img, type, quality) => (type === 'image/webp' ? browserCodec : ownCodec).encode(img, type, quality),
async encode(img, type, quality) {
if (type === 'image/webp') return browserCodec.encode(img, type, quality)
if (type === 'image/jpeg' || (type === 'image/png' && opaque(img))) {
try {
return await browserCodec.encode(img, type, quality)
} catch {
// too big for a canvas here, or out of memory: ours can still do it
}
}
return ownCodec.encode(img, type, quality)
},
}
Loading
Loading