Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/web.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ jobs:
- run: yarn test
- run: node scripts/notices.ts --check
- run: yarn build
- run: node scripts/seo-check.ts
- name: The Docker image builds, serves the app and sends the security headers
run: |
docker build -t ream:check .
Expand All @@ -56,6 +57,16 @@ jobs:
asset=$(docker exec ream sh -c 'ls /usr/share/nginx/html/assets | head -1')
headers "/assets/$asset" | grep -qi '^cache-control: public, max-age=31536000, immutable' || fail 'assets are not cached for good'
headers /no-such-file | grep -q '^HTTP/1.1 404' || fail 'a missing file is not a 404'
page() { curl -sS "http://localhost:8080$1"; }
headers /tools/pdf/compress-pdf/ | grep -q '^HTTP/1.1 200' || fail 'a tool page is not served'
page /tools/pdf/compress-pdf/ | grep -q '<title>Compress PDF' || fail 'the tool page has the wrong title'
page /tools/pdf/compress-pdf/ | grep -q 'rel="canonical" href="https://ream.deityhub.com/tools/pdf/compress-pdf/"' || fail 'the tool page has no canonical address'
headers /sitemap.xml | grep -q '^HTTP/1.1 200' || fail 'no sitemap'
page /sitemap.xml | grep -q '/tools/pdf/merge-pdf/' || fail 'the sitemap lacks a tool'
for gone in /tools/pdf/no-such-tool/ /tools/image/sign-pdf/ /tools/video/ /tool/compress-pdf/; do
headers "$gone" | grep -q '^HTTP/1.1 404' || fail "$gone is not a 404"
done
page /tools/pdf/no-such-tool/ | grep -q 'noindex' || fail 'the 404 page can be indexed'
[ "$(docker exec ream id -u)" != 0 ] || fail 'NGINX runs as root'
echo "image: $(docker image inspect ream:check --format '{{.Size}}') bytes"
- uses: actions/upload-artifact@v7
Expand Down
11 changes: 7 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,12 @@ yarn typecheck # tsc -p tsconfig.json (covers packages/*/src, pack
node scripts/pdf.ts <cmd> # dev CLI for pdf-core: info | merge | extract | remove | rotate | split | decrypt (-p <password>)
node scripts/contrast.ts # WCAG contrast check of every colour token in both modes
node scripts/notices.ts # rewrite THIRD-PARTY-NOTICES.md (--check: fail if out of date); fails on GPL/AGPL/unknown licences
node scripts/seo-check.ts # the built site as a crawler sees it (run `yarn build` first): every page's title, description, canonical, social card, JSON-LD, links and the sitemap; `check` runs it
node scripts/og.ts # redraw apps/web/public/og.png (the social card) with headless Chrome
node scripts/render-compare.ts # renderer vs macOS CoreGraphics (sips), fails above recorded limits (embedded fonts: real system fonts read at run time, never committed)
```

**Before reporting work as done, run `yarn typecheck`, `yarn test` and `yarn build`, all clean.** Run `node scripts/contrast.ts` after touching `packages/ui/src/theme.css`. Run `node scripts/render-compare.ts` after touching `packages/pdf-render`.
**Before reporting work as done, run `yarn typecheck`, `yarn test` and `yarn build`, all clean** (and `node scripts/seo-check.ts` after touching routes, pages or tool text). Run `node scripts/contrast.ts` after touching `packages/ui/src/theme.css`. Run `node scripts/render-compare.ts` after touching `packages/pdf-render`.

## Repository layout

Expand Down Expand Up @@ -69,7 +71,7 @@ Each tool is a folder in `packages/tools/src/<tool-id>/`:

Current tools: merge-pdf, split-pdf, images-to-pdf, watermark, page-numbers, crop-pdf, metadata, extract-images, compress-pdf, protect-pdf, unlock-pdf, sign-pdf, resize-image, convert-image, rotate-image, crop-image, merge-images, compress-image, animated-gif, image-metadata, watermark-image, blur-image, filters-image, meme, organize-pdf, bookmarks-pdf, repair-pdf, pdf-to-text, redact-pdf, fill-pdf-form, edit-pdf, pdf-to-image, pdf-to-word, pdf-to-excel, pdf-to-powerpoint, excel-to-pdf, powerpoint-to-pdf.

To add a tool: manifest, run, panel, three registry entries, and tests. Reuse the helpers above.
To add a tool: manifest, run, panel, three registry entries, its page text in `packages/tools/src/seo/` (`seo.test.ts` fails without it, and checks the lengths), and tests. Reuse the helpers above.

## PDF engine notes (`pdf-core`)

Expand Down Expand Up @@ -153,12 +155,13 @@ To add a tool: manifest, run, panel, three registry entries, and tests. Reuse th
- **Edit content mode:** the `content` host call gives the page's lines and pictures; picking one makes a `LineEditItem` or `PictureEditItem` (deleting marks it `deleted`, so it can be put back), and the Erase tool makes `EraseItem`s. `contentEdits` (`content.ts`) turns them into `editContent`'s input plus what is drawn over the page instead (a line its own font can't show, as a `TextItem` on its old baseline; a replacement picture). The run goes `editForm`, `editContent`, `decorate`.
- **What the editor shows:** the page is drawn *without* what was changed (`PdfPage hide`, the renderer's `hide`: letters and pictures whose middle is in a changed item's old box, the same test the saved file uses; erased areas as an even-odd clip), so there are no cover boxes. Everything is stacked in the saved file's order (`layerOf` in `geometry.ts`: the page's own content and changes, then added items in list order, then comments, then fields) with a z-index per item; page lines and pictures can be picked with Select in Add to pages too, under whatever was added over them. `stepOrder` / `canStep` move an item past the next one it overlaps in its layer.
- **Fonts:** the `fonts` host call (`documentFonts` in pdf-render) lists the PDF's families for `FontsPanel`, whose choices go in `o.fonts`. A changed line its own subset can't show is drawn in its family (`fallbackFont`), so the font book uses a font added for it or the look-alike; `viewFonts.loadFont` mirrors that in the editor.
- Routes (`App.tsx`, hash based): `#/tool/<id>` the tool page, `#/tools`, `#/tools/pdf` and `#/tools/image` the tools home for a category (`Home.tsx`, `filterHref` in `TopBar.tsx`), `#/licence` and `#/notices` (`Legal.tsx`: the repo's `LICENSE` and `THIRD-PARTY-NOTICES.md`, imported `?raw` when the page opens) and `#/` (`Landing.tsx`); any other hash shows the landing page. The web-only pages share `Marketing.tsx` (nav, footer, `primary`/`secondary` link styles); app pages render `TopBar` (logo on the left; search and theme on the right; 1600 px like the pages) and `Breadcrumb` under it (All tools, the category, the tool; every level a link). The breadcrumb replaces visible headings on the tools home and tool pages (the `h1` is `sr-only`); on All tools the PDF tools and Image tools section titles link to their category. `focus` in `TopBar.tsx` is the shared focus ring.
- Routes (`routes.ts`, `router.tsx`, `App.tsx`): real paths with the History API, no router dependency. `/` the landing page, `/tools/` the tools home, `/tools/pdf/` and `/tools/image/` a category (`Home.tsx`, `filterHref` in `TopBar.tsx`), `/tools/<kind>/<id>/` a tool (for example `/tools/pdf/sign-pdf/`; there is no `/tool/` route), `/licence/` and `/notices/` (`Legal.tsx`: the repo's `LICENSE` and `THIRD-PARTY-NOTICES.md`, imported `?raw` when the page opens); `parseRoute` checks the registry, so a tool that doesn't exist, or sits under the wrong kind, is `notFound` like any other unknown path and shows `NotFound` (the host answers 404 with `404.html` for the same addresses). The `tool` route carries the `Tool` itself; build links with `toolPath(tool)`. Paths end in a slash (what Pages and NGINX serve for `folder/index.html`); `parseRoute` accepts both. `parseRoute`/`pathOf`/`allRoutes` are pure and shared with the build. Links are `<Link to>` (a real `<a href>`; a plain click uses `navigate`, so new tabs and crawlers work), never `location.hash`. Old `#/...` links are turned into paths before the first render (`upgradeLegacyUrl`). The web-only pages share `Marketing.tsx` (nav, footer, `primary`/`secondary` link styles); app pages render `TopBar` (logo on the left; search and theme on the right; 1600 px like the pages) and `Breadcrumb` under it (All tools, the category, the tool; every level a link). The breadcrumb replaces visible headings on the tools home and tool pages (the `h1` is `sr-only`); on All tools the PDF tools and Image tools section titles link to their category. `focus` in `TopBar.tsx` is the shared focus ring.
- SEO (`seo.ts`, `seoHead.ts`, `seoPlugin.ts`, `ToolAbout.tsx`, `packages/tools/src/seo/`): the build (`seoPlugin`, after the bundle) writes one HTML file per route from the built `index.html` (`tool/merge-pdf/index.html`...): its own `<title>`, description, canonical address, Open Graph and Twitter tags, JSON-LD (landing: `WebSite` + `WebApplication`; tools: `WebApplication` + `BreadcrumbList`; marked `data-seo`) and plain text inside `#root` (`main.seo`, styled in `index.css`) that the app replaces when it starts. It also writes `404.html` (needed: without it Cloudflare Pages treats the site as an SPA and answers every unknown address with 200) and `sitemap.xml` (indexable routes only: not the licence, notices or 404 pages). `pageSeo(route)` is the one source for all of it; `seoHead.applyHead` (lazy) keeps the head in step while moving around the app. A tool's words (`title` at most 60 characters ending in "| Ream", `description` 70 to 155, `intro`, three `points`, two or more `faq`) live in `packages/tools/src/seo/{pdf,convert,image}.ts` and also show under the tool (`ToolAbout`, lazy: four framed cells, shown only while no file is added). Write only what the tool really does and where it stops (no OCR, no certificate signatures, Office conversions are approximate). `SITE` (`https://ream.deityhub.com`) in `seo.ts` is every canonical address, also from a self-hosted copy. The social card is `apps/web/public/og.png`, made by `node scripts/og.ts`.
- Tools home (`Home.tsx`, at most 1600 px wide) is drop-first: the `Dropzone` `bar` layout, and dropped files handed to a tool through `handoff.ts`, the same mechanism as "Use in another tool".
- Docker: `Dockerfile` builds the site with Node (on the build machine's platform) and serves it with `nginxinc/nginx-unprivileged` (non-root, port 8080). Its NGINX site is generated from `apps/web/public/_headers` by `scripts/nginx-conf.ts`, so change headers there only. The image build skips the typecheck (tests are left out by `.dockerignore`; CI typechecks anyway). `check` builds and runs it and checks the page, the headers, `sw.js` not being cached, the manifest's type, a 404 and a non-root user; the release pushes amd64 and arm64 images to `ghcr.io/deityhub/ream` and, with `DOCKERHUB_USERNAME`/`DOCKERHUB_TOKEN` set, `docker.io/<user>/ream`, tagged with the version and `latest`. Containers can't start in this Mac's Docker; test a server inside a throwaway `docker build` (start it in a `RUN`, query it with curl) or rely on CI.
- Versions: the root `package.json` `version` (semver, starting at 0.1.0) is the release version. A pull request into `main` must raise it and have its `CHANGELOG.md` section: `yarn release patch|minor|major` (`scripts/release.ts prepare`) writes both, the section from the pull requests merged since the last tag (first-parent merges of `dev`, their titles and links; earlier releases skipped), and `node scripts/release.ts check` fails on a version already tagged or older or one without a section. The merge deploys production, tags `v<version>` and creates a GitHub Release whose notes are that section (`release.ts notes`). `.github/workflows/prepare-release.yml` (Actions → Prepare release) runs `prepare` on `dev` and opens the "Release v<version>" pull request into `dev`, as the `RELEASE_TOKEN` secret so `check` and the preview run on it. The version is in the footer (`__APP_VERSION__`, defined in `vite.config.ts`, declared in `version.d.ts`) and in the offline cache's name. `.github/workflows/deploy-version.yml` builds a released tag again and deploys it to production (rollback when the Cloudflare dashboard's can't).
- Hosting: Cloudflare Pages, deployed by `.github/workflows/web.yml`: a pull request into `dev` runs `check` and gets a preview (its link is commented on the PR), a pull request into `main` runs `check`, a merge into `main` runs `check` and deploys production. Plain pushes run nothing. `check` is the status `.github/rulesets/protect-dev-main.json` requires (import it under Settings → Rules once the repo is public: rulesets aren't free on private repos). `apps/web/public/_headers` holds the security headers: the CSP allows only the site's own files, plus `api.fontsource.org` and `cdn.jsdelivr.net` in `connect-src` for Download free fonts, and no inline scripts or styles (`style-src 'self'`: React's `style` props are fine, `<style>` elements and `style="..."` strings are not). A new network host or inline style needs the policy changed, and the user asked first (the network rule above). Check with `wrangler pages dev` and a `securitypolicyviolation` listener.
- PWA: `manifest.webmanifest` and `icons/` in `apps/web/public` (PNGs from `apps/web/icons/icon.svg` by `node scripts/icons.ts`; square, full bleed, the mark inside the maskable safe zone). `apps/web/sw.js` is plain JS, outside `src`: the `serviceWorker` plugin in `vite.config.ts` writes it into `dist` after the build with every file's name (minus `.woff` fallbacks and the non-Latin Office fonts, which it keeps when first used) and a version hashed from their contents. The page is kept as `./` because Pages redirects `index.html` to `/`. Navigations are network first, everything else cache first. A new version waits until the user clicks Reload in `UpdateBar.tsx` (`pwa.ts` posts `switch`; the page reloads on `controllerchange` only after that, since the first worker's `clients.claim` changes the controller too). `pwa.ts` also keeps the `beforeinstallprompt` event for the landing page's Install Ream button. The worker is registered in production builds only: `yarn dev` has none. To test offline for real, serve the build with `wrangler pages dev`, let the worker fill its cache, then stop the server (CDP's offline emulation doesn't reach the worker's own requests).
- PWA: `manifest.webmanifest` and `icons/` in `apps/web/public` (PNGs from `apps/web/icons/icon.svg` by `node scripts/icons.ts`; square, full bleed, the mark inside the maskable safe zone). `apps/web/sw.js` is plain JS, outside `src`, registered as `/sw.js` with scope `/`: the `serviceWorker` plugin in `vite.config.ts` writes it into `dist` after the build with every file's name (minus `.woff` fallbacks, the non-Latin Office fonts, which it keeps when first used, and what only crawlers or the 404 need) and a version hashed from their contents. Each page is kept under its folder's address (`./`, `tool/merge-pdf/`) because Pages redirects `.../index.html` to the folder. A navigation that fails offline gets its own kept page, else `./` (the app routes by address). Cache lookups use `ignoreVary` (a host sending `Vary: Origin` made every module script miss its kept copy). Navigations are network first, everything else cache first. A new version waits until the user clicks Reload in `UpdateBar.tsx` (`pwa.ts` posts `switch`; the page reloads on `controllerchange` only after that, since the first worker's `clients.claim` changes the controller too). `pwa.ts` also keeps the `beforeinstallprompt` event for the landing page's Install Ream button. The worker is registered in production builds only: `yarn dev` has none. To test offline for real, serve the build with `wrangler pages dev`, let the worker fill its cache, then stop the server (CDP's offline emulation doesn't reach the worker's own requests).

## Design system

Expand Down
3 changes: 3 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,14 @@ Use Yarn, not npm or pnpm.
yarn typecheck
yarn test
yarn build
node scripts/seo-check.ts # the built site as a search engine sees it
node scripts/notices.ts --check # after changing dependencies, run it without --check and commit the result
```

These are what the `check` job runs (plus building and testing the Docker image). Also run `node scripts/contrast.ts` after changing `packages/ui/src/theme.css`, and `node scripts/render-compare.ts` (macOS) after changing the PDF renderer.

A new tool needs its page text (title, description, intro, points and questions) in `packages/tools/src/seo/`: it is what search results and the page under the tool say, so write what the tool really does and where it stops. `yarn test` fails until it is there.

Add tests with your change: engines and tools are tested in plain Node, and a tool's tests call its `run()` directly. For anything with a screen, look at it in a browser at desktop and phone widths, in light and dark.

## Branches and releases
Expand Down
6 changes: 3 additions & 3 deletions apps/web/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="icon" type="image/svg+xml" href="./favicon.svg" />
<link rel="manifest" href="./manifest.webmanifest" />
<link rel="apple-touch-icon" href="./icons/apple-touch-icon.png" />
<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
<link rel="manifest" href="/manifest.webmanifest" />
<link rel="apple-touch-icon" href="/icons/apple-touch-icon.png" />
<meta name="description" content="A private file toolkit that runs on your computer. Convert, edit and secure PDFs, images and documents without uploading them." />
<meta name="theme-color" content="#f2f2f3" media="(prefers-color-scheme: light)" />
<meta name="theme-color" content="#1d2d3d" media="(prefers-color-scheme: dark)" />
Expand Down
6 changes: 3 additions & 3 deletions apps/web/public/manifest.webmanifest
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
"name": "Ream: file toolkit",
"short_name": "Ream",
"description": "A private file toolkit that runs on your computer. Convert, edit and secure PDFs, images and documents without uploading them.",
"id": "./",
"start_url": "./#/tools",
"scope": "./",
"id": "/",
"start_url": "/tools/",
"scope": "/",
"display": "standalone",
"background_color": "#f2f2f3",
"theme_color": "#f2f2f3",
Expand Down
Binary file added apps/web/public/og.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
43 changes: 43 additions & 0 deletions apps/web/seoPlugin.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import type { OutputAsset } from 'rollup'
import type { Plugin } from 'vite'
import { tools } from '@ream/tools'
import { allRoutes, pathOf, type Route } from './src/routes'
import { headTags, pageSeo, sitemapXml, type PageSeo } from './src/seo'

/**
* Writes a page for every route (tool/merge-pdf/index.html and so on), a 404 page and sitemap.xml, from the built index.html:
* each has its own title, description, canonical address, social card tags and structured data in the head, and plain
* text inside #root that the app replaces when it starts. Search engines and link previews read these files; the app
* works the same from any of them.
*/
export const seoPages = (version: string): Plugin => ({
name: 'ream-seo',
apply: 'build',
generateBundle: {
order: 'post',
handler(_, bundle) {
const index = bundle['index.html'] as OutputAsset | undefined
if (!index) throw new Error('ream-seo: the build has no index.html to start from')
const template = String(index.source)
const page = (s: PageSeo) => {
const head = `<title>${escapeText(s.title)}</title>`
let html = template.replace(/<title>[^<]*<\/title>/, head).replace(/(<meta name="description" content=")[^"]*(")/, `$1${escapeAttr(s.description)}$2`)
html = html.replace('</head>', ` ${headTags(s)}\n </head>`).replace('<div id="root"></div>', `<div id="root">${s.html}</div>`)
if (html === template) throw new Error('ream-seo: the template has nothing to fill in')
return html
}
const emit = (fileName: string, source: string) => this.emitFile({ type: 'asset', fileName, source })
const routes: Route[] = allRoutes(tools)
for (const route of routes) {
const file = pathOf(route) === '/' ? 'index.html' : `${pathOf(route).slice(1)}index.html`
if (file === 'index.html') index.source = page(pageSeo(route, version))
else emit(file, page(pageSeo(route, version)))
}
emit('404.html', page(pageSeo({ page: 'notFound' }, version)))
emit('sitemap.xml', sitemapXml(routes, version))
},
},
})

const escapeText = (s: string) => s.replace(/&/g, '&amp;').replace(/</g, '&lt;')
const escapeAttr = (s: string) => escapeText(s).replace(/"/g, '&quot;')
Loading
Loading