Office documents in the portal: OnlyOffice Document Server extension

.github/workflows/test.yml

@@ -30,7 +30,10 @@ jobs:
       - run: npx playwright install --with-deps chromium
       # The extensions as the portal ships them, built: the tests list the ones in extensions/ by the halves they have.
       - run: npm run build:extensions
+      # OFFICE_IT_DOCKER=1 turns on 3 docker-gated office integration tests; they run in-process against a synthetic DS stand-in and only need a reachable docker CLI, which runners have.
       - run: npm test
+        env:
+          OFFICE_IT_DOCKER: "1"
 
   # The type check of the web app and its build, which is also what the image runs, the extensions' builds, and
   # the line between them and core (scripts/check-boundaries.mjs).

core/server/test/extension-ui.test.mjs

@@ -213,7 +213,7 @@ test("of two installed under one name, the one switched on is the one a plan fin
 
 test("what comes with the portal is listed to install from its own folder, without the subagent tool, which has a switch of its own in Settings → Tools", async () => {
   const { extensions } = await (await fetch(at("/api/extensions/shipped"))).json();
-  assert.deepEqual(extensions.map((e) => [e.name, e.kinds]), [["pithagoras-browser", ["pithagoras"]], ["pithagoras-canvas", ["pithagoras"]], ["pithagoras-creator", ["pithagoras"]], ["pithagoras-docker", ["pithagoras"]], ["pithagoras-docs", ["pithagoras"]], ["pithagoras-files", ["pithagoras"]], ["pithagoras-git", ["pithagoras"]], ["pithagoras-kanban", ["pithagoras"]], ["pithagoras-media", ["pithagoras"]], ["pithagoras-terminal", ["pithagoras"]], ["pithagoras-themes", ["themes"]], ["pithagoras-understory", ["pithagoras"]], ["pithagoras-voice", ["pithagoras"]], ["pithagoras-voice-servers", ["pithagoras"]]]);
+  assert.deepEqual(extensions.map((e) => [e.name, e.kinds]), [["pithagoras-browser", ["pithagoras"]], ["pithagoras-canvas", ["pithagoras"]], ["pithagoras-creator", ["pithagoras"]], ["pithagoras-docker", ["pithagoras"]], ["pithagoras-docs", ["pithagoras"]], ["pithagoras-files", ["pithagoras"]], ["pithagoras-git", ["pithagoras"]], ["pithagoras-kanban", ["pithagoras"]], ["pithagoras-media", ["pithagoras"]], ["pithagoras-office", ["pithagoras"]], ["pithagoras-terminal", ["pithagoras"]], ["pithagoras-themes", ["themes"]], ["pithagoras-understory", ["pithagoras"]], ["pithagoras-voice", ["pithagoras"]], ["pithagoras-voice-servers", ["pithagoras"]]]);
   for (const shipped of extensions) {
     assert.ok(path.isAbsolute(shipped.spec) && shipped.spec.endsWith(path.join("extensions", shipped.name.replace(/^pithagoras-/, ""))), shipped.spec);
     assert.ok(shipped.version && shipped.description, shipped.name);

extensions/office/.gitignore

@@ -0,0 +1,4 @@
+# Built by build.mjs (the portal's npm run build), not kept in git.
+/web/
+/server.js
+/node_modules/

extensions/office/README.md

@@ -0,0 +1,42 @@
+# pithagoras-office
+
+Office documents (.docx and .xlsx) edited in a real [OnlyOffice Document Server](https://github.com/ONLYOFFICE/Docker-DocumentServer) container that the portal runs of its own. The plan of record is `PLAN-office-280.md` (issue #280); this note says what ships in v1 and where everything lives.
+
+## What it is
+
+An opt-in package (it requires `pithagoras-docker`). In plain terms, v1 gives you:
+
+- **A document library** on the portal's Office tab: create blank Word/sheet documents, open them full-screen in the embedded OnlyOffice editor, sort the list by name / modified / size / type, and export (download) a copy of one. There is no manual import: every `.docx`/`.xlsx` found across the agent homes and projects (`portal().folders.list()`) is indexed **automatically at boot** (and on demand with the **Rescan workspace** button) — IN PLACE, so the scanned file IS the document and is edited through the same mtime-gated flow. Deleting a workspace-scanned document moves its real file into this library's trash, like everything else: reversible for 7 days.
+- **Agent tools** (`office_list`, `office_create`, `office_read`, `office_edit`, `office_comments`) registered in agent/project home chats. Agents read and edit the SAME files humans edit; writes are quote-based replacements gated on a revision number (the file's mtime), so a stale agent edit fails loudly instead of clobbering a human save.
+- **Co-editor model (no switch)**: the agent is just another user in the editor, like Office 365 — a human and the agent may edit the same document at the same time, and there is no setting to turn that on or off. Agent edits always proceed under the strict mtime/revision gate; if the file changed since they were read (for example a co-editor's save), NOTHING is written and the tool answers a loud `error:revision_mismatch` CONFLICT naming the open editor session when one stands — re-read and retry. Conflicts are surfaced, not prevented by refusal. Open editors learn about external changes through the existing 5 s polling loop (`live.ts pollState`). Every agent revision is recorded for the editor's version history.
+- **Wake on mention**: when a human comments `@agent` (the doc's owner) or `@<name>` / `@project:<name>` (a named agent/project) and saves, that home chat gets pinged with the quoted comment and gains an audited grant on the document.
+
+## Requirements
+
+- Docker on the host the portal runs from (shared networking: the DS container joins the same netns as the portal).
+- **~4 GB of RAM** for the Document Server container — the settings page warns before first start, and the status block shows live memory usage once it runs.
+
+## Setup
+
+1. Install/enable the package, then open Settings → office: enable it and press start. The pinned image (default `onlyoffice/documentserver:10.0` — never `:latest`; changing the tag can silently change callback/JWT behavior) is pulled on first start with progress shown in the UI.
+2. Documents appear under the Office tab: what you create here, plus every `.docx`/`.xlsx` found across the agent homes and projects — indexed automatically at boot (the scan runs after `settleKept`, so files that vanished while the portal slept are pruned, not re-added), with Rescan picking up anything written straight to disk afterwards. Open one to edit; agents work on it through their chat tools.
+
+## Where things live, and what is exposed
+
+All state is inside the extension's own data directory (`portal().dataDir()`): `settings.json`, `jwt-secret` (0600, created once), `ds.json` (container id + recorded origin), `index.json` + `docs/<docId>/<title>.<ext>` (the library), `.trash/`, `state/<docId>.json` (last-seen comment ids), and `history/<docId>/<rev>.zip` (change snapshots for the editor's version history).
+
+The DS container itself holds no session cookie, while every portal route is login-gated — so the two endpoints the container needs (a ticketed document GET and the callback POST) live on a **standalone loopback listener bound to 127.0.0.1 only**, on an ephemeral port minted per boot: `GET /doc/<ticket>` (HMAC + 10-minute expiry, no user context) and `POST /callback` (HS256-verified). Nothing else listens on it, and nothing outside the container's own network can reach it. The browser-facing half of the DS goes the other way: a **portal reverse proxy** (`GET /ds/<rest…>` under this extension's session-gated routes) forwards the editor traffic — api.js, the editor iframe, every asset — to the recorded DS origin, so remote browsers reach the editor through exactly the same URL prefix they use for the portal. That is why `document.url` and `editorConfig.callbackUrl` can stay absolute loopback URLs in the signed config: only the DS server itself fetches them.
+
+## Deferred (honest list)
+
+From the plan, intentionally NOT in v1:
+
+- **xlsx comments** (`xl/commentsN.xml` + VML layout) — reading and wakes for spreadsheet comments are M4; Word comments work now.
+- **Co-editing refresh handshake**: whether `DocEditor.refreshFile()` can be pushed programmatically was an M0 spike; v1 ships with the "agent updated — refresh" badge fallback. The M0 browser-reach question resolved to the portal reverse-proxy route (`/ds`, see Security notes) — the editor surface is same-origin.
+- `.pptx` reading, `office_comment_add`, per-comment grant expiry, postMessage live lock badges — M4 stretch items.
+- **Exporting into chat folders** — the export route serves a download of the stored bytes (fresh copy, served as an attachment); copying a document into a chat folder does not exist in v1.
+
+## Security notes
+
+- **The `/ds` proxy stays behind the session gate.** Browser editor traffic (api.js, iframe, assets) reaches the DS only through `GET /api/x/<extension>/ds/…`, mounted like every other route of this extension behind the portal's login — and it forwards to exactly one origin: the recorded DS origin in `ds.json` (`"container:<id>"` markers refuse with 422, an unreachable origin with 503). GET-only, no body, single destination.
+- **Host networking exposes the Document Server on every interface.** The DS container runs with the portal's shared network (`NetworkMode` from `docker.sharedNetwork()`; this deployment resolves it to `"host"`), so the image's own nginx serves the DS web origin (:80) on 0.0.0.0 — anything that can reach that machine's port 80 (the local host, the LAN, a tailnet peer) reaches the Document Server UI and API without going through any portal login gate: v1 has no authentication in front of that origin (the per-config JWTs sign requests the PORTAL mints; they are not an access-control layer for strangers). Operators should firewall :80 accordingly (e.g. reachable from the machine itself only), or override `/etc/nginx/conf.d` via a named volume to bind nginx to loopback — planned work, not something v1 does.

extensions/office/build.mjs

@@ -0,0 +1,27 @@
+/**
+ * Builds pithagoras-office: its browser half (src/client.tsx) into web/, with the Tailwind utilities its components use
+ * made into its own stylesheet from the portal's preset, and its server half (src/server.ts) into server.js. The
+ * portal's own `npm run build` runs this, so a checkout and the image build it the same way; what it makes is not kept
+ * in git.
+ *
+ * What it is built and runs with (react-icons for the browser half, jszip for the OOXML templates on the server
+ * side -- which deps.js ALSO installs at boot in the background, as pithagoras-docs does) is this package's own,
+ * installed here first when it is not there yet, or not at the version its lock file names (missingDependencies,
+ * shared with deps.js). esbuild, tailwindcss and postcss are the repository's.
+ */
+import { execFileSync } from "node:child_process";
+import path from "node:path";
+import { buildExtension } from "@pithagoras/extension-api/build";
+import { missingDependencies } from "./deps.js";
+
+const HERE = import.meta.dirname;
+
+// Missing, or not the versions the lock names (a later commit changed them): installed as the lock says.
+const missing = missingDependencies(HERE, { dev: true });
+if (missing.length) {
+  console.log(`pithagoras-office: installing its packages (${missing.length})`);
+  execFileSync(process.platform === "win32" ? "npm.cmd" : "npm", ["ci", "--ignore-scripts", "--no-audit", "--no-fund", "--prefer-offline"], { cwd: HERE, stdio: "inherit" });
+}
+
+await buildExtension({ dir: HERE, tailwind: true, server: "src/server.ts" });
+console.log("pithagoras-office: built web/ and server.js");

extensions/office/deps.d.ts

@@ -0,0 +1,4 @@
+/** Type surface of deps.js (its plain-JS twin beside this file): what the built server imports to install this
+ * package's own node_modules at boot, in the background. esbuild bundles the .js; tsc reads these declarations. */
+export function missingDependencies(dir?: string, opts?: { dev?: boolean }): string[];
+export function ensureDependencies(log: (line: string) => void, opts?: { dir?: string; npm?: string }): Promise<boolean>;

extensions/office/deps.js

@@ -0,0 +1,94 @@
+/**
+ * The extension's own dependencies, there before its server half needs them.
+ *
+ * pi installs a package by its path without installing what the package
+ * depends on, and the image installs them when it is built. A folder that
+ * comes without them (a checkout the portal runs from, a copy made by hand)
+ * has them installed here, once: jszip, for the minimal docx/xlsx templates
+ * its store writes, which the server bundle imports from its own node_modules
+ * at runtime (buildExtension leaves packages external there). The install is
+ * in the background, since a server half has a few seconds to start and an
+ * install takes longer.
+ */
+import { spawn } from "node:child_process";
+import { existsSync, readFileSync } from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const HERE = path.dirname(fileURLToPath(import.meta.url));
+
+function readJson(file) {
+  try {
+    return JSON.parse(readFileSync(file, "utf8"));
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * What the package in `dir` needs and does not have in its own node_modules as
+ * its lock file says: missing, or at another version (a later commit changed
+ * the lock, and nothing installed it since). Only what it needs to run, unless
+ * `dev`: then also what it is built with. Named as npm names them.
+ */
+export function missingDependencies(dir = HERE, { dev = false } = {}) {
+  const lock = readJson(path.join(dir, "package-lock.json"));
+  const wrong = new Set();
+  if (lock?.packages) {
+    for (const [where, entry] of Object.entries(lock.packages)) {
+      if (!where.startsWith("node_modules/") || entry.link || (entry.dev && !dev)) continue;
+      const installed = readJson(path.join(dir, where, "package.json"));
+      // An optional package is one npm leaves out where it does not fit (another system): only one that is there is checked.
+      if (!installed && entry.optional) continue;
+      if (installed?.version !== entry.version) wrong.add(where.slice(where.lastIndexOf("node_modules/") + "node_modules/".length));
+    }
+    return [...wrong];
+  }
+  // No lock: what the package names, there at all.
+  const meta = readJson(path.join(dir, "package.json")) ?? {};
+  return Object.keys({ ...meta.dependencies, ...(dev ? meta.devDependencies : {}) }).filter((name) => !existsSync(path.join(dir, "node_modules", name, "package.json")));
+}
+
+/**
+ * One install at a time per folder, for as long as the portal runs: kept for
+ * the process, since this module is read again with each revision of the
+ * server half, and an install a revision before started goes on after it.
+ */
+const installs = (globalThis[Symbol.for("pithagoras-office/installs")] ??= new Map());
+
+/**
+ * Installs what is missing or out of date with npm, as the lock file says, and says so in the
+ * portal's log. Resolves true once everything is there, false when the install
+ * failed (and a later start tries again).
+ */
+export function ensureDependencies(log, { dir = HERE, npm = process.platform === "win32" ? "npm.cmd" : "npm" } = {}) {
+  const missing = missingDependencies(dir);
+  if (!missing.length) return Promise.resolve(true);
+  if (installs.has(dir)) return installs.get(dir);
+  log(`Installing its dependencies (${missing.join(", ")}) with npm`);
+  const started = Date.now();
+  const install = new Promise((resolve) => {
+    // Only what it needs to run, and without install scripts: jszip is plain JS.
+    const child = spawn(npm, ["ci", "--omit=dev", "--ignore-scripts", "--no-audit", "--no-fund"], { cwd: dir, stdio: ["ignore", "ignore", "pipe"] });
+    let said = "";
+    child.stderr.on("data", (chunk) => (said = (said + chunk).slice(-2000)));
+    // A process that could not be started says so twice, as an error and as its end: it is told once.
+    let settled = false;
+    const done = (ok, why) => {
+      if (settled) return;
+      settled = true;
+      installs.delete(dir);
+      if (ok) log(`Installed its dependencies in ${Math.round((Date.now() - started) / 1000)} s`);
+      else log(`Could not install its dependencies: ${why}`);
+      resolve(ok);
+    };
+    child.on("error", (error) => done(false, error.message));
+    child.on("close", (code) => {
+      if (code !== 0) return done(false, `npm exited with ${code}${said.trim() ? `: ${said.trim().split("\n").slice(-3).join(" ")}` : ""}`);
+      const still = missingDependencies(dir);
+      done(!still.length, `still missing after npm: ${still.join(", ")}`);
+    });
+  });
+  installs.set(dir, install);
+  return install;
+}