Integrate wave 10 (sparql UPDATE, caldav free-busy, shortlink): 32 plugins, 363 tests

compose.test.js and serve.js boot all 32 (shortlink at /short, probed
in compose and on the dashboard). Docs folded: api.authorize gains its
4th consumer (caldav scheduling — Outbox→Inbox delivery is a write the
sender has no WAC right to make, loopback structurally can't express
it); the api.events entry gains two sharpenings (owning a write
endpoint doesn't buy a write-time index — plain-LDP writes bypass it;
a shared index must answer 'indexed under whose authority?'); the
conditional-write 'what worked' entry gains its measured caveat and a
5th filed-worthy bug — the host's If-Match check is check-then-write,
not atomic: two concurrent writers both told 2xx, one silently lost.
maxParamLength gains its 2nd consumer; new smaller note on
pluginDir-vs-pod state placement (no doctrine; 11 pluginDir witnesses).
REPORT.md updated throughout.

Author: melvincarvalho

AGENT.md

@@ -11,7 +11,7 @@ Out-of-tree plugins for [JavaScript Solid
 Server](https://github.com/JavaScriptSolidServer/JavaScriptSolidServer),
 built on its **#206 loader** (`createServer({ plugins })`, JSS ≥ 0.0.215).
 It is an *experiment*: prove the plugin api by using it, and treat every
-wall you hit as a finding, not a blocker. **31 plugins, 332 tests today.**
+wall you hit as a finding, not a blocker. **32 plugins, 363 tests today.**
 
 ### The one rule that makes the experiment valid
 
@@ -156,6 +156,7 @@ Full ranking in `NOTES.md`. The ones you'll hit:
 | a per-resource metadata/unfurl endpoint | `oembed/` | loopback resolve + local-URL-only guard |
 | a stateless request/response protocol (mail, sync, …) | `jmap/` | loopback CRUD + in-protocol refusal of push/delta |
 | a storage protocol with conditional writes | `remotestorage/` | If-Match/If-None-Match pass through loopback intact |
+| a redirect / metadata micro-service | `shortlink/` | pluginDir JSON table + wildcard slug routes |
 
 ## Footguns (every multi-boot suite rediscovered these)
 
@@ -177,7 +178,7 @@ Full ranking in `NOTES.md`. The ones you'll hit:
 
 ## Current state
 
-31 plugins (7 ports + 24 features), 332 tests, all green (`npm test`).
+32 plugins (7 ports + 25 features), 363 tests, all green (`npm test`).
 `compose.test.js` runs every one on a single server from pure config. Two
 core PRs (#590 `api.mountApp`, #591 `/idp/refresh`) sit upstream, unmerged,
 for the maintainer's call. Everything else lives here, by design.

ISSUES.md

@@ -21,7 +21,7 @@ evidence. Legend:
 | #382 / #379 | `corsproxy/` | CORS forward proxy, fail-closed SSRF defense |
 | #506 | `capability/` | scoped, time-bound, revocable capability URLs |
 | #507 | `webdav/` | mount a pod in Finder/Nautilus/Windows |
-| #509 | `sparql/` | read-time SPARQL over pod JSON-LD (write-index → 🔩 `api.events`) |
+| #509 | `sparql/` | SPARQL SELECT + UPDATE over pod JSON-LD (write-index → 🔩 `api.events`) |
 | #322 | `gitscratch/` | ephemeral Solid-authed git remotes (git-http-backend CGI) |
 | #515 / #516 | `mastodon/` | Mastodon-API shim — a client can log into its own pod (needs `appPaths` widened → 🔩) |
 | #505 | `otp/` | one-time-password session flow (account *recovery* → 🔩 core auth) |
@@ -42,6 +42,7 @@ evidence. Legend:
 | — | `oembed/` | oEmbed provider (discovery injection → 🔩 header/content hooks) |
 | — | `jmap/` | JMAP mail over the pod (push/delta → 🔩 `api.events`; blobs → 🔩 #583) |
 | #163 | `remotestorage/` | remoteStorage server — 7th port; witnessed the webfinger collision |
+| — | `shortlink/` | link shortener for pod URLs (pluginDir persistence, 11th witness) |
 
 Plus seven **ports of bundled features** proving the migration path for
 #564 / #164: `relay/` `webrtc/` `terminal/` `tunnel/` `notifications/`
@@ -105,11 +106,11 @@ exists.
 
 Of ~40 plugin-tagged issues: **14 built as plugins here** (plus `rss/`,
 `matrix/`, `search/`, `didweb/`, `s3/`, `micropub/`, `backup/`, `metrics/`,
-`dashboard/`, `oembed/`, `jmap/` — capability
+`dashboard/`, `oembed/`, `jmap/`, `shortlink/` — capability
 demonstrations with no single issue), **7 ported**, **5 shipped upstream**,
 **2 more plugin-able with no blocker**, **6 clusters blocked on a named
 seam** (each with a proof-of-need consumer), the rest core-by-nature or
-product-scale. **31 plugins total, 332 tests.** The plugin api reaches most
+product-scale. **32 plugins total, 363 tests.** The plugin api reaches most
 of the backlog today; ranked by demand, the seams that would unlock the
 most next are `api.authorize` (3 blocking consumers), `api.events` (7
 consumers — matrix `/sync` needs live push, jmap can't do push or delta

NEXT.md

@@ -8,15 +8,16 @@ plugin), `NOTES.md` (the findings/seams — the real deliverable), and
 
 ## Where things stand
 
-- **31 plugins, 332 tests, all green** (`npm test`), all pushed to
+- **32 plugins, 363 tests, all green** (`npm test`), all pushed to
   `github.com/JavaScriptSolidServer/plugins` (branch `gh-pages`).
-- `compose.test.js` boots all 31 on **one** JSS from pure config; `serve.js`
+- `compose.test.js` boots all 32 on **one** JSS from pure config; `serve.js`
   is the runnable demo. Both must be updated when you add a plugin.
 - Built so far: 7 ports (relay, webrtc, terminal, tunnel, notifications,
-  remotestorage, pay) + 24 features (nip05, corsproxy, capability, webdav,
-  sparql, gitscratch, otp, carddav, mastodon, bluesky, caldav, webfinger,
+  remotestorage, pay) + 25 features (nip05, corsproxy, capability, webdav,
+  sparql — now with UPDATE, gitscratch, otp, carddav, mastodon, bluesky,
+  caldav — now with free-busy, webfinger,
   activitypub, rss, matrix, search, didweb, s3, micropub, backup,
-  metrics, dashboard, oembed, jmap).
+  metrics, dashboard, oembed, jmap, shortlink).
 - Capability classes covered: realtime, WebDAV family, fediverse/social/chat
   (5 shims), IndieWeb publishing, identity, data/query/search, object
   storage, proxy, dev, pay, data portability, ops/observability, mail,
@@ -76,10 +77,10 @@ Still genuinely plugin-shaped and distinct:
 - **feed ingest** — subscribe to external RSS/Atom, store items in the pod
   (the inverse of `rss/`; uses a corsproxy-style fetch — outbound fetches:
   same careful-session caveat as webhooks/webmention).
-- **link-shortener**, **WebSub/PubSubHubbub** (needs `api.events` +
-  outbound POSTs — deferred),
-  **SPARQL UPDATE** (extends `sparql/`), **CalDAV scheduling/free-busy**
-  (extends `caldav/`).
+- **WebSub/PubSubHubbub** (needs `api.events` + outbound POSTs — deferred).
+- **CalDAV scheduling (RFC 6638)** — blocked on cross-user delivery
+  (api.authorize's issuer-authority case, or a deliver-to-inbox
+  primitive); free-busy is done.
 - **Bluesky/Mastodon/Matrix Phase-2** (federation, `/sync` live push) —
   these are blocked on `api.events` + `api.reservePath`; good once those
   seams exist, otherwise document the wall.
@@ -94,12 +95,14 @@ it sharpens the case.
 `NOTES.md` ranks candidate seams by how many independent plugins demanded
 each. Current top four (keep this current as you add consumers):
 
-1. **`api.authorize(request, path, mode)`** — 3 consumers; the top
+1. **`api.authorize(request, path, mode)`** — 4 consumers (caldav
+   scheduling joined); the top
    *blocking* seam (authority the requester doesn't drive).
-2. **`api.events.onResourceChange`** — 5 consumers (backup/ made it 5:
-   incremental/scheduled backup is unbuildable); matrix `/sync` needs
-   live push. Every "react to writes" plugin (webhooks, WebSub, indexing)
-   will want it — webhooks/ would make it 6.
+2. **`api.events.onResourceChange`** — 7 consumers (backup, jmap,
+   remotestorage joined); matrix `/sync` needs
+   live push, and sparql/'s UPDATE proved owning a write endpoint does
+   not buy a write-time index. Every "react to writes" plugin (webhooks,
+   WebSub, indexing) will want it.
 3. **`api.reservePath`** — every API-shim owns fixed roots outside its one
    prefix and can't self-exempt; didweb needs a *parameterized* form.
    (micropub/ is the counter-witness: client-discovered endpoints need no

NOTES.md

@@ -24,25 +24,31 @@ are candidates, each with a consumer in this repo attached.
   correct. This removes a whole class of would-be seams (`api.wac.check`)
   from the *necessary* list, leaving them merely *nice*.
 - **Conditional writes pass through loopback intact** (remotestorage/,
-  measured): `If-Match`/`If-None-Match` forwarded verbatim are honored by
+  measured; sparql/ is the second consumer): `If-Match`/`If-None-Match`
+  forwarded verbatim are honored by
   the host end-to-end — stale `If-Match` PUT/DELETE → 412,
   `If-None-Match: *` on an existing resource → 412 with the body proven
-  not to land, `If-None-Match` GET → 304. Atomic at the host; no
-  plugin-side conditional logic needed.
+  not to land, `If-None-Match` GET → 304. No plugin-side conditional
+  logic needed for the stale-writer case. **Caveat measured by sparql/:**
+  the host's check is check-then-write, not atomic — see the bugs section.
 
 ## Candidate seams (ranked by how many independent plugins demanded them)
 
 Twelve plugins in, the ranking is now empirical — a seam's rank is how many
 ports reached for it without coordinating.
 
 1. **`api.authorize(request, path, mode)`** — "would the host's WAC allow
-   this?" **Three independent consumers: notifications/, corsproxy/,
-   capability/.** The loopback trick (below) covers the case where the
-   *requester's own* credentials should decide (notifications, webdav,
+   this?" **Four independent consumers: notifications/, corsproxy/,
+   capability/, caldav/.** The loopback trick (below) covers the case where
+   the *requester's own* credentials should decide (notifications, webdav,
    sparql all use it), but it can't cover authorization the requester
    doesn't drive: a proxy governed by a *pod owner's* `.acl` (corsproxy
-   #382), or a capability exercising the *issuer's* authority
-   (capability #506). This is the most-requested seam and the one that
+   #382), a capability exercising the *issuer's* authority
+   (capability #506), or CalDAV scheduling (RFC 6638), where Outbox→Inbox
+   delivery is a write into the *recipient's* pod the sender has no WAC
+   right to make — loopback structurally cannot express it (a
+   server-mediated deliver-to-inbox primitive is the narrower
+   alternative). This is the most-requested seam and the one that
    moves the most backlog issues from "plugin-approximation" to "faithful".
 2. **`api.events.onResourceChange(cb)`** — **seven consumers now:
    notifications/, sparql/, search/, matrix/, backup/, jmap/,
@@ -60,6 +66,13 @@ ports reached for it without coordinating.
    (`eventSourceUrl`) and delta sync (`*/changes`, `Email/queryChanges`)
    are omitted/refused in-protocol), and remotestorage (rS's
    descendant-version propagation at depth ≥ 2 needs a write-time index).
+   sparql/'s UPDATE work sharpened the whole entry: **owning a write
+   endpoint does not buy the write-time index** — a plugin could index
+   its own writes synchronously, but plain-LDP writes bypass it, yielding
+   a silently-wrong index (worse than none); only the host's event stream
+   sees all writes. caldav/'s free-busy added a wrinkle: a shared
+   write-time index must answer "indexed under whose authority?" or it
+   leaks WAC-hidden data.
    Core already has the
    emitter internally (`src/notifications/events.js`); this is the seam
    every "react to pod writes" app (webhooks, indexing, sync, search, live
@@ -132,9 +145,10 @@ ports reached for it without coordinating.
    endpoints are client-discovered rather than protocol-fixed — the seam
    is specifically about protocols that pin absolute paths, not API shims
    per se.
-6. **Can't set fastify server options** — consumer: capability/ hit
+6. **Can't set fastify server options** — consumers: capability/ hit
    `maxParamLength` (100) silently 404ing long tokens in named params;
-   workaround is a wildcard route. A plugin has no way to raise per-route
+   shortlink/ dodged it pre-emptively the same way. Workaround is a
+   wildcard route. A plugin has no way to raise per-route
    limits. Minor, but sharp when it bites.
 7. **Internal utility modules plugins re-vendor** — consumer: relay/
    (`src/nostr/event.js` NIP-01 verify), pay/ (`src/mrc20.js`). Pure crypto.
@@ -233,6 +247,12 @@ this repo is its proof.
   candidate: when the basename is generic (`plugin`, `index`), derive from
   the parent directory (`relay/plugin.js` → `relay`). Small, backward-
   compatible, removes the most common footgun. **Filed-worthy.**
+- **The host's conditional write is check-then-write, not atomic**
+  (sparql/, measured live): two *concurrent* PUTs carrying the same
+  currently-valid `If-Match` both return 2xx and the loser is silently
+  overwritten; the same two PUTs run sequentially 412 correctly. Two
+  simultaneous writers can lose one with both told success — unfixable
+  plugin-side; the host needs an atomic stat+write. **Filed-worthy.**
 - **The loader's activate-failure wrap drops `err.code`**
   (remotestorage/): a route collision inside `activate` surfaces as
   `plugin <id>: activate() failed: <message>` with the original
@@ -280,3 +300,10 @@ this repo is its proof.
   exported (loopback enforces Control for free) but absolute-URL ACLs
   don't restore elsewhere — policy portability is a spec-level gap, not a
   plugin-api gap.
+- **pluginDir vs pod resources — no doctrine** (shortlink/): stateful
+  plugins choose between `pluginDir` (fast, private, invisible to WAC,
+  backup, and portability) and pod resources (WAC-governed, portable,
+  but a loopback round-trip per operation). The api offers one primitive
+  and no guidance; a design note in the plugin docs would spare every
+  author the same deliberation. Eleven plugins now persist in
+  `pluginDir` — the most settled seam in the api.

README.md

@@ -69,13 +69,13 @@ README and NOTES.md, ship the closest honest approximation.
 | `corsproxy/` | #382/#379 | forward proxy with fail-closed SSRF defense; per-pod ACL needs `api.authorize` |
 | `capability/` | #506 | macaroon-lite scoped/time-bound/revocable capability URLs |
 | `webdav/` | #507 | mount a pod in Finder/Nautilus/Windows; WebDAV↔LDP over loopback, Basic→Bearer |
-| `sparql/` | #509 | read-time SPARQL SELECT over pod JSON-LD; write-index needs `api.events` |
+| `sparql/` | #509 | SPARQL SELECT + UPDATE over pod JSON-LD; write-index still needs `api.events` |
 | `gitscratch/` | #322 | ephemeral Solid-authed git remotes via the `git-http-backend` CGI |
 | `otp/` | #505 | one-time-password session flow; account recovery needs core auth |
 | `carddav/` | #157 | contact sync (iOS/macOS/Thunderbird/DAVx5); the DAV bridge, generalized |
 | `mastodon/` | #515/#516 | Mastodon-API shim — a client logs into its own pod; needs `appPaths` widened |
 | `bluesky/` | #211 | AT-Protocol XRPC shim — same shape, `/xrpc` root; token bridge generalizes |
-| `caldav/` | #157-sib | calendar sync — completes the DAV family (webdav+carddav+caldav) |
+| `caldav/` | #157-sib | calendar sync + free-busy — completes the DAV family (webdav+carddav+caldav) |
 | `webfinger/` | #164 | `/.well-known/webfinger` — fediverse `acct:` resolution for the shims |
 | `activitypub/` | #51/#164 | federate a pod as an AS2 actor; outbound HTTP Signatures |
 | `rss/` | — | any pod container as an Atom/RSS feed |
@@ -89,9 +89,10 @@ README and NOTES.md, ship the closest honest approximation.
 | `dashboard/` | — | live plugin-status page; first consumer of the #463/#464 registry seam |
 | `oembed/` | — | oEmbed provider for pod resources (link unfurling; XSS-refusing) |
 | `jmap/` | — | JMAP mail over pod messages — the stateless/push line, sharpened |
+| `shortlink/` | — | link shortener for pod URLs (deliberately not an open redirector) |
 
-**332 tests, all green** (`npm test`), including `compose.test.js` — all
-thirty-one plugins on one server from pure config, pods + WAC intact
+**363 tests, all green** (`npm test`), including `compose.test.js` — all
+thirty-two plugins on one server from pure config, pods + WAC intact
 beside them. Findings consolidated in [NOTES.md](./NOTES.md); the full
 plugin-tagged backlog triaged in [ISSUES.md](./ISSUES.md); the build guide
 is [AGENT.md](./AGENT.md); the maintainer-facing summary — what to add

REPORT.md

@@ -1,4 +1,4 @@
-# The plugin api, 31 plugins later — a report for the maintainer
+# The plugin api, 32 plugins later — a report for the maintainer
 
 This document is the actionable summary of the whole experiment: what the
 #206 plugin api can already do, what it can't, and — ranked with evidence —
@@ -14,7 +14,7 @@ want it.
 ## Executive summary
 
 - The api as shipped in 0.0.215 (`createServer({ plugins })` + `prefix` +
-  `getAgent` + `pluginDir` + `ws.route`) is **sufficient for 31 real
+  `getAgent` + `pluginDir` + `ws.route`) is **sufficient for 32 real
   plugins across fifteen capability classes** — realtime, DAV,
   fediverse/chat shims, IndieWeb publishing, identity, query/search,
   object storage, proxy, dev tooling, pay, data portability,
@@ -30,10 +30,11 @@ want it.
   `api.events.onResourceChange`, `api.reservePath`, and `api.serverInfo`.
   Adding the first three moves essentially every remaining plugin-shaped
   issue from "honest approximation" to "faithful implementation".
-- Four small loader/core bugs are worth fixing regardless
+- Five small loader/core bugs are worth fixing regardless
   (generic-basename id derivation; dotted-prefix WS validation;
   `logger: false` breaking every plugin `onResponse` hook; the loader's
-  error wrap dropping `err.code`).
+  error wrap dropping `err.code`; the host's conditional write being
+  check-then-write rather than atomic).
 - One measured surprise: **plugins are not isolated from each other** —
   all entries share one Fastify register scope, so any plugin can hook
   every other plugin's routes (never core's). Worth a deliberate
@@ -62,9 +63,11 @@ Validated by use, not opinion:
    *not* on the ask list — only the cases loopback structurally can't
    cover are.
 6. **Conditional writes survive loopback intact** (measured by
-   `remotestorage/`): forwarded `If-Match`/`If-None-Match` are honored by
-   the host end-to-end (412 on stale, 304 on match, atomic). A protocol
-   that needs optimistic concurrency gets it for free.
+   `remotestorage/`; `sparql/`'s UPDATE is the second consumer):
+   forwarded `If-Match`/`If-None-Match` are honored by the host
+   end-to-end (412 on stale, 304 on match). A protocol that needs
+   optimistic concurrency gets the stale-writer case for free — but see
+   bug 5: under true concurrency the check is not atomic.
 
 ## The seams to add, ranked by independent demand
 
@@ -77,7 +80,8 @@ entry is written as a fileable issue: motivation, consumers, sketch, cost.
 *requester's credentials don't drive*.
 
 **Why loopback can't cover it:** loopback answers "may *this caller* read
-X?". Three plugins need "may *someone else* — a pod owner, a token issuer —
+X?". Four plugins need "may *someone else* — a pod owner, a token issuer,
+a message recipient —
 authorize this?":
 
 - `corsproxy/` (#382): per-pod proxy ACLs — the *pod owner's* `.acl`
@@ -87,6 +91,10 @@ authorize this?":
   can only enforce its own token scopes, not re-check the issuer's current
   WAC rights (revocation by ACL edit doesn't propagate).
 - `pay/`: the 402 gate wants to compose with WAC instead of replacing it.
+- `caldav/` (RFC 6638 scheduling): Outbox→Inbox delivery is a write into
+  the *recipient's* pod the sender has no WAC right to make — loopback
+  structurally cannot express it. (A narrower server-mediated
+  deliver-to-inbox primitive would also cover this one.)
 
 **Sketch:** `api.authorize({ agent, path, mode }) → Promise<boolean>` —
 agent id (not request) in, decision out, same engine the LDP path uses.
@@ -197,7 +205,7 @@ probe-port-then-boot dance for the same reason.
   Deliberately *not* asked for as a default-on hook: it's a bigger grant
   than route ownership. If ever, gate it: `capabilities: ['hooks']`.
 
-## Four loader/core bugs worth fixing regardless
+## Five loader/core bugs worth fixing regardless
 
 1. **Generic-basename id derivation.** Every plugin follows
    `<name>/plugin.js`, so every derived id is `plugin` and the (correct)
@@ -219,6 +227,12 @@ probe-port-then-boot dance for the same reason.
    failed: <message>` with `FST_ERR_DUPLICATED_ROUTE` stripped — callers
    can only string-match. Preserve `code` (or use `cause`) when
    re-wrapping (found by `remotestorage/`).
+5. **The conditional write is check-then-write, not atomic** (found live
+   by `sparql/`): two *concurrent* PUTs carrying the same currently-valid
+   `If-Match` both return 2xx and one is silently overwritten; the same
+   PUTs run sequentially 412 correctly. Both callers are told success but
+   one write is gone. Unfixable from a plugin; the LDP write path needs
+   an atomic stat+write.
 
 ## A measured surprise: plugins are not isolated from each other