Skip to content

Commit f19ab08

Browse files
Integrate wave 9 (oembed, jmap, remotestorage): 31 plugins, 332 tests
compose.test.js and serve.js boot all 31; remotestorage runs with claimWellKnown: false beside webfinger/ — the composed deployment living the collision finding it witnessed. Docs folded: api.events is now 7 hard consumers (jmap push/delta, remotestorage descendant versions; oembed a soft 8th via write-time dimension caching); api.serverInfo ~16; the header-injection seam gains its sharpest consumer (oembed per-resource <link> — content rewriting even a gated header hook can't do); the reservePath entry upgrades from predicted to WITNESSED collision (loud failure one order, silent link loss the other) with the link/JRD-registry seam as the real fix; a 4th filed-worthy bug (loader error wrap drops err.code); and a new 'what worked' entry — conditional writes (If-Match/If-None-Match) pass through loopback intact, measured. NOTES #564 table gains the remoteStorage row (7th port; core's bundled copy could move out-of-tree).
1 parent 712c5e9 commit f19ab08

8 files changed

Lines changed: 174 additions & 59 deletions

File tree

AGENT.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Out-of-tree plugins for [JavaScript Solid
1111
Server](https://github.com/JavaScriptSolidServer/JavaScriptSolidServer),
1212
built on its **#206 loader** (`createServer({ plugins })`, JSS ≥ 0.0.215).
1313
It is an *experiment*: prove the plugin api by using it, and treat every
14-
wall you hit as a finding, not a blocker. **28 plugins, 282 tests today.**
14+
wall you hit as a finding, not a blocker. **31 plugins, 332 tests today.**
1515

1616
### The one rule that makes the experiment valid
1717

@@ -153,6 +153,9 @@ Full ranking in `NOTES.md`. The ones you'll hit:
153153
| a data-export / archive download | `backup/` | loopback container walk streamed into a hand-rolled format |
154154
| an ops/observability endpoint | `metrics/` | node builtins + an `api.fastify` hook (scope: all plugins, never core) |
155155
| a meta/status page over siblings | `dashboard/` | anonymous loopback probes + an operator-declared list (no registry) |
156+
| a per-resource metadata/unfurl endpoint | `oembed/` | loopback resolve + local-URL-only guard |
157+
| a stateless request/response protocol (mail, sync, …) | `jmap/` | loopback CRUD + in-protocol refusal of push/delta |
158+
| a storage protocol with conditional writes | `remotestorage/` | If-Match/If-None-Match pass through loopback intact |
156159
157160
## Footguns (every multi-boot suite rediscovered these)
158161
@@ -174,7 +177,7 @@ Full ranking in `NOTES.md`. The ones you'll hit:
174177
175178
## Current state
176179
177-
28 plugins (6 ports + 22 features), 282 tests, all green (`npm test`).
180+
31 plugins (7 ports + 24 features), 332 tests, all green (`npm test`).
178181
`compose.test.js` runs every one on a single server from pure config. Two
179182
core PRs (#590 `api.mountApp`, #591 `/idp/refresh`) sit upstream, unmerged,
180183
for the maintainer's call. Everything else lives here, by design.

ISSUES.md

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,13 @@ evidence. Legend:
3939
|| `backup/` | pod → `.tar.gz` export (incremental backup → 🔩 `api.events`) |
4040
|| `metrics/` | `/healthz` + Prometheus exporter (core-pipeline metrics → 🔩 gated hooks) |
4141
|| `dashboard/` | plugin status page (sibling discovery → 🔩 app-registry #463/#464) |
42+
|| `oembed/` | oEmbed provider (discovery injection → 🔩 header/content hooks) |
43+
|| `jmap/` | JMAP mail over the pod (push/delta → 🔩 `api.events`; blobs → 🔩 #583) |
44+
| #163 | `remotestorage/` | remoteStorage server — 7th port; witnessed the webfinger collision |
4245

43-
Plus six **ports of bundled features** proving the migration path for #564 /
44-
#164: `relay/` `webrtc/` `terminal/` `tunnel/` `notifications/`, and `pay/`
46+
Plus seven **ports of bundled features** proving the migration path for
47+
#564 / #164: `relay/` `webrtc/` `terminal/` `tunnel/` `notifications/`
48+
`remotestorage/`, and `pay/`
4549
(the wall-report — it can't be a plugin, which is the point).
4650

4751
## Shipped upstream this line of work 🚢
@@ -99,15 +103,17 @@ exists.
99103

100104
## Tally
101105

102-
Of ~40 plugin-tagged issues: **13 built as plugins here** (plus `rss/`,
106+
Of ~40 plugin-tagged issues: **14 built as plugins here** (plus `rss/`,
103107
`matrix/`, `search/`, `didweb/`, `s3/`, `micropub/`, `backup/`, `metrics/`,
104-
`dashboard/` — capability
105-
demonstrations with no single issue), **6 ported**, **5 shipped upstream**,
108+
`dashboard/`, `oembed/`, `jmap/` — capability
109+
demonstrations with no single issue), **7 ported**, **5 shipped upstream**,
106110
**2 more plugin-able with no blocker**, **6 clusters blocked on a named
107111
seam** (each with a proof-of-need consumer), the rest core-by-nature or
108-
product-scale. **28 plugins total, 282 tests.** The plugin api reaches most
112+
product-scale. **31 plugins total, 332 tests.** The plugin api reaches most
109113
of the backlog today; ranked by demand, the seams that would unlock the
110-
most next are `api.authorize` (3 blocking consumers), `api.events` (5
111-
consumers — matrix `/sync` needs live push, backup can only pull-on-demand),
112-
`api.reservePath` (API-shims + didweb's parameterized case), and
113-
`api.serverInfo` (broadest: ~12 plugins hand-roll their own origin).
114+
most next are `api.authorize` (3 blocking consumers), `api.events` (7
115+
consumers — matrix `/sync` needs live push, jmap can't do push or delta
116+
sync, backup can only pull-on-demand),
117+
`api.reservePath` (API-shims + didweb's parameterized case + the
118+
**witnessed** webfinger collision), and
119+
`api.serverInfo` (broadest: ~16 plugins hand-roll their own origin).

NEXT.md

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,19 @@ plugin), `NOTES.md` (the findings/seams — the real deliverable), and
88

99
## Where things stand
1010

11-
- **28 plugins, 282 tests, all green** (`npm test`), all pushed to
11+
- **31 plugins, 332 tests, all green** (`npm test`), all pushed to
1212
`github.com/JavaScriptSolidServer/plugins` (branch `gh-pages`).
13-
- `compose.test.js` boots all 28 on **one** JSS from pure config; `serve.js`
13+
- `compose.test.js` boots all 31 on **one** JSS from pure config; `serve.js`
1414
is the runnable demo. Both must be updated when you add a plugin.
15-
- Built so far: 6 ports (relay, webrtc, terminal, tunnel, notifications,
16-
pay) + 22 features (nip05, corsproxy, capability, webdav, sparql,
17-
gitscratch, otp, carddav, mastodon, bluesky, caldav, webfinger,
15+
- Built so far: 7 ports (relay, webrtc, terminal, tunnel, notifications,
16+
remotestorage, pay) + 24 features (nip05, corsproxy, capability, webdav,
17+
sparql, gitscratch, otp, carddav, mastodon, bluesky, caldav, webfinger,
1818
activitypub, rss, matrix, search, didweb, s3, micropub, backup,
19-
metrics, dashboard).
19+
metrics, dashboard, oembed, jmap).
2020
- Capability classes covered: realtime, WebDAV family, fediverse/social/chat
2121
(5 shims), IndieWeb publishing, identity, data/query/search, object
22-
storage, proxy, dev, pay, data portability, ops/observability.
22+
storage, proxy, dev, pay, data portability, ops/observability, mail,
23+
link-embeds, remoteStorage.
2324
- **REPORT.md exists** — the maintainer-facing summary (ranked seams, each
2425
fileable nearly verbatim). Keep its consumer counts current as plugins
2526
land.
@@ -72,13 +73,13 @@ Verify each with `node --test --test-concurrency=1 <name>/test.js`, then:
7273
## Wave 8+ backlog (unbuilt ideas, roughly by value)
7374

7475
Still genuinely plugin-shaped and distinct:
75-
- **JMAP** email API over pod messages (HTTP/JSON, unlike stateful IMAP).
76-
- **oEmbed** provider for pod resources (link unfurling).
7776
- **feed ingest** — subscribe to external RSS/Atom, store items in the pod
78-
(the inverse of `rss/`; uses a corsproxy-style fetch).
79-
- **link-shortener**, **WebSub/PubSubHubbub** (needs `api.events`),
77+
(the inverse of `rss/`; uses a corsproxy-style fetch — outbound fetches:
78+
same careful-session caveat as webhooks/webmention).
79+
- **link-shortener**, **WebSub/PubSubHubbub** (needs `api.events` +
80+
outbound POSTs — deferred),
8081
**SPARQL UPDATE** (extends `sparql/`), **CalDAV scheduling/free-busy**
81-
(extends `caldav/`), **remoteStorage** (another bundled-feature port).
82+
(extends `caldav/`).
8283
- **Bluesky/Mastodon/Matrix Phase-2** (federation, `/sync` live push) —
8384
these are blocked on `api.events` + `api.reservePath`; good once those
8485
seams exist, otherwise document the wall.

NOTES.md

Lines changed: 45 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,12 @@ are candidates, each with a consumer in this repo attached.
2323
client's own credentials. Slower than an internal check but definitionally
2424
correct. This removes a whole class of would-be seams (`api.wac.check`)
2525
from the *necessary* list, leaving them merely *nice*.
26+
- **Conditional writes pass through loopback intact** (remotestorage/,
27+
measured): `If-Match`/`If-None-Match` forwarded verbatim are honored by
28+
the host end-to-end — stale `If-Match` PUT/DELETE → 412,
29+
`If-None-Match: *` on an existing resource → 412 with the body proven
30+
not to land, `If-None-Match` GET → 304. Atomic at the host; no
31+
plugin-side conditional logic needed.
2632

2733
## Candidate seams (ranked by how many independent plugins demanded them)
2834

@@ -38,8 +44,10 @@ ports reached for it without coordinating.
3844
#382), or a capability exercising the *issuer's* authority
3945
(capability #506). This is the most-requested seam and the one that
4046
moves the most backlog issues from "plugin-approximation" to "faithful".
41-
2. **`api.events.onResourceChange(cb)`****five consumers now:
42-
notifications/, sparql/, search/, matrix/, backup/**, in rising
47+
2. **`api.events.onResourceChange(cb)`****seven consumers now:
48+
notifications/, sparql/, search/, matrix/, backup/, jmap/,
49+
remotestorage/** (plus oembed/ as a soft eighth: a write-time image-
50+
dimension cache would replace per-unfurl byte parsing), in rising
4351
sharpness: notifications (a miss is a *late* notification), sparql (a
4452
miss is a *wrong* query result), search (a miss is *stale results*, the
4553
property users most expect to be fresh — the most user-visible
@@ -48,15 +56,19 @@ ports reached for it without coordinating.
4856
— so a stateless bridge can only do full-state `/sync` at all), and
4957
backup (with no change hook *and* no plugin-owned read authority,
5058
incremental and scheduled/server-initiated backup are both unbuildable —
51-
every backup is a caller-driven full crawl). Core already has the
59+
every backup is a caller-driven full crawl), jmap (JMAP push
60+
(`eventSourceUrl`) and delta sync (`*/changes`, `Email/queryChanges`)
61+
are omitted/refused in-protocol), and remotestorage (rS's
62+
descendant-version propagation at depth ≥ 2 needs a write-time index).
63+
Core already has the
5264
emitter internally (`src/notifications/events.js`); this is the seam
5365
every "react to pod writes" app (webhooks, indexing, sync, search, live
5466
chat) will want, and it's now clearly the #2 most-demanded after
5567
`api.authorize`.
5668
3. **`api.serverInfo` (`{ baseUrl, port }` at listen)****a dozen+
5769
consumers: notifications/, webdav/, carddav/, caldav/, sparql/, rss/,
5870
nip05/, webfinger/, mastodon/, bluesky/, activitypub/, micropub/,
59-
backup/** — essentially
71+
backup/, dashboard/, oembed/, jmap/, remotestorage/** — essentially
6072
every plugin that mints absolute URLs or reaches the host over loopback.
6173
All repeat the origin in config today; a wrong value fails quietly (nip05
6274
serves an empty identity map). The single most *broadly* needed seam
@@ -65,7 +77,9 @@ ports reached for it without coordinating.
6577
gitscratch/ sharpened it; micropub/ adds a blocked one. tunnel/ needed
6678
the raw *buffer*; git needs the raw *stream* piped to a subprocess
6779
gzip-and-all; micropub's **media endpoint** (multipart file upload) is
68-
simply not implemented until a plugin can pipe an un-drained body.
80+
simply not implemented until a plugin can pipe an un-drained body;
81+
jmap/ likewise omits blobs/attachments (`uploadUrl` absent,
82+
`maxSizeUpload: 0` advertised honestly).
6983
Whatever `api.mountApp` / raw-body mode ships must hand back the
7084
un-drained stream, not just a buffered body. (This is exactly what the
7185
merged loader's scoped pass-through parser does — the finding is to
@@ -97,10 +111,18 @@ ports reached for it without coordinating.
97111
an API shim, did:web can't escape to a fake root (the DID id is fixed
98112
by the method). It works only where the pod grants public Read. This is
99113
the sharpest form and needs a **parameterized** `api.reservePath`.
100-
- none of it has conflict detection: a future core route at a
101-
plugin-claimed path throws `FST_ERR_DUPLICATED_ROUTE` at boot (and a
102-
link registry — `api.webfinger.addLink` — is missing, so two plugins
103-
contributing `.well-known/webfinger` links would collide silently).
114+
- conflict detection: **the collision is now WITNESSED, both ways**
115+
(remotestorage/ + webfinger/ both own `/.well-known/webfinger`).
116+
Unguarded claim second → boot fails: `plugin remotestorage:
117+
activate() failed: Method 'GET' already declared for route
118+
'/.well-known/webfinger'` (Fastify's `FST_ERR_DUPLICATED_ROUTE`, but
119+
the loader's error wrap **drops `err.code`** — only the message
120+
identifies it). Reverse order → boot *succeeds* and webfinger/'s
121+
try/catch-guarded claim **silently loses**: its profile/actor/issuer
122+
links vanish with no error. Loud failure or silent loss — both wrong,
123+
because both plugins legitimately own *parts* of one discovery
124+
document. The missing seam is a link/JRD registry
125+
(`api.webfinger.addLink`).
104126
The seam: `api.reservePath('/xrpc')` / `api.reservePath('/:user/did.json')`
105127
(or `paths: [...]` in the entry) — the loader exempts *and* claims each
106128
deliberately and reports collisions. The seam **every API-shim plugin**
@@ -118,11 +140,16 @@ ports reached for it without coordinating.
118140
(`src/nostr/event.js` NIP-01 verify), pay/ (`src/mrc20.js`). Pure crypto.
119141
Export like auth.js (`javascript-solid-server/nostr.js`) or bless
120142
vendoring.
121-
8. **Response-header injection on core routes****two consumers:
122-
notifications/** (`Updates-Via` discovery) and **micropub/** (clients
143+
8. **Response-header injection on core routes****three consumers:
144+
notifications/** (`Updates-Via` discovery), **micropub/** (clients
123145
find the endpoint via `<link rel="micropub">` on the user's homepage —
124146
a core-owned resource the plugin can't decorate; the operator must
125-
advertise it by hand). A plugin can't add headers to routes it doesn't
147+
advertise it by hand), and **oembed/** — the sharpest form: oEmbed
148+
discovery wants a per-resource `<link>` in *every HTML resource's
149+
head*, which even a gated header hook couldn't provide (in-HTML
150+
injection is content rewriting — core's side of the #564 line; the
151+
`Link:` header variant would cover the rest). A plugin can't add
152+
headers to routes it doesn't
126153
own. NOT a default-on hook (bigger grant than route ownership); gate
127154
behind `capabilities: ['hooks']` if ever.
128155
9. **Plugin-to-plugin isolation is ZERO (measured — metrics/).** All
@@ -173,6 +200,7 @@ MODIFIES the request pipeline of routes it doesn't own.**
173200

174201
| Feature | Shape | Verdict | Evidence |
175202
|---|---|---|---|
203+
| remoteStorage | owns `/storage/:user/*` | ✅ plugin | remotestorage/ — 7th port; core's bundled copy could move out-of-tree |
176204
| nostr relay | owns `/relay` ws | ✅ plugin | relay/ — parity + persistence |
177205
| webrtc | owns `/webrtc` ws | ✅ plugin | webrtc/ — full parity, zero imports |
178206
| terminal | owns a ws | ✅ plugin | terminal/ — hardened beyond core |
@@ -205,6 +233,11 @@ this repo is its proof.
205233
candidate: when the basename is generic (`plugin`, `index`), derive from
206234
the parent directory (`relay/plugin.js``relay`). Small, backward-
207235
compatible, removes the most common footgun. **Filed-worthy.**
236+
- **The loader's activate-failure wrap drops `err.code`**
237+
(remotestorage/): a route collision inside `activate` surfaces as
238+
`plugin <id>: activate() failed: <message>` with the original
239+
`FST_ERR_DUPLICATED_ROUTE` code stripped — callers can only string-match
240+
the message. Preserve `code` (or `cause`) when re-wrapping.
208241
- **`logger: false` silently kills every plugin `onResponse` hook**
209242
(metrics/): core's access-log hook calls
210243
`request.log.isLevelEnabled('info')`, which doesn't exist on Fastify's

README.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ README and NOTES.md, ship the closest honest approximation.
5959
| `tunnel/` | `src/tunnel/` | ✅ parity | one deviation: single-prefix forces `{prefix}/connect` control path |
6060
| `notifications/` | `src/notifications/` | ✅ parity | **the seam-forcer** — WAC via loopback; forces `api.events`, `api.serverInfo` |
6161
| `pay/` | pay mode | 📋 wall-report | pipeline-modifying → **stays core**; draws the #564 line |
62+
| `remotestorage/` | `src/remotestorage.js` | ✅ parity | 7th port; **witnessed** the `/.well-known/webfinger` two-plugin collision, both directions |
6263

6364
**New features built straight onto the api** (from the `plugin`-tagged issue backlog):
6465

@@ -86,9 +87,11 @@ README and NOTES.md, ship the closest honest approximation.
8687
| `backup/` || pod container → `.tar.gz` download (data portability) |
8788
| `metrics/` || `/healthz` + Prometheus exporter; measured the plugin-hook scope |
8889
| `dashboard/` || live plugin-status page; first consumer of the #463/#464 registry seam |
90+
| `oembed/` || oEmbed provider for pod resources (link unfurling; XSS-refusing) |
91+
| `jmap/` || JMAP mail over pod messages — the stateless/push line, sharpened |
8992

90-
**282 tests, all green** (`npm test`), including `compose.test.js` — all
91-
twenty-eight plugins on one server from pure config, pods + WAC intact
93+
**332 tests, all green** (`npm test`), including `compose.test.js` — all
94+
thirty-one plugins on one server from pure config, pods + WAC intact
9295
beside them. Findings consolidated in [NOTES.md](./NOTES.md); the full
9396
plugin-tagged backlog triaged in [ISSUES.md](./ISSUES.md); the build guide
9497
is [AGENT.md](./AGENT.md); the maintainer-facing summary — what to add

0 commit comments

Comments
 (0)