Skip to content

Commit 3d93bd0

Browse files
melvincarvalhomelvincarvalho
committed
Integrate wave 6 (matrix, search, didweb, s3): 24 plugins, 244 tests
- compose.test.js + serve.js load all 24 (appPaths +/_matrix); 25-check compose. README/ISSUES/AGENT tables + counts updated. - NOTES: api.events now 4 consumers (notifications/sparql/search/matrix) in rising sharpness — matrix /sync needs live push, promoting it to the clear #2 seam. reserved-path gains its sharpest form: didweb's parameterized /<user>/did.json interleaves with the pod namespace and can't be exempted at all -> a PARAMETERIZED api.reservePath. The loopback bridge now has 7 witnesses; token bridge spans 4 protocols (mastodon/bluesky/matrix + activitypub).
1 parent bb27c25 commit 3d93bd0

6 files changed

Lines changed: 94 additions & 34 deletions

File tree

‎AGENT.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -143,9 +143,11 @@ Full ranking in `NOTES.md`. The ones you'll hit:
143143
| an API shim (a new social/chat/proto) | `mastodon/` or `bluesky/` | token bridge + fixed-root `appPaths` |
144144
| a `.well-known` discovery doc | `nip05/` or `webfinger/` | podsRoot scan + guarded absolute route |
145145
| a scoped-token / auth service | `capability/` or `otp/` | HMAC macaroon-lite + `pluginDir` |
146-
| a query/read-over-pod-data feature | `sparql/` or `rss/` | loopback container walk + forwarded auth |
146+
| a query/read/search over pod data | `sparql/`, `rss/`, `search/` | loopback container walk + forwarded auth |
147147
| a federation actor | `activitypub/` | keypair in `pluginDir` + loopback objects |
148148
| a proxy/gateway | `corsproxy/` | fetch upstream, SSRF gates, CORS |
149+
| an object-storage / S3-style gateway | `s3/` | loopback LDP + hand-rolled XML + SigV4 |
150+
| a DID / identity document | `didweb/` | podsRoot scan + key derivation |
149151
| a dev/tooling subsystem | `gitscratch/` | shell a system binary via CGI |
150152
151153
## Footguns (every multi-boot suite rediscovered these)
@@ -164,7 +166,7 @@ Full ranking in `NOTES.md`. The ones you'll hit:
164166
165167
## Current state
166168
167-
20 plugins (6 ports + 14 features), ~200 tests, all green (`npm test`).
169+
24 plugins (6 ports + 18 features), ~270 tests, all green (`npm test`).
168170
`compose.test.js` runs every one on a single server from pure config. Two
169171
core PRs (#590 `api.mountApp`, #591 `/idp/refresh`) sit upstream, unmerged,
170172
for the maintainer's call. Everything else lives here, by design.

‎ISSUES.md‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,10 @@ evidence. Legend:
3131
| #164 | `webfinger/` | `/.well-known/webfinger` — the WebFinger half of #164 |
3232
| #51/#164 | `activitypub/` | federate a pod as an AS2 actor (needs `appPaths` widened → 🔩) |
3333
| — | `rss/` | any pod container as an Atom/RSS feed |
34+
| — | `matrix/` | Matrix Client-Server API shim (chat) |
35+
| — | `search/` | full-text search over pod resources |
36+
| — | `didweb/` | `did:web` DID-document resolver |
37+
| — | `s3/` | S3-compatible object-storage gateway |
3438

3539
Plus six **ports of bundled features** proving the migration path for #564 /
3640
#164: `relay/` `webrtc/` `terminal/` `tunnel/` `notifications/`, and `pay/`
@@ -91,12 +95,14 @@ exists.
9195

9296
## Tally
9397

94-
Of ~40 plugin-tagged issues: **13 built as plugins here** (plus `rss/`, no
95-
issue), **6 ported**, **5 shipped upstream**, **2 more plugin-able with no
96-
blocker**, **6 clusters blocked on a named seam** (each with a proof-of-need
97-
consumer), the rest core-by-nature or product-scale. 20 plugins total, ~200
98-
tests. The plugin api reaches most of the backlog today; ranked by demand,
99-
the seams that would unlock the most next are `api.authorize` (3 blocking
100-
consumers), `api.reservePath` (3 API-shims can't self-exempt), `api.events`
101-
(2, one where a miss means *wrong* answers), and `api.serverInfo` (broadest:
102-
~10 plugins hand-roll their own origin).
98+
Of ~40 plugin-tagged issues: **13 built as plugins here** (plus `rss/`,
99+
`matrix/`, `search/`, `didweb/`, `s3/` — capability demonstrations with no
100+
single issue), **6 ported**, **5 shipped upstream**, **2 more plugin-able
101+
with no blocker**, **6 clusters blocked on a named seam** (each with a
102+
proof-of-need consumer), the rest core-by-nature or product-scale. **24
103+
plugins total, ~270 tests.** The plugin api reaches most of the backlog
104+
today; ranked by demand, the seams that would unlock the most next are
105+
`api.authorize` (3 blocking consumers), `api.events` (4 consumers — matrix
106+
`/sync` needs live push), `api.reservePath` (API-shims + didweb's
107+
parameterized case), and `api.serverInfo` (broadest: ~10 plugins hand-roll
108+
their own origin).

‎NOTES.md‎

Lines changed: 24 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -38,13 +38,18 @@ ports reached for it without coordinating.
3838
#382), or a capability exercising the *issuer's* authority
3939
(capability #506). This is the most-requested seam and the one that
4040
moves the most backlog issues from "plugin-approximation" to "faithful".
41-
2. **`api.events.onResourceChange(cb)`** — **two consumers: notifications/,
42-
sparql/**, and sparql is the *stronger* one: without a write hook a
43-
plugin index returns **wrong** query results, not merely late
44-
notifications, and `pluginDir` caching is uninvalidatable. Core already
45-
has the emitter internally (`src/notifications/events.js`). The seam
46-
every "react to pod writes" app (webhooks, indexing, sync, full-text)
47-
will want.
41+
2. **`api.events.onResourceChange(cb)`** — **four consumers now:
42+
notifications/, sparql/, search/, matrix/**, in rising sharpness:
43+
notifications (a miss is a *late* notification), sparql (a miss is a
44+
*wrong* query result), search (a miss is *stale results*, the property
45+
users most expect to be fresh — the most user-visible instance), and
46+
matrix (its `/sync?since=` long-poll needs server-side sync cursors +
47+
**live push on writes** — an event hook feeding `api.ws.route` — so a
48+
stateless bridge can only do full-state `/sync` at all). Core already has
49+
the emitter internally (`src/notifications/events.js`); this is the seam
50+
every "react to pod writes" app (webhooks, indexing, sync, search, live
51+
chat) will want, and it's now clearly the #2 most-demanded after
52+
`api.authorize`.
4853
3. **`api.serverInfo` (`{ baseUrl, port }` at listen)** — **eight+
4954
consumers: notifications/, webdav/, carddav/, caldav/, sparql/, rss/,
5055
nip05/, webfinger/, mastodon/, bluesky/, activitypub/** — essentially
@@ -78,14 +83,22 @@ ports reached for it without coordinating.
7883
prefixes"); activitypub sharpened it further — its natural layout wants
7984
paths **interleaved with** the pod's own `/<user>/` namespace, which no
8085
single mount prefix can carve out at all.
86+
- **parameterized paths can't be exempted at all** — didweb/ serves
87+
`/<user>/did.json`, which lives *inside* the pod's own WAC-governed
88+
`/<user>/` namespace. `appPaths` matches literal prefixes, so it can't
89+
carve out a parameterized route interleaved with pod paths; and unlike
90+
an API shim, did:web can't escape to a fake root (the DID id is fixed
91+
by the method). It works only where the pod grants public Read. This is
92+
the sharpest form and needs a **parameterized** `api.reservePath`.
8193
- none of it has conflict detection: a future core route at a
8294
plugin-claimed path throws `FST_ERR_DUPLICATED_ROUTE` at boot (and a
8395
link registry — `api.webfinger.addLink` — is missing, so two plugins
8496
contributing `.well-known/webfinger` links would collide silently).
85-
The seam: `api.reservePath('/xrpc')` (or `paths: [...]` in the entry) —
86-
the loader exempts *and* claims each deliberately and reports collisions.
87-
The seam **every API-shim plugin** structurally requires; third
88-
most-demanded after `api.authorize` and `api.events`.
97+
The seam: `api.reservePath('/xrpc')` / `api.reservePath('/:user/did.json')`
98+
(or `paths: [...]` in the entry) — the loader exempts *and* claims each
99+
deliberately and reports collisions. The seam **every API-shim plugin**
100+
structurally requires; third most-demanded after `api.authorize` and
101+
`api.events`.
89102
6. **Can't set fastify server options** — consumer: capability/ hit
90103
`maxParamLength` (100) silently 404ing long tokens in named params;
91104
workaround is a wildcard route. A plugin has no way to raise per-route

‎README.md‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -76,11 +76,16 @@ README and NOTES.md, ship the closest honest approximation.
7676
| `webfinger/` | #164 | `/.well-known/webfinger` — fediverse `acct:` resolution for the shims |
7777
| `activitypub/` | #51/#164 | federate a pod as an AS2 actor; outbound HTTP Signatures |
7878
| `rss/` | — | any pod container as an Atom/RSS feed |
79-
80-
**~200 tests, all green** (`npm test`), including `compose.test.js` — all
81-
twenty plugins on one server from pure config, pods + WAC intact beside
82-
them. Findings consolidated in [NOTES.md](./NOTES.md); the full
83-
plugin-tagged backlog triaged in [ISSUES.md](./ISSUES.md).
79+
| `matrix/` | — | Matrix Client-Server API shim (chat protocol) |
80+
| `search/` | — | full-text search over pod resources (TF-IDF) |
81+
| `didweb/` | — | `did:web` DID-document resolver for pods |
82+
| `s3/` | — | S3-compatible object-storage gateway (SigV4) |
83+
84+
**~270 tests, all green** (`npm test`), including `compose.test.js` — all
85+
twenty-four plugins on one server from pure config, pods + WAC intact
86+
beside them. Findings consolidated in [NOTES.md](./NOTES.md); the full
87+
plugin-tagged backlog triaged in [ISSUES.md](./ISSUES.md); the build guide
88+
is [AGENT.md](./AGENT.md).
8489

8590
```
8691
<name>/plugin.js the port <name>/test.js real-JSS tests <name>/README.md findings

‎compose.test.js‎

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ describe('composition: every plugin on one server', () => {
4646

4747
after(async () => { if (jss) await jss.close(); });
4848

49-
it('boots pods + idp + twenty plugins from config', async () => {
49+
it('boots pods + idp + twenty-four plugins from config', async () => {
5050
const port = await probePort();
5151
base = `http://127.0.0.1:${port}`;
5252
wsBase = `ws://127.0.0.1:${port}`;
@@ -56,9 +56,10 @@ describe('composition: every plugin on one server', () => {
5656
root,
5757
idp: true,
5858
// The protocol shims own fixed roots outside their prefix — mastodon
59-
// (/api,/oauth), bluesky (/xrpc), activitypub (/ap) — which a plugin
60-
// can't self-exempt (finding), so the operator widens appPaths.
61-
appPaths: ['/api', '/oauth', '/xrpc', '/ap'],
59+
// (/api,/oauth), bluesky (/xrpc), activitypub (/ap), matrix (/_matrix)
60+
// — which a plugin can't self-exempt (finding), so the operator
61+
// widens appPaths.
62+
appPaths: ['/api', '/oauth', '/xrpc', '/ap', '/_matrix'],
6263
// Explicit ids: the <name>/plugin.js convention makes every basename
6364
// reduce to 'plugin' — the loader's duplicate-id guard requires ids
6465
// here (finding: derive from the parent dir for generic basenames).
@@ -88,6 +89,10 @@ describe('composition: every plugin on one server', () => {
8889
{ id: 'webfinger', module: at('webfinger/plugin.js'), prefix: '/webfinger', config: { podsRoot: root, baseUrl: base } },
8990
{ id: 'activitypub', module: at('activitypub/plugin.js'), prefix: '/activitypub', config: { baseUrl: base, loopbackUrl: base } },
9091
{ id: 'rss', module: at('rss/plugin.js'), prefix: '/feed', config: { baseUrl: base, loopbackUrl: base } },
92+
{ id: 'matrix', module: at('matrix/plugin.js'), prefix: '/matrix', config: { baseUrl: base, loopbackUrl: base } },
93+
{ id: 'search', module: at('search/plugin.js'), prefix: '/search', config: { baseUrl: base, loopbackUrl: base } },
94+
{ id: 'didweb', module: at('didweb/plugin.js'), prefix: '/didweb', config: { podsRoot: root, baseUrl: base } },
95+
{ id: 's3', module: at('s3/plugin.js'), prefix: '/s3', config: { baseUrl: base, loopbackUrl: base } },
9196
],
9297
});
9398
assert.ok(jss.base);
@@ -250,6 +255,26 @@ describe('composition: every plugin on one server', () => {
250255
assert.strictEqual(res.status, 400);
251256
});
252257

258+
it('matrix: the versions endpoint answers (fixed /_matrix root)', async () => {
259+
const res = await fetch(`${base}/_matrix/client/versions`);
260+
assert.strictEqual(res.status, 200);
261+
});
262+
263+
it('search: missing query is 400 (endpoint alive)', async () => {
264+
const res = await fetch(`${base}/search`);
265+
assert.strictEqual(res.status, 400);
266+
});
267+
268+
it('didweb: unknown pod DID is 404 (resolver alive)', async () => {
269+
const res = await fetch(`${base}/didweb/nobody/did.json`);
270+
assert.strictEqual(res.status, 404);
271+
});
272+
273+
it('s3: an unauthenticated object GET is denied (gateway alive)', async () => {
274+
const res = await fetch(`${base}/s3/bucket/key`);
275+
assert.strictEqual(res.status, 403);
276+
});
277+
253278
it('pods still work beside all of it (idp register + WAC)', async () => {
254279
let res = await fetch(`${base}/idp/register`, {
255280
method: 'POST',

‎serve.js‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,10 @@ const fastify = createServer({
2929
idp: true,
3030
idpIssuer: PUBLIC_URL,
3131
// The protocol shims own fixed roots outside their prefix — mastodon
32-
// (/api,/oauth), bluesky (/xrpc), activitypub (/ap) — which a plugin
33-
// can't self-exempt from WAC, so the operator widens appPaths.
34-
appPaths: ['/api', '/oauth', '/xrpc', '/ap'],
32+
// (/api,/oauth), bluesky (/xrpc), activitypub (/ap), matrix (/_matrix) —
33+
// which a plugin can't self-exempt from WAC, so the operator widens
34+
// appPaths.
35+
appPaths: ['/api', '/oauth', '/xrpc', '/ap', '/_matrix'],
3536
// Explicit ids — the <name>/plugin.js convention collides on basename.
3637
plugins: [
3738
{ id: 'relay', module: at('relay/plugin.js'), prefix: '/relay' },
@@ -59,6 +60,10 @@ const fastify = createServer({
5960
{ id: 'webfinger', module: at('webfinger/plugin.js'), prefix: '/webfinger', config: { podsRoot: PODS, baseUrl: PUBLIC_URL } },
6061
{ id: 'activitypub', module: at('activitypub/plugin.js'), prefix: '/activitypub', config: { baseUrl: PUBLIC_URL, loopbackUrl: `http://127.0.0.1:${PORT}` } },
6162
{ id: 'rss', module: at('rss/plugin.js'), prefix: '/feed', config: { baseUrl: PUBLIC_URL, loopbackUrl: `http://127.0.0.1:${PORT}` } },
63+
{ id: 'matrix', module: at('matrix/plugin.js'), prefix: '/matrix', config: { baseUrl: PUBLIC_URL, loopbackUrl: `http://127.0.0.1:${PORT}` } },
64+
{ id: 'search', module: at('search/plugin.js'), prefix: '/search', config: { baseUrl: PUBLIC_URL, loopbackUrl: `http://127.0.0.1:${PORT}` } },
65+
{ id: 'didweb', module: at('didweb/plugin.js'), prefix: '/didweb', config: { podsRoot: PODS, baseUrl: PUBLIC_URL } },
66+
{ id: 's3', module: at('s3/plugin.js'), prefix: '/s3', config: { baseUrl: PUBLIC_URL, loopbackUrl: `http://127.0.0.1:${PORT}` } },
6267
],
6368
});
6469

@@ -92,3 +97,7 @@ console.log(` caldav: ${PUBLIC_URL}/caldav/ (calendar sync)`);
9297
console.log(` webfinger: GET ${PUBLIC_URL}/.well-known/webfinger?resource=acct:me@host`);
9398
console.log(` activitypub: GET ${PUBLIC_URL}/ap/<user>/actor`);
9499
console.log(` rss/atom: GET ${PUBLIC_URL}/feed/atom?container=<pod-container>`);
100+
console.log(` matrix: GET ${PUBLIC_URL}/_matrix/client/versions (point Element here)`);
101+
console.log(` search: GET ${PUBLIC_URL}/search?q=<terms>&container=<pod-container>`);
102+
console.log(` did:web: GET ${PUBLIC_URL}/.well-known/did.json`);
103+
console.log(` s3: ${PUBLIC_URL}/s3/<bucket>/<key> (aws-cli/rclone, SigV4)`);

0 commit comments

Comments
 (0)