Feat/importer

apps/importer/.env.example

@@ -0,0 +1,24 @@
+# Copy to .env, which is ignored, and keep it readable by nobody else:
+#
+#   cp .env.example .env && chmod 600 .env
+#
+# Anything already set in the shell wins over this file, so a single run can be
+# pointed at another relay without editing it.
+
+# The relays `publish:events` sends to, separated by commas, and what `--relay`
+# overrides. There is no default and there will not be one: where a corpus
+# signed by a key this project holds ends up is a decision, and one taken by
+# leaving a variable empty is not one.
+OPENSPECS_IMPORT_RELAYS=ws://127.0.0.1:7777
+
+# One key per corpus, named after its manifest in ./manifests. An nsec, or the
+# same key in hex. It signs the documents copied from that repository, and what
+# it proves is who copied them, never who wrote them: every event carries the
+# file and the commit it came from, and anyone can fetch the source and check
+# the copy against it.
+#
+# The importer refuses to sign with a key that is not the one its manifest
+# names, since a corpus published under the wrong key cannot be taken back.
+OPENSPECS_IMPORT_KEY_NIPS=
+OPENSPECS_IMPORT_KEY_BUDS=
+OPENSPECS_IMPORT_KEY_NUTS=

apps/importer/.gitignore

@@ -0,0 +1,3 @@
+.cache
+events
+.env

apps/importer/README.md

@@ -0,0 +1,81 @@
+# Importer
+
+Publishes specifications that live in git as ordinary Nostr documents: the NIPs, the BUDs
+and the NUTs, one addressable event each, signed by a key of their own per corpus.
+
+The key proves who copied a document, never who wrote it. What stands in for authorship
+is that the copy can be checked: every event names the file it came from in a `proxy` tag,
+pinned to a commit, and carries the sha256 of that file's bytes in an `x` tag. Fetch the
+source, hash it, compare.
+
+That only means something if the import is reproducible, so it is a pure function of the
+manifests and the source repository. Nothing is read from a clock, from the network, or
+from a model. Two runs at the same commits write the same bytes, and `manifests/README.md`
+is where the judgement went.
+
+## Commands
+
+```sh
+pnpm --filter @openspecs/importer build:events     # write the events, sign nothing
+pnpm --filter @openspecs/importer publish:events   # show what would be sent
+pnpm --filter @openspecs/importer publish:events --yes
+```
+
+`build:events` clones each repository into `.cache`, reads every file the manifest lists
+at the head of its branch, rewrites the links, and writes one unsigned event per document
+into `events/`. Both directories are ignored: one is a checkout, the other is output.
+
+It reports what left the corpus, which is the part worth reading. A link pinned to GitHub
+is either a file that is genuinely not a specification, or one the manifest forgot.
+
+## Links
+
+A document that only makes sense next to a website is not a document that lives on relays,
+so cross references are rewritten:
+
+| Link                                          | Becomes                          |
+| --------------------------------------------- | -------------------------------- |
+| To a document of any of the three manifests   | `nostr:naddr1...`, fragment kept |
+| To another file of the same repository        | GitHub, pinned to the commit     |
+| To anywhere else                              | left alone                       |
+| Into the document itself                      | left alone, `rehype-slug` resolves it |
+
+Fenced code is never touched: a link inside an example is part of the example.
+
+## Publishing
+
+`publish:events` reads what `build:events` wrote, signs it, and sends what the relays
+do not already hold. It prints its plan and stops; `--yes` is what sends.
+
+A run that changes nothing sends nothing. Every document is compared against the
+revision the relays serve, and only what differs is signed, which is what makes running
+it again safe and what running it again is for. A document that changed in its manifest
+rather than in git is stamped one second past the revision it replaces, since a relay
+keeps the older of two events sharing a timestamp.
+
+Keys come from the environment, one per corpus, `OPENSPECS_IMPORT_KEY_<CORPUS>`, an nsec
+or the same key in hex. Every key is resolved before the first event is sent, and a key
+that is not the one its manifest names is refused: a corpus published under the wrong key
+cannot be taken back.
+
+Relays come from `--relay`, repeatable, or `OPENSPECS_IMPORT_RELAYS`. There is no default
+and there will not be one. `infra/docker-compose.relay.yml` runs the relay this was
+written for.
+
+Both are read from `.env` beside this file, which is ignored, and anything already set in
+the shell wins over it:
+
+```sh
+cp .env.example .env && chmod 600 .env
+```
+
+A file rather than a variable typed in front of the command, because a command is written
+to a shell history and a key does not belong there. And a file of its own rather than
+`infra/.env`, because that one is read by Docker Compose, and nothing here is a service:
+publishing is a command somebody runs on purpose.
+
+## What it does not do yet
+
+Walk history. Every revision a document had before this import is still only in git, and
+until it is written as kind 1349 snapshots, publishing a new revision replaces the last
+one on the relays and nothing keeps what it said.

apps/importer/manifests/README.md

@@ -0,0 +1,57 @@
+# Manifests
+
+One file per corpus. Each lists every document to import, and carries everything the
+importer cannot read from the document itself.
+
+The importer is a pure function of these files and the source repository. Nothing here is
+computed at run time, so two runs at the same commits produce the same events, and a
+third party can rebuild the corpus and compare it byte for byte.
+
+## Fields
+
+| Field     | Meaning                                                            |
+| --------- | ------------------------------------------------------------------ |
+| `file`    | Path in the source repository                                       |
+| `d`       | Identifier of the addressable event, and the last part of its URL   |
+| `title`   | `title` tag                                                        |
+| `status`  | `s` tag, or null when the document claims none                     |
+| `summary` | `summary` tag                                                      |
+| `kinds`   | `k` tags, omitted when the document defines none                   |
+
+`topics` is declared once per corpus and applies to every document in it.
+
+## How these values were chosen
+
+**The title carries the number.** A listing row shows the title alone, and nobody looks
+for "Basic protocol flow description". `NIP-01: Basic protocol flow description`.
+
+**Status uses each corpus's own vocabulary**, read from the badge line under the heading:
+`draft`, `final`, `unrecommended` for NIPs, `draft` for BUDs, `mandatory` or `optional`
+for NUTs. A document with no badge gets null rather than a guess. The NIPs marked
+`unrecommended` are those struck through in the repository's own index, which is where
+that judgement is made and the only place it is written down.
+
+The site colours `draft`, `final` and the closed states, but not `unrecommended`. Adding
+it to `STATUS_TONE` in `spec-tags.tsx` is a one line change, and until then those
+documents render neutral.
+
+**Kinds come from the tables the repositories maintain**, the event kinds table in the
+NIPs index and the one in the Blossom readme. Only kinds a document actually defines are
+listed: NUT-27 carries 30078 because it specifies how that kind is used, and the other
+NUTs carry none.
+
+**Summaries are written here rather than derived.** The site can take a description from
+the first paragraph, but that paragraph is written for a reader who already knows what
+the document is.
+
+## Membership
+
+These files are the whole truth about what exists. `nips.json` holds 94 documents where
+the repository has 98 files: `12.md`, `16.md`, `20.md` and `33.md` were folded into
+NIP-01 and dropped from the index, so they are gone from here too. Files that are not
+specifications, `error_codes.md` and `suppl/` in the NUTs repository among them, are
+absent for the same reason.
+
+A new specification appears when a line is added. Removing one withdraws the document,
+by publishing an empty revision over it. Neither happens on its own, which is the point:
+both are a diff somebody read.

apps/importer/manifests/buds.json

@@ -0,0 +1,114 @@
+{
+  "name": "buds",
+  "title": "Blossom Upgrade Documents",
+  "npub": "npub1zvm2zlskr58g4u4k3m54454y087r30hedgtavyn75q4yp55dm9lqz4932r",
+  "repo": "https://github.com/hzrd149/blossom",
+  "branch": "master",
+  "license": "Unlicense",
+  "topics": ["blossom", "bud"],
+  "specs": [
+    {
+      "file": "buds/00.md",
+      "d": "bud-00",
+      "title": "BUD-00: Blossom Upgrade Documents",
+      "status": "draft",
+      "summary": "Defines the language every other BUD is written in, and what a blob is: raw binary data addressed by the sha256 hash of that data."
+    },
+    {
+      "file": "buds/01.md",
+      "d": "bud-01",
+      "title": "BUD-01: Server requirements and blob retrieval",
+      "status": "draft",
+      "summary": "The base every Blossom server implements: the CORS headers, the error reporting, and the GET and HEAD endpoints that serve a blob by its hash."
+    },
+    {
+      "file": "buds/02.md",
+      "d": "bud-02",
+      "title": "BUD-02: Blob upload",
+      "status": "draft",
+      "summary": "Defines PUT /upload and the blob descriptor, the JSON object describing a stored blob: its url, hash, size, mime type and upload time."
+    },
+    {
+      "file": "buds/03.md",
+      "d": "bud-03",
+      "title": "BUD-03: User Server List",
+      "status": "draft",
+      "summary": "Defines kind 10063, the replaceable event where a user lists the Blossom servers holding their blobs, most trusted first.",
+      "kinds": [
+        {
+          "kind": 10063,
+          "name": "User server list"
+        }
+      ]
+    },
+    {
+      "file": "buds/04.md",
+      "d": "bud-04",
+      "title": "BUD-04: Mirroring blobs",
+      "status": "draft",
+      "summary": "Defines PUT /mirror, which asks a server to fetch a blob from a URL itself instead of having the client upload the same bytes twice."
+    },
+    {
+      "file": "buds/05.md",
+      "d": "bud-05",
+      "title": "BUD-05: Media optimization endpoints",
+      "status": "draft",
+      "summary": "Defines PUT /media, where a server transcodes and optimises what it is given rather than storing the bytes untouched."
+    },
+    {
+      "file": "buds/06.md",
+      "d": "bud-06",
+      "title": "BUD-06: Upload requirements",
+      "status": "draft",
+      "summary": "Defines HEAD /upload, a preflight request telling a client whether an upload would be accepted before it sends the bytes."
+    },
+    {
+      "file": "buds/07.md",
+      "d": "bud-07",
+      "title": "BUD-07: Paid upload and download",
+      "status": "draft",
+      "summary": "How a server charges for an endpoint: a 402 response carrying the payment terms, for uploads, downloads or media processing."
+    },
+    {
+      "file": "buds/08.md",
+      "d": "bud-08",
+      "title": "BUD-08: Nostr File Metadata Tags",
+      "status": "draft",
+      "summary": "How a server returns NIP-94 file metadata alongside a blob descriptor, so a client can publish the file without recomputing it."
+    },
+    {
+      "file": "buds/09.md",
+      "d": "bud-09",
+      "title": "BUD-09: Blob Report",
+      "status": "draft",
+      "summary": "Defines PUT /report, where a client submits a signed NIP-56 report naming the blobs it objects to."
+    },
+    {
+      "file": "buds/10.md",
+      "d": "bud-10",
+      "title": "BUD-10: Blossom URI Schema",
+      "status": "draft",
+      "summary": "Defines the blossom: URI, which carries a blob hash, its extension, and hints about the servers where it can be found."
+    },
+    {
+      "file": "buds/11.md",
+      "d": "bud-11",
+      "title": "BUD-11: Nostr Authorization",
+      "status": "draft",
+      "summary": "Defines the kind 24242 event a user signs to prove to a server that they allowed an application to act on their behalf.",
+      "kinds": [
+        {
+          "kind": 24242,
+          "name": "Blossom authorization"
+        }
+      ]
+    },
+    {
+      "file": "buds/12.md",
+      "d": "bud-12",
+      "title": "BUD-12: Blob management endpoints",
+      "status": "draft",
+      "summary": "Defines the management endpoints: listing what a pubkey uploaded, and deleting a blob by its hash."
+    }
+  ]
+}