From 03788feff85e2bfb27baf20105d712c986535a85 Mon Sep 17 00:00:00 2001 From: ContextVM Date: Wed, 1 Apr 2026 18:00:11 +0100 Subject: [PATCH 1/6] docs: CEP-XX Open-Ended Stream Transfer --- mpp-specs | 1 + src/content/docs/spec/ceps/cep-xx.md | 437 +++++++++++++++++++++++++++ 2 files changed, 438 insertions(+) create mode 160000 mpp-specs create mode 100644 src/content/docs/spec/ceps/cep-xx.md diff --git a/mpp-specs b/mpp-specs new file mode 160000 index 0000000..b62c37d --- /dev/null +++ b/mpp-specs @@ -0,0 +1 @@ +Subproject commit b62c37dee384d0f8e9700d5aa8fbe739ac602d75 diff --git a/src/content/docs/spec/ceps/cep-xx.md b/src/content/docs/spec/ceps/cep-xx.md new file mode 100644 index 0000000..55b426d --- /dev/null +++ b/src/content/docs/spec/ceps/cep-xx.md @@ -0,0 +1,437 @@ +--- +title: CEP-XX Open-Ended Stream Transfer +description: Open-ended stream transfer for ContextVM using progress-notification framing +--- + +# Open-Ended Stream Transfer + +## Abstract + +This CEP defines an additive transport profile for open-ended streaming over ContextVM. It reuses MCP `notifications/progress` as the transfer envelope and uses the request `progressToken` as the stream identifier. + +Unlike bounded oversized-payload transfer in [`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md), this CEP defines a long-lived stream model where ordered fragments may continue until the sender explicitly closes or aborts the stream. The stream payload itself is the primary output; no rendered synthetic final JSON-RPC message is required. + +This CEP is intended for cases where data is naturally incremental, long-lived, or unbounded, and where representing the result as one reassembled MCP request or response would be artificial or inefficient. + +## Specification + +### Overview + +ContextVM currently transports MCP JSON-RPC messages through Nostr events. That model fits ordinary request and response exchange well, and [`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md) extends it for bounded reassembly of oversized logical messages. + +Some use cases are different in nature: + +- long-running generation that emits useful partial output over time +- event feeds or incremental result sets +- progressive delivery where partial consumption is desirable +- cases where no single final rendered payload is the right abstraction + +This CEP defines an open-ended stream-transfer profile that: + +- reuses the existing single-kind ContextVM transport model +- reuses MCP `notifications/progress` as the stream envelope +- uses the request `progressToken` as the stream identifier +- supports ordered `start`, `accept`, `chunk`, `close`, and `abort` frames +- treats the stream itself as the payload rather than a bounded reassembly artifact +- allows receivers to process fragments incrementally as they arrive + +This CEP is intentionally distinct from the bounded reassembly mechanism in [`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md). Implementations MUST NOT treat these two profiles as interchangeable. + +### Relationship to Bounded Oversized Transfer + +[`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md) defines transfer for one oversized logical JSON-RPC message that is reconstructed and validated before being surfaced. + +This CEP instead defines a stream profile where: + +- fragments are meaningful as they arrive +- no final rendered aggregate message is required +- integrity is defined by stream ordering and explicit lifecycle events rather than by a final digest over a single serialized message +- closure is explicit and semantic, not merely the last step of bounded reassembly + +When a sender needs to deliver one oversized request or response that should preserve ordinary MCP semantics, it SHOULD use [`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md), not this CEP. + +### Capability Advertisement and Negotiation + +Support for open-ended stream transfer MAY be advertised through the same additive discovery surfaces already used by ContextVM capabilities and transport features, following the patterns in [`src/content/docs/spec/ceps/cep-6.md`](src/content/docs/spec/ceps/cep-6.md) and [`src/content/docs/spec/ceps/cep-19.md`](src/content/docs/spec/ceps/cep-19.md). + +Peers MAY advertise support using one or more `support_open_stream_transfer` tags. + +Example tags only: + +```json +[["support_open_stream_transfer"]] +``` + +Advertisement surfaces: + +- **Public announcements:** Servers MAY advertise support in public server announcements. +- **Initialization:** Clients and servers SHOULD advertise support during MCP initialization when initialization is available. +- **Stateless operation:** Clients and servers MAY advertise support in tags on the first exchanged request or response when no prior initialization occurred. + +Support semantics: + +- `support_open_stream_transfer` indicates support for the open-ended stream-transfer profile defined by this CEP. + +### Request-Level Activation + +Open-ended stream transfer for a given logical exchange is available only when the initiating request includes a valid MCP `progressToken`. + +Activation rules: + +- Clients that want to permit open-ended streaming for a request MUST include a `progressToken`. +- Servers MUST NOT start an open-ended stream for a request that did not include a `progressToken`. +- When no `progressToken` is present, peers MUST use ordinary non-streaming behavior or fail cleanly. + +The `progressToken` is the stream identifier for the open-ended stream session. + +### Sender Behavior + +When open-ended stream transfer is used, the sender MUST emit an ordered sequence of MCP `notifications/progress` messages containing ContextVM stream frames. + +If the sender already knows that the receiver supports this CEP for the current exchange, it MAY proceed directly from `start` to `chunk`. + +If the sender does not know whether the receiver supports this CEP and wants to use open-ended stream transfer, it MUST wait for an `accept` frame before sending `chunk` frames. + +The sender: + +- MAY emit any number of `chunk` frames after stream startup +- MAY keep the stream open while useful incremental output continues +- MUST terminate the stream with either `close` or `abort` +- MUST NOT silently stop transmission without a terminal frame unless transport failure prevents completion + +### Progress Notification Framing + +Open-ended stream frames are carried inside MCP `notifications/progress` params. The MCP envelope remains valid and additive; ContextVM defines additional frame semantics inside the params object. + +Example conceptual envelope: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/progress", + "params": { + "progressToken": "req-123", + "progress": 1, + "message": "starting open stream", + "cvm": { + "type": "open-stream-transfer", + "frameType": "start", + "mode": "stream" + } + } +} +``` + +The sender MUST use `progress` values that increase monotonically across the stream, consistent with MCP progress rules. + +### Frame Types + +This CEP defines five frame types: + +- `start` +- `accept` +- `chunk` +- `close` +- `abort` + +#### Common Fields + +All open-stream-transfer frames MUST include a ContextVM-specific transport object with: + +- `type`: MUST be `open-stream-transfer` +- `frameType`: one of `start`, `accept`, `chunk`, `close`, `abort` + +The outer MCP progress params MUST include: + +- `progressToken` +- `progress` + +The outer MCP `total` and `message` fields MAY be used for UX hints or progress reporting, but they do not define stream correctness. + +#### `start` Frame + +The `start` frame begins the stream. + +Required fields: + +- `mode`: `stream` + +Rules: + +- If `mode` is omitted, receivers MAY reject the stream; senders SHOULD always provide it. +- In this CEP version, senders MUST use `mode: "stream"`. +- Receivers MUST reject unknown or unsupported stream modes. + +#### `accept` Frame + +The `accept` frame confirms that the receiver accepts the stream and that the sender may begin transmitting `chunk` frames. + +This frame is primarily intended for bootstrap in stateless sender-to-receiver flows where support is not yet known. + +Rules: + +- A receiver MAY send `accept` after `start`. +- A sender that is required to wait for confirmation MUST NOT send `chunk` frames before receiving `accept`. +- `accept` SHOULD remain minimal and does not negotiate additional stream parameters in v1. + +#### `chunk` Frame + +The `chunk` frame carries one ordered fragment of stream payload. + +Required fields: + +- `data`: chunk payload + +Optional fields: + +- `event` + +Rules: + +- For open-stream-transfer frames, MCP `progress` is the normative stream-ordering field. +- Each `chunk` frame MUST use a `progress` value greater than the preceding stream frame's `progress` value. +- The payload represented by `data` is one ordered fragment of stream output. +- If present, `event` names the semantic type of the fragment and MAY be used by receivers for routing or handling. + +#### `close` Frame + +The `close` frame signals successful sender-side closure of the stream. + +Rules: + +- `close` is required for successful stream completion. +- `close` indicates that no further `chunk` frames will be sent for the stream. +- `close` does not imply that the stream payload can or should be rendered as one aggregate MCP message. + +#### `abort` Frame + +The `abort` frame signals that the stream did not complete successfully. + +Optional fields: + +- `reason` + +Rules: + +- Receivers MUST treat `abort` as terminal for the stream. +- `reason` is advisory only. + +### Stream Semantics + +In this CEP: + +- the stream itself is the primary payload +- receivers MAY process `chunk` payloads incrementally +- receivers MUST NOT require a final rendered aggregate message +- successful completion occurs when a valid `close` frame is received + +This CEP does not require a final digest because it does not define a single canonical reassembled JSON-RPC message as its output. + +### Validation Rules + +#### Ordering and Lifecycle + +Receivers MUST validate stream ordering using MCP `progress`. + +Rules: + +- a stream MUST begin with `start` +- if confirmation is required for the stream, `accept` MUST be received before the first `chunk` +- `progress` values for open-stream-transfer frames MUST increase monotonically across the stream +- successful completion requires `close` +- if `close` arrives after malformed or non-monotonic ordering, the stream MUST fail + +This CEP does not define replay, selective retransmission, or repair. + +#### Post-Close Behavior + +After `close` or `abort`: + +- the stream is terminal +- receivers MUST ignore or reject later frames for the same terminated stream +- senders MUST NOT resume the same stream identifier + +### Receiver Behavior + +Receivers that support this CEP: + +- MUST track stream state by `progressToken` +- MUST process frames in stream order +- MUST reject or fail malformed frame sequences +- MUST treat `abort` as terminal +- MUST fail a stream if `close` is received before a valid monotonic `progress` sequence has been observed + +Receivers MAY expose stream fragments to applications incrementally as they arrive. + +### Stateless Operation + +This CEP is compatible with stateless ContextVM operation. + +In stateless operation: + +- peers MAY advertise support in tags on the first exchanged request or response +- stream state is correlated by `progressToken` +- receivers MUST NOT rely on a persistent connection-local session beyond temporary stream state + +For stateless client-to-server streaming where the client has not previously learned server support, the client MUST send `start` first and wait for `accept` before sending `chunk` frames. + +### Timeout and Keepalive Semantics + +Because stream frames are carried by MCP `notifications/progress`, receipt of valid open-stream-transfer frames MAY be treated by implementations as progress activity for request timeout handling, consistent with MCP lifecycle guidance. + +Implementations: + +- MAY reset soft request timeouts upon receiving valid stream frames +- SHOULD enforce a hard maximum timeout or other resource policy for long-lived streams +- MAY define local keepalive policies, but this CEP does not require a dedicated keepalive frame in v1 + +### Example: Server-to-Client Open Stream + +Client sends a request with a `progressToken`: + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": { + "name": "streaming_tool", + "arguments": {}, + "_meta": { + "progressToken": "req-123" + } + } +} +``` + +Server starts the stream: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/progress", + "params": { + "progressToken": "req-123", + "progress": 1, + "message": "starting stream", + "cvm": { + "type": "open-stream-transfer", + "frameType": "start", + "mode": "stream" + } + } +} +``` + +Server sends stream fragments: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/progress", + "params": { + "progressToken": "req-123", + "progress": 2, + "cvm": { + "type": "open-stream-transfer", + "frameType": "chunk", + "event": "token", + "data": "Hello" + } + } +} +``` + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/progress", + "params": { + "progressToken": "req-123", + "progress": 3, + "cvm": { + "type": "open-stream-transfer", + "frameType": "chunk", + "event": "token", + "data": " world" + } + } +} +``` + +Server closes the stream: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/progress", + "params": { + "progressToken": "req-123", + "progress": 4, + "message": "stream complete", + "cvm": { + "type": "open-stream-transfer", + "frameType": "close" + } + } +} +``` + +### Example: Stateless Client-to-Server Stream Bootstrap + +Client announces intent to begin a stream: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/progress", + "params": { + "progressToken": "req-789", + "progress": 1, + "message": "starting client stream", + "cvm": { + "type": "open-stream-transfer", + "frameType": "start", + "mode": "stream" + } + } +} +``` + +Server confirms support: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/progress", + "params": { + "progressToken": "req-789", + "progress": 2, + "message": "client stream accepted", + "cvm": { + "type": "open-stream-transfer", + "frameType": "accept" + } + } +} +``` + +After `accept`, the client sends `chunk` frames and eventually terminates the stream with `close` or `abort`. + +## Backward Compatibility + +This CEP introduces no breaking changes: + +- peers that do not advertise support continue using ordinary ContextVM request and response transport +- peers that do not include a `progressToken` on a request do not enable open-ended stream transfer for that exchange +- peers that do not understand the ContextVM-specific open-stream-transfer framing continue to interoperate for ordinary non-streaming messages + +## Dependencies + +- [CEP-6: Public Server Announcements](/spec/ceps/cep-6) +- [CEP-19: Ephemeral Gift Wraps](/spec/ceps/cep-19) +- [CEP-XX: Oversized Payload Transfer](/spec/ceps/cep-xx) + +## Reference Implementation + +A reference implementation is intended for the ContextVM SDK transport layer. From 65dd48eac3d23f5772ff6f6bb1435acc66a615b5 Mon Sep 17 00:00:00 2001 From: ContextVM Date: Thu, 30 Apr 2026 17:15:13 +0200 Subject: [PATCH 2/6] docs(cep): update open-ended stream specification with ping/pong frames - Add ping and pong frame types for stream keepalive and responsiveness probing - Simplify naming from open-stream-transfer to open-stream - Update CEP references to use proper CEP numbers (CEP-22, CEP-35) - Add idle timeout handling rules for active streams - Remove event field from chunk frames and streamline frame definitions - Remove redundant relationship and stream semantics sections --- src/content/docs/spec/ceps/cep-xx.md | 121 ++++++++++++++------------- 1 file changed, 62 insertions(+), 59 deletions(-) diff --git a/src/content/docs/spec/ceps/cep-xx.md b/src/content/docs/spec/ceps/cep-xx.md index 55b426d..6f3d09c 100644 --- a/src/content/docs/spec/ceps/cep-xx.md +++ b/src/content/docs/spec/ceps/cep-xx.md @@ -9,7 +9,7 @@ description: Open-ended stream transfer for ContextVM using progress-notificatio This CEP defines an additive transport profile for open-ended streaming over ContextVM. It reuses MCP `notifications/progress` as the transfer envelope and uses the request `progressToken` as the stream identifier. -Unlike bounded oversized-payload transfer in [`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md), this CEP defines a long-lived stream model where ordered fragments may continue until the sender explicitly closes or aborts the stream. The stream payload itself is the primary output; no rendered synthetic final JSON-RPC message is required. +Unlike bounded oversized-payload transfer in [`CEP-22`](/src/content/docs/spec/ceps/cep-22.md), this CEP defines a long-lived stream model where ordered fragments may continue until the sender explicitly closes or aborts the stream. The stream payload itself is the primary output; no rendered synthetic final JSON-RPC message is required. This CEP is intended for cases where data is naturally incremental, long-lived, or unbounded, and where representing the result as one reassembled MCP request or response would be artificial or inefficient. @@ -17,7 +17,7 @@ This CEP is intended for cases where data is naturally incremental, long-lived, ### Overview -ContextVM currently transports MCP JSON-RPC messages through Nostr events. That model fits ordinary request and response exchange well, and [`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md) extends it for bounded reassembly of oversized logical messages. +ContextVM currently transports MCP JSON-RPC messages through Nostr events. That model fits ordinary request and response exchange well, and [`CEP-22`](/src/content/docs/spec/ceps/cep-22.md) extends it for bounded reassembly of oversized logical messages. Some use cases are different in nature: @@ -35,31 +35,18 @@ This CEP defines an open-ended stream-transfer profile that: - treats the stream itself as the payload rather than a bounded reassembly artifact - allows receivers to process fragments incrementally as they arrive -This CEP is intentionally distinct from the bounded reassembly mechanism in [`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md). Implementations MUST NOT treat these two profiles as interchangeable. - -### Relationship to Bounded Oversized Transfer - -[`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md) defines transfer for one oversized logical JSON-RPC message that is reconstructed and validated before being surfaced. - -This CEP instead defines a stream profile where: - -- fragments are meaningful as they arrive -- no final rendered aggregate message is required -- integrity is defined by stream ordering and explicit lifecycle events rather than by a final digest over a single serialized message -- closure is explicit and semantic, not merely the last step of bounded reassembly - -When a sender needs to deliver one oversized request or response that should preserve ordinary MCP semantics, it SHOULD use [`src/content/docs/spec/ceps/cep-xx.md`](src/content/docs/spec/ceps/cep-xx.md), not this CEP. +This CEP is intentionally distinct from the bounded reassembly mechanism in [`CEP-22`](/src/content/docs/spec/ceps/cep-22.md). Implementations MUST NOT treat these two profiles as interchangeable. ### Capability Advertisement and Negotiation -Support for open-ended stream transfer MAY be advertised through the same additive discovery surfaces already used by ContextVM capabilities and transport features, following the patterns in [`src/content/docs/spec/ceps/cep-6.md`](src/content/docs/spec/ceps/cep-6.md) and [`src/content/docs/spec/ceps/cep-19.md`](src/content/docs/spec/ceps/cep-19.md). +Support for open-ended stream transfer MAY be advertised through the same additive discovery surfaces already used by ContextVM capabilities and transport features, following the patterns in [`CEP-35`](/src/content/docs/spec/ceps/informational/cep-35.md). -Peers MAY advertise support using one or more `support_open_stream_transfer` tags. +Peers MAY advertise support using one or more `support_open_stream` tags. Example tags only: ```json -[["support_open_stream_transfer"]] +[["support_open_stream"]] ``` Advertisement surfaces: @@ -70,7 +57,7 @@ Advertisement surfaces: Support semantics: -- `support_open_stream_transfer` indicates support for the open-ended stream-transfer profile defined by this CEP. +- `support_open_stream` indicates support for the open-ended stream-transfer profile defined by this CEP. ### Request-Level Activation @@ -88,9 +75,7 @@ The `progressToken` is the stream identifier for the open-ended stream session. When open-ended stream transfer is used, the sender MUST emit an ordered sequence of MCP `notifications/progress` messages containing ContextVM stream frames. -If the sender already knows that the receiver supports this CEP for the current exchange, it MAY proceed directly from `start` to `chunk`. - -If the sender does not know whether the receiver supports this CEP and wants to use open-ended stream transfer, it MUST wait for an `accept` frame before sending `chunk` frames. +If the sender already knows the receiver supports this CEP for the exchange, it MAY proceed directly from `start` to `chunk`. Otherwise it MUST wait for `accept` before sending `chunk` frames. The sender: @@ -112,9 +97,9 @@ Example conceptual envelope: "params": { "progressToken": "req-123", "progress": 1, - "message": "starting open stream", + "message": "(Optional) starting open stream", "cvm": { - "type": "open-stream-transfer", + "type": "open-stream", "frameType": "start", "mode": "stream" } @@ -126,11 +111,13 @@ The sender MUST use `progress` values that increase monotonically across the str ### Frame Types -This CEP defines five frame types: +This CEP defines seven frame types: - `start` - `accept` - `chunk` +- `ping` +- `pong` - `close` - `abort` @@ -138,8 +125,8 @@ This CEP defines five frame types: All open-stream-transfer frames MUST include a ContextVM-specific transport object with: -- `type`: MUST be `open-stream-transfer` -- `frameType`: one of `start`, `accept`, `chunk`, `close`, `abort` +- `type`: MUST be `open-stream` +- `frameType`: one of `start`, `accept`, `chunk`, `ping`, `pong`, `close`, `abort` The outer MCP progress params MUST include: @@ -182,16 +169,39 @@ Required fields: - `data`: chunk payload -Optional fields: - -- `event` - Rules: - For open-stream-transfer frames, MCP `progress` is the normative stream-ordering field. - Each `chunk` frame MUST use a `progress` value greater than the preceding stream frame's `progress` value. - The payload represented by `data` is one ordered fragment of stream output. -- If present, `event` names the semantic type of the fragment and MAY be used by receivers for routing or handling. + +#### `ping` Frame + +The `ping` frame probes whether the peer remains responsive for the active stream. + +Required fields: + +- `nonce` + +Rules: + +- Either peer MAY send `ping` on an active stream. +- `nonce` MUST identify the probe uniquely within the stream. +- `ping` carries no stream payload. + +#### `pong` Frame + +The `pong` frame acknowledges a received `ping` for the active stream. + +Required fields: + +- `nonce` + +Rules: + +- A receiver of `ping` MUST respond with `pong` for the same stream unless the stream has already terminated. +- `pong.nonce` MUST match the triggering `ping.nonce`. +- `pong` acknowledges peer responsiveness only and does not acknowledge delivery or processing of stream payload. #### `close` Frame @@ -201,7 +211,6 @@ Rules: - `close` is required for successful stream completion. - `close` indicates that no further `chunk` frames will be sent for the stream. -- `close` does not imply that the stream payload can or should be rendered as one aggregate MCP message. #### `abort` Frame @@ -213,20 +222,10 @@ Optional fields: Rules: +- Either peer MAY send `abort`. - Receivers MUST treat `abort` as terminal for the stream. - `reason` is advisory only. -### Stream Semantics - -In this CEP: - -- the stream itself is the primary payload -- receivers MAY process `chunk` payloads incrementally -- receivers MUST NOT require a final rendered aggregate message -- successful completion occurs when a valid `close` frame is received - -This CEP does not require a final digest because it does not define a single canonical reassembled JSON-RPC message as its output. - ### Validation Rules #### Ordering and Lifecycle @@ -238,6 +237,7 @@ Rules: - a stream MUST begin with `start` - if confirmation is required for the stream, `accept` MUST be received before the first `chunk` - `progress` values for open-stream-transfer frames MUST increase monotonically across the stream +- `pong` MUST correspond to an earlier `ping` on the same stream - successful completion requires `close` - if `close` arrives after malformed or non-monotonic ordering, the stream MUST fail @@ -277,13 +277,18 @@ For stateless client-to-server streaming where the client has not previously lea ### Timeout and Keepalive Semantics -Because stream frames are carried by MCP `notifications/progress`, receipt of valid open-stream-transfer frames MAY be treated by implementations as progress activity for request timeout handling, consistent with MCP lifecycle guidance. +Receipt of any valid open-stream-transfer frame counts as stream activity. -Implementations: +Implementations MUST maintain an idle timeout for each active stream. -- MAY reset soft request timeouts upon receiving valid stream frames -- SHOULD enforce a hard maximum timeout or other resource policy for long-lived streams -- MAY define local keepalive policies, but this CEP does not require a dedicated keepalive frame in v1 +Rules: + +- receipt of `start`, `accept`, `chunk`, `ping`, `pong`, `close`, or `abort` MUST reset the idle timeout +- if no valid frame is received before the idle timeout expires, the peer MUST send `ping` +- the receiver of `ping` MUST respond with `pong` carrying the same `nonce` +- if the probing peer does not receive a matching `pong` before its probe timeout expires, it MUST treat the stream as failed +- a peer that fails the stream due to probe timeout SHOULD send `abort` if it is still able to transmit +- implementations SHOULD enforce a hard maximum timeout or other resource policy for long-lived streams ### Example: Server-to-Client Open Stream @@ -315,7 +320,7 @@ Server starts the stream: "progress": 1, "message": "starting stream", "cvm": { - "type": "open-stream-transfer", + "type": "open-stream", "frameType": "start", "mode": "stream" } @@ -335,7 +340,6 @@ Server sends stream fragments: "cvm": { "type": "open-stream-transfer", "frameType": "chunk", - "event": "token", "data": "Hello" } } @@ -350,9 +354,8 @@ Server sends stream fragments: "progressToken": "req-123", "progress": 3, "cvm": { - "type": "open-stream-transfer", + "type": "open-stream", "frameType": "chunk", - "event": "token", "data": " world" } } @@ -370,7 +373,7 @@ Server closes the stream: "progress": 4, "message": "stream complete", "cvm": { - "type": "open-stream-transfer", + "type": "open-stream", "frameType": "close" } } @@ -390,7 +393,7 @@ Client announces intent to begin a stream: "progress": 1, "message": "starting client stream", "cvm": { - "type": "open-stream-transfer", + "type": "open-stream", "frameType": "start", "mode": "stream" } @@ -409,7 +412,7 @@ Server confirms support: "progress": 2, "message": "client stream accepted", "cvm": { - "type": "open-stream-transfer", + "type": "open-stream", "frameType": "accept" } } @@ -424,13 +427,13 @@ This CEP introduces no breaking changes: - peers that do not advertise support continue using ordinary ContextVM request and response transport - peers that do not include a `progressToken` on a request do not enable open-ended stream transfer for that exchange -- peers that do not understand the ContextVM-specific open-stream-transfer framing continue to interoperate for ordinary non-streaming messages +- peers that do not understand the ContextVM-specific open-stream framing continue to interoperate for ordinary non-streaming messages ## Dependencies - [CEP-6: Public Server Announcements](/spec/ceps/cep-6) - [CEP-19: Ephemeral Gift Wraps](/spec/ceps/cep-19) -- [CEP-XX: Oversized Payload Transfer](/spec/ceps/cep-xx) +- [CEP-22: Oversized Payload Transfer](/spec/ceps/cep-22) ## Reference Implementation From bfece6b0661f4cf35b8244d69b516e803fee90a5 Mon Sep 17 00:00:00 2001 From: ContextVM Date: Thu, 30 Apr 2026 17:22:38 +0200 Subject: [PATCH 3/6] docs(cep): rename cep-xx to cep-41 --- sdk | 1 + src/content/docs/spec/ceps/{cep-xx.md => cep-41.md} | 0 2 files changed, 1 insertion(+) create mode 160000 sdk rename src/content/docs/spec/ceps/{cep-xx.md => cep-41.md} (100%) diff --git a/sdk b/sdk new file mode 160000 index 0000000..7741878 --- /dev/null +++ b/sdk @@ -0,0 +1 @@ +Subproject commit 77418780663d634ac6999591333b2c480d92cdfa diff --git a/src/content/docs/spec/ceps/cep-xx.md b/src/content/docs/spec/ceps/cep-41.md similarity index 100% rename from src/content/docs/spec/ceps/cep-xx.md rename to src/content/docs/spec/ceps/cep-41.md From 401837818e8ad246a476056532d12a30bbd33040 Mon Sep 17 00:00:00 2001 From: ContextVM Date: Fri, 1 May 2026 12:35:19 +0200 Subject: [PATCH 4/6] docs(cep-41): update open-ended stream specification Add ping/pong frames, chunkIndex tracking, concurrent stream rules, and request completion semantics to the open-ended stream profile. Include relay rate guidance and clarify frame ordering requirements. --- src/content/docs/spec/ceps/cep-41.md | 90 ++++++++++++++++++++++++---- 1 file changed, 78 insertions(+), 12 deletions(-) diff --git a/src/content/docs/spec/ceps/cep-41.md b/src/content/docs/spec/ceps/cep-41.md index 6f3d09c..f9fa280 100644 --- a/src/content/docs/spec/ceps/cep-41.md +++ b/src/content/docs/spec/ceps/cep-41.md @@ -9,7 +9,7 @@ description: Open-ended stream transfer for ContextVM using progress-notificatio This CEP defines an additive transport profile for open-ended streaming over ContextVM. It reuses MCP `notifications/progress` as the transfer envelope and uses the request `progressToken` as the stream identifier. -Unlike bounded oversized-payload transfer in [`CEP-22`](/src/content/docs/spec/ceps/cep-22.md), this CEP defines a long-lived stream model where ordered fragments may continue until the sender explicitly closes or aborts the stream. The stream payload itself is the primary output; no rendered synthetic final JSON-RPC message is required. +Unlike bounded oversized-payload transfer in [`CEP-22`](/src/content/docs/spec/ceps/cep-22.md), this CEP defines a long-lived stream model where ordered fragments may continue until the sender explicitly closes or aborts the stream. The stream payload itself is the primary incremental output, but it does not replace the final JSON-RPC response for the originating request. This CEP is intended for cases where data is naturally incremental, long-lived, or unbounded, and where representing the result as one reassembled MCP request or response would be artificial or inefficient. @@ -26,12 +26,12 @@ Some use cases are different in nature: - progressive delivery where partial consumption is desirable - cases where no single final rendered payload is the right abstraction -This CEP defines an open-ended stream-transfer profile that: +This CEP defines an open-ended stream profile that: - reuses the existing single-kind ContextVM transport model - reuses MCP `notifications/progress` as the stream envelope - uses the request `progressToken` as the stream identifier -- supports ordered `start`, `accept`, `chunk`, `close`, and `abort` frames +- supports ordered `start`, `accept`, `chunk`, `ping`, `pong`, `close`, and `abort` frames - treats the stream itself as the payload rather than a bounded reassembly artifact - allows receivers to process fragments incrementally as they arrive @@ -57,7 +57,7 @@ Advertisement surfaces: Support semantics: -- `support_open_stream` indicates support for the open-ended stream-transfer profile defined by this CEP. +- `support_open_stream` indicates support for the open-ended stream profile defined by this CEP. ### Request-Level Activation @@ -84,6 +84,8 @@ The sender: - MUST terminate the stream with either `close` or `abort` - MUST NOT silently stop transmission without a terminal frame unless transport failure prevents completion +Multiple streams MAY exist concurrently between the same peers, but each active stream MUST use a distinct `progressToken`. A sender MUST NOT send a second `start` for a stream that is already active under the same `progressToken`. + ### Progress Notification Framing Open-ended stream frames are carried inside MCP `notifications/progress` params. The MCP envelope remains valid and additive; ContextVM defines additional frame semantics inside the params object. @@ -107,7 +109,7 @@ Example conceptual envelope: } ``` -The sender MUST use `progress` values that increase monotonically across the stream, consistent with MCP progress rules. +The sender MUST use `progress` values that increase monotonically across the stream, consistent with MCP progress rules. `progress` orders all stream frames, including control frames, and MUST NOT be interpreted as a chunk counter. ### Frame Types @@ -123,7 +125,7 @@ This CEP defines seven frame types: #### Common Fields -All open-stream-transfer frames MUST include a ContextVM-specific transport object with: +All open-stream frames MUST include a ContextVM-specific transport object with: - `type`: MUST be `open-stream` - `frameType`: one of `start`, `accept`, `chunk`, `ping`, `pong`, `close`, `abort` @@ -161,6 +163,16 @@ Rules: - A sender that is required to wait for confirmation MUST NOT send `chunk` frames before receiving `accept`. - `accept` SHOULD remain minimal and does not negotiate additional stream parameters in v1. +##### When `accept` Is Required + +`accept` is conditional bootstrap confirmation, not a universal requirement. + +This mirrors the `accept` semantics defined in [`CEP-22`](/src/content/docs/spec/ceps/cep-22.md), so implementations can reuse the same conceptual model for conditional bootstrap confirmation and avoid semantic drift between the two transfer profiles. + +- If the sender already knows that the receiver supports this CEP for the exchange through prior negotiation, explicit capability advertisement, or other valid context for the exchange, it MAY send `chunk` frames immediately after `start`. +- If support is not yet known for the exchange, the sender MUST wait for `accept` before sending the first `chunk` frame. +- In stateless bootstrap flows where no prior support knowledge exists, `accept` is required before the first `chunk`. + #### `chunk` Frame The `chunk` frame carries one ordered fragment of stream payload. @@ -168,12 +180,15 @@ The `chunk` frame carries one ordered fragment of stream payload. Required fields: - `data`: chunk payload +- `chunkIndex`: contiguous chunk index Rules: -- For open-stream-transfer frames, MCP `progress` is the normative stream-ordering field. +- For open-stream frames, MCP `progress` is the normative stream-ordering field for all frames. - Each `chunk` frame MUST use a `progress` value greater than the preceding stream frame's `progress` value. +- `chunkIndex` MUST start at `0` for the first `chunk` frame in the stream and increase contiguously by `1` for each subsequent `chunk` frame. - The payload represented by `data` is one ordered fragment of stream output. +- Receivers MUST use `chunkIndex`, not `progress`, to validate chunk contiguity and payload completeness. #### `ping` Frame @@ -187,6 +202,7 @@ Rules: - Either peer MAY send `ping` on an active stream. - `nonce` MUST identify the probe uniquely within the stream. +- Receivers SHOULD enforce a local maximum nonce size of `64 bytes` and MAY reject, ignore, or abort on oversized nonces. - `ping` carries no stream payload. #### `pong` Frame @@ -202,6 +218,8 @@ Rules: - A receiver of `ping` MUST respond with `pong` for the same stream unless the stream has already terminated. - `pong.nonce` MUST match the triggering `ping.nonce`. - `pong` acknowledges peer responsiveness only and does not acknowledge delivery or processing of stream payload. +- Implementations MAY apply local anti-abuse policy to `ping` handling, including ignoring, coalescing, rate-limiting, or aborting on redundant consecutive `ping` frames. +- Implementations SHOULD treat only the first valid `ping` in a consecutive run of `ping` frames as requiring a `pong`; later consecutive `ping` frames MAY be ignored until another valid non-keepalive stream frame is observed. #### `close` Frame @@ -236,8 +254,11 @@ Rules: - a stream MUST begin with `start` - if confirmation is required for the stream, `accept` MUST be received before the first `chunk` -- `progress` values for open-stream-transfer frames MUST increase monotonically across the stream +- `progress` values for open-stream frames MUST increase monotonically across the stream +- receivers MUST treat `progress` as the canonical frame-ordering field, not as a chunk count +- `chunk` frames MUST include contiguous `chunkIndex` values beginning at `0` - `pong` MUST correspond to an earlier `ping` on the same stream +- a second `start` received for an already active `progressToken` MUST cause the stream to fail - successful completion requires `close` - if `close` arrives after malformed or non-monotonic ordering, the stream MUST fail @@ -259,7 +280,8 @@ Receivers that support this CEP: - MUST process frames in stream order - MUST reject or fail malformed frame sequences - MUST treat `abort` as terminal -- MUST fail a stream if `close` is received before a valid monotonic `progress` sequence has been observed +- MUST allow a valid zero-chunk stream in which `close` follows `start` without any `chunk` frames +- MUST fail a stream if `close` is received before `start` or after malformed ordering Receivers MAY expose stream fragments to applications incrementally as they arrive. @@ -275,9 +297,21 @@ In stateless operation: For stateless client-to-server streaming where the client has not previously learned server support, the client MUST send `start` first and wait for `accept` before sending `chunk` frames. +### Request Completion Semantics + +Open-ended streaming supplements the lifecycle of the originating JSON-RPC request; it does not replace it. + +Rules: + +- A stream associated with a request MUST still conclude with exactly one final JSON-RPC response for that request. +- `close` indicates that no more stream frames will be sent, but it does not itself satisfy the JSON-RPC request/response lifecycle. +- After sending `close`, the sender MUST send the final JSON-RPC success response for the originating request. +- If a stream associated with a request is terminated with `abort`, the sender SHOULD send a final JSON-RPC error response when it is still able to do so. +- Implementations MUST NOT synthesize successful final JSON-RPC responses locally solely from receipt of `close`. + ### Timeout and Keepalive Semantics -Receipt of any valid open-stream-transfer frame counts as stream activity. +Receipt of any valid open-stream frame counts as stream activity. Implementations MUST maintain an idle timeout for each active stream. @@ -286,10 +320,22 @@ Rules: - receipt of `start`, `accept`, `chunk`, `ping`, `pong`, `close`, or `abort` MUST reset the idle timeout - if no valid frame is received before the idle timeout expires, the peer MUST send `ping` - the receiver of `ping` MUST respond with `pong` carrying the same `nonce` +- implementations MAY apply local anti-abuse policy to keepalive traffic, including ignoring or rejecting redundant consecutive `ping` frames and rejecting oversized `nonce` values - if the probing peer does not receive a matching `pong` before its probe timeout expires, it MUST treat the stream as failed - a peer that fails the stream due to probe timeout SHOULD send `abort` if it is still able to transmit - implementations SHOULD enforce a hard maximum timeout or other resource policy for long-lived streams +### Relay Rate and Flow-Control Guidance + +Nostr relays may impose different event-rate, buffering, or publication policies. + +Implementations: + +- MUST NOT assume that all relays accept the same sustained event rate +- SHOULD throttle frame emission conservatively enough to respect expected relay policies +- MAY apply local policy to abort, defer, or deprioritize streams that exceed relay-safety limits +- MUST NOT assume that this CEP provides transport-level backpressure signaling in v1 + ### Example: Server-to-Client Open Stream Client sends a request with a `progressToken`: @@ -338,8 +384,9 @@ Server sends stream fragments: "progressToken": "req-123", "progress": 2, "cvm": { - "type": "open-stream-transfer", + "type": "open-stream", "frameType": "chunk", + "chunkIndex": 0, "data": "Hello" } } @@ -356,6 +403,7 @@ Server sends stream fragments: "cvm": { "type": "open-stream", "frameType": "chunk", + "chunkIndex": 1, "data": " world" } } @@ -380,6 +428,24 @@ Server closes the stream: } ``` +Server returns the final JSON-RPC response for the originating request: + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": { + "content": [ + { + "type": "text", + "text": "Stream completed successfully" + } + ], + "isError": false + } +} +``` + ### Example: Stateless Client-to-Server Stream Bootstrap Client announces intent to begin a stream: @@ -419,7 +485,7 @@ Server confirms support: } ``` -After `accept`, the client sends `chunk` frames and eventually terminates the stream with `close` or `abort`. +After `accept`, the client sends `chunk` frames and eventually terminates the stream with `close` or `abort`. If the stream is associated with a JSON-RPC request, the exchange still concludes with the final JSON-RPC response for that request. ## Backward Compatibility From c5a7f84397483ccf8f7e2a1590e268d028fafdb7 Mon Sep 17 00:00:00 2001 From: ContextVM Date: Sun, 3 May 2026 13:18:52 +0200 Subject: [PATCH 5/6] docs(cep-41): clarify chunk payload semantics and add out-of-order handling rules Clarify that chunk data represents an ordered fragment of stream payload with reference to CEP-22 chunk-payload semantics. Add rules for buffering valid out-of-order chunk frames within bounded limits, tracking missing chunkIndex values as provisional gaps, and enforcing bounded buffering policies. Add rules for handling invalid pong frames with unknown, duplicate, expired, or already-satisfied nonces. Improve anti-abuse policy language for ping/pong handling. --- src/content/docs/spec/ceps/cep-41.md | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/src/content/docs/spec/ceps/cep-41.md b/src/content/docs/spec/ceps/cep-41.md index f9fa280..2d7772f 100644 --- a/src/content/docs/spec/ceps/cep-41.md +++ b/src/content/docs/spec/ceps/cep-41.md @@ -179,7 +179,7 @@ The `chunk` frame carries one ordered fragment of stream payload. Required fields: -- `data`: chunk payload +- `data`: chunk payload fragment - `chunkIndex`: contiguous chunk index Rules: @@ -187,8 +187,11 @@ Rules: - For open-stream frames, MCP `progress` is the normative stream-ordering field for all frames. - Each `chunk` frame MUST use a `progress` value greater than the preceding stream frame's `progress` value. - `chunkIndex` MUST start at `0` for the first `chunk` frame in the stream and increase contiguously by `1` for each subsequent `chunk` frame. -- The payload represented by `data` is one ordered fragment of stream output. +- `data` carries one ordered fragment of the stream payload, following the same chunk-payload semantics as [`CEP-22`](/src/content/docs/spec/ceps/cep-22.md). - Receivers MUST use `chunkIndex`, not `progress`, to validate chunk contiguity and payload completeness. +- Receivers MAY buffer valid out-of-order `chunk` frames within bounded local limits and process them once the contiguous `chunkIndex` sequence resumes. +- Receivers MAY track missing `chunkIndex` values as provisional gaps while the stream remains active. +- Receivers SHOULD enforce bounded buffering or equivalent local resource policy for unresolved chunk gaps. #### `ping` Frame @@ -218,8 +221,9 @@ Rules: - A receiver of `ping` MUST respond with `pong` for the same stream unless the stream has already terminated. - `pong.nonce` MUST match the triggering `ping.nonce`. - `pong` acknowledges peer responsiveness only and does not acknowledge delivery or processing of stream payload. -- Implementations MAY apply local anti-abuse policy to `ping` handling, including ignoring, coalescing, rate-limiting, or aborting on redundant consecutive `ping` frames. -- Implementations SHOULD treat only the first valid `ping` in a consecutive run of `ping` frames as requiring a `pong`; later consecutive `ping` frames MAY be ignored until another valid non-keepalive stream frame is observed. +- A `pong` with an unknown, duplicate, expired, or already-satisfied `nonce` is invalid for keepalive matching and MUST NOT be treated as evidence of stream liveness. +- Receivers MAY ignore invalid `pong` frames and MAY apply local logging or anti-abuse policy to them. +- Implementations MAY apply local anti-abuse policy to `ping` handling, including ignoring, coalescing, rate-limiting, or aborting on excessive keepalive traffic. #### `close` Frame @@ -243,6 +247,7 @@ Rules: - Either peer MAY send `abort`. - Receivers MUST treat `abort` as terminal for the stream. - `reason` is advisory only. +- A peer MAY send `abort` when local policy determines that successful continuation is no longer acceptable or no longer plausible, including resource exhaustion, excessive unresolved gaps, timeout, or anti-abuse conditions. ### Validation Rules @@ -257,6 +262,9 @@ Rules: - `progress` values for open-stream frames MUST increase monotonically across the stream - receivers MUST treat `progress` as the canonical frame-ordering field, not as a chunk count - `chunk` frames MUST include contiguous `chunkIndex` values beginning at `0` +- receivers MAY buffer valid out-of-order `chunk` frames within bounded local limits while awaiting missing earlier `chunkIndex` values +- receivers MAY treat missing `chunkIndex` positions as provisional gaps while the stream remains active +- receivers MUST NOT treat a gap alone as terminal failure while the stream remains active, except under local timeout or resource policy - `pong` MUST correspond to an earlier `ping` on the same stream - a second `start` received for an already active `progressToken` MUST cause the stream to fail - successful completion requires `close` @@ -282,6 +290,7 @@ Receivers that support this CEP: - MUST treat `abort` as terminal - MUST allow a valid zero-chunk stream in which `close` follows `start` without any `chunk` frames - MUST fail a stream if `close` is received before `start` or after malformed ordering +- MAY terminate a stream with `abort` when local timeout, buffering, relay-safety, or anti-abuse policy makes continued processing unacceptable Receivers MAY expose stream fragments to applications incrementally as they arrive. @@ -320,7 +329,7 @@ Rules: - receipt of `start`, `accept`, `chunk`, `ping`, `pong`, `close`, or `abort` MUST reset the idle timeout - if no valid frame is received before the idle timeout expires, the peer MUST send `ping` - the receiver of `ping` MUST respond with `pong` carrying the same `nonce` -- implementations MAY apply local anti-abuse policy to keepalive traffic, including ignoring or rejecting redundant consecutive `ping` frames and rejecting oversized `nonce` values +- implementations MAY apply local anti-abuse policy to keepalive traffic, including rate-limiting, coalescing, ignoring, or rejecting excessive `ping` traffic and rejecting oversized `nonce` values - if the probing peer does not receive a matching `pong` before its probe timeout expires, it MUST treat the stream as failed - a peer that fails the stream due to probe timeout SHOULD send `abort` if it is still able to transmit - implementations SHOULD enforce a hard maximum timeout or other resource policy for long-lived streams @@ -448,6 +457,8 @@ Server returns the final JSON-RPC response for the originating request: ### Example: Stateless Client-to-Server Stream Bootstrap +The following example shows the stream bootstrap phase only. As in [`Request-Level Activation`](#request-level-activation), the initiating JSON-RPC request for this exchange has already been sent and already supplied the `progressToken`. + Client announces intent to begin a stream: ```json From 6e65569268489bff6bb16b20fc75748600a18fd4 Mon Sep 17 00:00:00 2001 From: ContextVM Date: Sun, 3 May 2026 16:03:24 +0200 Subject: [PATCH 6/6] docs(cep-41): add optional fields and clarify stream frame semantics Add optional `contentType` and `contentEncoding` fields to `start` frame, and `lastChunkIndex` to `close` frame. Clarify that `data` MUST be a JSON string in v1, define the logical stream payload as byte-preserving concatenation of `data` values, and add rules for out-of-order chunk handling and stream completion validation. Define "fail a stream" behavior for error handling. --- src/content/docs/spec/ceps/cep-41.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/src/content/docs/spec/ceps/cep-41.md b/src/content/docs/spec/ceps/cep-41.md index 2d7772f..d8137ed 100644 --- a/src/content/docs/spec/ceps/cep-41.md +++ b/src/content/docs/spec/ceps/cep-41.md @@ -145,11 +145,19 @@ Required fields: - `mode`: `stream` +Optional fields: + +- `contentType` +- `contentEncoding` + Rules: - If `mode` is omitted, receivers MAY reject the stream; senders SHOULD always provide it. - In this CEP version, senders MUST use `mode: "stream"`. - Receivers MUST reject unknown or unsupported stream modes. +- `contentType`, when present, SHOULD identify the media type of the logical stream payload. +- `contentEncoding`, when present, SHOULD identify how `data` string fragments are encoded for interpretation by the application. +- `contentType` and `contentEncoding` are advisory metadata only and do not define stream correctness. #### `accept` Frame @@ -229,10 +237,16 @@ Rules: The `close` frame signals successful sender-side closure of the stream. +Optional fields: + +- `lastChunkIndex` + Rules: - `close` is required for successful stream completion. - `close` indicates that no further `chunk` frames will be sent for the stream. +- If the stream included one or more `chunk` frames, `close.lastChunkIndex` MUST equal the greatest `chunkIndex` sent for the stream. +- If the stream included no `chunk` frames, `close.lastChunkIndex` MUST be omitted. #### `abort` Frame @@ -255,6 +269,8 @@ Rules: Receivers MUST validate stream ordering using MCP `progress`. +To fail a stream means to treat it as unsuccessfully terminated, release local state for it, and NOT treat it as successfully completed. A peer that fails a stream SHOULD send `abort` with an advisory `reason` when it is still able to transmit. + Rules: - a stream MUST begin with `start` @@ -268,6 +284,9 @@ Rules: - `pong` MUST correspond to an earlier `ping` on the same stream - a second `start` received for an already active `progressToken` MUST cause the stream to fail - successful completion requires `close` +- if `close.lastChunkIndex` is present, receivers MUST treat it as the completeness bound for the stream payload +- when `close` is received and one or more `chunk` frames were sent, successful completion requires receipt of every `chunkIndex` from `0` through `lastChunkIndex` +- if gaps remain when `close` is received, receivers MAY wait a bounded local grace period for delayed chunks or MAY fail immediately under local policy - if `close` arrives after malformed or non-monotonic ordering, the stream MUST fail This CEP does not define replay, selective retransmission, or repair. @@ -511,6 +530,7 @@ This CEP introduces no breaking changes: - [CEP-6: Public Server Announcements](/spec/ceps/cep-6) - [CEP-19: Ephemeral Gift Wraps](/spec/ceps/cep-19) - [CEP-22: Oversized Payload Transfer](/spec/ceps/cep-22) +- [CEP-35: Discoverability Patterns for ContextVM Capabilities](/spec/ceps/informational/cep-35) ## Reference Implementation