Repository navigation
Conversation
…es working createX402Fetch wrapped every paid 2xx body with getReader(), which a Node-stream body (node-fetch v2) lacks. The TypeError surfaced after the payment as a retryable transport failure, so maxRetries signed new payments. Such bodies are now returned unwrapped; GlassnodeAPI already never retries a paid x402 body failure. Responses without text() (a minimal custom fetch exposing only json(), accepted by 1.0.0) are parsed with json() again instead of failing as malformed JSON.
martende
left a comment
There was a problem hiding this comment.
Solid fix overall — separating response.text() from JSON.parse(text), guarding paid x402 streams, and tagging unpaid responses with Symbol.for('glassnode-api.x402.safe-body-retry') cleanly addresses #59, #60, and #61.
Left a few inline comments on a test helper bug in test/x402.spec.ts and a couple of edge cases in guardPaidBody / GlassnodeAPI.send.
| function nodeStreamResponse(text: () => Promise<string>): Response { | ||
| const body = Readable.from([]); | ||
| return { | ||
| ok: true, | ||
| status: 200, | ||
| statusText: 'OK', | ||
| headers: new Headers(), | ||
| body, | ||
| bodyUsed: false, | ||
| text, | ||
| } as unknown as Response; | ||
| } | ||
|
|
||
| it('paid request -> 200 with a Node-stream body: success, one payment', async () => { | ||
| const baseFetch = scripted(() => | ||
| nodeStreamResponse(async () => JSON.stringify(['/market/price_usd_close'])) | ||
| ); | ||
| const signTypedData = vi.fn(async (): Promise<`0x${string}`> => FAKE_SIGNATURE); | ||
| const paidFetch = await createX402Fetch({ | ||
| account: fakeAccount(signTypedData), | ||
| fetch: baseFetch as typeof fetch, | ||
| }); | ||
| const result = await paidApi(paidFetch, { maxRetries: 2 }).getMetricList(); | ||
| expect(result).toEqual(['/market/price_usd_close']); | ||
| expect(signTypedData).toHaveBeenCalledTimes(1); | ||
| }); | ||
|
|
||
| it('paid request -> 200 with a Node-stream body that fails in transit: one payment, not retried', async () => { | ||
| const baseFetch = scripted(() => | ||
| nodeStreamResponse(async () => { | ||
| throw new TypeError('terminated'); | ||
| }) | ||
| ); | ||
| const signTypedData = vi.fn(async (): Promise<`0x${string}`> => FAKE_SIGNATURE); | ||
| const paidFetch = await createX402Fetch({ | ||
| account: fakeAccount(signTypedData), | ||
| fetch: baseFetch as typeof fetch, | ||
| }); | ||
| const err = await caught(paidApi(paidFetch, { maxRetries: 2 }).getMetricList()); | ||
| expect(err).toBeInstanceOf(GlassnodeError); | ||
| expect(signTypedData).toHaveBeenCalledTimes(1); | ||
| expect(baseFetch.mock.calls.filter(isPaidCall)).toHaveLength(1); | ||
| }); |
There was a problem hiding this comment.
nodeStreamResponse initializes bodyUsed: false and never flips it to true when text() is called (unlike real node-fetch v2, which sets bodyUsed = true as soon as consumption starts).
Because bodyUsed remains false, bodyReadStarted (readableBody && response.bodyUsed) in src/glassnode-api.ts:829 evaluates to false, so 'paid request -> 200 with a Node-stream body that fails in transit: one payment, not retried' actually falls through to GlassnodeValidationError (line 848) instead of GlassnodeNetworkError (line 836) and never exercises the (!this.x402 || isUnpaidResponse(response)) retry guard on line 845 (expect(err).toBeInstanceOf(GlassnodeError) masks this).
| function nodeStreamResponse(text: () => Promise<string>): Response { | |
| const body = Readable.from([]); | |
| return { | |
| ok: true, | |
| status: 200, | |
| statusText: 'OK', | |
| headers: new Headers(), | |
| body, | |
| bodyUsed: false, | |
| text, | |
| } as unknown as Response; | |
| } | |
| it('paid request -> 200 with a Node-stream body: success, one payment', async () => { | |
| const baseFetch = scripted(() => | |
| nodeStreamResponse(async () => JSON.stringify(['/market/price_usd_close'])) | |
| ); | |
| const signTypedData = vi.fn(async (): Promise<`0x${string}`> => FAKE_SIGNATURE); | |
| const paidFetch = await createX402Fetch({ | |
| account: fakeAccount(signTypedData), | |
| fetch: baseFetch as typeof fetch, | |
| }); | |
| const result = await paidApi(paidFetch, { maxRetries: 2 }).getMetricList(); | |
| expect(result).toEqual(['/market/price_usd_close']); | |
| expect(signTypedData).toHaveBeenCalledTimes(1); | |
| }); | |
| it('paid request -> 200 with a Node-stream body that fails in transit: one payment, not retried', async () => { | |
| const baseFetch = scripted(() => | |
| nodeStreamResponse(async () => { | |
| throw new TypeError('terminated'); | |
| }) | |
| ); | |
| const signTypedData = vi.fn(async (): Promise<`0x${string}`> => FAKE_SIGNATURE); | |
| const paidFetch = await createX402Fetch({ | |
| account: fakeAccount(signTypedData), | |
| fetch: baseFetch as typeof fetch, | |
| }); | |
| const err = await caught(paidApi(paidFetch, { maxRetries: 2 }).getMetricList()); | |
| expect(err).toBeInstanceOf(GlassnodeError); | |
| expect(signTypedData).toHaveBeenCalledTimes(1); | |
| expect(baseFetch.mock.calls.filter(isPaidCall)).toHaveLength(1); | |
| }); | |
| function nodeStreamResponse(text: () => Promise<string>): Response { | |
| const body = Readable.from([]); | |
| const response = { | |
| ok: true, | |
| status: 200, | |
| statusText: 'OK', | |
| headers: new Headers(), | |
| body, | |
| bodyUsed: false, | |
| text: () => { | |
| response.bodyUsed = true; | |
| return text(); | |
| }, | |
| }; | |
| return response as unknown as Response; | |
| } | |
| it('paid request -> 200 with a Node-stream body: success, one payment', async () => { | |
| const baseFetch = scripted(() => | |
| nodeStreamResponse(async () => JSON.stringify(['/market/price_usd_close'])) | |
| ); | |
| const signTypedData = vi.fn(async (): Promise<`0x${string}`> => FAKE_SIGNATURE); | |
| const paidFetch = await createX402Fetch({ | |
| account: fakeAccount(signTypedData), | |
| fetch: baseFetch as typeof fetch, | |
| }); | |
| const result = await paidApi(paidFetch, { maxRetries: 2 }).getMetricList(); | |
| expect(result).toEqual(['/market/price_usd_close']); | |
| expect(signTypedData).toHaveBeenCalledTimes(1); | |
| }); | |
| it('paid request -> 200 with a Node-stream body that fails in transit: one payment, not retried', async () => { | |
| const baseFetch = scripted(() => | |
| nodeStreamResponse(async () => { | |
| throw new TypeError('terminated'); | |
| }) | |
| ); | |
| const signTypedData = vi.fn(async (): Promise<`0x${string}`> => FAKE_SIGNATURE); | |
| const paidFetch = await createX402Fetch({ | |
| account: fakeAccount(signTypedData), | |
| fetch: baseFetch as typeof fetch, | |
| }); | |
| const err = await caught(paidApi(paidFetch, { maxRetries: 2 }).getMetricList()); | |
| expect(err).toBeInstanceOf(GlassnodeNetworkError); | |
| expect(signTypedData).toHaveBeenCalledTimes(1); | |
| expect(baseFetch.mock.calls.filter(isPaidCall)).toHaveLength(1); | |
| }); |
| const bodyReadStarted = readableBody && response.bodyUsed; | ||
| const explicitAbort = | ||
| typeof parseError === 'object' && | ||
| parseError !== null && | ||
| ABORT_NAMES.has(String((parseError as { name?: unknown }).name)); | ||
| if (failure && (bodyReadStarted || explicitAbort)) { | ||
| retryAfterMs = undefined; | ||
| lastError = new GlassnodeNetworkError( | ||
| `Glassnode API error: ${this.redact(failure.message)}`, | ||
| { | ||
| cause: parseError, | ||
| timedOut: failure.timedOut || (attemptSignal.signal?.aborted ?? false), | ||
| } | ||
| ); |
There was a problem hiding this comment.
Line 840 checks timedOut: failure.timedOut || (attemptSignal.signal?.aborted ?? false), but the if condition on line 834 only checks if (failure && (bodyReadStarted || explicitAbort)).
If the per-attempt timeout signal aborts while reading a body from a custom fetch/polyfill (where response.body is absent or bodyUsed isn't flipped) and response.text() rejects with a generic TypeError('terminated') rather than AbortError/TimeoutError, bodyReadStarted || explicitAbort is false and the timeout is misclassified as a non-retryable GlassnodeValidationError.
| const bodyReadStarted = readableBody && response.bodyUsed; | |
| const explicitAbort = | |
| typeof parseError === 'object' && | |
| parseError !== null && | |
| ABORT_NAMES.has(String((parseError as { name?: unknown }).name)); | |
| if (failure && (bodyReadStarted || explicitAbort)) { | |
| retryAfterMs = undefined; | |
| lastError = new GlassnodeNetworkError( | |
| `Glassnode API error: ${this.redact(failure.message)}`, | |
| { | |
| cause: parseError, | |
| timedOut: failure.timedOut || (attemptSignal.signal?.aborted ?? false), | |
| } | |
| ); | |
| const bodyReadStarted = readableBody && response.bodyUsed; | |
| const attemptTimedOut = attemptSignal.signal?.aborted ?? false; | |
| const explicitAbort = | |
| typeof parseError === 'object' && | |
| parseError !== null && | |
| ABORT_NAMES.has(String((parseError as { name?: unknown }).name)); | |
| if (failure && (bodyReadStarted || explicitAbort || attemptTimedOut)) { | |
| retryAfterMs = undefined; | |
| lastError = new GlassnodeNetworkError( | |
| `Glassnode API error: ${this.redact(failure.message)}`, | |
| { | |
| cause: parseError, | |
| timedOut: failure.timedOut || attemptTimedOut, | |
| } | |
| ); |
(Minor related note: in the fetchFn catch block slightly above at line 758, timedOut: failure.timedOut could also use failure.timedOut || (attemptSignal.signal?.aborted ?? false) for the same reason.)
| return response.ok && typeof response.body?.getReader === 'function' | ||
| ? guardPaidBody( | ||
| response, | ||
| keys, | ||
| init?.signal ?? (input instanceof Request ? input.signal : undefined) | ||
| ) | ||
| : response; |
There was a problem hiding this comment.
Two subtle points here:
- Post-payment synchronous throw protection:
guardPaidBodyruns afterpaymentSent === trueoutside thetry / catch. Ifresponse.bodyis already locked/consumed (response.body.locked/response.bodyUsed) ornew Response(body, ...)throwsTypeError, that error would escapecreateX402Fetchunclassified and causeGlassnodeAPI(whenmaxRetries > 0) to retry and sign a second payment. Checking!response.bodyUsed && !response.body.locked(and/or wrappingguardPaidBodyconstruction intry / catchfalling back toreturn response) guarantees it can never throw synchronously after payment. - Cross-realm
Requestsignal:input instanceof Requestcan befalsefor cross-realmRequests (which the rest ofsrc/x402.tssupports via duck-typing innoRedirect/hasPaymentHeader). Using a duck-typed signal fallback keeps it consistent.
| const reader = response.body!.getReader(); | ||
| const body = new ReadableStream<Uint8Array>({ | ||
| async pull(controller) { | ||
| try { | ||
| const { done, value } = await reader.read(); | ||
| if (done) { | ||
| reader.releaseLock(); | ||
| controller.close(); | ||
| } else controller.enqueue(value); | ||
| } catch (error) { | ||
| reader.releaseLock(); | ||
| const paymentError = toPaidTransportError(error, keys); | ||
| // Some fetches reject body reads with AbortError even for a timeout signal. | ||
| if (signal?.aborted && signal.reason?.name === 'TimeoutError' && !paymentError.timedOut) { | ||
| controller.error( | ||
| new GlassnodePaymentError(paymentError.message, { | ||
| cause: error, | ||
| paymentMayHaveSettled: true, | ||
| timedOut: true, | ||
| }) | ||
| ); | ||
| } else controller.error(paymentError); | ||
| } | ||
| }, | ||
| async cancel(reason) { | ||
| try { | ||
| await reader.cancel(reason); | ||
| } finally { | ||
| reader.releaseLock(); | ||
| } | ||
| }, | ||
| }); |
There was a problem hiding this comment.
If guarded.body.cancel() is called while await reader.read() is pending in pull(), reader.cancel(reason) resolves the pending reader.read() with { done: true, value: undefined }. pull() then resumes and calls controller.close() on a stream controller that was already closed by cancel(), which throws TypeError [ERR_INVALID_STATE]: Invalid state: Controller is already closed into catch (error) and triggers toPaidTransportError + controller.error(...).
Tracking a cancelled flag in cancel() avoids closing/enqueuing or erroring after cancellation:
| const reader = response.body!.getReader(); | |
| const body = new ReadableStream<Uint8Array>({ | |
| async pull(controller) { | |
| try { | |
| const { done, value } = await reader.read(); | |
| if (done) { | |
| reader.releaseLock(); | |
| controller.close(); | |
| } else controller.enqueue(value); | |
| } catch (error) { | |
| reader.releaseLock(); | |
| const paymentError = toPaidTransportError(error, keys); | |
| // Some fetches reject body reads with AbortError even for a timeout signal. | |
| if (signal?.aborted && signal.reason?.name === 'TimeoutError' && !paymentError.timedOut) { | |
| controller.error( | |
| new GlassnodePaymentError(paymentError.message, { | |
| cause: error, | |
| paymentMayHaveSettled: true, | |
| timedOut: true, | |
| }) | |
| ); | |
| } else controller.error(paymentError); | |
| } | |
| }, | |
| async cancel(reason) { | |
| try { | |
| await reader.cancel(reason); | |
| } finally { | |
| reader.releaseLock(); | |
| } | |
| }, | |
| }); | |
| const reader = response.body!.getReader(); | |
| let cancelled = false; | |
| const body = new ReadableStream<Uint8Array>({ | |
| async pull(controller) { | |
| try { | |
| const { done, value } = await reader.read(); | |
| if (cancelled) return; | |
| if (done) { | |
| reader.releaseLock(); | |
| controller.close(); | |
| } else controller.enqueue(value); | |
| } catch (error) { | |
| reader.releaseLock(); | |
| if (cancelled) return; | |
| const paymentError = toPaidTransportError(error, keys); | |
| // Some fetches reject body reads with AbortError even for a timeout signal. | |
| if (signal?.aborted && signal.reason?.name === 'TimeoutError' && !paymentError.timedOut) { | |
| controller.error( | |
| new GlassnodePaymentError(paymentError.message, { | |
| cause: error, | |
| paymentMayHaveSettled: true, | |
| timedOut: true, | |
| }) | |
| ); | |
| } else controller.error(paymentError); | |
| } | |
| }, | |
| async cancel(reason) { | |
| cancelled = true; | |
| try { | |
| await reader.cancel(reason); | |
| } finally { | |
| reader.releaseLock(); | |
| } | |
| }, | |
| }); |
| try { | ||
| void response.body?.cancel().catch(() => {}); | ||
| } catch { | ||
| // Cleanup must not replace the HTTP failure that caused the retry. | ||
| } |
There was a problem hiding this comment.
When fetch is node-fetch v2, response.body is a Node Readable stream (which has .destroy(), not .cancel()). response.body?.cancel() throws a TypeError (swallowed by catch), leaving the underlying socket/stream unclosed on discarded 429/5xx responses.
| try { | |
| void response.body?.cancel().catch(() => {}); | |
| } catch { | |
| // Cleanup must not replace the HTTP failure that caused the retry. | |
| } | |
| try { | |
| if (typeof response.body?.cancel === 'function') { | |
| void response.body.cancel().catch(() => {}); | |
| } else { | |
| (response.body as { destroy?: () => void } | null | undefined)?.destroy?.(); | |
| } | |
| } catch { | |
| // Cleanup must not replace the HTTP failure that caused the retry. | |
| } |
| readableBody = !!response.body && !response.bodyUsed && !response.body.locked; | ||
| text = await response.text(); | ||
| } catch (parseError) { | ||
| // A paid response stream classifies its own failures; never repeat its payment. | ||
| if (isGlassnodeError(parseError)) throw parseError; | ||
| // Reading the body was cut off by the caller's abort: that is not a malformed body. | ||
| if (signal?.aborted) throw aborted(); |
There was a problem hiding this comment.
Nit: trace.lastDurationMs is recorded when headers arrive (line 771) and isn't updated if response.text() (or readErrorDetail on line 797) fails after a slow/stalled body transfer. Updating trace.lastDurationMs = performance.now() - sentAt in the catch path before emitting onRetry / throwing to onError would make hook durationMs reflect the actual time spent before the attempt failed.
| export function markUnpaidResponse(response: Response): Response { | ||
| Object.defineProperty(response, SAFE_BODY_RETRY, { value: true }); | ||
| return response; | ||
| } |
There was a problem hiding this comment.
Nit: if a custom fetch or test double returns a frozen Response-like object (Object.freeze(...)), Object.defineProperty will throw a TypeError. Wrapping in try { ... } catch {} makes markUnpaidResponse safe on non-extensible responses (failing closed to isUnpaidResponse === false).
Successful headers could be followed by a timeout or connection reset reported as a validation error. Caller cancellation during a terminal error body was swallowed, and discarded retryable responses retained their bodies.
Read response text separately from JSON parsing: body-transfer failures follow the network policy, while malformed JSON always fails validation, including with node-fetch v2. Cancel discarded 429/5xx bodies without waiting on cleanup and preserve caller cancellation after error-detail reading.
With createX402Fetch, paid-body failures remain never-retried payment errors. In x402 mode, successful body transfers can only be retried when createX402Fetch positively identifies the response as unpaid; a plain payment fetch's successful body failure is never retried, even with an explicit maxRetries. Library error classifications also survive separate package copies. Version and changelog are prepared for 1.0.1.
Fixes #59
Fixes #60
Fixes #61
Validation:
Follow-up fixes (72244ee):
createX402Fetchover a base fetch whose body is a Node stream (node-fetch v2) no longer throws after the payment was sent. Before, thatTypeErrorlooked like a retryable transport failure, and withmaxRetries: 2one call signed three payments (regression test added). Such bodies are returned unwrapped;GlassnodeAPInever retries a paid x402 body failure.json()(notext()) works again as in 1.0.0, instead of failing as malformed JSON in a patch release.