Cloudflare Stream API integration#103
Conversation
Webhook is the primary signal but deliveries can be lost. Add a scheduled reconciliation that GETs the Stream API for any video stuck in `encoding`, applies the same status/HLS/thumbnail mapping the webhook would have, and is wired into the existing daily cron. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (8)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR. |
There was a problem hiding this comment.
Code Review
This pull request introduces a polling mechanism to reconcile Cloudflare Stream videos stuck in the 'encoding' state, acting as a safety net for missed webhooks. The changes include the core polling logic in a new module, integration into the scheduled worker handler, and associated unit tests. Feedback identifies an opportunity to use the new STREAM_ENABLED flag to gate the polling logic, suggests removing a redundant filter in the database results processing, and recommends using encodeURIComponent for safer API URL construction.
| CF_STREAM_WEBHOOK_SECRET?: string; | ||
| CF_STREAM_API_TOKEN?: string; | ||
| CLOUDFLARE_ACCOUNT_ID?: string; | ||
| STREAM_ENABLED?: string; |
There was a problem hiding this comment.
The STREAM_ENABLED environment variable is added to the bindings but is not used to gate the polling logic or the webhook handler in this file. If this is intended to be a feature flag to enable/disable Cloudflare Stream integration, it should be checked before executing pollStreamForEncodingVideos(env) in the scheduled handler.
| .bind(limit) | ||
| .all<EncodingRow>(); | ||
|
|
||
| const candidates = (rows.results ?? []).filter((r) => r.stream_video_id); |
There was a problem hiding this comment.
| for (const row of candidates) { | ||
| try { | ||
| const res = await fetchImpl( | ||
| `https://api.cloudflare.com/client/v4/accounts/${accountId}/stream/${row.stream_video_id}`, |
There was a problem hiding this comment.
The URL for the Cloudflare Stream API is constructed using template literals without encoding the accountId or row.stream_video_id. While these are expected to be safe strings, it is best practice to use encodeURIComponent to ensure the URL is correctly formed if these values ever contain special characters.
| `https://api.cloudflare.com/client/v4/accounts/${accountId}/stream/${row.stream_video_id}`, | |
| `https://api.cloudflare.com/client/v4/accounts/${encodeURIComponent(accountId)}/stream/${encodeURIComponent(row.stream_video_id)}`, |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4844b7b074
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const allowedFrom = (VIDEO_STATUSES as readonly VideoStatus[]).filter((from) => | ||
| canTransition(from, next), | ||
| ); |
There was a problem hiding this comment.
Restrict poll updates to rows still in encoding
This sweep computes allowedFrom from the full state machine, so when next resolves to encoding it permits failed as a predecessor. In a race where the row was selected as encoding but a webhook flips it to failed before this UPDATE, the stale poll result can move it back to encoding and effectively resurrect a failed job. Since this worker is only reconciling rows that were originally encoding, the update predicate should keep status = 'encoding' (or equivalent optimistic check) to avoid backward transitions caused by stale poll reads.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Code Review
This pull request introduces a polling mechanism to reconcile Cloudflare Stream encoding jobs that may have missed webhook notifications. It adds a new pollStreamForEncodingVideos utility, integrates it into the scheduled worker tasks, and includes comprehensive unit tests. I have reviewed the implementation and suggest reducing the default batch limit to 20 to ensure compatibility with Cloudflare Workers' subrequest limits.
| } | ||
|
|
||
| const fetchImpl = deps.fetch ?? fetch; | ||
| const limit = deps.limit ?? 25; |
There was a problem hiding this comment.
The default limit of 25, combined with one fetch and one DB.run per row, results in 51 subrequests (1 initial SELECT + 25 GETs + 25 UPDATEs). This exceeds the 50-subrequest limit for Cloudflare Workers on the Free plan. Consider lowering the default limit to 20 to ensure the sweep runs successfully across all plan types.
| const limit = deps.limit ?? 25; | |
| const limit = deps.limit ?? 20; |
…ack (ALO-135)
Wire the upload→encode→playback path end to end against the Cloudflare
Stream API:
- encoding.ts now POSTs to /stream/copy (was /stream) with an HTTPS
source URL. The previous r2:// URL was unfetchable by Stream — the
pipeline only worked because nothing exercised it.
- New /api/internal/stream-source endpoint streams R2 bytes to Stream,
authorized by an HMAC-signed query string (reuses
CF_STREAM_WEBHOOK_SECRET so secrets rotate together).
- New stream-poll.ts reconciles rows that missed their webhook
callback. Wired into the daily scheduled handler — any row stuck in
`encoding` for >5 minutes gets a direct GET /stream/{uid} and the
HLS manifest URL persists onto the videos row.
Webhook flow remains the primary path; polling is a safety net so a
dropped callback doesn't pin a row in `encoding` forever.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR. |
Closes ALO-135.
Summary
End-to-end Cloudflare Stream integration on top of the existing webhook
plumbing:
encoding.tsnow POSTs to/stream/copy(wasthe wrong
/streamendpoint) with an HTTPS source URL Stream canactually fetch. The previous
r2://...body would have failed inproduction — nothing exercised the path.
/api/internal/stream-sourceendpoint feeds R2bytes to Stream, authorized by an HMAC-signed query string. Reuses
CF_STREAM_WEBHOOK_SECRETso the inbound webhook secret and theoutbound source signing key rotate together.
pollStuckEncodingsreconciles rows thatmissed their webhook callback. Wired into the existing daily cron —
any row in
encodingfor >5 minutes gets a directGET /accounts/{id}/stream/{uid}and the HLS manifest URL persistsonto the row. Status transitions go through
canTransitionso astale poll cannot drag a
readyrow backwards.Webhook (
stream-webhook.ts) remains the primary signal; polling is asafety net so a dropped delivery doesn't pin a row in
encodingforever.
Files
src/workers/encoding.ts—/stream/copy+ signed source URL builder +verifyStreamSourceSignaturesrc/workers/stream-source.ts— public R2 source endpointsrc/workers/stream-poll.ts— stuck-row reconciliation sweepsrc/workers/index.ts— wires the source route + polling cronwrangler.toml— documentsSTREAM_SOURCE_ORIGINand the dual role ofCF_STREAM_WEBHOOK_SECRETTest plan
npm test— 518 passing (47 files)npm run lint— 0 warnings, 0 errorsnpm run type-check— clean