Skip to content

Cloudflare Stream API integration#103

Open
aloewright wants to merge 2 commits into
mainfrom
conductor/alo-135-cloudflare-stream-api-integration
Open

Cloudflare Stream API integration#103
aloewright wants to merge 2 commits into
mainfrom
conductor/alo-135-cloudflare-stream-api-integration

Conversation

@aloewright
Copy link
Copy Markdown
Owner

@aloewright aloewright commented May 8, 2026

Closes ALO-135.

Summary

End-to-end Cloudflare Stream integration on top of the existing webhook
plumbing:

  • Trigger from R2. encoding.ts now POSTs to /stream/copy (was
    the wrong /stream endpoint) with an HTTPS source URL Stream can
    actually fetch. The previous r2://... body would have failed in
    production — nothing exercised the path.
  • Source URL. New /api/internal/stream-source endpoint feeds R2
    bytes to Stream, authorized by an HMAC-signed query string. Reuses
    CF_STREAM_WEBHOOK_SECRET so the inbound webhook secret and the
    outbound source signing key rotate together.
  • Polling fallback. pollStuckEncodings reconciles rows that
    missed their webhook callback. Wired into the existing daily cron —
    any row in encoding for >5 minutes gets a direct
    GET /accounts/{id}/stream/{uid} and the HLS manifest URL persists
    onto the row. Status transitions go through canTransition so a
    stale poll cannot drag a ready row backwards.

Webhook (stream-webhook.ts) remains the primary signal; polling is a
safety net so a dropped delivery doesn't pin a row in encoding
forever.

Files

  • src/workers/encoding.ts/stream/copy + signed source URL builder + verifyStreamSourceSignature
  • src/workers/stream-source.ts — public R2 source endpoint
  • src/workers/stream-poll.ts — stuck-row reconciliation sweep
  • src/workers/index.ts — wires the source route + polling cron
  • wrangler.toml — documents STREAM_SOURCE_ORIGIN and the dual role of CF_STREAM_WEBHOOK_SECRET
  • Unit tests for all three new modules

Test plan

  • npm test — 518 passing (47 files)
  • npm run lint — 0 warnings, 0 errors
  • npm run type-check — clean
  • AI Gateway guard — 0 findings

Webhook is the primary signal but deliveries can be lost. Add a
scheduled reconciliation that GETs the Stream API for any video stuck
in `encoding`, applies the same status/HLS/thumbnail mapping the
webhook would have, and is wired into the existing daily cron.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 8, 2026 14:47
@aloewright aloewright added the conductor Conductor-managed PR label May 8, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

Warning

Rate limit exceeded

@aloewright has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 39 minutes and 56 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cce00c45-1f70-49a8-9a39-05e1f48c8249

📥 Commits

Reviewing files that changed from the base of the PR and between 4d3c13f and c97e834.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (8)
  • src/workers/encoding.test.ts
  • src/workers/encoding.ts
  • src/workers/index.ts
  • src/workers/stream-poll.test.ts
  • src/workers/stream-poll.ts
  • src/workers/stream-source.test.ts
  • src/workers/stream-source.ts
  • wrangler.toml
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch conductor/alo-135-cloudflare-stream-api-integration

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ecc-tools
Copy link
Copy Markdown
Contributor

ecc-tools Bot commented May 8, 2026

ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a polling mechanism to reconcile Cloudflare Stream videos stuck in the 'encoding' state, acting as a safety net for missed webhooks. The changes include the core polling logic in a new module, integration into the scheduled worker handler, and associated unit tests. Feedback identifies an opportunity to use the new STREAM_ENABLED flag to gate the polling logic, suggests removing a redundant filter in the database results processing, and recommends using encodeURIComponent for safer API URL construction.

Comment thread src/workers/index.ts
CF_STREAM_WEBHOOK_SECRET?: string;
CF_STREAM_API_TOKEN?: string;
CLOUDFLARE_ACCOUNT_ID?: string;
STREAM_ENABLED?: string;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The STREAM_ENABLED environment variable is added to the bindings but is not used to gate the polling logic or the webhook handler in this file. If this is intended to be a feature flag to enable/disable Cloudflare Stream integration, it should be checked before executing pollStreamForEncodingVideos(env) in the scheduled handler.

Comment thread src/workers/stream-poll.ts Outdated
.bind(limit)
.all<EncodingRow>();

const candidates = (rows.results ?? []).filter((r) => r.stream_video_id);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This filter is redundant because the SQL query already includes AND stream_video_id IS NOT NULL. Removing it simplifies the code.

Suggested change
const candidates = (rows.results ?? []).filter((r) => r.stream_video_id);
const candidates = rows.results ?? [];

Comment thread src/workers/stream-poll.ts Outdated
for (const row of candidates) {
try {
const res = await fetchImpl(
`https://api.cloudflare.com/client/v4/accounts/${accountId}/stream/${row.stream_video_id}`,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The URL for the Cloudflare Stream API is constructed using template literals without encoding the accountId or row.stream_video_id. While these are expected to be safe strings, it is best practice to use encodeURIComponent to ensure the URL is correctly formed if these values ever contain special characters.

Suggested change
`https://api.cloudflare.com/client/v4/accounts/${accountId}/stream/${row.stream_video_id}`,
`https://api.cloudflare.com/client/v4/accounts/${encodeURIComponent(accountId)}/stream/${encodeURIComponent(row.stream_video_id)}`,

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4844b7b074

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +102 to +104
const allowedFrom = (VIDEO_STATUSES as readonly VideoStatus[]).filter((from) =>
canTransition(from, next),
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restrict poll updates to rows still in encoding

This sweep computes allowedFrom from the full state machine, so when next resolves to encoding it permits failed as a predecessor. In a race where the row was selected as encoding but a webhook flips it to failed before this UPDATE, the stale poll result can move it back to encoding and effectively resurrect a failed job. Since this worker is only reconciling rows that were originally encoding, the update predicate should keep status = 'encoding' (or equivalent optimistic check) to avoid backward transitions caused by stale poll reads.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a polling mechanism to reconcile Cloudflare Stream encoding jobs that may have missed webhook notifications. It adds a new pollStreamForEncodingVideos utility, integrates it into the scheduled worker tasks, and includes comprehensive unit tests. I have reviewed the implementation and suggest reducing the default batch limit to 20 to ensure compatibility with Cloudflare Workers' subrequest limits.

Comment thread src/workers/stream-poll.ts Outdated
}

const fetchImpl = deps.fetch ?? fetch;
const limit = deps.limit ?? 25;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The default limit of 25, combined with one fetch and one DB.run per row, results in 51 subrequests (1 initial SELECT + 25 GETs + 25 UPDATEs). This exceeds the 50-subrequest limit for Cloudflare Workers on the Free plan. Consider lowering the default limit to 20 to ensure the sweep runs successfully across all plan types.

Suggested change
const limit = deps.limit ?? 25;
const limit = deps.limit ?? 20;

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

…ack (ALO-135)

Wire the upload→encode→playback path end to end against the Cloudflare
Stream API:

- encoding.ts now POSTs to /stream/copy (was /stream) with an HTTPS
  source URL. The previous r2:// URL was unfetchable by Stream — the
  pipeline only worked because nothing exercised it.
- New /api/internal/stream-source endpoint streams R2 bytes to Stream,
  authorized by an HMAC-signed query string (reuses
  CF_STREAM_WEBHOOK_SECRET so secrets rotate together).
- New stream-poll.ts reconciles rows that missed their webhook
  callback. Wired into the daily scheduled handler — any row stuck in
  `encoding` for >5 minutes gets a direct GET /stream/{uid} and the
  HLS manifest URL persists onto the videos row.

Webhook flow remains the primary path; polling is a safety net so a
dropped callback doesn't pin a row in `encoding` forever.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@ecc-tools
Copy link
Copy Markdown
Contributor

ecc-tools Bot commented May 9, 2026

ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conductor Conductor-managed PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants