fix(cli): add source discriminator to telemetry events#762
fix(cli): add source discriminator to telemetry events#762miguel-heygen wants to merge 3 commits into
Conversation
Addresses the gap where LLMs default to slide-like layouts (centered text over dark background repeated for every scene). The main skill now teaches: - Temporal map: write what the viewer sees per second before any HTML - Slideshow trap: explicit anti-patterns and how to break them - Scene variety: table of layout types to rotate between - One focus per frame: billboard-per-beat principle - Beat duration guide: impact/content/atmosphere timing - Easing vocabulary: intent-based ease selection instead of power2.out Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Agents default to CSS rectangles for illustrations, producing amateur visuals. The skill now: - Mandates inline SVG over CSS shapes for any non-text visual - Provides a table of SVG patterns per visual need (diagrams, node graphs, data viz, icons, decoratives, waveforms) - Requires 3-layer depth per scene (background + content + accent) - Includes the stroke draw-on pattern inline since it's the most commonly needed SVG animation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PostHog dashboard inflates render counts (~10k) vs Temporal (~400) because CLI local renders and web app renders share the same `render_complete` event name with no way to filter them apart. Adds `source: "cli"` to all CLI telemetry events at the client level so dashboards can filter by `source = "web"` to show only production renders. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
vanceingalls
left a comment
There was a problem hiding this comment.
First review at 33f809f0. CI is red on three required checks — pulling those first.
CI status (Rule 5)
File size check— failed at25767325783Render on windows-latest— failedTests on windows-latest— failed
The PR body doesn't address any of these. Per Rule 5, this can't APPROVE while a required check is red.
Audited
packages/cli/src/telemetry/client.ts:75(the one-line telemetry change)skills/hyperframes/SKILL.md(the 109/-4 skills diff — bundled with this PR)
Important — this PR bundles two unrelated changes that should ship as separate PRs
The PR body describes a single-purpose telemetry discriminator fix (one line, one file). The actual diff is:
packages/cli/src/telemetry/client.ts +1 / 0 (telemetry — in PR body)
skills/hyperframes/SKILL.md +109 / -4 (skills update — NOT in PR body)
The skills change is the same +109/-4 SKILL.md diff as #716 ("feat(skills): teach temporal thinking and visual variety") — same author, same commit shas 3073d0ab and e466b8d3. #762's history is [3073d0ab, e466b8d3, 33f809f0]; #716's is [3073d0ab, e466b8d3]. #762 is a strict superset of #716.
Pick one shape:
- Drop the skills commits from this PR, leave just
33f809f0(telemetry only) — single-purpose. #716 then lands the skills work. - Land this PR (skills + telemetry together), close #716 as merged-by-superset.
Either is fine, but the current state is two PRs with overlapping history — confusing for the merge queue and easy to double-merge.
Telemetry change — correctness review
packages/cli/src/telemetry/client.ts:75 injects source: "cli" into the properties spread inside trackEvent. Spread order is:
properties: {
...properties, // caller's props
source: "cli", // <-- new
cli_version: VERSION,
os: process.platform,
...
}source is BEFORE the system-metadata fields, so a caller passing { source: "user-override" } would have its value overwritten by the hardcoded "cli". PR body asserts this is intentional ("per-event props can't accidentally overwrite it"). That's the right shape — the discriminator should be authoritative.
Verified the dashboard impact: the companion hyperframes-internal PR mentioned in the body needs to land too, otherwise web-app renders still come through without source, defeating the filter. Worth flagging in the merge order.
Important — CI must be addressed before merge
The three failing checks need a look:
File size checkfailing on a +1/-0 telemetry change is unusual; almost certainly the skills SKILL.md bumping past a size cap or the bundled .gitignore changes. If it's the skill, that's a structural decision (split the file vs. bump the cap).- Windows render/test failures on this branch — pull the job logs; if these are flake, kick a re-run. If they're real, the skills bundle change may have broken windows-side path handling.
Verdict
Verdict: REQUEST CHANGES
Reasoning: Three required CI checks are red (File size check, Render on windows-latest, Tests on windows-latest); PR bundles a telemetry one-liner with the entire skills-temporal-thinking diff from #716, creating overlapping PR history. Either split this PR (telemetry only, drop the skills commits) or close #716 as superseded.
— Vai
Summary
source: "cli"to all CLI telemetry events at the client levelProblem
PostHog dashboard shows ~10k renders but Temporal shows <400. The discrepancy exists because CLI local renders (
hyperframes render) and web app renders share the samerender_completeevent name in PostHog with no way to filter them apart.CLI events affected:
cli_command,render_complete,render_error,init_template,browser_install— all now carrysource: "cli".Fix
Single-line change in
packages/cli/src/telemetry/client.ts— thetrackEventwrapper already injects system metadata into all events, so addingsource: "cli"there tags every CLI event automatically.Companion PR on
hyperframes-internaladdssource: "web"to the web app's analytics wrapper. Together, dashboards can filtersource = "web"to show only production renders.Test plan
trackEventcall site —sourceis injected before user properties spread, so per-event props can't accidentally overwrite it🤖 Generated with Claude Code