chore(agents): add instrumenting-first-party-metrics skill

[DanielVisca]

Problem

No skill covers how to emit first-party metrics from this repo's backend code. The canonical pattern (a prometheus_client instrument plus an OtelInstrumentFactory twin) lives only in the posthog/otel_metrics.py docstring, three call sites, and internal design docs. Agents hitting this cold discover the OTel SDK and start adding exporters and env vars that already exist as charts-level deployment config.

Changes

Adds .agents/skills/instrumenting-first-party-metrics/SKILL.md, a repo-engineering skill covering:

• the two sinks (Grafana scrape, PostHog Metrics product) and the twin pattern that feeds both from one declaration
• what is already wired: :8001 scraping, PROMETHEUS_MULTIPROC_DIR, per-PID OTLP provider init and its env-var gating
• what not to do: no env vars, no provider/exporter setup, no import-time OTel instruments, no customer-SDK path inside the monorepo
• adjacent mechanisms (pushgateway for short-lived jobs, the Node twin) and how to verify

How did you test this code?

Doc-only change. hogli lint:skills can't run in my sandbox (no flox python), so I validated the frontmatter parses with name and a 496-char description, and confirmed the skill is visible through the .claude/skills symlink. CI's skills lint covers the rest.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Written by Claude (PostHog Code session) after an investigation into how web/celery Prometheus metrics reach the Metrics product, prompted by an agent wrongly insisting env vars were needed. Invoked /writing-skills for conventions. Companion PRs: customer-facing docs in PostHog/posthog.com#18700 and the instrument-metrics skill family in PostHog/context-mill#255.

---

Created with PostHog Code

[github-actions]

Hey @DanielVisca! 👋

It looks like your git author email on this PR isn't your @posthog.com address (danielvisca96@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[greptile-apps]

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
.agents/skills/instrumenting-first-party-metrics/SKILL.md:79
**Verification Excludes Python Metrics**

`bin/verify-metrics-pipe` filters for `logs-ingestion`, `metrics-ingestion`, and `nodejs`, but Python metrics normally use `posthog-python`. Following this step for a Django, Celery, or Temporal metric therefore reports no results even when export works; document a Python-specific check or update the script first.

_{Reviews (1): Last reviewed commit: "chore(agents): add instrumenting-first-p..." | Re-trigger Greptile}

[stamphog]

Note

🤖 stamphog reviewed bebeb19448ee9cf2474a9811754239bf689f7baa — verdict: REFUSED

Confirmed via the actual script that greptile's unresolved concern is accurate: bin/verify-metrics-pipe filters to logs-ingestion/metrics-ingestion/nodejs service names, excluding Python (Django/Celery/Temporal) — the exact stack this skill targets — so following the doc's verification step for a Python metric would falsely suggest export isn't working.

• greptile-apps[bot] reviewed the current head.
• Unresolved substantive review comment from @greptile-apps on SKILL.md line 79: the documented verification command (bin/verify-metrics-pipe) doesn't cover Python-emitted metrics, which is the primary audience of this skill — following the doc as written for a Django/Celery/Temporal metric would misleadingly suggest export failed, risking the exact env-var/exporter confusion the skill is meant to prevent.

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	0L, 0F substantive, 82L/1F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1b-small (82L, 1F, single-area, chore)
stamphog 2.0.0b3		.stamphog/policy.yml @ 492ad4a · reviewed head bebeb19

[greptile-apps]

Thanks for confirming with the script directly. Here's a targeted fix for the "Verifying" section — remove the misleading bin/verify-metrics-pipe reference for Python and replace it with the approach that actually works for Django/Celery/Temporal:

- **End to end locally**: for the Prometheus sink, hit `http://localhost:8001/metrics` (web or celery worker) and grep for your instrument name. For the OTLP/Metrics-product sink, check the dev collector logs (`docker compose logs otel-collector`) for `ScopeMetrics` entries from your area — `bin/verify-metrics-pipe` only covers `logs-ingestion`/`metrics-ingestion`/`nodejs` service names and will show nothing for Python services. `otel-collector-config.dev.yaml` shows the dev collector setup.

The key changes:

1. Replaces the bare bin/verify-metrics-pipe call with the correct per-sink steps.
2. Explicitly calls out what bin/verify-metrics-pipe does cover so a future reader knows why it's not used here.
3. Keeps the otel-collector-config.dev.yaml reference for context on collector setup.

[DanielVisca]

Reoriented the skill per feedback: it's now Metrics-product-first — environment decision table (SDK the customer way per the public docs, OTel factory only as fallback), explicit per-SDK version gates (posthog-python 7.23.0 / posthog-node 5.43.0 / posthog-js ~1.399.0, with a runtime check for js), what's already wired internally (#72430 + posthog-python#753), and a validation section covering destination project, the disabled-in-DEBUG/TEST gotcha, and arrival checks via metric-names-list/query-metrics/SQL. Grafana + pushgateway demoted to an 'adjacent, not this skill's job' note.

[stamphog]

Doc-only addition of a new agent skill file; no production code, dependency, or risky-surface changes. The one reviewer concern (bin/verify-metrics-pipe not covering Python) was addressed in the current diff and greptile acknowledged the fix with a 👍.

• 👍 on the PR from greptile-apps[bot], hex-security-app[bot].

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	0L, 0F substantive, 74L/1F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1b-small (74L, 1F, single-area, chore)
stamphog 2.0.0b3		.stamphog/policy.yml @ ced1a6a · reviewed head 4229dd4

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-21 21:41 UTC	Run
prod-us	✅ Deployed	2026-07-21 21:52 UTC	Run
prod-eu	✅ Deployed	2026-07-21 21:56 UTC	Run