Skip to content

Add PostHog telemetry #293

Description

@mando

Goal

Measure how many desktop clients check for updates, with breakdowns by app version and platform, and establish a small product-usage telemetry foundation for the desktop app and dashboard.

Proposed implementation

  • Instrument the native updater path in desktop/src-tauri/src/lib.rs, centered on check_available_update.
  • Send a best-effort PostHog Capture API event from the desktop app rather than using the browser-only posthog-js integration.
  • Emit an update-check event with a result such as started, current, update_available, or error.
  • Track update lifecycle events where practical: install_started and install_completed.
  • Add the additional activation, product-usage, and reliability events listed below.
  • Include only operational properties: app version, OS, architecture, update channel, check source (background or manual), result, and available version.
  • Keep telemetry asynchronous and bounded by a short timeout so it never delays or breaks update checks.
  • Make the PostHog project token a release-build variable; builds without a token remain a no-op.

PostHog Capture API reference: https://posthog.com/docs/api/capture

Additional events to consider

Activation funnel

  • desktop_started
  • onboarding_completed
  • first_data_indexed
  • dashboard_viewed
  • session_detail_viewed
  • returned_installation

The primary funnel should be: desktop_startedonboarding_completedfirst_data_indexeddashboard_viewedsession_detail_viewed, with 7-day and 30-day returning-installation rates.

Product usage

  • search_used
  • filter_applied
  • label_created
  • session_hidden
  • sync_configured
  • sync_completed
  • interpretation_completed

Reliability

  • index_failed
  • sync_failed
  • serve_failed
  • interpretation_failed

Use a coarse failure category and duration rather than raw error text, paths, or payloads.

Identity and privacy

  • Generate a separate random per-installation ID for PostHog distinct_id.
  • Do not reuse sync identifiers, email addresses, hostnames, local paths, session data, or transcript content.
  • Send $process_person_profile: false so this remains anonymous operational telemetry.
  • Add a documented telemetry setting or consent/opt-out path before enabling collection for users.
  • Document the event schema and data flow in the privacy/configuration documentation.
  • Consider a separate PostHog project or event namespace for desktop telemetry so it does not mix with docs-site analytics.
  • Do not instrument raw prompts, responses, session titles, summaries, repository names, or tool/MCP names.

Metrics to expose in PostHog

  • Total update checks per day/week.
  • Unique checking installations per day/week/month.
  • Checks by app version, OS, architecture, and source.
  • Update-available rate and check error rate.
  • Update install start/completion rate.
  • Activation funnel conversion from launch to first useful dashboard view.
  • Feature adoption for search, filters, labels, sync, and interpretations.
  • Indexing, sync, serving, and interpretation failure rates.
  • Weekly and monthly returning installations.

Acceptance criteria

  • Background and manual checks are represented without double-counting concurrent checks.
  • Failed checks are distinguishable from successful checks and from an up-to-date result.
  • Activation and reliability events have stable names and documented properties.
  • Telemetry failure does not affect updater or dashboard behavior.
  • No telemetry is sent in builds where the token is absent or collection is disabled.
  • Tests cover event payload shape, privacy filtering, opt-out behavior, and the non-blocking failure path.
  • Release/build documentation explains where the public PostHog project token is configured.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions