Skip to content

feat: add telemetry#274

Draft
hubgan wants to merge 9 commits into
mainfrom
feat/telemetry
Draft

feat: add telemetry#274
hubgan wants to merge 9 commits into
mainfrom
feat/telemetry

Conversation

@hubgan

@hubgan hubgan commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds anonymous, opt-out telemetry for Argent installer, update/uninstall flows, CLI telemetry controls, tool-server lifecycle, and tool invocation health signals.

What Changed

  • Added a new private @argent/telemetry package
  • Introduced typed telemetry events with runtime property allowlisting/sanitization.
  • Added lazy anonymous identity creation in ~/.argent/telemetry-id.
  • Added consent handling via: argent telemetry enable, argent telemetry disable, argent telemetry status
  • Added local debug logging with ARGENT_TELEMETRY_DEBUG=1.
  • Tracks installer funnel events for init, global install/update decisions, editor selection, allowlist decisions, skill installation, rules/agents copy, update, and uninstall.
  • Tracks tool-server start/stop and tool invocation/complete/fail events.
  • Adds per-tool invocation IDs so request metadata can be correlated without relying on ordering.
  • Records only coarse platform metadata (ios / android) for device tools
  • Ensures short-lived commands drain telemetry with bounded shutdown before exit.
  • Wires telemetry package into bundling and injects full Argent CLI version metadata at build time.

Tests Added / Updated

  • Telemetry unit coverage for consent precedence, identity creation, base props, sanitization, PostHog config, debug logging, shutdown, opt-out, and forget behavior.
  • Registry/tool-server tests for invocation IDs, platform metadata boundaries, startup failure flushing, and tool lifecycle telemetry.
  • Installer tests for final telemetry drain, update action classification, and uninstall identity cleanup ordering.

@hubgan hubgan changed the title feat: add telemetry for installer and tool usage feat: add telemetry Jun 1, 2026

@pFornagiel pFornagiel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README.md right now states:

Argent does not collect or transmit any user data. No telemetry, no analytics, no crash reporting.

This will have to be updated as well

Comment on lines +47 to +51
export function isCi(env: NodeJS.ProcessEnv = process.env): boolean {
if (env.CI === "false") return false;
if (GENERIC_CI_ENV_VARS.some((name) => Boolean(env[name]))) return true;
return isKnownVendorCi(env);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check here can do lots of false-positives from my perspective. The ordering should be:

  1. env.CI == "false
  2. isKnownVendorCi(env)
  3. GENERIC_CI_ENV_VARS

Also, the GENERIC_CI_ENV_VARS can be a little too genereic - BUILD_ID, BUILD_NUMBER, RUN_ID could simply be present in normal setup. Additionally, I am not sure whether ENV variables mentioning CI are not sometimes present on the users end. Can we do a bit more research here, or provide some source as to why this will not produce too many false-positives?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generic list, the CI === "false" bypass, and the vendor table are all taken verbatim from ci-info v4.4.0. Added a comment noting this. Kept BUILD_ID/BUILD_NUMBER/RUN_ID for parity. Narrowing them risks false negatives (Jenkins/TeamCity set only these), and this just gates a telemetry property, not behavior. CI === "false" is the escape hatch for anyone hitting a false positive. Ordering is a no-op — checks 2 and 3 are an OR, and the current order matches ci-info's.

Comment thread packages/telemetry/src/events.ts Outdated
Comment thread packages/telemetry/src/events.ts Outdated
Comment thread packages/telemetry/src/events.ts Outdated
Comment on lines +97 to +116
export interface ToolInvokeProps {
tool: string;
tool_invocation_id: string;
platform?: "ios" | "android";
}

export interface ToolCompleteProps {
tool: string;
tool_invocation_id: string;
platform?: "ios" | "android";
duration_ms: number;
}

export interface ToolFailProps {
tool: string;
tool_invocation_id: string;
platform?: "ios" | "android";
duration_ms: number;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to discussion: Would it not be better to consolidate this into ToolInvokeProps entry with information about success / failure? Or would that be less clear / problematic to implement?

@pFornagiel

Copy link
Copy Markdown
Contributor

Possibly found something - at the moment it seems like every time when we do run-sequence and possibly flow we are emitting standalone events, even though they are part of these tools invocations. Maybe there would be a way to mark those events as executed using flow or run-sequence or not gather them? Up to discussion.

@pFornagiel

Copy link
Copy Markdown
Contributor

Additionally, maybe we could also add information whether the source of invocation was MCP or CLI. If you believe this is a scope for another PR - it is also fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants