Skip to content

[DX-1319] Add @elevenlabs/dts-breaking-changes engine + composite action - #917

Draft
kraenhansen wants to merge 15 commits into
mainfrom
kh/dx-1319-dts-breaking-changes
Draft

[DX-1319] Add @elevenlabs/dts-breaking-changes engine + composite action#917
kraenhansen wants to merge 15 commits into
mainfrom
kh/dx-1319-dts-breaking-changes

Conversation

@kraenhansen

@kraenhansen kraenhansen commented Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Variance-aware breaking-change detection for our public TypeScript packages: an analysis engine (packages/dts-breaking-changes) and a composite action (.github/actions/dts-breaking-changes). The workflow that drives it in this repo is #918.

Engine

Given two directories of already-built .d.ts (base and head), it reports whether the head breaks existing consumers. It builds nothing — the caller supplies both trees.

  • Models each package's public surface as a module type and uses the TypeScript compiler's structural assignability to classify changes. Consumer direction (New assignable to Old) failures are breaking; forward direction is informational. Adding a required field to an input type is breaking; to an output type is not.
  • MethodsToProperties reconstructs method signatures so parameters are checked contravariantly (methods are otherwise bivariant) and drops private/protected members (not part of the consumer contract).
  • Workspace discovery: enumerates packages from pnpm-workspace.yaml and resolves each package's public type entrypoints via the compiler's own module resolution — honoring exports, the types condition, implicit sibling .d.ts, and every subpath (e.g. @scope/pkg, @scope/pkg/internal). A package can drop subpaths with ignoreEntrypoints in its dts-breaking-changes.json.
  • Type-only packages: compareTypeOnlyExports compares pure type exports by name (for packages like @elevenlabs/types that have no value surface). Removal/narrowing is breaking; an added required field is a warning.
  • Renders one combined report with a ## <title> section per surface. External JSON (surfaces, config files) is validated with zod.

Action

Runs the engine and posts one sticky PR comment (a section per package), failing the check on an unacknowledged consumer-breaking change. Three input modes: workspace (base-root — discovers everything), explicit (surfaces), or single-surface (old-dir/new-dir). A break is acknowledged (downgraded to a warning) by the breaking label or a package's major changeset. Zero external action dependencies (node/npm/gh/jq/bash).

Test plan

  • Engine unit tests: variance (in/out), private/protected members across two builds, dropped overloads, type-only add/remove/narrow, entrypoint discovery (explicit/implicit/subpath/legacy), combined reporting, full elevenlabs-js surface with no TS2589
  • turbo check-types / turbo lint green, including whole-repo lint with the package added
  • Verified end-to-end via [DX-1319] Generalize type-surface check to the packages matrix #918: workspace discovery finds all public entrypoints (incl. @elevenlabs/client's ././internal/./internal/unity) and reports them in one comment

🤖 Generated with Claude Code

kraenhansen added a commit to elevenlabs/elevenlabs-js that referenced this pull request Aug 6, 2026
@kraenhansen @claude
Point type-surface action at the public packages repo
7ee0ec3 
The action moved from the private elevenlabs-dx to the public elevenlabs/packages
repo (a public repo cannot consume an action from a private one). References the
feature branch until elevenlabs/packages#917 merges.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen kraenhansen mentioned this pull request Aug 6, 2026
Draft
2 tasks
kraenhansen added a commit that referenced this pull request Aug 6, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
f7e9fb7 
Single job that builds head and the merge-base (git worktree) once with turbo,
discovers packages with a public type entry, and runs the dts-breaking-changes
action once to report all packages in one PR comment. A break is acknowledged by
the `breaking` label or a `major` changeset for the package.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine and multi-surface action this uses live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit that referenced this pull request Aug 6, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
228f84b 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit that referenced this pull request Aug 6, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
838dc43 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen
kraenhansen force-pushed the kh/dx-1319-dts-breaking-changes branch from a029ea0 to af7ec2d Compare August 6, 2026 14:54
kraenhansen added a commit that referenced this pull request Aug 6, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
d07f773 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen and others added 8 commits August 6, 2026 17:53
@kraenhansen @claude
[DX-1319] Add @elevenlabs/dts-breaking-changes engine + composite action
d5e3ae1 
Variance-aware breaking-change detection for the public TypeScript packages
(DX-1319). Hosted here (public) rather than the private docs repo, because a
public repo cannot consume a GitHub Action from a private one.

Engine (packages/dts-breaking-changes): pure analysis. Given two dirs of
already-built .d.ts (base and head), it reports whether the head breaks existing
consumers. It builds nothing; the caller supplies both trees. Models each
package's public surface as a module type and uses the compiler's structural
assignability to classify input vs output changes: consumer direction (New
assignable to Old) breaks are gated; forward direction is informational. The
MethodsToProperties transform reconstructs method signatures so parameters are
checked contravariantly (identity mapped types keep method bivariance, which
would hide added-required-input breaks) and drops private/protected members.

Type-only exports (interfaces/type aliases with no value meaning) are compared
by name in type space behind compareTypeOnlyExports (default false), because a
package like @elevenlabs/types is ~150 interfaces with no values and would look
empty to the value surface. A bare type's variance is ambiguous, so its findings
use a heuristic (removal/narrowing breaking, added-required warning) kept
separate from the value-surface convention.

Composite action (.github/actions/dts-breaking-changes): runs the engine, posts
a sticky PR comment, gates the check; the `breaking` label downgrades a failure
to a warning. Zero external action dependencies (node/npm/gh/jq/bash only).

Wired into the pnpm workspace with packages conventions (test, check-types,
lint:prettier; prettier reformatted to this repo's config). 12 tests pass
(variance, private members, overloads, identical, type-only add/remove/narrow,
full elevenlabs-js surface no TS2589), verified via turbo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
[DX-1319] Filter nominal private/protected member artifacts
1995dd0 
The pilot surfaced false positives on a PR that changes no .d.ts: every finding
bottomed out at "Types have separate declarations of a private property 'level'".
A Logger class with a private member is reachable only through a parameter
position (Client -> Options -> fetcher -> Args -> logging -> Logger), which
MethodsToProperties keeps positional (to preserve variance/overloads) and so
never strips. Comparing two independent builds then treats those private members
as nominally distinct.

Private/protected members are not part of the consumer-visible contract, so such
a mismatch can never be a real breaking change. Drop any finding whose message
is a "separate declarations of a private/protected property" (or "is private/
protected in type") artifact.

The self-comparison surface test missed this because comparing a dir to itself
shares module identity. Harden it to compare two separate copies (the real
workflow scenario), and add a small nominal-private-param fixture that
reproduces the bug directly. 13 tests pass; two identical elevenlabs-js builds
now diff clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
docs: align action example node-version with pinned LTS
946f439 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
Un-backtick the base SHA so GitHub auto-links it to the commit
3432186 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
Add a workflow-run provenance footer to the sticky comment
9b5c8e1 
A <small> footer linking to the run that generated the comment gives it
provenance without changing the bot identity (which is fixed to the
github-actions token). The run URL is CI-specific, so it lives in the action,
not the engine.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
[DX-1319] Multi-surface action + combined report
1f39906 
Extend the engine and action to check several packages in one run and report
them in a single comment. Moved here (from the packages-workflow PR) so the
action is complete in its home PR.

- Engine: `renderCombined` renders one report with a `## <title>` heading per
  package; the CLI gains `--surfaces <file>` (analyze N + combine + gate) and
  `--label-acknowledged`. External JSON (surfaces manifest, config files) is
  validated with zod instead of cast.
- Action: accepts a `surfaces` JSON array (or the single-surface inputs, wrapped
  as one), runs the engine once, posts one sticky comment with a section per
  package, and gates on the combined result. A break is acknowledged by the
  `breaking` label (all surfaces) or a surface's `allowBreaking`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
[DX-1319] Discover workspace type entrypoints (TS resolver)
2ca355c 
Replace the brittle package/entrypoint heuristic with proper resolution in the
engine, and add a workspace mode to the action.

- `discover.ts`: enumerate workspace packages (pnpm-workspace.yaml globs or
  package.json workspaces) and resolve each package's public type entrypoints
  using the TS compiler's own module resolution (self-reference from inside the
  package dir). This honors `exports`, the `types` condition, implicit sibling
  `.d.ts`, and every export subpath — so `@elevenlabs/client` yields `.`,
  `./internal`, and `./internal/unity`, and implicitly-typed packages like
  `@elevenlabs/react-native` are no longer skipped. A package can drop subpaths
  via `ignoreEntrypoints` in its dts-breaking-changes.json.
- CLI `--discover --base-root <dir> --head-root <dir> [--allow-breaking-packages
  <csv>]` emits the surfaces JSON; each (package, subpath) is its own surface.
- Action: `base-root`/`head-root`/`allow-breaking-packages` inputs run discovery,
  so callers no longer hand-build the surfaces list.

Adds `yaml` for pnpm-workspace parsing. Fixtures cover explicit types, subpaths,
implicit siblings, legacy types/main, and no-types packages.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
[DX-1319] Fix two false positives surfaced by the packages matrix
67a2a2a 
Discovered when the workspace run flagged @elevenlabs/client (which changes no
source) as breaking:

- Protected-member nominal artifact: TS phrases the protected variant of the
  cross-build nominal mismatch as "Property 'x' is protected but type 'Y' is not
  a class derived from 'Y'", which the private-only filter missed. Broaden
  isNominalAccessArtifact to catch any private/protected member nominal message.
- Type-only re-export of a value: `export type { X }` where X resolves to a
  class is not in the value namespace, but getAliasedSymbol resolves through the
  `type` modifier and marked it a value export, so per-symbol localization
  indexed a name absent from `typeof import()` and raised a spurious TS2339.
  Honor the export-specifier's type-only modifier when classifying exports.

Regression fixture nominal-reexport reproduces both; the real @elevenlabs/client
surfaces (`.`, `./internal`, `./internal/unity`) now self-compare clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen
kraenhansen force-pushed the kh/dx-1319-dts-breaking-changes branch from af7ec2d to 67a2a2a Compare August 6, 2026 15:54
kraenhansen added a commit that referenced this pull request Aug 6, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
daac86e 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen and others added 2 commits August 7, 2026 10:59
@kraenhansen @claude
[DX-1319] Test callback assignability (contravariant cases)
25b8051 
Callbacks the consumer supplies are checked doubly-contravariantly (the event
the library passes is itself a parameter). Lock down the variance: a callback
event gaining a field is safe, losing one breaks; requiring a callback to return
a value breaks; a new required callback on an options bag breaks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
[DX-1319] Enumerate export conditions and group the report by package
d2137c9 
Model each type surface as (package, subpath, condition). Discovery now
enumerates the export conditions in each subpath's exports value and resolves
each via the TS compiler's customConditions, deduping by resolved .d.ts — so a
condition pointing at a different type surface (e.g. @elevenlabs/react-native's
`react-native` -> index.react-native.d.ts) is checked too. Conditions resolving
to the same file as `default` are folded into it.

The combined report groups results by package with a subsection per entrypoint
(subpath, plus the condition when it isn't the default), and the summary counts
packages rather than entrypoints. Fixtures cover a condition-divergent package
and the grouped rendering.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit that referenced this pull request Aug 7, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
2369600 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
feat(dts-breaking-changes): opt-in API-additions diff summary
db7a0c1 
Add an `apiSummary` mode that attaches a structural, member-level
summary of API additions, removals, and changes to the report, rendered
as a GitHub ```diff block. It is a review aid only: the gate stays
authored entirely by tsc, and the differ enumerates rather than judges.

The differ (`diffApi`) walks both module surfaces' value exports via the
compiler's symbol table, recursing one level into changed containers so
a new method or a changed member signature shows up as its own line
(e.g. `+ Client.cancel: (id: string) => void`). Class instance exports
always recurse structurally, since `typeToString` renders their nominal
name and would otherwise compare equal.

Wired through the config (`apiSummary`, default false), the CLI
(`--api-summary`), and the composite action (`api-summary` input).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit that referenced this pull request Aug 7, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
75c0fc7 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
feat(dts-breaking-changes): collapse unchanged packages into one line
8f66082 
The combined comment rendered a full heading and a "No type-surface
changes" block for every package (and every entrypoint), which drowned
the packages that actually changed. Now only surfaces with findings or
API changes get a rendered section; the rest roll into a single
`<small>` line so a reviewer can still confirm they were considered.

Also fixes the summary wording when a package has additive API changes
but no breaking findings: it now reads "No consumer-breaking changes"
rather than claiming "No type-surface changes".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit that referenced this pull request Aug 7, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
75fde12 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
fix(dts-breaking-changes): stop false API diffs from build-tree paths…
7290218 
…; list entrypoints considered

The API diff rendered spurious changes for exports typed as
`typeof import("…")` (e.g. namespace re-exports of type-only modules):
the base and head trees resolve under different absolute roots, so an
unchanged module read as changed. Normalize `import("…")` to the bare
module name before comparing and displaying, which also reads better.

Also expand the "considered" footer to list each unchanged package's
entrypoints and conditions, not just the package name, so a reviewer can
confirm every surface was checked.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit that referenced this pull request Aug 7, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
c420be4 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit to elevenlabs/elevenlabs-js that referenced this pull request Aug 7, 2026
@kraenhansen @claude
Point type-surface action at the public packages repo
a4f099c 
The action moved from the private elevenlabs-dx to the public elevenlabs/packages
repo (a public repo cannot consume an action from a private one). References the
feature branch until elevenlabs/packages#917 merges.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
refactor(dts-breaking-changes): tidy the combined comment layout
e3ecb10 
- Drop the title from the sticky marker (`<!-- dts-breaking-changes -->`):
  there is one combined comment per PR, so it needs no disambiguator.
- Move the "Compared against base <sha>" line to the footer, next to the
  generated-by line, so the verdict leads.
- Reword the unchanged-packages ledger from "No type-surface changes in
  …" to a terse "Unchanged: …" so it reads as detail under the verdict
  rather than repeating it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit that referenced this pull request Aug 7, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
1518cf6 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen @claude
refactor(dts-breaking-changes): one provenance footer line
3736a3d 
Render "Compared against base <sha>" and "Generated by the … workflow"
on a single `<small>` line. The comment step now owns the base SHA (bare,
so it still auto-links) and no longer passes `--base-sha` to the engine,
which keeps the whole provenance note in one place.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kraenhansen added a commit that referenced this pull request Aug 7, 2026
@kraenhansen @claude
[DX-1319] Add the packages type-surface workflow
872e605 
Single job that builds head and the merge-base (git worktree) once with turbo,
then runs the dts-breaking-changes action in workspace mode: the action
discovers every package's public type entrypoints and reports them in one PR
comment. Major changesets are detected and passed as `allow-breaking-packages`.

`@elevenlabs/types` is type-only, so it ships dts-breaking-changes.json with
`compareTypeOnlyExports: true`.

The engine, discovery, and multi-surface action live in the base PR (#917).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kraenhansen kraenhansen self-assigned this Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@kraenhansen kraenhansen

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kraenhansen