feat(dependency-impact): implement multi-step analysis with inline review comments#13
Conversation
… pipeline Add types.ts with interfaces for the three-step analysis workflow: - Step1Result: breaking change extraction from release notes - Step2Result: cross-reference of breaking changes with codebase usage - DependencyAssessment: final synthesized output with risk table, action items, and inline annotations https://claude.ai/code/session_01DcQfYZFCR9zyLMot2crYdN
Add classifyUpgrade() to classify version bumps as major/minor/patch based on semver segments. Add findDepLineInPatch() to deterministically locate the line number of a dependency's version change in a diff patch, used for placing inline review comments. https://claude.ai/code/session_01DcQfYZFCR9zyLMot2crYdN
Add prompts.ts with focused prompt builders for each analysis step: - buildStep1Prompt: extract breaking changes from release notes - buildStep2Prompt: cross-reference breaking changes with codebase usage - buildStep3Prompt/buildStep3NoUsagePrompt: synthesize final assessment - buildLegacyPrompt: fallback preserving current single-prompt behavior Each prompt requests structured JSON output and has a single focused task instead of the current monolithic prompt. https://claude.ai/code/session_01DcQfYZFCR9zyLMot2crYdN
…omments Replace the single LLM call with a three-step pipeline: 1. Extract breaking changes from release notes 2. Cross-reference with codebase usage (skipped when unnecessary) 3. Synthesize structured assessment Switch PR output from postComment() to createReview() with: - Summary table with per-dependency risk ratings - Inline comments on manifest file version-change lines - Action items referencing impacted source files - Graceful fallback to legacy single-prompt mode on parse failure https://claude.ai/code/session_01DcQfYZFCR9zyLMot2crYdN
dortort
force-pushed
the
claude/improve-dependency-impact-yoBf6
branch
from
490f5e2 to
6f6573e
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 490f5e2b92
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…odule Move buildReviewBody() and buildInlineComments() from index.ts into a dedicated review.ts module so they can be unit tested independently of the GitHub Actions runtime. https://claude.ai/code/session_01DcQfYZFCR9zyLMot2crYdN
…dering Add prompts.test.ts (21 tests) covering all 5 prompt builder functions: - buildStep1Prompt, buildStep2Prompt, buildStep3Prompt, buildStep3NoUsagePrompt, buildLegacyPrompt Add review.test.ts (13 tests) covering: - buildReviewBody: risk badge, summary table, action items, narrative - buildInlineComments: line mapping, missing deps, missing patches, one-comment-per-dep deduplication Total test count: 32 existing + 34 new = 66 https://claude.ai/code/session_01DcQfYZFCR9zyLMot2crYdN
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9872bbae0e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Instead of copying the entire bot PR body into every dependency's release notes (wasting tokens when multiple deps are bumped), parse the <details> blocks and extract only the sections that mention each dependency name. Falls back to the full body for single-dep PRs or when no matching block is found. https://claude.ai/code/session_01DcQfYZFCR9zyLMot2crYdN
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7bf6709ebf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… in findDepLineInPatch Use a word-boundary regex instead of raw includes() so a short name like "aws" no longer matches "aws-sdk" lines in the diff patch. Also skip the "\\ No newline at end of file" diff marker when counting line numbers to prevent off-by-one errors in inline comment placement. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… files buildInlineComments now filters prFiles to known manifest filenames (package.json, go.mod, requirements.txt, etc.) before searching for the dependency version-change line. This prevents annotations from being placed on source or doc files that happen to mention the dependency name. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…llback When no GitHub release notes are found for any dep, the code fell back to storing the full PR body for every changed dependency. In multi-dep PRs this embedded the same content N times in the Step 1 prompt, inflating token usage and mixing unrelated notes under each dependency section. Now the PR body is attached only to the first dep that has no dedicated release notes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…step results parseJsonResponse only validates JSON syntax, not schema shape. If the LLM returns a structurally valid JSON object that is missing required fields (dependencySummaries, actionItems, inlineAnnotations, deprecations, etc.) the action would throw a TypeError outside any try-catch, bypassing runLegacyFallback. - Use optional chaining for d.deprecations/d.notableChanges in the hasBreakingChanges check so a partial step-1 result doesn't crash step 2 - Wrap buildReviewBody/buildInlineComments in a try-catch that falls back to the legacy prompt when the step-3 assessment is structurally incomplete Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6d71659719
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…Step2Prompt buildStep2Prompt accessed d.deprecations.length and d.notableChanges.length directly. Since parseJsonResponse only validates JSON syntax, a partial LLM response missing these arrays throws a TypeError that silently swallows Step 2 via the catch handler even when confirmed breaking changes exist. Use optional chaining and nullish coalescing throughout the filter and the spread map so a schema-incomplete step-1 result degrades gracefully. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ection block.includes(depName) caused substring collisions in grouped Dependabot PRs — e.g. searching for "aws" could match a block for "aws-sdk", attaching the wrong release notes to the dependency and propagating misattributed breaking-change analysis through all three steps. Apply the same word-boundary regex used in findDepLineInPatch so only blocks that contain the exact package name (not a prefix or suffix of it) are selected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ent under-analysis The previous fix attached pr.body only to the first dependency to reduce prompt token usage. This caused the remaining dependencies in multi-dep PRs to receive no release notes context in Step 1, leading to falsely low risk summaries. The non-Dependabot fallback is already less token-intensive than the original Dependabot case (which duplicated the entire body before extractDependabotSection was introduced), so restoring it for all deps is the correct trade-off. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Regenerate dist/index.js to include all recent source changes: manifest file filter in buildInlineComments, word-boundary regex in findDepLineInPatch, no-newline marker skipping, optional chaining for partial LLM results, and extractDependabotSection exact matching. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ce8e46cd8a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… notes map releaseNotesPerDep was keyed by dep.name alone. In monorepos or PRs that update identically named packages across different ecosystems, later entries silently overwrote earlier ones, causing Step 1/3 to attribute breaking changes from the wrong upgrade. Key by "name::ecosystem" instead to keep each dependency's notes isolated. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…okup riskLabel is keyed by lowercase RiskLevel values. parseJsonResponse does no schema validation, so the LLM can return "Low", "HIGH", etc. and the direct lookup silently produces undefined, rendering "**undefined**" in the review body header and summary table without triggering the legacy fallback. Introduce safeRiskLabel() which lowercases the value before the lookup and falls back to the uppercased raw value if the key is still not found. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rmalization Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 366c353008
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… prevent map key collisions parseDependencyChanges can emit multiple entries with the same name and ecosystem when a package appears in several manifests within a monorepo. This caused releaseNotesPerDep.set() to overwrite earlier notes for the same package, and buildInlineComments' enrichedDeps.find() to always bind annotations to the first occurrence regardless of ecosystem. Deduplicate depChanges by name::ecosystem immediately after parsing, keeping the first occurrence. Analysis and release-note lookups apply to the package identity, not to individual manifest files, so this is the correct unit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 046c285225
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…Upgrade In 0.x packages, semver allows breaking changes on minor version increments (0.4.0 → 0.5.0). Treating them as "minor" caused Step 1 to be primed with "minor upgrades rarely have breaking changes", leading to systematic under-analysis of pre-1.0 packages. Return "major" when the major segment is 0 and the minor segment changes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…lt is malformed parseJsonResponse could succeed (valid JSON) but return an object missing required fields like impacts. The previous code assigned the parsed value to step2Result before accessing .impacts.length, so a TypeError in that access entered the catch block with step2Result already holding the malformed object instead of the safe default. Step 3 then received garbage data despite the warning saying "empty impact analysis". Move the step2Result assignment after all field accesses so the safe default is preserved whenever parsing or validation throws. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ep-2 fix Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1cfe5b0de
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
… PRs Use name::ecosystem::fromVersion::toVersion as the dedup key so that the same package upgraded to different ranges across manifests is analyzed once per distinct range instead of only once per package. Update depNoteKey to match so release notes are fetched per range. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If GitHub rejects the review (e.g. an inline comment line falls outside the diff range), catch the error and post the analysis body as a plain comment so the review is never silently lost. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
2 participants
Replace the single monolithic LLM call with a three-step pipeline that
improves prompt quality, enables short-circuiting, and communicates
results more effectively within PRs.
Multi-step workflow:
changes or no usage detected)
Improved prompts:
expectations for breaking change likelihood
Better PR communication:
New files: types.ts (structured interfaces), prompts.ts (prompt builders)
New parsers: classifyUpgrade(), findDepLineInPatch()
https://claude.ai/code/session_01DcQfYZFCR9zyLMot2crYdN
Note
Medium Risk
Reworks the action’s core analysis/publishing flow (multi-step LLM calls, JSON parsing, and PR review creation), which could change output quality or fail to post comments if edge cases slip through (mitigated by a legacy fallback).
Overview
Refactors
dependency-impactto run a 3-step, JSON-driven LLM pipeline: extract breaking changes from per-dependency release notes, optionally cross-reference against sampled code usage, then synthesize a structured assessment (risk, summaries, action items, and inline annotations), with a legacy single-prompt fallback on failures.Enhances PR output by rendering a review body with a summary table and (when possible) creating a GitHub PR review with inline comments on manifest version-change lines (via new
findDepLineInPatch), plus improved dependency handling (dedupe identical upgrades, semver-basedclassifyUpgrade, and Dependabot per-dependency release-note extraction).Written by Cursor Bugbot for commit f48bd03. This will update automatically on new commits. Configure here.