Skip to content

feat: label test/story file context on diagnostic sites#795

Merged
aidenybai merged 4 commits into
mainfrom
devin/1781245564-file-context-labels
Jun 12, 2026
Merged

feat: label test/story file context on diagnostic sites#795
aidenybai merged 4 commits into
mainfrom
devin/1781245564-file-context-labels

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

A finding in a spec or story file doesn't mean users crash — but the report framed every site identically, so rn-no-raw-text-style findings in non-shipping code read as production impact. This adds file-context labels driven by the same path heuristics that already power test-noise auto-suppression (isTestFilePath), so the label and the suppression can never disagree.

  • classifyFileContext(relativePath): "test" | "story" | "production" in core — .stories.*/.story.* suffix wins as story, everything else isTestFilePath matches is test.
  • buildDiagnosticPipeline stamps fileContext?: "test" | "story" on surviving diagnostics (omitted for production, so the JSON report shape is purely additive — Diagnostic schema gains fileContext: Schema.optional(Schema.Literals(["test", "story"])), no schemaVersion bump). The pipeline's old isTest cache is now the context cache; auto-suppression behavior is unchanged.
  • Renderers tag sites: src/utils/foo.spec.tsx:3 (test file) in the terminal cluster locations and the per-rule Files: text dumps.
  • Telemetry: diagnosticsInTestFiles / diagnosticsInStoryFiles counts on the run wide event, so the next Lorenzo-style eval can check whether labeled findings change behavior before judging if the scheme reduces confusion.

Product-thinking note: no new flag/config — the label is unconditional, additive output on an existing surface. The classification half is fully tested; whether the labeling reduces confusion is the eval's call.

Tests: classify-file-context.test.ts (classification incl. fixture-project and Windows paths), pipeline stamping + "rn-no-raw-text in a spec never shows" in filter-diagnostics.test.ts, and renderer tagging in render-file-context-tag.test.ts. Changeset: react-doctor minor.

Link to Devin session: https://app.devin.ai/sessions/92feef39863a4c84a5a105959d757c84
Requested by: @aidenybai


Note

Low Risk
Additive reporting and a refactor of existing test-path heuristics; suppression behavior stays aligned via a single classifier, with broad test coverage and no Score API shape change.

Overview
Adds file-context labeling so diagnostics in non-shipping files are clearly distinguished from production code.

Core: New classifyFileContext centralizes path heuristics (.stories.*"story", test/spec/fixture suffixes and test dirs → "test", else "production"). isTestFilePath now delegates to it so test-noise auto-suppression and labels always use the same rules. The diagnostic pipeline stamps optional fileContext: "test" | "story" on survivors; the JSON Diagnostic schema gains the same optional field (additive, no schema version bump). Score API requests still strip local path-derived fields via stripLocalFileFields.

CLI: Terminal cluster locations and per-rule Files: dumps append dim (test file) / (story file) tags. Run telemetry adds diagnosticsInTestFiles and diagnosticsInStoryFiles counts.

Tests cover classification (including fixture projects and Windows paths), pipeline stamping, and renderer output.

Reviewed by Cursor Bugbot for commit 190df5c. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@pkg-pr-new

pkg-pr-new Bot commented Jun 12, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/eslint-plugin-react-doctor@795
npm i https://pkg.pr.new/oxlint-plugin-react-doctor@795
npm i https://pkg.pr.new/react-doctor@795

commit: 190df5c

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

No React Doctor issues found. 🎉

Reviewed by React Doctor for commit 190df5c.

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

End-to-end test results

Ran the built CLI from this branch against a fixture project containing the identical key={index} bug in src/List.tsx (production), src/List.test.tsx, and src/List.stories.tsx.

  • Terminal labels — passed: src/List.test.tsx:4 (test file), src/List.stories.tsx:4 (story file), production site untagged
  • JSON report — passed: fileContext: "test" / "story" on the non-shipping diagnostics; key omitted entirely for production (Object.hasOwn === false), so the shape is purely additive
  • rn-no-raw-text framing guard — covered at unit level (filter-diagnostics.test.ts): the rule is test-noise and auto-suppressed in specs, so no runtime path frames it as user-facing
Terminal: tags on test/story sites, none on production JSON: fileContext only for test/story
Terminal output with (test file)/(story file) tags JSON report fileContext fields
Notes
  • All 19 CI checks pass (one pnpm format failure was fixed and pushed).
  • Whether the labeling reduces confusion is deferred to the next eval per task scope.

Devin session

Makes the file-context classifier the single owner of the test/story
path heuristics so the label and test-noise suppression agree by
construction. Also strips the new fileContext field from the Score API
payload, folds the per-rule summary's context tags into the single
site-grouping pass, drops a dead guard in the pipeline stamping, and
counts test/story diagnostics for the run event in one pass.
@aidenybai aidenybai merged commit 04e72a4 into main Jun 12, 2026
20 checks passed
@aidenybai aidenybai deleted the devin/1781245564-file-context-labels branch June 12, 2026 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant