Add feedback tone check (#69) - #1013
Open
SaimM2007 wants to merge 8 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #69
Adds a tone check step to PathReview's feedback generation pipeline. Previously,
ContentFilteronly caught explicitly harmful content (self-harm, slurs, illegal activity) via regex — it had no concept of "constructive vs. discouraging" tone, and it wasn't even called from the review generation flow. This meant harsh-but-not-technically-harmful feedback could reach users unfiltered.What changed
safety/content_filter.pyToneChecker, an LLM-based classifier that judges whether a piece of feedback is CONSTRUCTIVE (actionable, specific, encouraging — even if critical) or NEGATIVE (discouraging, vague, dismissive).ToneCheckResultdataclass (is_constructive: bool,raw_response: str) as the return type.rag/generator/review_generator.py_generate_section_once().generate_section()now calls_generate_section_once(), runs the result throughToneChecker, and retries generation up toMAX_RETRIES(2) times if the tone check fails.structlogwarning logged so the fallback is visible rather than silent.Tests
tests/unit/test_content_filter.py— reproduction test showing the original gap (ContentFilterhas no tone concept), plus new tests forToneChecker: constructive feedback, negative feedback, critical-but-actionable feedback (must not be falsely flagged), and empty content.tests/unit/test_review_generator_tone_check.py(new) — coversgenerate_section()passing on the first attempt, succeeding after one regeneration, and falling back correctly after exhausting retries.mainin unrelated modules (review_service,resume_parser,security,skill_extractor, etc.). This PR introduces 0 new test failures and 0 new lint errors.Open questions
Tone classification strictness is a judgment call — the current prompt is tuned to distinguish "critical but specific" from "vague/dismissive," but I'd welcome feedback on whether it's too strict or too lenient in practice.