Repository navigation
the-tests-passed-the-feature-had-never-worked #441
Replies: 1 comment
|
This highlights an important gap in AI-assisted development: passing unit tests don't necessarily prove that the actual user workflow works. I'd add another layer to this: after an AI-generated fix is applied, the review should be repeated against the updated code, and the relevant tests should actually execute. I also think test outcomes need to distinguish PASSED, FAILED, ERROR, TIMED_OUT and NOT_RUN. Otherwise, a skipped or unsuccessful validation can create false confidence. How do you approach validating the user-facing path after an AI-generated patch? |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
the-tests-passed-the-feature-had-never-worked
With an AI assistant, the test, the example and the docs may all come from the process that wrote the feature. Two ways to check.
https://foojay.io/today/the-tests-passed-the-feature-had-never-worked/
All reactions