Skip to content

test: gem.bats tests 5-6 are grep-based regression guards, not functional unit tests #273

Description

@gtrabanco

Problem

PR #270 (fix/265-gem-update-false-positive) added two bats tests in tests/package/gem.bats that use grep -q on the source file to verify the fix is present, rather than calling gem::update_apps and asserting on its behavior:

@test "gem::update_apps checks exit code of gem outdated (not just stdout)" {
    local gem_script="$SLOTH_PATH/scripts/package/src/package_managers/gem.sh"
    [[ -f "$gem_script" ]]
    grep -q 'gem_outdated_exit' "$gem_script"
    grep -q 'gem_outdated_exit.*-ne 0' "$gem_script"
}

@test "gem::update_apps discards stderr to avoid parsing error text as packages" {
    local gem_script="$SLOTH_PATH/scripts/package/src/package_managers/gem.sh"
    [[ -f "$gem_script" ]]
    grep -q 'gem outdated 2> /dev/null' "$gem_script"
}

These tests verify implementation details (presence of specific strings in source) rather than functional behavior. They are fragile — renaming gem_outdated_exit or reformatting the gem outdated line breaks the test even if the actual bug fix is still correct.

Why not fix now

gem::update_apps shells out to the external gem command. Writing a true functional test requires mocking gem outdated to:

…then asserting on gem::update_apps return code and output. This needs a mock/stub harness for gem that the test suite does not currently have.

Acceptance criteria

  • Tests 5-6 in tests/package/gem.bats call gem::update_apps (or a sourced version of it) instead of grepping the source file
  • A mock/stub for the gem command is provided (e.g. a fake gem on PATH or a function override) that can simulate the three exit-code/stdout scenarios above
  • Tests assert on the return code and/or output of gem::update_apps, not on source-file string presence
  • Existing grep-based tests are removed or converted (not duplicated)

Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions