Skip to content

No tests cover the CLI: exit codes, --dry-run and --help in bin/binary_sdk_bridge.dart are unverified #4

Description

@saimskywalker

The problem

bin/binary_sdk_bridge.dart is 156 lines of argument parsing, validation and
exit-code handling, and nothing tests it. test/generator_test.dart covers
BridgeSpec and BridgeGenerator thoroughly, but never invokes the CLI;
coverage/lcov.info contains six SF: records, all under lib/, none for
bin/.

Behaviours that are contracts a user hits, and that no test pins today:

  • Exit code 64 (EX_USAGE) on a missing --name/--org, a bad
    --min-sdk/--compile-sdk/--java, or an unknown option. Anyone scripting
    around this tool depends on the code, and nothing would catch it changing.
  • --dry-run writes nothing. It prints the plan and returns, but no test
    asserts that the output directory is still empty afterwards. That is the one
    property of --dry-run that matters.
  • --help and a bare invocation print usage and exit 0.
  • --min-sdk foo prints error: --min-sdk must be a whole number. rather
    than a Dart stack trace. The comment above the int.tryParse block says this
    shipped broken once — CONTRIBUTING asks for a test that fails without the
    fix, and this one never got one.

There is also something a test would settle. --flavor is declared with
allowed: ['flutter', 'native'], so parser.parse throws a FormatException
for a bad value and _fail handles it — which means the later
BridgeFlavor.parse(...) == null branch and its --flavor must be "flutter" or "native" message appear to be unreachable from the CLI. Confirmed by hand on
main:

$ dart run bin/binary_sdk_bridge.dart --name a_b --org com.example --ios-framework X --flavor nonsense
error: "nonsense" is not an allowed value for option "--flavor".
$ echo $?
64

Worth a test that records which message actually answers, and then either
deleting the dead branch or keeping it deliberately (it still guards the
library API, where BridgeFlavor.parse is public).

Why it matters

The CLI is the whole user interface. Everything a user of this tool touches
before they see a generated file goes through this one untested file, and the
error paths — the parts most likely to regress unnoticed — are exactly the
parts with no coverage.

Suggested approach

Add test/cli_test.dart. Two viable shapes; pick one and say in the PR why:

  1. Out of process. Run the real entry point with
    Process.run(Platform.resolvedExecutable, ['run', 'bin/binary_sdk_bridge.dart', ...])
    into a Directory.systemTemp.createTempSync output dir, and assert on
    exitCode, stdout and stderr. Slower, but it tests the actual contract
    including the exit codes.
  2. In process. Refactor main into something like
    int run(List<String> args, {StringSink out, StringSink err}) with main
    as a thin wrapper, and call it directly. Faster and easier to assert on, at
    the cost of a slightly larger diff to bin/.

A group per concern (usage errors, --dry-run, --help) keeps it readable.

Files involved

  • new test/cli_test.dart
  • bin/binary_sdk_bridge.dart — only if you take approach 2

How to verify

dart pub get
dart analyze --fatal-infos
dart format .
dart test

The new tests should fail if you, say, change exitCode = 64 to 1 in
_fail, or make --dry-run fall through to generator.write. Try both
mutations locally before opening the PR — a test that passes either way is not
covering anything.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomerstestsTest coverage and test infrastructure

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions