Skip to content

Add end-to-end Smart Test Picker demo project #26

Description

@ljubisap

Goal

Add a small, self-contained demo application to the Smart Test Picker repository that demonstrates the complete workflow from a production code change to affected test selection and execution.

The demo should make it possible for a new user to understand what Smart Test Picker does and see it working within a few minutes.

Motivation

We already have the individual building blocks for runtime dependency observation and per-test attribution, and we use Spring PetClinic as a more realistic validation target.

What is still missing is a simple user-facing example that demonstrates the complete Smart Test Picker workflow:

code change
    ->
detect changed production classes
    ->
find affected tests
    ->
execute affected tests only

Spring PetClinic should remain a real-world integration/validation target. The new demo should instead be intentionally small, deterministic and easy to understand.

Proposed structure

Add a demo project under the main repository, for example:

examples/
    spring-demo/
        build.gradle
        src/main/java/...
        src/test/java/...

The application should contain only a few components, for example:

Controller
    ->
Service
    ->
Repository

and approximately 5-10 tests with easily understandable dependencies.

Demo scenario

The demo should support the following workflow.

1. Run the complete test suite

Run all tests while Smart Test Picker records runtime dependencies.

Example dependency information:

UserControllerTest#getUser
    -> UserController
    -> UserService
    -> UserRepository

UserServiceTest#updateUser
    -> UserService
    -> UserRepository

2. Change production code

For example:

UserService.java

3. Determine affected tests

Smart Test Picker should use the changed production classes and previously collected runtime dependencies to determine the affected tests.

Example:

Changed classes:
UserService

Selected tests:
UserControllerTest#getUser
UserServiceTest#updateUser

4. Execute only the selected tests

The demo should provide a simple command that runs only the affected tests.

The complete workflow should not require manual inspection or modification of generated dependency data.

Requirements

  • Demo lives in the main Smart Test Picker repository.
  • Demo is self-contained and reproducible.
  • Use a small application rather than copying Spring PetClinic.
  • Runtime dependencies are collected through the Smart Test Picker runtime observation mechanism.
  • Dependencies are attributed to individual tests.
  • Production code changes are detected from Git.
  • Changed classes are mapped back to affected tests.
  • Only affected tests can be executed.
  • Full test execution remains available for comparison.
  • Demo uses the same public Smart Test Picker artifacts and APIs that a real user would use.
  • Avoid demo-specific shortcuts in the Smart Test Picker implementation.

Documentation

Add a short README explaining:

  1. What the demo application contains.
  2. How to run the initial full test execution.
  3. Where Smart Test Picker stores the collected dependency information.
  4. How to make a sample production code change.
  5. How to calculate the affected tests.
  6. How to execute the selected tests.
  7. How to compare selected execution with the full suite.

A user unfamiliar with the Smart Test Picker internals should be able to complete the example without reading the implementation.

Expected output

Where practical, provide a human-readable summary such as:

Smart Test Picker

Changed production classes: 1
Affected tests:             2
Total tests:                8

Selected: 25%
Skipped:  75%

Selected tests:
- UserControllerTest#getUser
- UserServiceTest#updateUser

The exact output format is not part of this issue and can be adjusted based on the existing Smart Test Picker APIs.

Validation

The demo should include at least the following scenarios:

  • Change a class used by one test.
  • Change a class used by multiple tests.
  • Change an unrelated class and verify unrelated tests are not selected.
  • Run with no relevant production changes.
  • Run the full test suite and compare the result with the selected execution.

For the supported demo scenarios, the selected execution must not miss a test that would fail in the full suite because of the corresponding production code change.

Out of scope

This issue does not attempt to solve all framework-specific runtime dependency cases.

In particular, the following can be handled separately:

  • complex Spring Data proxy behavior
  • generated classes
  • reflection-heavy frameworks
  • asynchronous execution
  • additional test frameworks
  • performance optimization for large repositories
  • advanced test-selection algorithms

The purpose of this issue is to demonstrate the existing Smart Test Picker concept end-to-end before expanding framework coverage.

Follow-up

Once the local demo works reliably, a follow-up issue can add a GitHub Actions PR demonstration showing:

Changed production classes: 2
Affected tests:             7
Total tests:               42

Selected: 16.7%
Skipped:  83.3%

including execution-time comparison between the full and selected test suites.

Acceptance criteria

  • A small demo application exists under examples/.
  • Full test execution records per-test runtime dependencies.
  • Production changes can be detected from Git.
  • Changed production classes can be mapped to affected tests.
  • Only affected tests can be executed.
  • The workflow uses normal Smart Test Picker APIs/artifacts rather than demo-specific implementation shortcuts.
  • README documents the complete workflow.
  • At least one example demonstrates a production change selecting a subset of the test suite.
  • Full-suite and selected-suite results can be compared.
  • The demo is suitable as the primary getting-started example for the open-source project.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions