Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 14 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -645,24 +645,28 @@ jobs:
# with them. Each app's list of what its release ships is held to what it
# really resolves, and a notice copied from one release of a library is
# held to that release, so neither can drift out of date unnoticed. The
# watch's rules for its tile and complication, and the car's reading of
# what is typed into it, are tested here too.
# car's screens, every template it is handed and what it hands on, and its
# reading of what is typed into it, are tested here too; the watch's tile
# and complication run with its screens below.
- name: Every library each release ships is in its notices
run: ./gradlew --no-daemon :app:checkNotices :wear:checkNotices :automotive:checkNotices :notices:testDebugUnitTest :wear:testDebugUnitTest :car:testDebugUnitTest
run: ./gradlew --no-daemon :app:checkNotices :wear:checkNotices :automotive:checkNotices :notices:testDebugUnitTest :car:testDebugUnitTest
# Every screen drawn here, by Android's own graphics stack under
# Robolectric, and compared with the image kept in the repository: light
# and dark, a small phone to a tablet, the largest text, a longer language
# and right to left. A screen that moved fails, and the old, the new and
# the difference between them are kept below. The app's other unit tests
# run in the same pass.
# Robolectric, and compared with the image kept in the repository: the
# phone in light and dark, a small phone to a tablet, the largest text, a
# longer language and right to left; and the watch on a round face, its
# tile as the watch's own renderer draws it. A screen that moved fails,
# and the old, the new and the difference between them are kept below.
# Each app's other unit tests run in the same pass.
- name: Every screen matches its reference image
run: ./gradlew --no-daemon :app:verifyRoborazziDebug
run: ./gradlew --no-daemon :app:verifyRoborazziDebug :wear:verifyRoborazziDebug
- name: Keep the screens that no longer match
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: screens-that-moved
path: android/app/build/outputs/roborazzi
path: |
android/app/build/outputs/roborazzi
android/wear/build/outputs/roborazzi
if-no-files-found: ignore
# Lint's errors fail the build in every module; its warnings do not. The
# errors it has found so far were screens reading the device's language,
Expand Down
29 changes: 28 additions & 1 deletion android/TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ world, is set out at the end as a guide to work through before each release.
| Design | Every screen against a reference image: light and dark, a small phone, a phone, a foldable and a tablet, the largest text, a longer language and right to left | `app/src/test/kotlin/com/gridpointcode/screens`, images in `app/src/test/screenshots` | CI, every PR |
| Robustness | StrictMode and leak detection around every device test, a seeded stress run of random input, the process ended and the screen put back | `NoStrictModeViolations`, LeakCanary, `RandomInputTest`, `app/src/test/.../ProcessDeathTest.kt` | CI, every PR |
| Performance | The baseline profile the release carries, and benchmarks of start-up and frames with it and without it | `benchmark`, `app/src/release/generated/baselineProfiles` | CI checks the profile reaches the release; benchmarks by hand |
| The watch and the car | Their screens, the tile, the complication and the car's templates | next | |
| The watch and the car | The car's templates and what they hand on; the watch's tile and complication, their timelines and how they draw; the watch's screens on a round face | `car/src/test`, `wear/src/test`, images in `wear/src/test/screenshots` | CI, every PR |

## Running them

Expand Down Expand Up @@ -159,6 +159,33 @@ folder when the check fails.
waiting to be compiled. Whether the profile helps a reader is for a real
phone to say, a low-end one most of all; see below.

### How the watch and the car are checked

- **The car's templates.** A car draws an app's screens itself, from templates,
and refuses one that breaks its rules. `CarScreensTest` hands each screen to
the Car App Library's own test car and reads what comes back, row by row and
action by action: a saved place's code, how it is said, how far and which way
with the directions to the door; where the car is, with how close and no way
to navigate to it; the list before a fix, newest first; a car with no phone;
search, a code offered as it is typed and a short form waiting for a fix; and
a car older than level 7. Navigate is pressed, and what it hands the driver's
navigation app is checked: a geo URI of the place, never the code.
- **The watch's tile and complication.** Each is asked as a watch asks, over a
last fix planted in their store: the place for its hour, then the offer to
find one, both in one timeline whose first entry ends when the hour does. An
entry left without an end ends at zero, and the watch drew nothing; the tests
hold that. The tile is drawn by the Tiles renderer the watch itself uses, on a
round face, at a time held still, and kept as an image.
- **The watch's screens.** Here, asking for location, finding, and with a place;
the saved places with their arrows; walking to one. Drawn on a round face
with the time along the top held at ten past ten, and verified with the
phone's images by `:wear:verifyRoborazziDebug`.
- **One process, many tests.** Robolectric keeps the process from one test to
the next. Android's default factory for view models is kept for the process
too, so it handed every later watch test the first test's application, and
its files; the watch's screen tests make their view model with a factory of
their own.

### How the device tests are written

- **As a reader, not as the code.** `Reader` drives the app by what the screen
Expand Down
10 changes: 10 additions & 0 deletions android/car/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ android {
targetCompatibility = JavaVersion.VERSION_17
}

testOptions {
// The templates are built from the car module's own strings and icons.
unitTests.isIncludeAndroidResources = true
// Robolectric's Android 16 reaches into the JDK's own file descriptors.
unitTests.all { it.jvmArgs("--add-opens=java.base/jdk.internal.access=ALL-UNNAMED") }
}

sourceSets {
getByName("main") {
// The palette as the resources design/build-tokens.mjs writes, day and
Expand All @@ -37,4 +44,7 @@ dependencies {
api(libs.kotlinx.coroutines.core)

testImplementation(libs.kotlin.test.junit)
testImplementation(libs.robolectric)
testImplementation(libs.androidx.test.ext.junit)
testImplementation(libs.car.app.testing)
}
173 changes: 173 additions & 0 deletions android/car/src/test/kotlin/com/gridpointcode/car/CarScreensTest.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
package com.gridpointcode.car

import android.content.Context
import android.os.Looper
import androidx.car.app.CarContext
import androidx.car.app.HandshakeInfo
import androidx.car.app.versioning.CarAppApiLevels
import androidx.car.app.OnDoneCallback
import androidx.car.app.model.Action
import androidx.car.app.model.Distance
import androidx.car.app.model.DistanceSpan
import androidx.car.app.model.ListTemplate
import androidx.car.app.model.PaneTemplate
import androidx.car.app.model.Row
import androidx.car.app.model.SearchTemplate
import androidx.car.app.testing.TestCarContext
import androidx.car.app.testing.TestScreenManager
import androidx.test.core.app.ApplicationProvider
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.gridpointcode.core.Point
import com.gridpointcode.core.SavedPlace
import com.gridpointcode.core.Source
import com.gridpointcode.core.aloud
import com.gridpointcode.core.formatted
import com.gridpointcode.core.metresBetween
import com.gridpointcode.core.selectionOf
import com.gridpointcode.core.spellingFor
import java.util.Locale
import kotlinx.coroutines.flow.MutableStateFlow
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.Shadows.shadowOf

/**
* The car's screens as the car is handed them: templates, checked row by row
* and action by action on the Car App Library's own test car. A car draws a
* template in its own style and refuses one that breaks its rules, so what is
* checked is what each screen says and does, and what it hands to other apps.
*/
@RunWith(AndroidJUnit4::class)
class CarScreensTest {

private val car = carAt(CarAppApiLevels.LEVEL_7)

/** A test car that has shaken hands at [level], as a real one does before it asks for a template. */
private fun carAt(level: Int) = TestCarContext.createCarContext(ApplicationProvider.getApplicationContext()).apply {
updateHandshakeInfo(HandshakeInfo("com.gridpointcode.test.host", level))
}
private val speaker by lazy { CarSpeaker(car) }
private val spelling = spellingFor(Locale.ENGLISH)

private val market = SavedPlace("G3RJM8X3L1", "Market, north door", "North door, beside the bakery stall", savedAt = 1)
private val kensington = SavedPlace("G3RJL5FRCR", "Kensington Market", "", savedAt = 2)

/** The car at Union Station, to within four metres. */
private val union = CarFix(Point(43.64546, -79.38063), 4)

private fun places(
vararg saved: SavedPlace,
savedHere: Boolean = true,
notices: ((Context) -> String)? = null,
) = CarPlaces(MutableStateFlow(saved.toList()), { spelling }, savedHere, notices)

private fun Row.lines(): List<String> = listOf(title.toString()) + texts.map { it.toString() }

private fun click(action: Action) = sent { action.onClickDelegate!!.sendClick(it) }

private fun click(row: Row) = sent { row.onClickDelegate!!.sendClick(it) }

/** Sends a host's call into the app and lets the main thread run it, as the car's would. */
private fun sent(call: (OnDoneCallback) -> Unit) {
call(object : OnDoneCallback {})
shadowOf(Looper.getMainLooper()).idle()
}

@Test
fun aSavedPlaceSaysItsCodeHowFarWhichWayAndHowToFindTheDoor() {
val template = PlaceScreen(car, Shown.saved(market), union, places(market), speaker).onGetTemplate() as PaneTemplate
val rows = template.pane.rows
assertEquals(listOf(formatted(market.code), aloud(market.code, spelling)), rows[0].lines())
// The car writes the distance itself, from a span, in the driver's units.
val far = rows[1].title!!.spans.single().carSpan as DistanceSpan
val metres = metresBetween(union.point, selectionOf(market.code, Source.SAVED).point)
assertTrue("the market is under a kilometre from the station: $metres", metres < 1000)
assertEquals(Distance.UNIT_METERS, far.distance.displayUnit)
assertEquals(Math.round(metres).toDouble(), far.distance.displayDistance, 0.0)
assertEquals("north-east · North door, beside the bakery stall", rows[1].texts.single().toString())
assertEquals(listOf("Navigate", "Read aloud"), template.pane.actions.map { it.title.toString() })
assertEquals("Market, north door", template.header?.title?.toString() ?: template.title.toString())
}

@Test
fun navigateHandsThePointToTheDriversOwnNavigationApp() {
val template = PlaceScreen(car, Shown.saved(market), union, places(market), speaker).onGetTemplate() as PaneTemplate
click(template.pane.actions.first { it.title.toString() == "Navigate" })
val sent = car.startCarAppIntents.single()
val point = selectionOf(market.code, Source.SAVED).point
assertEquals(CarContext.ACTION_NAVIGATE, sent.action)
assertEquals(String.format(Locale.ROOT, "geo:%.6f,%.6f", point.latitude, point.longitude), sent.dataString)
}

@Test
fun whereTheCarIsSaysHowCloseAndOffersNoWayToNavigateThere() {
val template = PlaceScreen(car, Shown.here(union), union, places(), speaker).onGetTemplate() as PaneTemplate
assertEquals(formatted(union.code), template.pane.rows[0].title.toString())
assertEquals("±4 m", template.pane.rows[1].title.toString())
assertEquals(listOf("Read aloud"), template.pane.actions.map { it.title.toString() })
assertEquals("Where am I", template.header?.title?.toString() ?: template.title.toString())
}

@Test
fun beforeAFixTheSavedPlacesAreAListNewestFirstUnderWhereTheCarIs() {
// As the shelf keeps them: the newest first.
val template = PlacesScreen(car, places(kensington, market), speaker).onGetTemplate() as ListTemplate
val rows = template.singleList!!.items.map { it as Row }
assertEquals("Where am I", rows[0].title.toString())
assertEquals("Allow location to find where the car is.", rows[0].texts.single().toString())
assertEquals(listOf("Kensington Market", "Market, north door"), rows.drop(1).map { it.title.toString() })
assertEquals(formatted(market.code), rows[2].texts.last().toString())
}

@Test
fun anOlderCarGetsTheTitleTheWayItKnows() {
val older = carAt(CarAppApiLevels.LEVEL_6)
val template = PlaceScreen(older, Shown.saved(market), union, places(market), CarSpeaker(older)).onGetTemplate() as PaneTemplate
// Built the way a car before level 7 takes it, with the title and the
// header action set on their own, and without asking for a header.
@Suppress("DEPRECATION")
assertEquals("Market, north door", template.title.toString())
}

@Test
fun aSavedPlaceOpensItsOwnScreen() {
val template = PlacesScreen(car, places(market), speaker).onGetTemplate() as ListTemplate
click(template.singleList!!.items[1] as Row)
val pushed = car.getCarService(TestScreenManager::class.java).screensPushed.single()
val opened = (pushed as PlaceScreen).onGetTemplate() as PaneTemplate
assertEquals(formatted(market.code), opened.pane.rows[0].title.toString())
}

@Test
fun aCarWithNoPhoneSaysWhereTheSavedPlacesAre() {
val template = PlacesScreen(car, places(savedHere = false, notices = { "The notices." }), speaker).onGetTemplate() as ListTemplate
val rows = template.singleList!!.items.map { it as Row }
assertEquals("Saved places come from the phone app, with Android Auto.", rows.last().title.toString())
val actions = template.actionStrip!!.actions
assertEquals(2, actions.size)
assertEquals("Notices", actions.last().title.toString())
}

@Test
fun aCodeTypedIsOfferedAsItIsTypedAndOpensWhenSent() {
val screen = SearchScreen(car, union, places(), speaker)
val empty = screen.onGetTemplate() as SearchTemplate
assertEquals("The place appears here as it is typed.", empty.itemList!!.noItemsMessage.toString())
sent { empty.searchCallbackDelegate.sendSearchTextChanged("g3rjm 8x3l1", it) }
val offered = (screen.onGetTemplate() as SearchTemplate).itemList!!.items.single() as Row
assertEquals(listOf(formatted(market.code), "Open"), offered.lines())
sent { empty.searchCallbackDelegate.sendSearchSubmitted("g3rjm 8x3l1", it) }
assertTrue(car.getCarService(TestScreenManager::class.java).screensPushed.single() is PlaceScreen)
}

@Test
fun aShortFormWaitsForAFixRatherThanNamingADoorElsewhere() {
val screen = SearchScreen(car, null, places(), speaker)
val template = screen.onGetTemplate() as SearchTemplate
sent { template.searchCallbackDelegate.sendSearchTextChanged("-8X3L1", it) }
val row = (screen.onGetTemplate() as SearchTemplate).itemList!!.items.single() as Row
assertEquals("A short form is read against where the car is, which is not known yet.", row.title.toString())
}
}
2 changes: 2 additions & 0 deletions android/car/src/test/resources/robolectric.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Android 16, the release the device tests also run on.
sdk=36
6 changes: 6 additions & 0 deletions android/gradle/libs.versions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,12 @@ leakcanary-android-instrumentation = { module = "com.squareup.leakcanary:leakcan
androidx-profileinstaller = { module = "androidx.profileinstaller:profileinstaller", version.ref = "profileinstaller" }
androidx-benchmark-macro-junit4 = { module = "androidx.benchmark:benchmark-macro-junit4", version.ref = "benchmark" }

# The car's templates and the watch's tile, checked on the computer: the Car App
# Library's test car, and the tile service's test client and renderer.
car-app-testing = { module = "androidx.car.app:app-testing", version.ref = "carApp" }
wear-tiles-testing = { module = "androidx.wear.tiles:tiles-testing", version.ref = "wearTiles" }
wear-tiles-renderer = { module = "androidx.wear.tiles:tiles-renderer", version.ref = "wearTiles" }

[plugins]
android-application = { id = "com.android.application", version.ref = "agp" }
android-library = { id = "com.android.library", version.ref = "agp" }
Expand Down
29 changes: 29 additions & 0 deletions android/wear/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
plugins {
alias(libs.plugins.android.application)
alias(libs.plugins.kotlin.compose)
alias(libs.plugins.roborazzi)
}

// The same four values the phone app is signed with for upload, and the same
Expand Down Expand Up @@ -42,6 +43,14 @@ android {
compose = true
}

testOptions {
// The watch's screens, tile and complication, drawn and read on the
// computer, need its strings, fonts and drawables.
unitTests.isIncludeAndroidResources = true
// Robolectric's Android 16 reaches into the JDK's own file descriptors.
unitTests.all { it.jvmArgs("--add-opens=java.base/jdk.internal.access=ALL-UNNAMED") }
}

signingConfigs {
if (upload.values.all { it != null }) {
create("upload") {
Expand Down Expand Up @@ -102,7 +111,27 @@ dependencies {
implementation(libs.androidx.concurrent.futures)

testImplementation(libs.kotlin.test.junit)
testImplementation(platform(libs.compose.bom))
testImplementation(libs.compose.ui.test.junit4)
testImplementation(libs.androidx.test.ext.junit)
testImplementation(libs.robolectric)
testImplementation(libs.roborazzi)
testImplementation(libs.roborazzi.compose)
testImplementation(libs.roborazzi.junit.rule)
testImplementation(libs.wear.tiles.testing)
// The renderer the watch draws tiles with, for the tile's reference images.
// In the debug build rather than the tests alone, because a library's own
// resources are made only for a build it is part of; the release never has it.
debugImplementation(libs.wear.tiles.renderer)
debugImplementation(libs.compose.ui.test.manifest)
}

// Holds the watch's notices to what its release really ships.
apply(from = rootProject.file("gradle/notices.gradle.kts"))

// The watch's reference images, beside the phone's: its screens on a round
// face and its tile as the watch draws it. ./gradlew :wear:recordRoborazziDebug
// draws them again; :wear:verifyRoborazziDebug, which CI runs, compares.
roborazzi {
outputDir.set(file("src/test/screenshots"))
}
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,16 @@ import java.util.Date
* entry of its timeline. Read aloud opens the app, which reads the next place
* it finds: where the wrist is now, not the code on the tile.
*/
class CodeTile : TileService() {
class CodeTile(
// The time, for a test to hold still. The system makes the tile with none
// given, through the constructor with no arguments Kotlin writes for it.
private val clock: () -> Long = System::currentTimeMillis,
) : TileService() {

override fun onTileRequest(request: RequestBuilders.TileRequest): ListenableFuture<TileBuilders.Tile> {
val device = request.deviceConfiguration
val last = LastFixStore(this).read()
val now = System.currentTimeMillis()
val now = clock()
val timeline = TimelineBuilders.Timeline.Builder()
if (last != null && last.shown(now)) {
timeline.addTimelineEntry(entry(found(this, device, last), end = last.shownUntil))
Expand Down
Loading
Loading