diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 4384cc1..a0d0c72 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -43,6 +43,7 @@ dotnet test tests/FFmpegKit.Android.PackageTests # validate the packed .nup Everything runs on **ubuntu-latest**. `pr.yml` and `release.yml` both call the reusable `build.yml` (`verify` input: true on PRs, false on releases). - Pull request → pack all eight variants as `-beta..`, package tests, MAUI sample builds (net8 + net10), emulator smoke tests (all eight variants on net10, plus Video and Audio on net8), a run of the sample on the emulator, then publish the betas to nuget.org. Forked PRs build and test but skip publishing. Betas cannot be deleted, only unlisted. +- Older tracks run only when a pull request needs them: `.github/scripts/tracks-to-verify.sh` picks a track when the pull request adds its release note or changes its mapping row, and every shipping track when it changes the checks themselves. Each one packs as `-beta..`, runs package tests, Audio and Video on net8 on API 31 and 35, and the sample. Those packages are never published. - Merging `docs/release-notes/.md` to `main` is the release: `auto-release.yml` tags the merge `v` and dispatches `release.yml`, whose `guard` job proves the commit is an ancestor of the default branch before anything publishes. - Tags name the **FFmpeg** version: `v6.1.6.5` publishes 6.1.6.5 and binds FFmpegKit 6.0.4 from the mapping; a version outside `build/native-versions.tsv` fails the release with the list of known builds. - `upstream-drift.yml` (daily) runs `build/check-upstream.sh` over `build/upstream.tsv` and files one issue per group. diff --git a/.github/instructions/native-versions.instructions.md b/.github/instructions/native-versions.instructions.md index b2d64b5..ddb7fd4 100644 --- a/.github/instructions/native-versions.instructions.md +++ b/.github/instructions/native-versions.instructions.md @@ -8,7 +8,7 @@ The `.aar` files come from Maven Central (`dev.ffmpegkit-maintained`) and are fe 1. **Establish the FFmpeg version the release actually contains.** Read it out of the shipped `libavcodec.so` (`jni/arm64-v8a/` inside the `.aar`), the way `.github/scripts/native-build-info.sh` does. Upstream release prose mentions other FFmpeg versions in passing and must not be used as the source. 2. **Record the checksum baseline:** `./build/update-checksums.sh ` writes `build/checksums/.sha256` (Maven Central `.sha256` sidecars where published, download-and-hash otherwise). Review the diff and commit it. This is deliberately never run in CI — CI verifies against a baseline a human recorded. -3. **Add the mapping row** to `build/native-versions.tsv`: `\t`, tab-separated, keeping the existing comment header. There is one row per FFmpeg version, because the lookup stops at the first match and a test requires both columns to be unique. A new FFmpeg gets a new row. A respin of the same FFmpeg replaces that row: note the old pair in the header comment, keep its checksum baseline, and say in the release note that the old FFmpegKit release can no longer be built from `main`. Existing tags are unaffected, because a release reads the file as it stood at its tag. Never drop a track. +3. **Add the mapping row** to `build/native-versions.tsv`: `\t`, tab-separated, keeping the existing comment header. There is one row per FFmpeg version, because the lookup stops at the first match and a test requires both columns to be unique. A new FFmpeg gets a new row. A respin of the same FFmpeg replaces that row: note the old pair in the header comment, keep its checksum baseline, and say in the release note that the old FFmpegKit release can no longer be built from `main`. Existing tags are unaffected, because a release reads the file as it stood at its tag. Never drop a track. A row that is not the default makes `pr.yml` run that track on the emulator, on the pull request that changes it. 4. **Then**, if this is the default line, bump `FFmpegVersion` and `FFmpegKitNativeVersion` in `Directory.Build.props` (both, together — they are different schemes). Older lines leave the props alone. Either way, fetch: `./src/FFmpegKit.Android/Jars/FetchJars.sh `. 5. Verify and pack: `./src/FFmpegKit.Android/Jars/FetchJars.sh --verify`, then `./src/FFmpegKit.Android/BuildNugets.sh`, then `dotnet test tests/FFmpegKit.Android.PackageTests` (it asserts the mapping and the props agree). diff --git a/.github/instructions/workflows.instructions.md b/.github/instructions/workflows.instructions.md index 3a85d73..39ed8f3 100644 --- a/.github/instructions/workflows.instructions.md +++ b/.github/instructions/workflows.instructions.md @@ -14,5 +14,6 @@ applyTo: ".github/workflows/*.yml" - The pack job caches `src/FFmpegKit.Android/Jars` keyed on the native version, and re-runs `FetchJars.sh --verify` even on a cache hit. Keep that verification step. - Emulator jobs must stay `x86_64` (the `.aar` ships arm64-v8a and x86_64 only), keep KVM enabled and keep the disk-cleanup step; the AVD needs ~7.2 GB free. - The `e2e` matrix runs every variant, one job each: each is a different `.aar`, and nothing else loads its native code. Do not narrow it to save time, and keep the Audio leg on API 31: it is the only one that can see an x86_64-specific load failure, which API 35 hides. `sample-run` deploys the sample because the device checks once passed on a package that froze it; keep it a gate on publishing betas. +- `older-tracks` in `pr.yml` is what makes the release's `verify: false` honest for a track that is not the default: merging a note is how a track ships, and `tracks-to-verify.sh` runs the track on the pull request that adds it (a tag pushed by hand skips this). Keep that trigger, keep both API levels (the failure it exists for only shows on API 31), and keep `publish` from taking its packages: the `artifact-suffix` input is what separates them. `publish` needs `older-tracks` and must keep accepting `skipped`, or every pull request that runs no track stops publishing. `tracks-to-verify.sh` lists the checks whose changes run every track; a new script the track legs depend on belongs in that list. New scripts need `git update-index --chmod=+x`, because the working volume does not track file modes. - `upstream-drift.yml` only orchestrates: what is watched belongs in `build/upstream.tsv`, how it is checked in `build/check-upstream.sh`. - Keep the explanatory comments — they record failures that have already happened once (NETSDK1112, emulator boot timeouts, silent OIDC failures). diff --git a/.github/scripts/run-sample-test.sh b/.github/scripts/run-sample-test.sh index a831cd4..5d9dec9 100755 --- a/.github/scripts/run-sample-test.sh +++ b/.github/scripts/run-sample-test.sh @@ -107,15 +107,31 @@ screen_shows() { # Takes the bounds from the dump instead of fixed coordinates, so that a different screen or font # cannot move a tap off its target. Reads the latest dump, so it follows a screen_shows. -tap_text() { +tap_node() { local left top right bottom # bounds="[left,top][right,bottom]" - read -r left top right bottom < <(grep -o "]*text=\"$1\"[^>]*>" "${UI_DUMP}" \ + read -r left top right bottom < <(grep -o "]*$1[^>]*>" "${UI_DUMP}" \ | sed -n 's/.*bounds="\[\([0-9]*\),\([0-9]*\)\]\[\([0-9]*\),\([0-9]*\)\]".*/\1 \2 \3 \4/p') \ - || fail "'$1' is not on screen" + || fail "$1 is not on screen" adb shell input tap "$(( (left + right) / 2 ))" "$(( (top + bottom) / 2 ))" } +tap_text() { + tap_node "text=\"$1\"" +} + +# hide_error_dialogs stops new dialogs but does not close one that is already up, and uiautomator +# only reads the focused window. A launcher that stalled during boot left the sample fully loaded +# behind its dialog, and this script timed out waiting for text it could not see. By resource id +# because the label is translated. An ANR of the sample's own would still be in the log, so closing +# its dialog here would not hide it from require_healthy. +close_leftover_dialog() { + if grep 'resource-id="android:id/aerr_wait"' "${UI_DUMP}" > /dev/null; then + echo "==> closing a system dialog that was already up" + tap_node 'resource-id="android:id/aerr_wait"' + fi +} + # The freeze needs a garbage collection after FFmpeg has run, and with Mono's default 4 MB nursery # this small app can finish a conversion and sit idle without one: the published 8.1.2.5, which does # freeze, passed this script three times out of three until this was added. A tiny nursery makes @@ -157,6 +173,7 @@ until screen_shows 'text="Source: '; do [ "${SECONDS}" -lt "${deadline}" ] || fail "the sample did not load within ${LAUNCH_TIMEOUT}s" # Nothing has been tapped yet, so a slow first frame cannot be an ANR. require_running "while loading" + close_leftover_dialog sleep 2 done diff --git a/.github/scripts/tracks-to-verify.sh b/.github/scripts/tracks-to-verify.sh new file mode 100755 index 0000000..df76ee5 --- /dev/null +++ b/.github/scripts/tracks-to-verify.sh @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Works out which older tracks a pull request has to run on a device, and prints them as a JSON array +# for a workflow matrix: [{"ffmpeg":"7.1.5","ffmpegkit":"7.1.7"}]. Most pull requests print []. +# +# tracks-to-verify.sh +# +# release.yml packs a tag and verifies nothing, on the grounds that its commit was verified on its +# pull request. That only holds for a track the pull request built, and it used to build the default +# track alone: 7.1.5.4 reached nuget.org with a freeze on x86_64, and Audio 7.1.5.4 could not load on +# API 31. So a track runs here when one of these is true: +# +# - a release note for it is added, because merging that note is what releases it; +# - its row in native-versions.tsv is added or replaced, because that changes the .aar it binds; +# - the checks below change, because nothing else would try the change on the older tracks. This +# runs every track that still ships. +# +# Not when the shared binding code changes, which is the price of running this on so few pull +# requests: a change that breaks an older track shows up on that track's next release note, still +# before it ships, rather than on the pull request that caused it. +# +# The default track is left out: pr.yml always runs it, on more variants than an older track gets. + +BASE="${1:?a base commit is required}" +HEAD="${2:?a head commit is required}" + +cd "$(dirname "$0")/../.." + +CHECKS=( + .github/scripts/run-device-tests.sh + .github/scripts/run-sample-test.sh + .github/scripts/tracks-to-verify.sh + .github/workflows/build.yml + .github/workflows/pr.yml +) + +# Checked first because every question below is a git diff: one that cannot read its commits would +# otherwise look like a pull request with nothing to verify, and the run would be green for it. +for commit in "${BASE}" "${HEAD}"; do + if ! git rev-parse --verify --quiet "${commit}^{commit}" > /dev/null; then + echo "::error::'${commit}' is not a commit in this checkout" >&2 + exit 1 + fi +done + +# Four parts only, the same rule auto-release.yml applies before it tags a note. Added only, too: an +# edit corrects a release that already happened. +note_tracks() { + git diff --name-only --diff-filter=A "${BASE}" "${HEAD}" -- 'docs/release-notes/*.md' \ + | sed -nE 's:^docs/release-notes/([0-9]+\.[0-9]+\.[0-9]+)\.[0-9]+(-[0-9A-Za-z.-]+)?\.md$:\1:p' +} + +# Rows start with a digit, which also skips the diff's own +++ line and any change to a comment. +row_tracks() { + git diff --unified=0 "${BASE}" "${HEAD}" -- build/native-versions.tsv \ + | awk '/^\+[0-9]/ { print substr($1, 2) }' +} + +# The newest FFmpeg of each major.minor. An older one stays in the mapping so that it can be rebuilt, +# as 8.1.2 did when 8.1.3 replaced it, but a row like that is not a track. +shipping_tracks() { + awk '/^[[:space:]]*#/ { next } NF >= 2 { print $1 }' build/native-versions.tsv \ + | sort -t. -k1,1n -k2,2n -k3,3n \ + | awk -F. '{ newest[$1 "." $2] = $0 } END { for (track in newest) print newest[track] }' +} + +notes="$(note_tracks)" +rows="$(row_tracks)" +shipping="" +if ! git diff --quiet "${BASE}" "${HEAD}" -- "${CHECKS[@]}"; then + shipping="$(shipping_tracks)" +fi + +echo "==> release notes added for:" ${notes:-nothing} >&2 +echo "==> mapping rows changed for:" ${rows:-nothing} >&2 +echo "==> checks changed, so every shipping track:" ${shipping:-no} >&2 + +default_ffmpeg="$(sed -n 's:.*\(.*\).*:\1:p' Directory.Build.props | head -1)" + +tracks="" +for ffmpeg in $(printf '%s\n' "${notes}" "${rows}" "${shipping}" | sort -u); do + [ "${ffmpeg}" != "${default_ffmpeg}" ] || continue + + # An unmapped version fails here rather than at the release, which would already have been + # tagged by the time it found out. + if ! resolved="$(./build/resolve-versions.sh --ffmpeg "${ffmpeg}" 2>&1)"; then + echo "::error::${resolved}" >&2 + exit 1 + fi + eval "${resolved}" + + tracks="${tracks:+${tracks},}{\"ffmpeg\":\"${ffmpeg}\",\"ffmpegkit\":\"${ffmpegkit}\"}" +done + +echo "[${tracks}]" diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 03f4629..d505a76 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -49,6 +49,22 @@ on: required: false default: '["net10.0-android36.0"]' type: string + e2e-api-levels: + description: > + Which Android API levels every variant runs on, as a JSON array. API 35 by default, with + the one API 31 leg listed in the e2e job. An older track passes both, because a library + that loads on API 35 can still be refused on API 31. + required: false + default: '[35]' + type: string + artifact-suffix: + description: > + Appended to the name of every artifact this run uploads. A workflow that calls this more + than once needs it, since artifact names are unique across a whole run. It is also what + keeps an older track's packages out of the nuget-packages artifact that pr.yml publishes. + required: false + default: '' + type: string env: DOTNET_NOLOGO: 'true' @@ -136,7 +152,7 @@ jobs: - name: Upload packages uses: actions/upload-artifact@v4 with: - name: nuget-packages + name: nuget-packages${{ inputs.artifact-suffix }} path: | artifacts/*.nupkg artifacts/*.snupkg @@ -147,7 +163,7 @@ jobs: if: ${{ inputs.verify && (always()) }} uses: actions/upload-artifact@v4 with: - name: package-test-results + name: package-test-results${{ inputs.artifact-suffix }} path: '**/*.trx' if-no-files-found: ignore retention-days: 7 @@ -208,7 +224,7 @@ jobs: - name: Download packages uses: actions/download-artifact@v4 with: - name: nuget-packages + name: nuget-packages${{ inputs.artifact-suffix }} path: artifacts - name: Build the ${{ matrix.tfm }} sample against the packed package @@ -241,7 +257,7 @@ jobs: matrix: variant: ${{ fromJSON(inputs.e2e-variants) }} target-framework: ${{ fromJSON(inputs.e2e-target-frameworks) }} - api-level: [35] + api-level: ${{ fromJSON(inputs.e2e-api-levels) }} include: - variant: Video target-framework: net8.0-android34.0 @@ -289,7 +305,7 @@ jobs: - name: Download packages uses: actions/download-artifact@v4 with: - name: nuget-packages + name: nuget-packages${{ inputs.artifact-suffix }} path: artifacts - name: Free disk space for the emulator @@ -337,7 +353,7 @@ jobs: if: always() uses: actions/upload-artifact@v4 with: - name: device-test-logs-${{ matrix.variant }}-${{ matrix.target-framework }}-api${{ matrix.api-level }} + name: device-test-logs-${{ matrix.variant }}-${{ matrix.target-framework }}-api${{ matrix.api-level }}${{ inputs.artifact-suffix }} path: device-tests-logcat.txt if-no-files-found: ignore retention-days: 7 @@ -380,7 +396,7 @@ jobs: - name: Download packages uses: actions/download-artifact@v4 with: - name: nuget-packages + name: nuget-packages${{ inputs.artifact-suffix }} path: artifacts - name: Free disk space for the emulator @@ -421,7 +437,7 @@ jobs: if: always() uses: actions/upload-artifact@v4 with: - name: sample-test-logs + name: sample-test-logs${{ inputs.artifact-suffix }} path: | sample-test-logcat.txt sample-test-screen.png diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index ae3442d..20a9fb0 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -13,12 +13,16 @@ permissions: jobs: version: - name: resolve beta version + name: resolve beta version and older tracks runs-on: ubuntu-latest outputs: version: ${{ steps.resolve.outputs.version }} + tracks: ${{ steps.tracks.outputs.tracks }} steps: - uses: actions/checkout@v4 + with: + # The merge commit and its parents, which is all the older-track diff below reads. + fetch-depth: 2 - id: resolve run: | @@ -35,6 +39,14 @@ jobs: echo "version=${version}" >> "$GITHUB_OUTPUT" echo "Building ${version}" >> "$GITHUB_STEP_SUMMARY" + - id: tracks + run: | + # This runs on the merge commit, so its first parent is the base branch as this pull + # request would merge into it, and the diff is exactly what merging would change. + tracks="$(./.github/scripts/tracks-to-verify.sh HEAD^1 HEAD)" + echo "tracks=${tracks}" >> "$GITHUB_OUTPUT" + echo "Older tracks to verify: \`${tracks}\`" >> "$GITHUB_STEP_SUMMARY" + build: name: build needs: version @@ -42,13 +54,52 @@ jobs: with: version: ${{ needs.version.outputs.version }} + # The tracks that are not the default, so no pull request ever packed them, and release.yml packs a + # tag without verifying it. On demand rather than on every pull request: nothing reaches an older + # track until its release note merges, and its .aar only changes with its mapping row, so + # tracks-to-verify.sh runs a track on exactly those. Running them all the time would double this + # workflow for pull requests that ship nothing on those tracks, and the extra emulators are more + # chances for a slow boot to hold up the betas. A public repository pays nothing for the minutes. + # + # A beta of the track's own FFmpeg, because run-device-tests.sh reads the FFmpeg the package has to + # contain from the version. They stay in this run: publish below only takes the default track's. + older-tracks: + name: older track ${{ matrix.track.ffmpeg }} + needs: version + if: needs.version.outputs.tracks != '[]' + strategy: + # Tracks bind different .aar files, so one failing says nothing about the other. + fail-fast: false + matrix: + track: ${{ fromJSON(needs.version.outputs.tracks) }} + uses: ./.github/workflows/build.yml + with: + version: ${{ matrix.track.ffmpeg }}-beta.${{ github.event.number }}.${{ github.run_number }} + native-version: ${{ matrix.track.ffmpegkit }} + artifact-suffix: -${{ matrix.track.ffmpeg }} + # Both API levels, because FFmpegKit 7.1.6 was only refused on API 31, and both variants, + # because that hit Video on both ABIs and Audio on x86_64 alone. net8 is where those failures + # and the freeze were reproduced, and sample-run covers net10. + e2e-variants: '["Audio", "Video"]' + e2e-target-frameworks: '["net8.0-android34.0"]' + e2e-api-levels: '[31, 35]' + publish: name: publish beta to nuget.org - needs: [version, build] + needs: [version, build, older-tracks] runs-on: ubuntu-latest # Forked pull requests get no OIDC token for this repository, so the exchange would fail with # a confusing error. Those PRs still build and test; only the publish step is skipped. - if: github.event.pull_request.head.repo.full_name == github.repository + # + # The rest is here because older-tracks is skipped on most pull requests, and a skipped + # dependency skips this job as well unless the condition says what it accepts. !cancelled() is + # what lets the condition be evaluated at all. A failed version job fails this too: it skips + # build, and build has to have succeeded. + if: >- + !cancelled() + && needs.build.result == 'success' + && contains(fromJSON('["success", "skipped"]'), needs.older-tracks.result) + && github.event.pull_request.head.repo.full_name == github.repository # Must match the Environment on the nuget.org trusted publishing policy for this workflow. environment: nuget.org permissions: diff --git a/README.md b/README.md index 42b6a33..9dccb69 100644 --- a/README.md +++ b/README.md @@ -54,6 +54,8 @@ Both constraints are surfaced at build time in consuming apps by a small `.targe Tags name the FFmpeg version: **`v6.1.6.5` publishes `6.1.6.5`** and builds against the FFmpegKit release that packages FFmpeg 6.1.6, looked up in [`build/native-versions.tsv`](build/native-versions.tsv). The fourth component is the binding revision and any prerelease suffix is ignored when resolving (`v8.1.2.4-rc.1` → FFmpeg `8.1.2`). A tag naming a version that is not in the mapping fails the release with the list of known builds, so an old-style tag like `v8.1.7.4` cannot be published by mistake. No branch or `Directory.Build.props` edit is needed. +Release through a pull request that adds the note, as for any other version. That pull request packs the track and runs it on an emulator, which nothing else does: a pull request without one builds the default track only, and a release verifies nothing itself. A tag pushed by hand skips the check. + Locally, pass the native version as the second argument: ```sh @@ -312,7 +314,7 @@ It is deliberately **not** in `FFmpegKit.sln`, so that `dotnet build FFmpegKit.s | Workflow | Trigger | What it does | | --- | --- | --- | -| [`pr.yml`](.github/workflows/pr.yml) | pull request | Builds and packs all 8 variants as `-beta..`, runs package tests, the emulator smoke tests (all 8 variants on net10, plus Video and Audio on net8) and a run of the sample on an emulator, then publishes the betas to nuget.org. Forked PRs build and test but skip publishing, since they cannot read secrets. | +| [`pr.yml`](.github/workflows/pr.yml) | pull request | Builds and packs all 8 variants as `-beta..`, runs package tests, the emulator smoke tests (all 8 variants on net10, plus Video and Audio on net8) and a run of the sample on an emulator, then publishes the betas to nuget.org. A pull request that adds a release note for an older track or changes its row in the mapping also runs that track, as does one that changes these checks, which runs them all: all 8 variants packed and tested, Audio and Video on net8 on API 31 and 35, and the sample. Those packages are not published. Forked PRs build and test but skip publishing, since they cannot read secrets. | | [`release.yml`](.github/workflows/release.yml) | tag `v*` | Packs at the tag's version, the tests having already run on that commit's pull request, publishes to nuget.org, then creates a GitHub release with the changelog since the previous tag and links to every package. | Both call the reusable [`build.yml`](.github/workflows/build.yml).