Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@ Repeated physical-device reference runs can be captured with
post-run memory and thermal snapshots from sampled peaks and power measurements.
The first retained 10-run capture is summarized in the
[Samsung SM-A356E repeated-device report](reports/android_sm_a356e_reference_10_run_v0_1_4.md).
The APK/JNI reference path was also executed for 10 cold launches on an Android
15 ARM64 emulator whose runtime reported `PAGE_SIZE=16384`; the raw evidence and
claim boundaries are summarized in the
[Android 16 KB runtime report](reports/android_16kb_emulator_reference_v0_1_7.md).
Every successful CI run also assembles a versioned end-to-end release-evidence
bundle containing native baseline/fused benchmark JSON, the verified APK, the
16 KiB alignment report, a Git revision, and SHA-256 checksums. The acceptance
Expand Down
5 changes: 5 additions & 0 deletions docs/end_to_end_acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,11 @@ for reviewers.
checks;
- all retained files are bound to a Git revision with SHA-256 digests.

The tracked Android 16 KB evidence separately demonstrates that the v0.1.7
reference APK/JNI path launches and completes repeated benchmarks on an API 35
ARM64 emulator reporting `PAGE_SIZE=16384`. It is runtime-compatibility proof,
not physical-device performance, QNN placement, thermal, or power evidence.

## What requires a physical device

CI does not claim NPU placement, thermals, or power. Attach physical-device
Expand Down
67 changes: 67 additions & 0 deletions reports/android_16kb_emulator_reference_v0_1_7.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Android 16 KB runtime evidence: API 35 ARM64 emulator

Captured on 2026-08-27 from EdgeGenBench Android v0.1.7 (version code 8)
using `scripts/profile_android_reference.sh` for 10 automated cold launches.

## Provenance and scope

| Field | Observed value |
|---|---|
| Environment | Google Android Emulator `sdk_gphone16k_arm64` |
| Android / SDK | Android 15 / SDK 35 |
| ABI | `arm64-v8a` |
| Runtime page size | **16,384 bytes** |
| App version | 0.1.7 (code 8) |
| Backend | Deterministic reference backend, **not QNN** |
| QNN/NPU placement | Not tested |
| Power | Not measured |

The runtime page size was read from the booted environment with
`adb shell getconf PAGE_SIZE`. Every retained machine-readable benchmark result
also records `runtime_page_size_bytes=16384`. Before installation, Android SDK
`zipalign -c -P 16 -v 4` passed and all `PT_LOAD` segments in both packaged
native libraries reported `0x4000` alignment.

This closes the Android 16 KB **runtime compatibility** gap for the reference
APK/JNI path. An emulator is not physical-device performance evidence, and
these timings must not be compared with the Samsung or Qualcomm measurements.

## Ten-run results

Each outer run force-stopped and cold-launched the activity. The native
benchmark then performed 100 measured iterations.

| Metric | Mean | Median | Minimum | Maximum |
|---|---:|---:|---:|---:|
| Native cold latency (ms) | 0.022621 | 0.014333 | 0.011500 | 0.060667 |
| Native warm mean (ms) | 0.002608 | 0.002440 | 0.002259 | 0.003924 |
| Native warm p95 (ms) | 0.003333 | 0.002667 | 0.002458 | 0.006250 |
| Baseline preprocessing (ms) | 26.361593 | 23.912975 | 21.260857 | 38.929105 |
| Fused preprocessing (ms) | 12.570465 | 12.233062 | 11.625028 | 14.820786 |
| Preprocessing speedup | 2.092613x | 1.951007x | 1.772588x | 2.852528x |
| Android launch total (ms) | 4032.9 | 3948.0 | 3190 | 5258 |
| Post-run PSS (KiB) | 60,623 | 60,698 | 60,233 | 60,853 |
| Post-run RSS (KiB) | 157,038 | 157,096 | 156,608 | 157,232 |

All ten runs reported zero preprocessing drift and zero downstream output
drift. PSS and RSS are post-run process snapshots, not sampled peaks.

## Thermal and power boundaries

The emulator reported thermal status 0 and a constant synthetic test sensor.
Those values are emulator diagnostics, not physical thermal measurements.
Power remains `not measured`; no energy or power-savings claim is made.

## Retained evidence

The complete capture is stored under
`reports/device/android-16kb-api35-reference-10-runs/` and includes:

- device/app identity and the observed runtime page size;
- ten activity-start, logcat, memory, and thermal records;
- the aggregate latency/memory CSV;
- pre/post thermal snapshots and the final screenshot;
- explicit reference-backend, QNN, memory, thermal, and power claim boundaries.

The remaining hardware-only gaps are an end-to-end QNN APK run on a supported
Snapdragon device and a calibrated physical power measurement.
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
captured_at=2026-08-27T20:01:33Z
manufacturer=Google
model=sdk_gphone16k_arm64
android=15
sdk=35
abi=arm64-v8a
page_size=16384
versionCode=8 minSdk=28 targetSdk=35
versionName=0.1.7
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
run,cold_ms,warm_mean_ms,warm_p95_ms,baseline_preprocess_mean_ms,fused_preprocess_mean_ms,preprocess_speedup_x,preprocess_max_abs_drift,output_max_abs_drift,launch_total_ms,total_pss_kib,total_rss_kib
1,0.033250,0.002658,0.002667,38.929105,13.647231,2.852528,0.000000,0.000000,4575,60548,156992
2,0.013958,0.002545,0.002750,21.260857,11.686140,1.819322,0.000000,0.000000,4036,60695,157104
3,0.011500,0.002315,0.002584,23.230263,12.460185,1.864359,0.000000,0.000000,3190,60247,156688
4,0.020083,0.002310,0.002667,25.604844,12.850200,1.992564,0.000000,0.000000,5258,60701,157088
5,0.012625,0.002312,0.002583,23.244588,12.173451,1.909449,0.000000,0.000000,3484,60717,157168
6,0.060667,0.002259,0.002458,30.244593,14.820786,2.040688,0.000000,0.000000,4001,60788,157232
7,0.035708,0.003924,0.006250,21.789849,12.292673,1.772588,0.000000,0.000000,4840,60233,156608
8,0.014709,0.002855,0.006041,21.970452,12.135276,1.810462,0.000000,0.000000,3804,60853,157232
9,0.011625,0.002335,0.002708,32.760015,11.625028,2.818059,0.000000,0.000000,3895,60642,157056
10,0.012084,0.002570,0.002625,24.581362,12.013676,2.046115,0.000000,0.000000,3246,60807,157216
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
backend=reference
qnn-npu-placement=not-tested
power=not-measured
memory=post-run-snapshots-not-sampled-peaks
thermal=platform-sensor-snapshots-not-calibrated-power
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Starting: Intent { cmp=dev.edgegenbench/.MainActivity (has extras) }
Status: ok
LaunchState: COLD
Activity: dev.edgegenbench/.MainActivity
TotalTime: 4575
WaitTime: 4837
Complete
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
--------- beginning of main
08-27 16:01:35.017 7683 7683 I EdgeGenBench: MainActivity.onCreate: rendering high-contrast benchmark UI
08-27 16:01:36.400 7683 7683 I EdgeGenBench: MainActivity.onCreate: content view attached
08-27 16:01:36.401 7683 7683 I EdgeGenBench: MainActivity.onCreate: auto_run requested
08-27 16:01:44.154 7683 7705 I EdgeGenBench: backend=reference
08-27 16:01:44.154 7683 7705 I EdgeGenBench: CPU fallback=false
08-27 16:01:44.154 7683 7705 I EdgeGenBench: cold_ms=0.033250
08-27 16:01:44.154 7683 7705 I EdgeGenBench: warm_mean_ms=0.002658
08-27 16:01:44.154 7683 7705 I EdgeGenBench: warm_p95_ms=0.002667
08-27 16:01:44.154 7683 7705 I EdgeGenBench: runs=100
08-27 16:01:44.154 7683 7705 I EdgeGenBench: baseline_preprocess_mean_ms=38.929105
08-27 16:01:44.154 7683 7705 I EdgeGenBench: fused_preprocess_mean_ms=13.647231
08-27 16:01:44.154 7683 7705 I EdgeGenBench: preprocess_elements=262144
08-27 16:01:44.154 7683 7705 I EdgeGenBench: preprocess_speedup_x=2.852528
08-27 16:01:44.154 7683 7705 I EdgeGenBench: preprocess_max_abs_drift=0.000000
08-27 16:01:44.154 7683 7705 I EdgeGenBench: output_max_abs_drift=0.000000
08-27 16:01:44.154 7683 7705 I EdgeGenBench: output=0.156160
08-27 16:01:44.154 7683 7705 I EdgeGenBench: power=not measured
08-27 16:01:44.154 7683 7705 I EdgeGenBench: placement={"backend":"reference","operators":{"ReferenceLinear":1},"cpu_fallback":false,"hardware_measurement":false}
08-27 16:01:44.221 7683 7683 I EdgeGenBench: benchmark_json={"schema_version":1,"backend":"reference","cpu_fallback":false,"cold_ms":0.03325,"warm_mean_ms":0.00265791,"warm_p95_ms":0.002667,"runs":100,"baseline_preprocess_mean_ms":38.9291046,"fused_preprocess_mean_ms":13.6472313,"preprocess_elements":262144,"preprocess_speedup_x":2.852527648,"preprocess_max_abs_drift":0,"output_max_abs_drift":0,"output":0.156159997,"runtime_page_size_bytes":16384,"power":"not measured","captured_at_epoch_ms":1787860904166}
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
Applications Memory Usage (in Kilobytes):
Uptime: 16882397 Realtime: 16882397

** MEMINFO in pid 7683 [dev.edgegenbench] **
Pss Private Private SwapPss Rss Heap Heap Heap
Total Dirty Clean Dirty Total Size Alloc Free
------ ------ ------ ------ ------ ------ ------ ------
Native Heap 17258 17216 0 77 18080 24384 17314 1938
Dalvik Heap 4283 4160 32 333 5168 6687 3344 3343
Dalvik Other 6454 2464 0 169 10688
Stack 928 928 0 0 928
Ashmem 65 0 0 0 608
Other dev 80 0 32 0 688
.so mmap 4299 528 384 21 34032
.jar mmap 3489 0 400 0 40352
.apk mmap 8869 64 8560 0 10224
.ttf mmap 266 0 0 0 1024
.dex mmap 22 0 0 0 912
.oat mmap 69 0 0 0 1664
.art mmap 10466 9312 512 113 25808
Other mmap 2143 16 0 0 5152
Unknown 1130 1024 80 14 1664
TOTAL 60548 35712 10000 727 156992 31071 20658 5281

App Summary
Pss(KB) Rss(KB)
------ ------
Java Heap: 13984 30976
Native Heap: 17216 18080
Code: 9968 96208
Stack: 928 928
Graphics: 0 0
Private Other: 3616
System: 14836
Unknown: 10800

TOTAL PSS: 60548 TOTAL RSS: 156992 TOTAL SWAP PSS: 727

Objects
Views: 13 ViewRootImpl: 1
AppContexts: 7 Activities: 1
Assets: 20 AssetManagers: 0
Local Binders: 14 Proxy Binders: 55
Parcel memory: 5 Parcel count: 20
Death Recipients: 0 WebViews: 0

SQL
MEMORY_USED: 0
PAGECACHE_OVERFLOW: 0 MALLOC_SIZE: 0

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
IsStatusOverride: false
ThermalEventListeners:
callbacks: 5
killed: false
broadcasts count: -1
ThermalStatusListeners:
callbacks: 3
killed: false
broadcasts count: -1
Thermal Status: 0
Cached temperatures:
Temperature{mValue=30.8, mType=3, mName=test temperature sensor, mStatus=0}
HAL Ready: true
HAL connection:
ThermalHAL 2.0 connected: yes
Current temperatures from HAL:
Temperature{mValue=30.8, mType=3, mName=test temperature sensor, mStatus=0}
Current cooling devices from HAL:
CoolingDevice{mValue=100, mType=0, mName=test cooling device}
Temperature static thresholds from HAL:
TemperatureThreshold{mType=3, mName=test temperature sensor, mHotThrottlingThresholds=[NaN, NaN, NaN, 48.0, NaN, NaN, 60.0], mColdThrottlingThresholds=[NaN, NaN, NaN, NaN, NaN, NaN, NaN]}
Temperature headroom thresholds:
[NaN, NaN, NaN, NaN, NaN, NaN, 1.4]
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Starting: Intent { cmp=dev.edgegenbench/.MainActivity (has extras) }
Status: ok
LaunchState: COLD
Activity: dev.edgegenbench/.MainActivity
TotalTime: 4036
WaitTime: 4042
Complete
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
--------- beginning of main
08-27 16:01:48.212 7797 7797 I EdgeGenBench: MainActivity.onCreate: rendering high-contrast benchmark UI
08-27 16:01:49.011 7797 7797 I EdgeGenBench: MainActivity.onCreate: content view attached
08-27 16:01:49.012 7797 7797 I EdgeGenBench: MainActivity.onCreate: auto_run requested
08-27 16:01:54.321 7797 7814 I EdgeGenBench: backend=reference
08-27 16:01:54.321 7797 7814 I EdgeGenBench: CPU fallback=false
08-27 16:01:54.321 7797 7814 I EdgeGenBench: cold_ms=0.013958
08-27 16:01:54.321 7797 7814 I EdgeGenBench: warm_mean_ms=0.002545
08-27 16:01:54.321 7797 7814 I EdgeGenBench: warm_p95_ms=0.002750
08-27 16:01:54.321 7797 7814 I EdgeGenBench: runs=100
08-27 16:01:54.321 7797 7814 I EdgeGenBench: baseline_preprocess_mean_ms=21.260857
08-27 16:01:54.321 7797 7814 I EdgeGenBench: fused_preprocess_mean_ms=11.686140
08-27 16:01:54.321 7797 7814 I EdgeGenBench: preprocess_elements=262144
08-27 16:01:54.321 7797 7814 I EdgeGenBench: preprocess_speedup_x=1.819322
08-27 16:01:54.321 7797 7814 I EdgeGenBench: preprocess_max_abs_drift=0.000000
08-27 16:01:54.321 7797 7814 I EdgeGenBench: output_max_abs_drift=0.000000
08-27 16:01:54.321 7797 7814 I EdgeGenBench: output=0.156160
08-27 16:01:54.321 7797 7814 I EdgeGenBench: power=not measured
08-27 16:01:54.321 7797 7814 I EdgeGenBench: placement={"backend":"reference","operators":{"ReferenceLinear":1},"cpu_fallback":false,"hardware_measurement":false}
08-27 16:01:54.375 7797 7797 I EdgeGenBench: benchmark_json={"schema_version":1,"backend":"reference","cpu_fallback":false,"cold_ms":0.013958,"warm_mean_ms":0.00254542,"warm_p95_ms":0.00275,"runs":100,"baseline_preprocess_mean_ms":21.26085666,"fused_preprocess_mean_ms":11.68614039,"preprocess_elements":262144,"preprocess_speedup_x":1.819322373,"preprocess_max_abs_drift":0,"output_max_abs_drift":0,"output":0.156159997,"runtime_page_size_bytes":16384,"power":"not measured","captured_at_epoch_ms":1787860914336}
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
Applications Memory Usage (in Kilobytes):
Uptime: 16892536 Realtime: 16892536

** MEMINFO in pid 7797 [dev.edgegenbench] **
Pss Private Private SwapPss Rss Heap Heap Heap
Total Dirty Clean Dirty Total Size Alloc Free
------ ------ ------ ------ ------ ------ ------ ------
Native Heap 17258 17216 0 78 18080 24384 17303 2055
Dalvik Heap 4267 4144 32 333 5152 6687 3344 3343
Dalvik Other 6726 2752 0 169 10944
Stack 880 880 0 0 880
Ashmem 65 0 0 0 608
Other dev 80 0 32 0 688
.so mmap 4291 528 384 21 34016
.jar mmap 3488 0 400 0 40352
.apk mmap 8864 64 8560 0 10208
.ttf mmap 266 0 0 0 1024
.dex mmap 22 0 0 0 912
.oat mmap 69 0 0 0 1664
.art mmap 10466 9312 512 113 25808
Other mmap 2143 16 0 0 5152
Unknown 1082 1024 32 14 1616
TOTAL 60695 35936 9952 728 157104 31071 20647 5398

App Summary
Pss(KB) Rss(KB)
------ ------
Java Heap: 13968 30960
Native Heap: 17216 18080
Code: 9984 96160
Stack: 880 880
Graphics: 0 0
Private Other: 3840
System: 14807
Unknown: 11024

TOTAL PSS: 60695 TOTAL RSS: 157104 TOTAL SWAP PSS: 728

Objects
Views: 13 ViewRootImpl: 1
AppContexts: 7 Activities: 1
Assets: 20 AssetManagers: 0
Local Binders: 14 Proxy Binders: 55
Parcel memory: 4 Parcel count: 16
Death Recipients: 0 WebViews: 0

SQL
MEMORY_USED: 0
PAGECACHE_OVERFLOW: 0 MALLOC_SIZE: 0

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
IsStatusOverride: false
ThermalEventListeners:
callbacks: 5
killed: false
broadcasts count: -1
ThermalStatusListeners:
callbacks: 3
killed: false
broadcasts count: -1
Thermal Status: 0
Cached temperatures:
Temperature{mValue=30.8, mType=3, mName=test temperature sensor, mStatus=0}
HAL Ready: true
HAL connection:
ThermalHAL 2.0 connected: yes
Current temperatures from HAL:
Temperature{mValue=30.8, mType=3, mName=test temperature sensor, mStatus=0}
Current cooling devices from HAL:
CoolingDevice{mValue=100, mType=0, mName=test cooling device}
Temperature static thresholds from HAL:
TemperatureThreshold{mType=3, mName=test temperature sensor, mHotThrottlingThresholds=[NaN, NaN, NaN, 48.0, NaN, NaN, 60.0], mColdThrottlingThresholds=[NaN, NaN, NaN, NaN, NaN, NaN, NaN]}
Temperature headroom thresholds:
[NaN, NaN, NaN, NaN, NaN, NaN, 1.4]
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Starting: Intent { cmp=dev.edgegenbench/.MainActivity (has extras) }
Status: ok
LaunchState: COLD
Activity: dev.edgegenbench/.MainActivity
TotalTime: 3190
WaitTime: 3454
Complete
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
--------- beginning of main
08-27 16:01:57.633 7896 7896 I EdgeGenBench: MainActivity.onCreate: rendering high-contrast benchmark UI
08-27 16:01:58.442 7896 7896 I EdgeGenBench: MainActivity.onCreate: content view attached
08-27 16:01:58.442 7896 7896 I EdgeGenBench: MainActivity.onCreate: auto_run requested
08-27 16:02:03.924 7896 7914 I EdgeGenBench: backend=reference
08-27 16:02:03.924 7896 7914 I EdgeGenBench: CPU fallback=false
08-27 16:02:03.924 7896 7914 I EdgeGenBench: cold_ms=0.011500
08-27 16:02:03.924 7896 7914 I EdgeGenBench: warm_mean_ms=0.002315
08-27 16:02:03.924 7896 7914 I EdgeGenBench: warm_p95_ms=0.002584
08-27 16:02:03.924 7896 7914 I EdgeGenBench: runs=100
08-27 16:02:03.924 7896 7914 I EdgeGenBench: baseline_preprocess_mean_ms=23.230263
08-27 16:02:03.924 7896 7914 I EdgeGenBench: fused_preprocess_mean_ms=12.460185
08-27 16:02:03.924 7896 7914 I EdgeGenBench: preprocess_elements=262144
08-27 16:02:03.924 7896 7914 I EdgeGenBench: preprocess_speedup_x=1.864359
08-27 16:02:03.924 7896 7914 I EdgeGenBench: preprocess_max_abs_drift=0.000000
08-27 16:02:03.924 7896 7914 I EdgeGenBench: output_max_abs_drift=0.000000
08-27 16:02:03.924 7896 7914 I EdgeGenBench: output=0.156160
08-27 16:02:03.924 7896 7914 I EdgeGenBench: power=not measured
08-27 16:02:03.924 7896 7914 I EdgeGenBench: placement={"backend":"reference","operators":{"ReferenceLinear":1},"cpu_fallback":false,"hardware_measurement":false}
08-27 16:02:03.982 7896 7896 I EdgeGenBench: benchmark_json={"schema_version":1,"backend":"reference","cpu_fallback":false,"cold_ms":0.0115,"warm_mean_ms":0.00231455,"warm_p95_ms":0.002584,"runs":100,"baseline_preprocess_mean_ms":23.23026294,"fused_preprocess_mean_ms":12.46018544,"preprocess_elements":262144,"preprocess_speedup_x":1.864359327,"preprocess_max_abs_drift":0,"output_max_abs_drift":0,"output":0.156159997,"runtime_page_size_bytes":16384,"power":"not measured","captured_at_epoch_ms":1787860923937}
Loading
Loading