Revalidate the current ONNX model on Qualcomm QNN

.github/workflows/ci.yml

@@ -141,6 +141,12 @@ jobs:
         run: |
           python scripts/validate_tracked_qualcomm_int8.py
 
+      - name: Build cross-runtime portfolio acceptance matrix
+        run: |
+          python scripts/build_portfolio_acceptance.py \
+            --output-json /tmp/portfolio-acceptance.json \
+            --output-markdown /tmp/portfolio-acceptance.md
+
   release-acceptance:
     name: End-to-end release acceptance
     runs-on: ubuntu-latest

.gitignore

@@ -21,6 +21,8 @@ data/processed/*
 reports/*
 !reports/.gitkeep
 !reports/qualcomm_qnn_v0_1.json
+!reports/portfolio_acceptance.json
+!reports/portfolio_acceptance.md
 
 # Deployment artifacts
 *.onnx

README.md

@@ -165,6 +165,11 @@ Cross-provider timings should not be interpreted as a direct hardware ranking.
 
 ## Qualcomm QNN / Snapdragon deployment
 
+The consolidated [portfolio acceptance matrix](reports/portfolio_acceptance.md)
+separates validated CI/device lanes from evidence still pending. It also checks
+that the model hash in the Qualcomm report matches the current ONNX artifact.
+The current-model AI Hub rerun restores that provenance chain.
+
 EdgeGenBench now includes a validated Qualcomm QNN deployment path for the
 compact neural surrogate.
 
@@ -177,13 +182,13 @@ profiled layers were placed on the NPU for every validated graph.
 
 | Batch | AI Hub profile latency | Peak memory | Compute units |
 |---:|---:|---:|---|
-| 1 | 38 us | 122,937,344 B | NPU: 9 |
-| 32 | 34 us | 122,888,192 B | NPU: 9 |
-| 256 | 57 us | 123,211,776 B | NPU: 9 |
+| 1 | 38 us | 122,855,424 B | NPU: 9 |
+| 32 | 40 us | 122,880,000 B | NPU: 9 |
+| 256 | 47 us | 122,896,384 B | NPU: 9 |
 
 On all 900 held-out rows, the linked batch-1 QNN deployment retained mean R2
-of 0.996953 versus 0.996955 for the local FP32 ONNX reference. Maximum
-normalized deployment drift was 0.003636.
+of 0.996954 versus 0.996956 for the local FP32 ONNX reference. Maximum
+normalized deployment drift was 0.003234.
 
 These values are device-specific AI Hub model-profile measurements. They are
 not presented as end-to-end Android application latency or as a same-hardware
@@ -193,6 +198,13 @@ Complete provenance and linked-graph validation are stored in:
 
 `reports/qualcomm_qnn_v0_1.json`
 
+The authenticated current-model run is reproducible with:
+
+```bash
+PYTHONPATH=src python scripts/rerun_qualcomm_qnn_current_model.py
+python scripts/build_portfolio_acceptance.py
+```
+
 ### Qualcomm-native INT8 decision
 
 The follow-up INT8/QDQ study was executed on the same Snapdragon 8 Elite

docs/qualcomm_ai_hub_qnn.md

@@ -90,11 +90,11 @@ containing three statically specialized graph variants:
 
 Linked target model:
 
-`mnl7771jm`
+`mnjjex79n`
 
 Link job:
 
-`jp16x3285`
+`jgk4d8lvp`
 
 The linked artifact retains:
 
@@ -107,9 +107,9 @@ The linked artifact retains:
 
 | Graph | Batch | AI Hub profile latency | Derived model throughput | Peak memory | Compute units |
 |---|---:|---:|---:|---:|---|
-| `edgegenbench_batch1` | 1 | 38 us | 26,315.8 samples/s | 122,937,344 B | NPU: 9 |
-| `edgegenbench_batch32` | 32 | 34 us | 941,176.5 samples/s | 122,888,192 B | NPU: 9 |
-| `edgegenbench_batch256` | 256 | 57 us | 4,491,228.1 samples/s | 123,211,776 B | NPU: 9 |
+| `edgegenbench_batch1` | 1 | 38 us | 26,315.8 samples/s | 122,855,424 B | NPU: 9 |
+| `edgegenbench_batch32` | 32 | 40 us | 800,000.0 samples/s | 122,880,000 B | NPU: 9 |
+| `edgegenbench_batch256` | 256 | 47 us | 5,446,808.5 samples/s | 122,896,384 B | NPU: 9 |
 
 The throughput values are derived from the configured batch size divided by
 AI Hub's estimated model-inference time. They are not end-to-end Android
@@ -125,8 +125,8 @@ All 900 held-out test rows were evaluated through the linked batch-1 graph.
 
 | Metric | Local FP32 ONNX | Linked Snapdragon QNN |
 |---|---:|---:|
-| Mean R2 | 0.996955004 | 0.996953249 |
-| Mean NRMSE | 0.050432628 | 0.050444571 |
+| Mean R2 | 0.996955989 | 0.996954462 |
+| Mean NRMSE | 0.050425306 | 0.050449587 |
 
 Deployment drift:
 
@@ -193,16 +193,16 @@ Install Qualcomm-specific dependencies with:
 pip install -e '.[qualcomm]'
 ```
 
-Validate the linked multi-graph target:
+Compile, link, profile, and validate the current ONNX model:
 
 ```bash
-python scripts/validate_qualcomm_qnn_multigraph.py
+PYTHONPATH=src python scripts/rerun_qualcomm_qnn_current_model.py
 ```
 
 Build the canonical evidence report:
 
 ```bash
-python scripts/build_qualcomm_qnn_report.py
+python scripts/build_portfolio_acceptance.py
 ```
 
 The complete machine-readable result is stored in:

reports/portfolio_acceptance.json

@@ -0,0 +1,82 @@
+{
+  "schema_version": 1,
+  "project": "EdgeGenBench",
+  "lanes": {
+    "native_cpp": {
+      "status": "validated_in_ci",
+      "claim": "C++17 reference runtime, tests, CLI, and fused preprocessing acceptance."
+    },
+    "android_reference": {
+      "status": "validated_physical_device",
+      "device": "Samsung SM-A356E",
+      "report": "reports/android_sm_a356e_reference_10_run_v0_1_4.md",
+      "claim": "Reference JNI/application measurements; not QNN."
+    },
+    "qualcomm_ai_hub_qnn": {
+      "status": "validated_ai_hub_physical_qnn",
+      "device": "Snapdragon 8 Elite QRD",
+      "backend": "QNN HTP",
+      "qairt_version": "2.45.0.260326154327",
+      "context_model_id": "mnjjex79n",
+      "link_job_id": "jgk4d8lvp",
+      "graphs": [
+        {
+          "graph": "edgegenbench_batch1",
+          "batch_size": 1,
+          "latency_ms": 0.038,
+          "throughput_samples_per_second": 26315.78947368421,
+          "peak_memory_bytes": 122855424,
+          "compute_units": {
+            "NPU": 9
+          },
+          "max_normalized_drift": 0.003233889676953719,
+          "profile_job_id": "j567vw3np",
+          "inference_job_id": "jg9m8d3q5"
+        },
+        {
+          "graph": "edgegenbench_batch32",
+          "batch_size": 32,
+          "latency_ms": 0.04,
+          "throughput_samples_per_second": 800000.0,
+          "peak_memory_bytes": 122880000,
+          "compute_units": {
+            "NPU": 9
+          },
+          "max_normalized_drift": 0.0028659358491131483,
+          "profile_job_id": "jpyxnmvr5",
+          "inference_job_id": "jpez2yl7p"
+        },
+        {
+          "graph": "edgegenbench_batch256",
+          "batch_size": 256,
+          "latency_ms": 0.047,
+          "throughput_samples_per_second": 5446808.510638298,
+          "peak_memory_bytes": 122896384,
+          "compute_units": {
+            "NPU": 9
+          },
+          "max_normalized_drift": 0.0028659358491131483,
+          "profile_job_id": "jglx7lllg",
+          "inference_job_id": "jgk4d82op"
+        }
+      ],
+      "reported_source_model_sha256": "191927e05b5f82a539f0ad35c78dafe7f969cb4e57e9c76556cb9f35053e658e",
+      "current_source_model_sha256": "191927e05b5f82a539f0ad35c78dafe7f969cb4e57e9c76556cb9f35053e658e",
+      "source_model_hash_origin": "committed_git_blob",
+      "source_model_matches_repository": true,
+      "claim_boundary": "Physical AI Hub model profiling; not Android APK end-to-end latency."
+    },
+    "android_qnn_apk": {
+      "status": "implementation_complete_evidence_pending",
+      "claim": "Build/JNI/capture paths exist; requires a supported Snapdragon APK run."
+    },
+    "android_16kb_runtime": {
+      "status": "packaging_validated_runtime_pending",
+      "claim": "ELF/APK alignment passes; runtime PAGE_SIZE=16384 evidence is pending."
+    },
+    "power": {
+      "status": "not_measured",
+      "claim": "No power-savings claim is made without a named calibrated tool."
+    }
+  }
+}

reports/portfolio_acceptance.md

@@ -0,0 +1,24 @@
+# EdgeGenBench portfolio acceptance
+
+| Evidence lane | Status | Claim boundary |
+|---|---|---|
+| `native_cpp` | `validated_in_ci` | C++17 reference runtime, tests, CLI, and fused preprocessing acceptance. |
+| `android_reference` | `validated_physical_device` | Reference JNI/application measurements; not QNN. |
+| `qualcomm_ai_hub_qnn` | `validated_ai_hub_physical_qnn` | Physical AI Hub model profiling; not Android APK end-to-end latency. |
+| `android_qnn_apk` | `implementation_complete_evidence_pending` | Build/JNI/capture paths exist; requires a supported Snapdragon APK run. |
+| `android_16kb_runtime` | `packaging_validated_runtime_pending` | ELF/APK alignment passes; runtime PAGE_SIZE=16384 evidence is pending. |
+| `power` | `not_measured` | No power-savings claim is made without a named calibrated tool. |
+
+## Validated Qualcomm QNN results
+
+Device: **Snapdragon 8 Elite QRD**; backend: **QNN HTP**; QAIRT: `2.45.0.260326154327`.
+Source-model provenance match: **True**.
+
+| Batch | AI Hub latency (ms) | Throughput (samples/s) | Peak memory (bytes) | Placement | Max normalized drift |
+|---:|---:|---:|---:|---|---:|
+| 1 | 0.038000 | 26315.789 | 122855424 | NPU × 9 | 0.003233890 |
+| 32 | 0.040000 | 800000.000 | 122880000 | NPU × 9 | 0.002865936 |
+| 256 | 0.047000 | 5446808.511 | 122896384 | NPU × 9 | 0.002865936 |
+
+AI Hub measurements are physical-device model profiles, not Android application end-to-end timings. Current-model acceptance requires source-model provenance to match the repository, as reported above.
+Power remains unmeasured. The two pending proof items are a supported-device QNN APK run and a runtime page size of 16384 bytes.