Skip to content

evidence: add structured hardware qualification workflow - #90

Merged
Syax89 merged 11 commits into
Syax89:mainfrom
avgbob:upstream/hardware-testing-workflow
Oct 18, 2026
Merged

Syax89 merged 11 commits into
Syax89:mainfrom
avgbob:upstream/hardware-testing-workflow

Conversation

@avgbob

@avgbob avgbob commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add a focused, read-only hardware qualification workflow so independent Surface Laptop AMD results can be collected in a consistent form.

This PR does not change driver behavior.

What it adds

  • structured GitHub hardware-test issue form;
  • independent tester guide and hardware-testing hub;
  • bounded capture_beta_multitouch.sh helper;
  • beta-MT passthrough in the existing Linux trace bundle and blinded-session wrapper;
  • failure propagation so a requested evtest capture cannot silently fail while the parent session reports success;
  • CI contract test for the evidence helpers.

Beta multitouch identity

The new beta capture helper deliberately identifies the driver-owned input device by:

  • exact input name: MSHW0231 Touchscreen;
  • EV_ABS;
  • BTN_TOUCH;
  • ABS_MT_TRACKING_ID.

It does not assume an eventN number.

That distinction matters because the standard HID child and the heatmap-backed MT bridge can coexist; a qualification run must not accidentally record the wrong device.

The beta helper is intentionally scoped to the exact MSHW0231 identity currently used by the heatmap tracker. The general report form also records MSHW0230/MSHW0162 so unrelated standard-driver results are not conflated.

Failure semantics

If --capture-beta-multitouch is requested and that capture fails:

  • the trace bundle records the exit status;
  • the trace bundle exits non-zero;
  • the blinded-session wrapper records status=trace-failed.

This avoids a misleading "completed" evidence session with a missing input capture.

Safety

All added evidence helpers are read-only:

  • no sudo invocation;
  • no install/load/unload/bind/configure operations;
  • bounded capture duration;
  • refusal to overwrite an existing output path;
  • dynamic input discovery rather than fixed device nodes.

Tests

Adds hardware_evidence_contract_test.py to make -C tests test.

It pins:

  • the MSHW0231 beta input identity;
  • multitouch capability checks;
  • no hard-coded event node;
  • capture failure propagation;
  • blinded-session passthrough;
  • executable permissions on evidence helpers.

This is independent of driver PRs #87/#88 and installer PR #89.

Why this qualification workflow exists

The need for structured independent hardware evidence came directly out of the Windows reverse-engineering/qualification campaign:

This PR is the mechanism for testing whether conclusions derived from that work generalize beyond the original machine, rather than treating one-device success as broad compatibility.

Review follow-up

The README overview now explicitly says that direct touch, beta multitouch, and stylus evtest captures are opt-in. No runtime helper behavior was changed in response to review; the existing requested-capture failure propagation remains intact.

@Syax89

Syax89 commented Oct 16, 2026

Copy link
Copy Markdown
Owner

Thanks for adding the structured hardware evidence workflow. Two independent review passes plus controller checks found no verified introduced runtime regression at this revision. This is a draft-review comment, not hardware qualification or formal merge approval.

Reviewed head: 428781a9f2a2f53ff2004791905870ceadd78a87; base: 72b3492ba08fdee481b5dd1fa5781025264583f9.

Minor documentation note

tools/hardware_evidence/README.md:26–27 lists only direct touch/stylus as opt-in; please include the new beta multitouch capture there too. The exact flag is already documented in usage/examples at :44–52 and the new hub, so this is an overview omission rather than a functional defect. Index/link and abbreviated identity wording are editorial improvements only.

Error propagation is intentional and should stay

A requested capture failure now makes the bundle/session fail instead of reporting a completed capture. A base/head comparison reproduced the change from exit 0 to exit 1 for missing direct-touch input. This is the intended fail-closed correction, not a regression to revert.

The real scripts were also exercised in a redirected sysfs/dev sandbox, with explicitly stubbed evtest/journalctl:

Scenario Observed result
Beta identity correct, evtest succeeds exit 0
Beta identity correct, evtest fails exit 7
Bundle with requested beta capture succeeding exit 0
Bundle with requested beta capture failing exit 1; manifest records beta exit 7
Blinded session with requested beta capture failing exit 1; status trace-failed
Wrong standard-HID name despite MT capabilities exit 1; no output file

Failure artifacts/manifests remain available. README :73–75 documents nonzero/trace-failed; the not observed wording in docs/HARDWARE_VALIDATION.md:81–82 does not require a successful exit or a completed status for an incomplete requested capture.

Other reviewed concerns

  • Driver-owned beta identity matches the publisher rather than the standard HID child.
  • Exact-name/single-node rejection is intentional: it avoids recording the wrong device.
  • Capability arithmetic follows the existing x86-64 helper pattern; Bash invocation is harmless; status assignment executes in the same shell.
  • No driver/protocol bytes change in this PR.

Executed checks / limits

make -C tests test, tests/hardware_evidence_contract_test.py, Bash syntax checks for all three touched capture/session scripts, and diff whitespace checks passed. The six sandbox scenarios above passed their expected outcomes. Sandbox results are script behavior tests, not captured hardware evidence.

No real Surface/evtest gestures, device journal traffic, cold boot/resume, kernel build or hardware equivalence was tested. The issue-form YAML was text-reviewed; no YAML-validator result is claimed. Remote checks were absent at review time. No source changes were made.

avgbob commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — fixed the documentation mismatch.

Current head: 5bf984656937f72d7764039531b298596cbfcf23.

The overview now says that direct touch, beta multitouch, and stylus evtest captures are opt-in. I did not change the helper/error-propagation behavior you reviewed; requested capture failures still propagate to the trace/session result.

@Syax89
Syax89 marked this pull request as ready for review October 18, 2026 10:40
@Syax89
Syax89 merged commit be25620 into Syax89:main Oct 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants