Repository navigation
Conversation
Findings
|
There was a problem hiding this comment.
Actionable comments posted: 2

Review info
-
Run configuration-
- Source mode: Auto
- Repository: Unavailable
- Branch: Unavailable
- File: No configuration file from this repository applied
- Commit: Unavailable
- Layers: Repository UI
- Review profile: CHILL
- Plan: Advanced
- Run ID:
ae48c0cb-4fa0-49bb-9ab6-126a22ec25e6
Files ignored due to path filters: 1-
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
Files selected for processing: 45-
config/scripts/vitest-node-runtime-files.mjsdocs/optimize/simulator-avcc-autonomous-report.mddocs/optimize/simulator-stream-hillclimb-report.mddocs/optimize/simulator-video-quality-report.mddocs/plans/2026-11-01-0139-perf-simulator-avcc-stream-plan.mdpackage.jsonsrc/main/emulator/avcc-frame-parser.test.tssrc/main/emulator/avcc-frame-parser.tssrc/main/emulator/avcc-frame-stream.test.tssrc/main/emulator/avcc-frame-stream.tssrc/main/emulator/backends/ios-emulator-backend.test.tssrc/main/emulator/backends/ios-emulator-backend.tssrc/main/emulator/emulator-bridge-types.tssrc/main/emulator/emulator-bridge.test.tssrc/main/emulator/emulator-bridge.tssrc/main/emulator/emulator-session-registry.tssrc/main/emulator/emulator-types.tssrc/main/emulator/serve-sim-endpoint-readiness.tssrc/main/emulator/serve-sim-runtime-materializer.test.tssrc/main/emulator/serve-sim-runtime-materializer.tssrc/main/ipc/emulator-control-stream.test.tssrc/main/ipc/emulator-control-stream.tssrc/main/ipc/emulator-video-stream.test.tssrc/main/ipc/emulator-video-stream.tssrc/main/ipc/register-core-handlers/register-core-handlers.test.tssrc/main/ipc/register-core-handlers/register-core-handlers.tssrc/main/runtime/orca-runtime-emulator.tssrc/preload/api/emulator-api.tssrc/preload/api/emulator-bridge.tssrc/renderer/src/components/emulator-pane/EmulatorPane.tsxsrc/renderer/src/components/emulator-pane/emulator-device-frame-visibility.test.tsxsrc/renderer/src/components/emulator-pane/emulator-device-frame.input.test.tsxsrc/renderer/src/components/emulator-pane/emulator-device-frame.tsxsrc/renderer/src/components/emulator-pane/emulator-pane-session-view.tssrc/renderer/src/components/emulator-pane/emulator-pane-types.tssrc/renderer/src/components/emulator-pane/emulator-screen-stream-content.test.tsxsrc/renderer/src/components/emulator-pane/emulator-screen-stream-content.tsxsrc/renderer/src/components/emulator-pane/emulator-screen-surface.tsxsrc/renderer/src/components/emulator-pane/use-emulator-control-stream.tssrc/renderer/src/components/emulator-pane/use-emulator-pane-session.tssrc/renderer/src/components/emulator-pane/use-emulator-video-stream.tssrc/renderer/src/web/preload-api/web-browser-api.emulator.test.tssrc/renderer/src/web/preload-api/web-browser-api.tssrc/shared/emulator-control-stream.tssrc/shared/emulator-video-stream.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.
-
Declined the percentage-only expansion. The repository asks for concise comments only where code cannot express the constraint. The parser, ownership, fallback and cleanup boundaries retain their necessary documentation; adding docstrings that restate the code would add review and maintenance cost without improving the streaming behavior. |
There was a problem hiding this comment.
Actionable comments posted: 1

Review info
-
Run configuration-
- Source mode: Auto
- Repository: Unavailable
- Branch: Unavailable
- File: No configuration file from this repository applied
- Commit: Unavailable
- Layers: Repository UI
- Review profile: CHILL
- Plan: Advanced
- Run ID:
8de99e9b-ad01-405d-8f25-07c1f7c2862f
Files ignored due to path filters: 1-
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
Files selected for processing: 5-
docs/optimize/simulator-avcc-autonomous-report.mdsrc/main/emulator/emulator-bridge.test.tssrc/main/ipc/emulator-video-stream.test.tssrc/main/runtime/orca-runtime-emulator.tssrc/renderer/src/components/emulator-pane/use-emulator-pane-session.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.
-
|
Adversarial review: REWORK (risk: Medium). Not closing; the H.264 path is a real improvement in principle.
Reviewed at e04bd4d. |
|
Closing: too large to land as one change (about 4k lines, more test code than production, and a 240-line drag-input proxy that is a separate feature). It also has a security gap to fix before any split: main opens the stream URL the renderer sends and checks only scheme and path, so the URL should come from the session registry by device id. A split that lands the H.264 stream path alone, with the registry-sourced URL, decode-queue backpressure, and an idle-stall fallback to JPEG, would be welcome, with the drag-input proxy as its own PR. |
ELI5
The local iOS Simulator preview delivers smoother video and live drag input without native embedding. On the owned fixture, H.264 raised distinct decoded updates from about 25 to 60 per second with less measured CPU, traffic and summed RSS; held drags reached the source before pointer-up, and JPEG remains the fallback.
The pinned helper does not include the separate periodic-watchdog fix in serve-sim PR #187. That requires an upstream release and another dependency update.
What Changed
serve-sim@0.1.49-beta.139.1, advertise optionalavccStreamUrl, and parse its AVCC configuration and H.264 packets insrc/main/emulator/avcc-frame-{parser,stream}.ts. Reuse Android's existing main-process video IPC, preload bridge, renderer hook, decoder, and canvas.streamUrland compatibility metadata. Verify the live helper before enabling AVCC. Leave ready helpers reporting 404 or a different version running on JPEG. Cancel unread non-OK diagnostic bodies and replace stale materialized packages even under the same Orca app version.Android retains its Annex-B/scrcpy path. The current-main integration preserves orientation, stream-end handling, trusted renderer rebinding, and lifecycle behavior. No native embedding, custom encoder, adaptive bitrate controller, new tunnel, or browser/mobile video feature is added.
Detach metadata does not establish helper ownership. A version mismatch is therefore not permission to stop a working helper. Only an actual endpoint/helper-readiness failure retains the existing retry. An inconclusive diagnostic leaves a new session on JPEG; an already verified, reachable session remains reusable. Non-OK diagnostic responses use
cancelUnreadResponseBodybefore returning.Unsupported browser video start now fails explicitly instead of waiting for the new 10-second AVCC timeout. This does not add browser video or JPEG delivery. Browser control rejection already existed and is unchanged.
Changed tests cover packet boundaries, video/control ownership, fallback, lifecycle, helper identity/cache behavior, and unsupported browser capabilities. The measurement report preserves receipts, methods, negative results, and the separate watchdog findings.
Why
The JPEG frame gate discarded useful updates. Raising its limit required more independent image encoding, transport, and decoding. The newer helper supplies H.264 configuration, key frames, and delta frames, so Orca can reuse its video path rather than build another streaming system.
Measured codec/helper result
These historical measurements belong to the preserved AVCC integration, before the later compatibility corrections and current-main merge. They are not measurements of the final publication head. The comparison changes both the helper version and codec; it cannot assign a separate gain percentage to either.
The table reports medians from three held-out MJPEG runs and three integrated-production H.264 runs. Both use the same owned UIKit fixture at native 1206 × 2622, unchanged encoder defaults, and fixed exact-source-frame quality limits. Each run contributes its worse workload value for source age and resources; cadence is reported separately per workload.
Ratios use unrounded receipts. Lower-is-better reduction is
100 × (1 − new / baseline); cadence increase is100 × (new / baseline − 1). The measured CPU was about one quarter of baseline, encoded IPC was 5.21× lower, and distinct-frame cadence was about 2.4× higher. These results do not establish an overall 3× or 4× Simulator speedup.H.264 is lossy. Its worst measured native RGB PSNR was 38.7114 dB and displayed luma SSIM 0.994210, above the fixed gates of 35.9933 dB and 0.992423. Held-out JPEG minima were higher at 39.6509 dB / 0.997654. Passing the gates is not pixel identity or a quality guarantee for every app. The comparator uses a stable app-content region and fixed mathematical display-size reduction, not an ICC-managed perceptual comparison.
Measurement boundaries:
Separate input result
The newer helper rejected the renderer's cross-origin control WebSocket with HTTP 403. Video could still work, but a held drag was buffered and replayed at pointer-up. Moving control to Electron main fixes that integration error while preserving upstream security checks. Splitting codec adoption from this fix would leave input broken under the newer helper.
In the controlled 1.5-second held-drag reproduction, three broken trials had median times of 1526 ms to UIKit receipt and 1553 ms to decoded visible response. Ten warm fixed trials had medians of 10 ms and 36.5 ms. All ten delivered input while held without fallback replay. These are reproduction timings, not a universal input-latency ratio, cold-start result, or physical input-to-photon measurement.
Linked Issue
Fixes #37183
Visual Proof
All four clips show actual decoded pixels from the owned native UIKit motion fixture. Source streams remain 1206 × 2622. A recording-only mirror uses the actual 402 × 874 CSS viewport to separate recording overhead from production. These are illustrative browser captures, not recordings of full Orca windows or physical scanout, and they do not establish browser-client support.
Original presentation timestamps and holds are preserved, without interpolation or speed changes. Recording adds drawing/encoding overhead, so the clips are not the formal benchmark. Different warmups can select different initial counters; movie zero is not exact source-frame alignment. The distinct-source-frame benchmark is measured separately.
The codec clips are separate real-time 10-second recordings using Electron 43.8.4 / Node 24.22.1 and actual helper middleware without the CLI watchdog:
Codec before: MJPEG input / helper 0.1.47
orca-codec-before.mp4
Codec after: AVCC H.264 input / helper 0.1.49-beta.139.1
orca-codec-after.mp4
The separate real-CLI watchdog clips measure the initial upstream
d87475fchange. They do not show the pinned Orca helper or validate the later upstream cache/test-harness follow-up:Watchdog before: synchronous boot-device query
serve-sim-watchdog-before.mp4
Watchdog after: locally patched asynchronous query
serve-sim-watchdog-after.mp4
All four published MP4s were downloaded anonymously and matched their inspected hashes. Signed-out GitHub video players loaded and played them without errors. The codec-before MP4 records MJPEG input, not H.264 input. Public receipts and media provenance.
The earlier Native Simulator versus Orca recordings show only the pre-watchdog-fix state. They are diagnostic evidence, not after footage. Benchmark tables, screenshots, and timing overlays do not replace these demo videos.
Testing
Historical evidence in the preserved worktree:
Current publication-source checks: 192 tests in 18 focused files and all five typecheck projects passed. On committed head
e04bd4d23b53d61f000253438332a748989172c2,ORCA_BACKGROUND_LAUNCH=1 pnpm check:changed --base=1c40f5404db6eaa26a598c72aa57185b937cf965 --maxWorkers=2passed all 41 composite checks over the 46 changed files, including format, changed-code quality, typechecks, related unit tests and repository ratchets (451.7 s). The report separately records the original publication checkpoint's fullpnpm lintrun; that is not the same command ascheck:changed.A fresh native smoke on the conflict-repaired source used the actual
0.1.47and0.1.49-beta.139.1helpers, the dedicated UIKit fixture, and a freshly bundled production screen consumer. Both renderer windows stayed hidden, unfocused and non-focusable; H.264 decoded native 1206×2622 frames with advancing source counters, no JPEG fallback or page/decoder errors, and an empty final decoder queue. The recording was visually inspected and all owned producers stopped; the guest returned to Shutdown with its fixture/data preserved. This is a component/native stream smoke, not a packaged full-app, physical scanout, Windows/Linux desktop, SSH/mobile or native-parity test. Packaging/build verification remains for CI; no local full-app build pass is claimed.Reproduction requires the ownership receipts and a dedicated fixture device/private helper state. Do not aim the benchmark at an existing user Simulator or shared helper. Frozen instruments and raw-result paths are listed in the report.
Risk
Level: High. This adopts a prerelease helper and changes shared video/control ownership contracts. A regression could leave a local iOS preview blank, select JPEG unexpectedly, or leave input/key state stale; the shared decoder/IPC changes could also affect Android or inactive/replaced sessions. Packet bounds, owner/stream guards, deterministic JPEG fallback, key-release cleanup and lifecycle regressions limit these risks but do not establish every platform or app's behavior.
Optional metadata preserves older helpers/clients. Ready adopted helpers must not be stopped merely for a version mismatch. Local iOS hosting remains macOS-only; browser/mobile video is not added. The lossy-quality tradeoff and untested platform boundaries are stated above.
Rollback starts with reverting the code and dependency pin, but that alone does not stop running helpers or replace a helper already cached under the same Orca app version. Stop only Orca-owned session helpers and replace only the Orca-owned, version-scoped materialized helper cache before rematerializing the reverted pin. Preserve shared helpers, guests and workspace data. No saved-setting or database-schema migration is introduced.
AI Disclosure
AI assisted this work. The benchmark and publication text were prepared with
openai-codex/gpt-6.1-sol. The report identifies the separately verified review models and the revisions they reviewed.Review
The report documents three-model reviews of the original AVCC integration and the separate input fix, plus the original inline simplification pass. Those checks are historical, not a three-model review of the exact publication head. The later compatibility corrections preserve ready helpers, cancel unread diagnostic bodies, and reject unsupported browser video. The publication source passed the local checks, fresh native streaming smoke and exact committed-head changed-file gate listed above.
The separate upstream three-model review covers initial
d87475f, not the subsequent cache/test-harness changes. Reviewer agreement is not runtime proof.This is a broad end-to-end change across the helper dependency, main process, IPC/preload, renderer, and tests. The small/focused checklist item remains unchecked. The control and compatibility work is needed to adopt the newer helper without knowingly breaking input or disturbing working helpers; the breadth still increases review burden.
Agent skill upstream boundary
Notes
0.1.49-beta.140.1was rejected by the repository's minimum-release-age policy;0.1.49-beta.139.1was accepted without bypassing that policy. It remains a prerelease dependency.d87475f, not its later cache/test-harness follow-up. Source-build manifests report0.1.46; this is not a benchmark of a published watchdog-fixed beta.0.1.49-beta.139.1and does not contain the watchdog fix. Adoption requires an upstream release and dependency update. Zero gaps over 33 ms belongs only to the locally patched upstream A/B/A interval, not this unpatched Orca dependency. No generated/minified upstream bundle patch is added here.Checklist
N/Awith reasonpnpm lint,pnpm typecheck,pnpm test, andpnpm buildpass (or CI will cover; local preferred)