Skip to content

Add Depth probing and live depth-map preview - #337

Merged
TTolsun merged 5 commits into
mainfrom
codex/depth-probe-live
Oct 19, 2026
Merged

TTolsun merged 5 commits into
mainfrom
codex/depth-probe-live

Conversation

@TTolsun

@TTolsun TTolsun commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

PROBE now reports whether each camera advertises DEPTH_OUTPUT, its DEPTH16 resolutions, and whether the HAL allows color and depth in the same request. Missing metadata stays unknown, and declared support is distinguished from a verified stream combination.

LIVE gains a Depth mode with a resolution picker. It closes the color engine before opening a depth-only Camera2 session, so the viewer can also target cameras that require exclusive depth requests. The map displays distance and confidence, center distance, and the valid sample percentage. Photo/Video restore color capture; media capture, PIP, zoom, manual controls and CLI tuning are disabled in depth view.

The change includes pure Kotlin depth decoding, bounded display work, buffer and camera cleanup, user documentation, and reviewed/generated documentation freshness records. Benchmark contracts and pixel-free incident exports are preserved.

Validation

  • App JVM tests: 907 passed, including depth confidence, stride/crop, buffer bounds, support/unknown states, exports and readout.
  • lintDebug: 0 errors, 0 warnings; debug and instrumentation APKs built.
  • Android 16 / API 36 emulator: unsupported-camera flow, PROBE search, and synthetic depth display instrumentation (DEPTH_MAP_CHECKS_OK).
  • Samsung SM-S936N, Android 16 / API 36: installed without deleting app data; PROBE succeeded for all 6 reported cameras, with JSON/TXT size and SHA-256 verified. Every camera reports no DEPTH_OUTPUT and no DEPTH16 sizes. All 4 public camera previews and unsupported Depth sheets passed; PROBE search and result navigation passed. Fixed a device-found Video → Depth → Off transition bug and verified Photo preview recovery over 3 cycles plus background/resume. Synthetic renderer instrumentation also passed on this device. See device validation.
  • Documentation tests: 37 passed; docflow check --build and docflow check --ci passed.
  • Code review covered camera close/cancel ordering, late frame filtering, UI mode/CLI transitions, buffer interpretation and pixel storage boundaries. No unresolved code findings.

Hardware limits

The connected SM-S936N exposes no Camera2 DEPTH_OUTPUT stream. Sustained sensor frames, resolution-specific session support, orientation and repeated color/depth transitions still need hardware validation. The viewer uses depth-only sessions; color/depth registration and bokeh composition are outside this change.

Documentation acceptance was explicitly authorized by the user and recorded as codex-user-authorized-depth-review; it does not claim real-depth measurements or independent human code review.

Screenshots

PROBE Depth search:

PROBE depth capability search

Unsupported camera in LIVE:

LIVE depth unavailable explanation

Synthetic DEPTH16 fixture (not real sensor output):

Synthetic depth map display

Summary by Sourcery

Add depth capability probing and a depth-only LIVE preview while keeping color capture workflows and existing benchmark artifacts isolated.

New Features:

  • Add PROBE reporting for depth output support, DEPTH16 resolutions, color/depth request declarations, and unknown metadata states.
  • Add a LIVE Depth mode that previews depth maps with distance, confidence, center distance, and valid-sample metrics.

Bug Fixes:

  • Ensure depth-to-color mode transitions restore the correct Photo or Video state and recover the color preview after repeated transitions or resume.

Enhancements:

  • Introduce a dedicated depth-only Camera2 engine with bounded frame processing, safe resource cleanup, and disabled capture, tuning, zoom, PIP, and benchmark pixel inputs during depth viewing.
  • Preserve depth metadata and display constraints across telemetry, architecture models, and UI state transitions without storing depth pixels in benchmark or incident artifacts.

Documentation:

  • Document depth capability discovery, LIVE depth viewing, engine behavior, limitations, and validation results.

Tests:

  • Add JVM coverage for DEPTH16 confidence decoding, crop and stride handling, buffer bounds, bounded sampling, support states, and stream readouts.
  • Add instrumentation coverage for synthetic depth rendering, center-distance accessibility output, and display validation.

Chores:

  • Refresh generated documentation evidence and freshness records for the depth-related architecture and user-facing guidance.

@sourcery-ai

sourcery-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown

Sorry @TTolsun, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 1 hour by commenting @sourcery-ai review. Upgrade to get a review now.

@sourcery-ai

sourcery-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown

Reviewer's Guide

This PR adds end-to-end DEPTH16 capability reporting and a depth-only LIVE preview: metadata is modeled conservatively, frames are decoded and bounded into a non-persisted map, lifecycle transitions isolate depth from color capture, and UI/CLI controls are restricted accordingly. It also adds JVM and synthetic instrumentation coverage plus synchronized documentation and validation records; real DEPTH_OUTPUT hardware behavior remains unverified.

Sequence diagram for switching from color to depth view

sequenceDiagram
    actor User
    participant MainActivity
    participant ColorEngine
    participant DepthCameraEngine
    participant ImageReader
    participant DepthMapView

    User->>MainActivity: chooseDepth()
    MainActivity->>ColorEngine: close(done)
    ColorEngine-->>MainActivity: close complete
    MainActivity->>DepthCameraEngine: start()
    DepthCameraEngine->>DepthCameraEngine: depthSupport()
    DepthCameraEngine->>DepthCameraEngine: open()
    DepthCameraEngine->>ImageReader: create DEPTH16 reader
    DepthCameraEngine->>ImageReader: setRepeatingRequest()
    ImageReader-->>DepthCameraEngine: image available
    DepthCameraEngine->>DepthFrame: render()
    DepthFrame-->>MainActivity: depth frame
    MainActivity->>DepthMapView: bind()
    DepthMapView-->>User: distance, confidence, valid percentage
Loading

Sequence diagram for returning from depth to color capture

sequenceDiagram
    actor User
    participant MainActivity
    participant DepthCameraEngine
    participant ColorEngine

    User->>MainActivity: selectMode(video)
    MainActivity->>DepthCameraEngine: close(done)
    DepthCameraEngine-->>MainActivity: close complete
    MainActivity->>ColorEngine: start()
    ColorEngine-->>MainActivity: color preview ready
    MainActivity-->>User: Photo or Video controls enabled
Loading

Flow diagram for conservative depth capability reporting

flowchart TD
    CameraCharacteristics --> DepthCapabilities
    DepthCapabilities --> DepthSupport
    DepthSupport --> ProbeRows
    DepthSupport --> LiveDepthChoice
    DepthSupport -->|missing or read failure| Unknown
    DepthSupport -->|advertised and DEPTH16 sizes available| Available
    Available --> DepthOnlySession
    DepthOnlySession -.->|combination still requires verification| SessionCheck
Loading

File-Level Changes

Change Details Files
Adds depth capability inspection to PROBE with explicit support, unknown, error, resolution, exclusivity, and color/depth declaration states.
  • Reads DEPTH_OUTPUT, DEPTH16 sizes, exclusivity, and color output metadata independently.
  • Exports the same depth rows through screen, text, and JSON representations.
  • Distinguishes HAL declarations from verified output-combination support and flags physical cameras that cannot be selected directly.
app/src/main/java/dev/halcamera/camera/CameraProbeReader.kt
app/src/main/java/dev/halcamera/camera/DepthCapabilities.kt
app/src/main/java/dev/halcamera/camera/DepthSupport.kt
app/src/test/java/dev/halcamera/camera/DepthSupportTest.kt
Introduces a depth-only Camera2 engine and bounded DEPTH16 decoding/display pipeline.
  • Opens an exclusive depth session after the previous camera engine closes, with availability waiting, first-frame timeout, late-frame filtering, and deterministic cleanup.
  • Decodes little-endian DEPTH16 distance/confidence values while honoring crop, row stride, pixel stride, buffer bounds, and buffer ownership.
  • Downsamples display work to a maximum edge of 640 pixels and throttles UI delivery to 10 frames per second.
  • Renders orientation/mirroring-aware maps with center distance, valid-sample percentage, confidence handling, accessibility text, and no persisted pixels.
app/src/main/java/dev/halcamera/camera/DepthCameraEngine.kt
app/src/main/java/dev/halcamera/camera/DepthFrame.kt
app/src/main/java/dev/halcamera/ui/DepthMapView.kt
app/src/test/java/dev/halcamera/camera/DepthFrameTest.kt
Adds LIVE Depth mode, resolution selection, and color/depth lifecycle transitions.
  • Adds a Depth picker and unsupported-camera messaging backed by PROBE capability data.
  • Switches from CameraX or color Camera2 to the depth-only engine and restores color capture through Photo, Video, or Off.
  • Disables media capture, PIP, zoom, manual controls, touch metering, metrics, and CLI tuning while depth is active.
  • Preserves depth selection across activity recreation and synchronizes Lab navigation and camera-close completion.
app/src/main/java/dev/halcamera/MainActivity.kt
app/src/main/java/dev/halcamera/MainDepthBridge.kt
app/src/main/java/dev/halcamera/MainCliBridge.kt
.omm/ui-tool-handoff/description.md
.omm/ui-zoom/description.md
Adds synthetic instrumentation and updates architecture, usage, validation, and generated documentation.
  • Adds a pixel-free synthetic DEPTH16 display check covering decoding, center readout, rendering, and accessibility output.
  • Documents depth data flow, engine contracts, UI restrictions, Probe semantics, hardware-validation limits, and preservation of benchmark/incident export boundaries.
  • Refreshes generated documentation freshness metadata and regression expectations.
app/src/androidTest/java/dev/halcamera/cli/CliStoreInstrumentation.kt
app/src/androidTest/java/dev/halcamera/ui/DepthMapChecks.kt
.omm/data-flow/description.md
.omm/overall-architecture/camera-engines/depth-engine/description.md
.omm/overall-architecture/camera-engines/description.md
.omm/overall-architecture/camera-engines/meta.yaml
.omm/overall-architecture/screens/expert-screen/description.md
.omm/overall-architecture/screens/probe-screen/probe-model/description.md
.omm/state-transitions/engine-lifecycle/description.md
.omm/ui-tool-handoff/description.md
.omm/ui-zoom/description.md
docs/architecture.html
docs/design/APP-UI.md
docs/live.html
docs/probe.html
docs/validation/depth-20261020/README.md
guide/_bindings.yaml
guide/_content/_reference/engine/camera2.md
guide/_content/_reference/engine/camerax.md
guide/_content/_reference/engine/comparison.md
guide/_content/_reference/engine/contract.md
guide/_content/architecture/constraints.md
guide/_content/architecture/module-roles.md
guide/_content/architecture/overview.md
guide/_content/architecture/runtime-flow.md
guide/_reference/engine.md
guide/_reference/evidence.md
guide/_reference/troubleshooting.md
guide/architecture.md
guide/live.md
guide/probe.md
tools/docgen/regression.test.mjs
tools/docgen/state/evidence.json

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@TTolsun
TTolsun merged commit 446910e into main Oct 19, 2026
7 checks passed
@TTolsun
TTolsun deleted the codex/depth-probe-live branch October 22, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant