Skip to content

Add per-player controller type configuration on macOS and iOS - #1348

Draft
dandelion711 wants to merge 1 commit into
seedds:mainfrom
dandelion711:feat/apple-controller-configuration
Draft

dandelion711 wants to merge 1 commit into
seedds:mainfrom
dandelion711:feat/apple-controller-configuration

Conversation

@dandelion711

@dandelion711 dandelion711 commented Nov 5, 2026 •

Copy link
Copy Markdown

What and why

Previously motion games that required Joy-Cons were unaccessible to players with regular gamepads. When a game requested a controller change (Joy-con required, but a Pro Controller is detected), nothing would happen. This PR aims to close these gaps.

Add session-only controller configuration to native macOS and iOS game apps. Games such as Super Mario Party Jamboree can require individual Joy-Cons or a Joy-Con pair, while a connected DualSense, DualShock or other gamepad is normally exposed as Pro Controller or Handheld. Players can choose the player and game-facing type, form a two-controller pair, and apply changes while playing. The engine continues to own physical input, motion and rumble.

This builds on upstream’s existing controller allocation, joining popup and live input handling. Player and controller type default to Automatic: leaving these choices alone preserves the existing automatic routing policy. Configuration edits remain drafts until confirmed; runtime changes take effect only after Apply. A game-requested menu may still appear to resolve an incompatible setup, which is the intentional new workflow described below.

The PR contains nxnative source, tests and documentation only. Controller choices are per connection/session. It adds no per-game assignment store, behavior-mode switch or button-mapping editor.

Features and changes

1. Controller configuration on the game launch screen

  1. Problem and previous behavior: The generated game's launcher offered general settings but no way to choose how each connected controller should appear to the game.
  2. New behavior: Configure controllers appears under Controls on Mac and iOS, including launchers without an existing Controls section. Device details stay inside the configuration sheet. For example, a DualSense can be assigned to Player 1 as Joy-Con (R) before Play.
  3. Implementation rationale: Reuse the game’s existing launch settings UI and allocator. Automatic keeps existing player allocation, one-player merging, keyboard/touch fallback, hotplug and optional press-to-join behavior in control. Opening the sheet or applying unchanged defaults does not create manual assignments. Returning all player/type choices to Automatic restores the inherited routing policy after manual configuration.

2. Explicit game-facing controller styles

  1. Problem and previous behavior: Display mode normally determined Pro Controller or Handheld, and the guest-facing single-Joy-Con paths could not supply the required state.
  2. New behavior: Each device offers Automatic, Pro Controller, Joy-Con pair, Joy-Con (L) and Joy-Con (R). HID state, attributes, assignment metadata, style events, six-axis handles and vibration handles represent the selected supported style. Single-Joy-Con button/stick conversion supplies the corresponding controls, including SL/SR.
  3. Implementation rationale: Implement the distinction in the engine's HID and source-routing layers. Changing a label alone would not satisfy a game's controller-style or sensor queries.

3. Upright full gamepads when a virtual single Joy-Con rotates

  1. Problem and previous behavior: A game's Joy-Con hold-orientation change could force a person holding a DualSense to rotate the whole gamepad to keep directions/buttons usable.
  2. New behavior: A full-size controller assigned as Joy-Con (L) or (R) compensates its buttons and stick for the game's hold orientation, with opposite quarter-turns for the two sides. Physical Joy-Cons retain their previous handling.
  3. Implementation rationale: Adapt the virtual control frame at the engine boundary. This does not patch the game, remove a minigame's motion requirements or change the physical gyro source.

4. Runtime opening and game-requested controller changes

  1. Problem and previous behavior: With default settings, a full-size gamepad such as a DualSense is normally exposed as Pro Controller or Handheld. In Super Mario Party Jamboree, entering a motion minigame or mode can require Joy-Cons instead. The contributor reports that selecting the game’s controller-change action previously did nothing useful: no configuration editor appeared, and the controller remained incompatible.
  2. New behavior: The same game action now reaches the native controller configuration workflow. When the current setup cannot satisfy the reported requirements and upstream would skip its joining popup, our controller menu opens directly. The player can choose an accepted Joy-Con style and Apply, then return to the game. If upstream would show its joining popup because more players are needed, that popup stays first and its Configure controllers action opens the editor as a child. The card is hidden while editing and resumes afterward. Already compatible requests continue without an added screen. Manual access remains Controllers → Configure Controllers… on Mac and Configure controllers in the existing iOS gear menu.
  3. Implementation rationale: Extend the existing live workflow rather than replace it. Both ShowControllerSupport overloads use the same compatibility evaluation. Each game request owns one presentation route and one result, so the joining popup and editor do not open independently for the same request. The game thread waits outside the HID lock, and Apply checks the current requirements again. The trigger is the game’s reported player/type/pair requirements, not a Jamboree-specific rule or an assumption that a physical gamepad has motion hardware. Compatible initial draft suggestions remain uncommitted until Apply.

5. Stable device identity and explicit player assignments

  1. Problem and previous behavior: Connection order, identical device names and hotplug made a name- or row-based assignment unreliable.
  2. New behavior: The registry identifies each live connection with a session UUID. The player selector supports Automatic, Unassigned and numbered players. Reordering two identically named DualSense controllers does not exchange their draft choices; reconnecting is a new connection rather than a saved identity.
  3. Implementation rationale: Keep physical device ownership in the host and bind overrides to connections. Names and list positions are presentation data, not identity.

6. Requirements, readiness and actionable validation

  1. Problem and previous behavior: Users could see a disabled Apply action without knowing which player count, slot, type or incomplete pair prevented it.
  2. New behavior: Runtime configuration shows the requested player range, accepted styles, relevant player slots and ready count. Incompatible devices show • Not ready after their name on iOS and a ✕ Not ready badge on Mac, with corrective guidance. An unrestricted selectable style set is summarized as Any. Apply checks ready logical players and complete pairs against current game requirements. Recommended labels identify a sole accepted style. When the accepted styles are exactly Joy-Con (L) and Joy-Con (R), both receive the label.
  3. Implementation rationale: Use the same acceptance rules for preview and final validation. A two-source pair is one player. The editor cannot reduce a genuine two-player requirement, change a game's roster or invent a missing controller; it has no player-count override.

7. Native controller-support argument flags

  1. Problem and previous behavior: Reading only raw minimum/maximum counts ignored the native single-player and JoyDual-permission flags.
  2. New behavior: A request with single mode enabled has effective limits of one player even if its raw range says 2–4. A request that forbids JoyDual does not accept a pair. These temporary constraints apply to the pending setup request and are released when it finishes.
  3. Implementation rationale: Expose scoped requirements through dm_controller_setup_requirements without changing the game's ordinary supported-style/ID queries. Both UI validation and the HID result enforce the effective limits.

8. Identify feedback without a new settings surface

  1. Problem and previous behavior: Identical names made it difficult to tell which physical device a row represented.
  2. New behavior: Each row has Identify, targeting that connection's supported output. Successful requests show Identify • signal sent. Unsupported or failed signaling reports an error rather than claiming physical feedback occurred.
  3. Implementation rationale: Reuse existing output capabilities. There is no new mapping editor, motion toggle, motion-status indicator or instructional flash/vibration text.

9. Normal pair buttons from either full gamepad

  1. Problem and previous behavior: The left source converted face buttons into directions and filtered out right-side shoulders; the right source dropped D-pad input. An L+R prompt therefore required buttons across both gamepads.
  2. New behavior: Either full gamepad in a two-source pair can send the normal buttons of both virtual halves: D-pad remains left, face buttons remain right, and shoulders, triggers, menu and stick clicks retain their meanings. L+R works on one gamepad. Held state remains per source, so releasing one does not release another's held button. Native Joy-Con sources keep the side path.
  3. Implementation rationale: Add dm_pad_set_pair_gamepad and share the locked publication logic with native-side input. Primary sticks, motion, Shake and rumble keep their assigned roles: pressing R on the left gamepad does not move the right gyro. Ordinary button bits already identify the virtual side.

10. Motion for single Joy-Cons and Joy-Con pairs

  1. Problem and previous behavior: A game accepting a Joy-Con style must also receive sensor data from the correct physical controller. One gamepad has one motion source and cannot represent two independently moving hands.
  2. New behavior: A motion-capable gamepad selected as Joy-Con (L) or Joy-Con (R) supplies that virtual controller’s gyro and accelerometer data from its own device. Separate players keep separate motion sources. With Joy-Con pair → One controller, both virtual Joy-Con sensors read that one gamepad’s motion, so their motion is shared. With two devices assigned to the same player as Left controller and Right controller, the pair’s left sensor reads only the left source and its right sensor reads only the right source, with independent histories. Each source’s primary stick, motion, synthetic Shake and rumble keep its assigned side even though ordinary buttons from either full gamepad can control both halves. For example, pressing R on the left gamepad does not supply motion to the right Joy-Con.
  3. Implementation rationale: Reuse the existing GameController motion capture, supported passive Pro path and sensor fusion. Preserve available sample timestamps and reject queued samples from a previous assignment. Existing motion enable/disable settings are respected. A controller without a usable gyro does not gain one by selecting a Joy-Con style. Existing iOS phone-motion fallback is retained for the ordinary Player 1 path where applicable, but is not duplicated into the two independent side histories of a two-device pair. Hold-orientation compensation changes buttons/sticks, not the physical sensor source. The game’s physical gesture requirements remain in effect, and this does not claim identical calibration or full Joy-Con hardware features across controller models.

11. Extensible source-registry and launch-channel contracts

  1. Problem and previous behavior: Direct GameController-only enumeration cannot describe another frontend's already-owned raw/split input sources, while a second allocator or settings store would compete with existing behavior.
  2. New behavior: cs_update_sources accepts transport-owned descriptors with optional physicalSide, automaticPlayer and reservedPlayers; cs_run_registry supplies host discovery, Identify and final acceptance through the existing private launch protocol. Ordinary clients retain cs_update.
  3. Implementation rationale: Provide a compatible extension point without importing NX2Apple's backend into this PR. The standalone Mac frontend retains GameController discovery alongside current-main native Joy-Con and Switch 2 support. The optional registry contract does not claim to import NX2Apple's complete HID/Steam transport integration.

Screenshots

These are earlier feature illustrations, not newly captured images of the rebased head. The three Mac images are actual standalone-branch SwiftUI/AppKit captures driven by synthetic device/requirement fixtures. The iOS image is the contributor’s original landscape screenshot from Downloads. Each illustrates a different task. No image is repeated, and none is evidence of sensor accuracy or exhaustive gameplay coverage.

Launch access: Controllers are opened from Controls, without listing all devices inline.

Mac game launch screen with Configure controllers under Controls

Requirements and corrective guidance: A two-player L/R-only fixture explains why one controller is not ready and why Apply is disabled.

Runtime requirements and one incompatible controller with Apply disabled

Two physical sources, one logical player: Both full-size controllers share Player 1 with complementary Pair input roles. Identify feedback is also visible. The taller capture shows both complete rows without cropping.

Two gamepads assigned as left and right members of Player 1

iOS menu and Automatic labels: The supplied landscape image shows native Cancel/Apply actions, a DualSense, and resolved Automatic player/type choices. Its requirements say Any with no player count requested, so it is not presented as a capture of Jamboree’s incompatible-controller prompt.

iOS controller menu showing a DualSense with Player 1 and Pro Controller Automatic choices

Checked

Published source: 16b97568d0f7994d70533f0accdfefffb0e38172 (tree 655e4fe881f651073074a003bc0ac704b20198c1), rebuilt as one commit on upstream 7479adf86cd7997c806d45c4c99a875ed4a6038e. The previous branch is preserved at dandelion711/nxnative:backup/pr1348-before-remedies-20261109. Genuine upstream attribution remains in the base, including the original Pro Controller deduplication commit a00f03b and its coauthor, rather than carrying its redundant cherry-pick as a feature commit.

Runtime/build/test sources are identical to tested 0da0413. The final amend only clarifies the native-versus-GameController discovery wording in docs/frontends.md; documentation checks were rerun afterward. Fresh local checks on Apple M1 / macOS 27.2 / Xcode 27.2:

  • make -C dm/host/tests GAME=0100965017338000 run: full host suite passed with the final runtime sources. HID and its ThreadSanitizer target each passed 21,211 checks, zero failed, with no ThreadSanitizer report. Native controller fixtures include assignment, registry admission, joining, motion and haptics. New haptics ownership regressions cover switching from an external Switch 2 output sink to a configured pair and then an ordinary controller.
  • 20 focused Python controller/launch/docs/usage checks passed. The full tools run executed 1,416 tests, 23 skipped; low internal-disk space caused errors/failure in two unrelated test modules. Both modules were rerun with temporary files on the external volume: 56 tests passed. The initial full invocation was not all green. Final documentation checks: 11 passed. iOS pipeline checks: 8 passed, including a new framework-filter regression observed failing before the fix. C-lint contract tests: 19 passed.
  • dm/tools/lint_c.py compdb and tidy --update --against origin/main, using CI's SHA-verified LLVM 23.1.4: all 273 sources checked, zero file/check pairs above or below the baseline. The baseline shrinks from 223 to 222 findings, with no increase. A separate same-toolchain run on current main passes at 223 findings. No unrelated graphics rewrite or broad baseline expansion was used.
  • dm/tools/lint_python.py --against origin/main, using pinned ruff 0.17.2 / pyright 1.1.414: zero new or stale findings (336 existing baseline findings).
  • make -C dm/host GAME=0100965017338000 dmhost: production Mac host built. DM_GAME=0100965017338000 dm/ios/tools/build_smoke.py: unsigned production iOS app compiled and linked with SDK 27.2 and the iOS 26 target, using synthetic modules, without installation. The rebase exposed Mac-only IOBluetooth being forwarded into the iOS project; filtering it fixes the observed linker failure while retaining CoreBluetooth.
  • make -C dm/gfx/tests GAME=0100965017338000 run built all 124 test programs but is not green. t_render_counter_overlap fails 1 of 42 checks (CopyBuffer0); continuing the remaining 39 targets finds t_render_enable_queue failing 9 of 46. Both targets reproduce with the same counts on unmodified baseline 7479adf using the same Mac/SDK. All other targets complete; six reflection-sample skips remain. The PR changes no graphics source. These are inherited observed failures, not a claimed all-GPU pass.
  • scripts/regress.py impact origin/main..HEAD: the controller/HID changes reach the four manifested titles below, with 149 profiles unknown because they lack manifests. The missing pad_hid_mac.h:hidpad_gc_controllers rule is now present. The only final unmapped file is the non-runtime lint-baseline metadata.

Selected game regression: Jamboree for the motivating Joy-Con workflow, Hades for ordinary input/rumble, Breath of the Wild for assignment/six-axis, Pokémon Scarlet for initialization/state, and Tears of the Kingdom for disconnect/six-axis. The five-game check exits 2 because only Jamboree has a usable local Mac build. The initial full Jamboree check on runtime-identical 0da0413 exited 2 before candidate comparison (the local cache has since been repaired):

Game Baseline → candidate Result
Jamboree 0100965017338000 7479adf → 0da0413 Initial attempt blocked by a legacy local shader package; regenerated package verified in the follow-up below

Jamboree also has no committed smoke route. No fresh boot/gameplay pass, validated.tsv record or five-game compatibility claim is made. The shader-chain code was not changed. Cache-repair follow-up: regenerated the local shader package, verified all 25,693 mapped records and all four registered tables, and preserved the old package as a backup. The retry passed package validation and began compiling shaders, then was stopped at the contributor’s request. The full comparison remains incomplete; no gameplay pass is claimed. No fresh physical-controller calibration, UIKit simulator test or exhaustive screen check was performed.

Hosted CI for 16b9756: tests and lint-c are queued; lint-python has passed. The queued workflows are pending results, not passes. Local results do not imply maintainer review or acceptance.

Contributor-reported gameplay testing:

Game Device Operating system Controllers
Super Mario Party Jamboree 2.3.0 iPhone 16 iOS 27 DualSense, DualShock
Super Mario Party Jamboree 2.3.0 M1 MacBook Air macOS 27.2 DualSense, DualShock

Front ends

  • Front-ends: interface: Optional connection descriptors and registry callbacks allow another frontend to reuse its existing transport/slot authority. UUIDs and physical device references stay host-owned; names are not identifiers.
  • Front-ends: new: Full-size two-source pair members publish through dm_pad_set_pair_gamepad; native halves retain dm_pad_set_side.
  • Front-ends: new: dm_controller_setup_requirements exposes scoped effective setup limits, and dm_ios_controller_open opens the runtime editor from the in-process gear menu.
  • Existing front_ctrl_support / DM_CTRL_PROMPT_TEXT joining behavior retains priority. Configure on that popup opens the child editor. The configuration hook opens directly only when upstream skips joining and the current setup cannot satisfy the request.

Docs

  • Mechanisms and limitations are in docs/reference/services.md and docs/frontends.md, linked from the Jamboree note. The iOS framework-link finding is in docs/reference/packing.md.
  • Symptom index regenerated and documentation checks passed.
  • No shader-chain tool, game files, keys or extracted game data are included.

Related work already identified for this PR includes #1127, #1268, #480, #585, #1143, #1322 and #221. The snapshot does not depend on those branches. The controller-joining prompt has been integrated into the current local candidate. This PR builds around that popup instead of introducing a competing applet result or an additional controller-settings store.

@github-actions github-actions Bot added host The Horizon host: SVCs, services, threads, memory (dm/host) packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers ios The iPhone and iPad app (dm/ios) mac The Mac front end and app (dm/mac, front_mac.m) tools Tools and scripts (dm/tools, scripts) docs Documentation game: 0100965017338000 Super Mario Party Jamboree labels Nov 5, 2026
@dandelion711 dandelion711 changed the title Configure per-player controller types in native Apple game apps Add per-player controller type configuration on macOS and iOS Nov 5, 2026
@dandelion711
dandelion711 marked this pull request as draft November 6, 2026 11:07
@dandelion711
dandelion711 force-pushed the feat/apple-controller-configuration branch from 2452673 to d2010e4 Compare November 8, 2026 16:46
@dandelion711
dandelion711 marked this pull request as ready for review November 8, 2026 16:47
Games that require Joy-Cons can now configure ordinary controllers by player and game-facing type before launch or while playing. Reuse the existing allocator, joining prompt, motion and rumble paths. Apply validates one draft and publishes one HID transaction, preserves unchanged owners, and rejects stale samples. Automatic remains the session default.

Support independent sources for a Joy-Con pair, normal full-gamepad buttons from either source, and native Mac/iOS configuration. Keep current-main native Joy-Con splitting, Switch 2 styles, mouse sensors and Bluetooth rumble. Silence transport output when a configured gamepad or pair takes ownership. Guard absent registry sources, invalid player indexes and capture output failures. Exclude the Mac-only IOBluetooth framework from generated iOS projects so the current-main host dependencies still link on iPhone.

Reconstructed from the final feature tree on upstream 7479adf, without synchronization merges or misattributed feature commits. Upstream commits retain their original authorship in the base history. HID passes 21,211 checks. Native haptics ownership-transition tests and 20 controller, launch, documentation and usage tests pass. LLVM 23.1.4 passes over all 273 sources, shrinking one resolved finding without adding any. Pinned Python lint, the Mac host build and the unsigned iOS production smoke build pass. The iOS framework regression fails before the fix and passes afterward. Broader final-revision verification and unavailable game/hardware checks are recorded in the PR description.

Front-ends: interface: optional session source descriptors and registry callbacks let external frontends retain their transport and slot authority.

Front-ends: new: dm_pad_set_pair_gamepad publishes ordinary full-gamepad buttons for a pair member while motion and rumble retain the assigned side.

Front-ends: new: dm_controller_setup_requirements exposes scoped setup limits, and dm_ios_controller_open opens the existing runtime editor.
@dandelion711
dandelion711 force-pushed the feat/apple-controller-configuration branch from 874bcba to 16b9756 Compare November 9, 2026 04:56
@dandelion711
dandelion711 marked this pull request as draft November 10, 2026 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0100965017338000 Super Mario Party Jamboree game: 0400486024DEC000 A game host The Horizon host: SVCs, services, threads, memory (dm/host) ios The iPhone and iPad app (dm/ios) mac The Mac front end and app (dm/mac, front_mac.m) packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant