[#303] Add Controller Pose Transformations by prikhi · Pull Request #382 · Supreeeme/xrizer

prikhi · 2026-07-04T03:48:34Z

Add new values to the BoundPoseType enum & make it a public type: Base, Handgrip, Grip, OpenxrHandmodel, OpenxrPinch, OpenxrPoke, OpenxrAim, OpenxrGrip.

Modify the InteractionProfile type to include a new pose_transformation function that returns optional left & right transformation matrices for each BoundPoseType via a new PoseTransformations type.

Implement the new pose_transformation function for the Index Controller, Vive Focus3 Controller, & Oculus Touch Controller profiles - all other profiles always return a None.

Extend the ProfileData struct to expose a pose_transformation function that returns the transformation matrix for a sepcific pose & hand.

Modify the get_controller_pose function to take an optional BoundPoseType & use it to transform the raw controller location before returning it.

Modify the GetPoseActionDataForNextFrame IVRInput interface function to use the BoundPoseType when calculating & writing the Pose ActionData. This required modifying the type signature of multiple functions in the callstack from GetPoseActionDataForNextFrame to get_controller_pose. All other uses of the modified functions simply pass in a pose type of None.

Modify TrackedDevice.get_pose to ignore the pose_cache when its pose_type argument is defined. This allows the
GetPoseActionDataForNextFrame call in a frame to supply the transformed controller locations when get_pose has already been called for a device.

Fixes the issue described in #303 where games that use poses other than Raw have the hands/controllers pointing in wrong direction.

Add new values to the `BoundPoseType` enum & make it a public type: Base, Handgrip, Grip, OpenxrHandmodel, OpenxrPinch, OpenxrPoke, OpenxrAim, OpenxrGrip. Modify the `InteractionProfile` type to include a new `pose_transformation` function that returns optional left & right transformation matrices for each `BoundPoseType` via a new `PoseTransformations` type. Implement the new `pose_transformation` function for the Index Controller, Vive Focus3 Controller, & Oculus Touch Controller profiles - all other profiles always return a `None`. Extend the `ProfileData` struct to expose a `pose_transformation` function that returns the transformation matrix for a sepcific pose & hand. Modify the `get_controller_pose` function to take an optional `BoundPoseType` & use it to transform the raw controller location before returning it. Modify the `GetPoseActionDataForNextFrame` `IVRInput` interface function to use the `BoundPoseType` when calculating & writing the Pose ActionData. This required modifying the type signature of multiple functions in the callstack from `GetPoseActionDataForNextFrame` to `get_controller_pose`. All other uses of the modified functions simply pass in a pose type of `None`. Modify `TrackedDevice.get_pose` to ignore the `pose_cache` when its `pose_type` argument is defined. This allows the `GetPoseActionDataForNextFrame` call in a frame to supply the transformed controller locations when `get_pose` has already been called for a device. Fixes the issue described in Supreeeme#303 where games that use poses other than `Raw` have the hands/controllers pointing in wrong direction.

prikhi

I'm not a rust, matrix math, or openvr/openxr expert so lemme know what you need me to fixup here.

prikhi · 2026-07-04T03:49:44Z

+        "base" => BoundPoseType::Base,
        "gdc2015" => BoundPoseType::Gdc2015,
+        "handgrip" => BoundPoseType::Handgrip,
+        "grip" => BoundPoseType::Grip,
+        "openxr_handmodel" => BoundPoseType::OpenxrHandmodel,
+        "openxr_pinch" => BoundPoseType::OpenxrPinch,
+        "openxr_poke" => BoundPoseType::OpenxrPoke,
+        "openxr_aim" => BoundPoseType::OpenxrAim,
+        "openxr_grip" => BoundPoseType::OpenxrGrip,


found all these in various rendermodels json files. dunno if openvr games would ever use the openxr_* poses but figured it doesn't hurt to include them, esp if games support launching in both openvr & openxr.

I don't really want to include them if they are unused. It makes the code harder to understand imo. I think it'll be easy enough to add them if a game starts using them.

I have ~30 games installed atm & do see some uses, worth keeping around?

$ rg 'pose/open_xr' **/*.json Dungeons of Eternity/DoE_Data/StreamingAssets/SteamVR/binding_psvr2.json 151: "path" : "/user/hand/left/pose/openxr_aim" 155: "path" : "/user/hand/right/pose/openxr_aim" 159: "path" : "/user/hand/left/pose/openxr_grip" 163: "path" : "/user/hand/right/pose/openxr_grip" # ...

prikhi · 2026-07-04T03:52:20Z

            .inverse(),
        }
    }
+


Expand the section above this chunk to see offset_grip_pose. It seems like that is just the inverse matrices of pose_transformation(BoundPoseType::Grip)? Did I duplicate some existing work?

Should I kill that function & just use this one? What's the inverse() in that call for - is there some math trick that improves performance by using the inverse instead of the actual transformation matrix?

offset_grip_pose is the raw pose transformation. The comment on this function in the InteractionProfile trait definition explains why it's the inverse. The transformation matrix presumably transforms from the raw pose to the grip pose, so using the inverse would get us back to the raw pose as expected by games.

hmm interesting i'll peek more at it(just saw the definition site similarity, didn't peek at the callsites) - just seemed odd to have & apply the inverse of a transformation that we didn't have or apply 😅

seems like some are defined as the inverse of Grip, some are defined as inverse of OpenxrGrip. Tempted to replace their defs with something like pose_transformation(BoundPoseType::Grip).map_or(Mat4::IDENTITY, ...).inverse() so that we still have the "this is derived from Grip/OpenxrGrip" annotation but don't have to repeat the actual matrices twice in the code.

prikhi · 2026-07-04T03:57:50Z

+    fn pose_transformation(_pose: BoundPoseType) -> Option<PoseTransformations> {
+        None
+    }


I just found this in SteamVR/resources/rendermodels/vr_controller_vive_1_5/vr_controller_vive_1_5.json I could add it but I dunno why there's one file & not separate left & right matrices... the openxr_* ones have openxr_*_r matrices but not the normal base/grip/handgrip etc.

The VIVE wands are ambidextrous controllers, there isn't a separate left and right hand controller

word, I'll work those in, interesting that openxr ones do have separate inline left/right poses:

{ // ... "openxr_poke": { "component_local" : { "origin": [-0.038, -0.058, 0.005], "rotate_xyz" : [45.00, 0.00, 0.00] } }, "openxr_poke_r": { "component_local" : { "origin": [0.038, -0.058, 0.005], "rotate_xyz" : [45.00, 0.00, 0.00] } }, // ... }

I haven't seen any games that bind a openxr_*_r pose though so I'm guessing it's safe to just put those as the right hand of the openxr_* pose instead of adding an R variant of them to the BoundPoseType

transforms for ViveWand pushed, should we use one of these for the offset_grip_pose as well?

prikhi · 2026-07-04T04:02:52Z

+        let raw_pose = location.pose;
+        let pose_mat = Mat4::from_rotation_translation(
+            Quat::from_xyzw(
+                raw_pose.orientation.x,
+                raw_pose.orientation.y,
+                raw_pose.orientation.z,
+                raw_pose.orientation.w,
+            ),
+            Vec3 {
+                x: raw_pose.position.x,
+                y: raw_pose.position.y,
+                z: raw_pose.position.z,
+            },
+        );
+        let (_, new_quat, new_vec) =
+            (pose_mat * location_transform).to_scale_rotation_translation();
+        let [quat_x, quat_y, quat_z, quat_w] = new_quat.to_array();
+        location.pose = xr::Posef {
+            orientation: xr::Quaternionf {
+                x: quat_x,
+                y: quat_y,
+                z: quat_z,
+                w: quat_w,
+            },
+            position: xr::Vector3f {
+                x: new_vec.x,
+                y: new_vec.y,
+                z: new_vec.z,
+            },


I dunno if there is a smarter, more efficient way of applying the pose transforms, but I have verified this fixes the issue w/ backwards pointing hands in Dungeons of Eternity.

prikhi · 2026-07-04T04:05:54Z

+            && pose_type.is_none()
+        {
            return Some(pose);
        }


Multiple interface functions can(& do) call a device's get_pose function within a single frame. In Dungeons of Eternity, the GetPoseActionDataForNextFrame call happens after calls w/ no pose_type defined, which makes the transforms not work since we just pull untransformed poses from the cache.

So I skipped the fetch if we have a pose_type.

But that opens up some questions:

Should the cache be indexed by Option<BoundPoseType>? What's that look like in rust?

If we don't cache per-type, should pose_type calls write to the cache or leave it unchanged?

Should we require the pose_type arg & force the current None callers to pass in Raw?

It would make the most sense to me to cache the untransformed pose and transform as necessary each time.

makes sense, can do

prikhi · 2026-07-04T04:14:04Z

-                        (origin, hand)
-                    }
-                }
+                (origin, hand, pose_type)


I ignored this TODO & just include the pose_type in the return block cause I don't think it's possible to do that transformation in this scope? I think I tried but some required object properties were private.

prikhi · 2026-07-04T04:14:17Z

        unsafe {
            let pose = self
-                .get_controller_pose(hand, Some(origin))
+                .get_controller_pose(hand, Some(origin), pose_type)


This is the only place where we actually pass a pose_type

prikhi · 2026-07-04T04:15:54Z

This is the script I used to generate the oculus_touch transforms:

#!/usr/bin/env python
"""
./render_models_to_pose_transforms.py <left-rendermodels>.json <right-rendermodels>.json
"""

import json
import sys

def main():
    [left_filename, right_filename] = sys.argv[1:3]
    left_json = {}
    right_json = {}
    with open(left_filename) as left_file:
        left_json = json.load(left_file)['components']
    with open(right_filename) as right_file:
        right_json = json.load(right_file)['components']

    pairs = [
        ("raw", "BoundPoseType::Raw"),
        ("tip", "BoundPoseType::Tip"),
        ("base", "BoundPoseType::Base"),
        ("gdc2015", "BoundPoseType::Gdc2015"),
        ("handgrip", "BoundPoseType::Handgrip"),
        ("grip", "BoundPoseType::Grip"),
        ("openxr_handmodel", "BoundPoseType::OpenxrHandmodel"),
        ("openxr_pinch", "BoundPoseType::OpenxrPinch"),
        ("openxr_poke", "BoundPoseType::OpenxrPoke"),
        ("openxr_aim", "BoundPoseType::OpenxrAim"),
        ("openxr_grip", "BoundPoseType::OpenxrGrip"),
        ]
    for (json_pose, xrizer_pose) in pairs:
        if (json_pose in left_json) != (json_pose in right_json):
            print(f"Found pair in one file but not other: {json_pose}", file=sys.stderr)
            continue
        if not (json_pose in left_json and json_pose in right_json):
            print(f"            {xrizer_pose} => None,")
            continue
        left_pose = left_json[json_pose]['component_local']
        right_pose = right_json[json_pose]['component_local']
        print(f"""
            {xrizer_pose} => Some(PoseTransformations {{
                left_hand: Mat4::from_rotation_translation(
                    Quat::from_euler(
                        EulerRot::XYZ,
                        {left_pose['rotate_xyz'][0]}_f32.to_radians(),
                        {left_pose['rotate_xyz'][1]}_f32.to_radians(),
                        {left_pose['rotate_xyz'][2]}_f32.to_radians(),
                    ),
                    Vec3::new({left_pose['origin'][0]}, {left_pose['origin'][1]}, {left_pose['origin'][2]}),
                ),
                right_hand: Mat4::from_rotation_translation(
                    Quat::from_euler(
                        EulerRot::XYZ,
                        {right_pose['rotate_xyz'][0]}_f32.to_radians(),
                        {right_pose['rotate_xyz'][1]}_f32.to_radians(),
                        {right_pose['rotate_xyz'][2]}_f32.to_radians(),
                    ),
                    Vec3::new({right_pose['origin'][0]}, {right_pose['origin'][1]}, {right_pose['origin'][2]}),
                ),
            }}),
              """)


if __name__ == '__main__':
    main()

jaoreir · 2026-07-19T07:40:34Z

I tested this patch to fix a controller rotation offset issue with vivecraft. The tip transformation worked like a charm for me. I hope this pr gets merged soon :)

Supreeeme

If you want to put that script in a resources folder in the repo or something that would be useful.

Supreeeme · 2026-07-22T01:09:58Z

+        "base" => BoundPoseType::Base,
        "gdc2015" => BoundPoseType::Gdc2015,
+        "handgrip" => BoundPoseType::Handgrip,
+        "grip" => BoundPoseType::Grip,
+        "openxr_handmodel" => BoundPoseType::OpenxrHandmodel,
+        "openxr_pinch" => BoundPoseType::OpenxrPinch,
+        "openxr_poke" => BoundPoseType::OpenxrPoke,
+        "openxr_aim" => BoundPoseType::OpenxrAim,
+        "openxr_grip" => BoundPoseType::OpenxrGrip,


I don't really want to include them if they are unused. It makes the code harder to understand imo. I think it'll be easy enough to add them if a game starts using them.

Supreeeme · 2026-07-22T01:14:08Z

+            && pose_type.is_none()
+        {
            return Some(pose);
        }


It would make the most sense to me to cache the untransformed pose and transform as necessary each time.

Supreeeme · 2026-07-22T01:19:25Z

            .inverse(),
        }
    }
+


offset_grip_pose is the raw pose transformation. The comment on this function in the InteractionProfile trait definition explains why it's the inverse. The transformation matrix presumably transforms from the raw pose to the grip pose, so using the inverse would get us back to the raw pose as expected by games.

prikhi

thanks for the review, prob won't have time to mess w/ this til the weekend.

prikhi · 2026-07-22T02:21:15Z

+        "base" => BoundPoseType::Base,
        "gdc2015" => BoundPoseType::Gdc2015,
+        "handgrip" => BoundPoseType::Handgrip,
+        "grip" => BoundPoseType::Grip,
+        "openxr_handmodel" => BoundPoseType::OpenxrHandmodel,
+        "openxr_pinch" => BoundPoseType::OpenxrPinch,
+        "openxr_poke" => BoundPoseType::OpenxrPoke,
+        "openxr_aim" => BoundPoseType::OpenxrAim,
+        "openxr_grip" => BoundPoseType::OpenxrGrip,


I have ~30 games installed atm & do see some uses, worth keeping around?

$ rg 'pose/open_xr' **/*.json Dungeons of Eternity/DoE_Data/StreamingAssets/SteamVR/binding_psvr2.json 151: "path" : "/user/hand/left/pose/openxr_aim" 155: "path" : "/user/hand/right/pose/openxr_aim" 159: "path" : "/user/hand/left/pose/openxr_grip" 163: "path" : "/user/hand/right/pose/openxr_grip" # ...

prikhi · 2026-07-22T02:21:52Z

+            && pose_type.is_none()
+        {
            return Some(pose);
        }


makes sense, can do

prikhi · 2026-07-22T02:37:41Z

            .inverse(),
        }
    }
+


hmm interesting i'll peek more at it(just saw the definition site similarity, didn't peek at the callsites) - just seemed odd to have & apply the inverse of a transformation that we didn't have or apply 😅

seems like some are defined as the inverse of Grip, some are defined as inverse of OpenxrGrip. Tempted to replace their defs with something like pose_transformation(BoundPoseType::Grip).map_or(Mat4::IDENTITY, ...).inverse() so that we still have the "this is derived from Grip/OpenxrGrip" annotation but don't have to repeat the actual matrices twice in the code.

Extract a `transform_pose` method from the `get_controller_pose` method that translates a device's pose given some `BoundPoseType`. Call this from the `get_controller_pose` method. Modify the `get_pose` method so it always gets & puts the Raw pose into the `pose_cache`. When a `pose_type` is passed, we transform the cached pose and return it, instead of potentially caching a non-Raw pose.

Add a new `offset_grip_pose_from_pose_type` function to the `input::profiles` module. This lets us generate the inverse grip matrix via the matrices defined in a profile's `pose_transformation` method - instead of having to specify the matrix twice. Use this new helper in the knuckles, touch, & focus3 profiles.

Add a `resources/generate_pose_transforms.py` script that will take the left & right rendermodel JSON files for a SteamVR devices and output all the `BoundPoseType` match branches for use in a `InteractionProfile`'s `pose_transformation` method.

prikhi · 2026-07-26T05:11:53Z

Pushed 3 commits to always & only cache Raw poses, simplify the offset_grip_pose implementations, and add the script.

prikhi · 2026-07-26T05:48:39Z

hmm, test is failing because openvr's HmdMatrix34_t -> Posef -> HmdMatrix34_t conversion functions are not the identity function 😬

#[test]
fn pose_to_from_hmd34_is_identity() {
    let expected = xr::Posef {
        orientation: xr::Quaternionf {
            x: 0.5,
            y: 0.0,
            z: 0.0,
            w: 0.0,
        },
        position: xr::Vector3f {
            x: 0.0,
            y: 0.0,
            z: 0.0,
        },
    };
    assert_eq!(xr::Posef::from(HmdMatrix34_t::from(expected)), expected);
}

---- convert::pose_to_from_hmd34_is_identity stdout ----

thread 'convert::pose_to_from_hmd34_is_identity' (567339) panicked at openvr/src/convert.rs:167:5:
assertion `left == right` failed
  left: Posef { orientation: Quaternionf { x: 0.0, y: 0.0, z: 0.0, w: 0.8660254 }, position: Vector3f { x: 0.0, y: 0.0, z: 0.0 } }
 right: Posef { orientation: Quaternionf { x: 0.5, y: 0.0, z: 0.0, w: 0.0 }, position: Vector3f { x: 0.0, y: 0.0, z: 0.0 } }

And I use the assumption of that property here:

1992ecf#diff-a08e377509cb45c7cab7793e9248a2305c025b918d77ff8d469c5fe6366eabfdR219-R221

prikhi · 2026-08-01T21:21:45Z

ok, i have done some digging. I guess valid rotation quarternions are supposed to have a length of 1 so the fact that conversion isn't reversible for invalid quarternions({ 0.5, 0, 0, 0 } used in the existing test) seems less relevant.

The thing I shoulda caught is that the position vector in that assertion is all 0, while the test specifies 0.5 for all of em. Some printf debugging showed we're hitting the bPoseIsValid: false branch of space_relation_to_openvr_pose, so I'm going to tweak this to only transform valid cached poses, and write a test so we exercise both the valid & invalid branches.

Modify `TrackedDevice.get_pose()` so we only transform locations for valid poses. Invalid poses generate a default pose, which contains an invalid quaternion, resulting in drift between the return skeletal pose & hand pose. Extend the test suite to ensure `get_controller_pose` & `GetPoseActionDataForNextFrame`(hand & skeletal versions) return the same raw poses when the device has both valid & invalid poses.

prikhi · 2026-08-02T00:42:29Z

Mkay, test should pass now & prob ready for re-review @Supreeeme. Will retest against Dungeons of Eternity tonight.

…ntroller-pose-transforms

prikhi · 2026-08-04T04:01:36Z

Updated to account for new MetaTouchPlus profile from #308

…ntroller-pose-transforms

prikhi commented Jul 4, 2026

View reviewed changes

Supreeeme reviewed Jul 22, 2026

View reviewed changes

prikhi commented Jul 22, 2026

View reviewed changes

prikhi added 3 commits July 26, 2026 00:20

prikhi requested a review from Supreeeme July 26, 2026 05:11

prikhi added 2 commits August 3, 2026 23:55

Merge branch 'main' of github.com:Supreeeme/xrizer into prikhi/303-co…

c486abb

…ntroller-pose-transforms

Add pose_transformation For New MetaTouchPlus Profile

d619f75

prikhi mentioned this pull request Aug 4, 2026

hl2-vr: multiple fixes (fixes #266) (fixes #369) #395

Closed

skryvel mentioned this pull request Aug 6, 2026

hl2-vr: multiple fixes #397

Open

Add pose_transformation For ViveWand Profile

3db192b

prikhi requested a review from ImSapphire August 16, 2026 20:55

Merge branch 'main' of github.com:Supreeeme/xrizer into prikhi/303-co…

17bcd6e

…ntroller-pose-transforms

Uh oh!

Conversation

prikhi commented Jul 4, 2026

Uh oh!

prikhi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prikhi Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prikhi commented Jul 4, 2026

Uh oh!

jaoreir commented Jul 19, 2026

Uh oh!

Supreeeme left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prikhi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prikhi commented Jul 26, 2026

Uh oh!

prikhi commented Jul 26, 2026

Uh oh!

prikhi commented Aug 1, 2026

Uh oh!

prikhi commented Aug 2, 2026

Uh oh!

prikhi commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

prikhi Aug 16, 2026 •

edited

Loading