XR Hand Tracking

Working systems

Ultralink, Meta Quest, Apple Vision Pro, Rokid

APIs

- OpenXR
- SteamVR Skeletal Input
- Android XR_ANDROID_hand_mesh
- MetaHorizon vrapi
- visionOS/ARKit
- xreal NRSDK

Implementations

- AVP

- Monado OpenXR optical hand tracking
  Is Monado's Ultralink driver an OpenXR layer or compiled into their OpenXR run time?

- OpenXR (unity, unreal, OpenXR Toolkit)

  - godot

- SteamLink (Quest to OpenXR/Skeletal Input)
  passes hand tracking to any PC VR app
  using OpenXR hand tracking or SteamVR Skeletal Input;
  also supported in SteamVR's interface:
  - You see only a pointer, not your hands;
  - point and pinch to select and scroll menu items.
  - Bring up the SteamVR Dashboard at any time
    by pinching left thumb to left index finger.

  ST.URIEL
 "SteamVR 2.8 update doesn’t actually bring
  full hand-tracking support to SteamVR’s UI,
  but rather allows Quest users to DIY setups for each game
  by experimenting with input bindings."
  Uses Steam Link For Meta Quest

  speculations

Since SteamVR supports only OpenVR runtimes,
then Steam Link must emulate that.
Supposing Quest employs a flavor of OpenXR hand tracking
while SteamVR runs non-OpenXR games,
then Steam Link For Meta Quest presumably is an ADD-ON
providing a hand tracking controller and OpenXR run time.

- Ultraleap OpenXR - SteamLink compatible??

- Virtual Desktop   mbucchia 24 Jun 2024:
  The most common issue is developers having Ultraleap software,
  with an OpenXR API that will "mask" any other hand tracker.
  With Ultraleap installed, Virtual Desktop hand tracking data
  is blocked behind Ultraleap's, even if Ultraleap isn't connected.

Input device emulations

- touchscreen (similar to visionOS?)

- mouse/laser pointer (IMO doomed)