XR Hand Tracking

Working systems

Ultraleap, Meta Quest, Apple Vision Pro

APIs

- OpenXR
- SteamVR Skeletal Input
- Android XR_ANDROID_hand_mesh
- MetaHorizon vrapi
- visionOS/ARKit
- xreal NRSDK

Implementations

- ALVR (Rust source)

- AVP

- HTCC Hand tracking for flight simulators

- Monado OpenXR optical hand tracking
  Is Monado's Ultralink driver an OpenXR layer
  or compiled into their OpenXR run time?

- OpenXR (unity, unreal, OpenXR Toolkit)

  - godot

- Steam Link (Quest to OpenXR/Skeletal Input)
  the most complete Windows and SteamVR implementation?
  Passes hand tracking to any PC VR app
  using OpenXR hand tracking or SteamVR Skeletal Input;
  also supported in SteamVR's interface:
 
  SteamVR detects a Hand Tracking Controller
 
  - point and pinch to select and scroll menu items.
  - Bring up the SteamVR Dashboard at any time
    by pinching left thumb to left index finger.

  ST.URIEL
 "SteamVR 2.8 update doesn’t actually bring
  full hand-tracking support to SteamVR’s UI,
  but rather allows Quest users to DIY setups for each game
  by experimenting with input bindings."
  Uses Steam Link For Meta Quest

  speculations

Since SteamVR supports only OpenVR runtimes,
then Steam Link must emulate that (back to Quest).
Supposing Quest employs a flavor of OpenXR hand tracking
while SteamVR runs non-OpenXR games,
then Steam Link For Meta Quest presumably is an ADD-ON
providing a hand tracking controller and OpenXR run time.

- Ultraleap OpenXR - SteamLink compatible??

- Virtual Desktop   mbucchia 24 Jun 2024:
  The most common issue is developers having Ultraleap software,
  with an OpenXR API that will "mask" any other hand tracker.
  With Ultraleap installed, Virtual Desktop hand tracking data
  is blocked behind Ultraleap's, even if Ultraleap isn't connected.

Input device emulations

- touchscreen (similar to visionOS?)

- mouse/laser pointer (IMO doomed)

Compromise?

- SteamVR seems addicted to laser pointer..

- point using left hand, input by HOTAS with right?