overlays and MR

overlays

OVR Toolkit overlay desktop windows in SteamVR (not OpenXR)

window overlay in OpenXR

XR_EXTX_overlay is an experimental extension

OpenXR Toolkit FPS display or advanced timings overlay

WebXR implementation in WebKit targeting the OpenXR API

OpenXR overlay support via Monado XR_EXTX_overlay

Monado video see-through overlay projection layer for OpenXR

MR - Mixed Reality:  binocular video (e.g. from cameras) overlay

OpenBrush (Valve Index camera passthru)

Hand tracking videos

AC Mixed Reality Passthrough with Hand tracking

Monado's Mercury OpenXR optical hand tracking ready for use

OpenXR Toolkit with UltraLeap

Somnium VR1 mixed reality cameras

VR1 Tool integrates (Ultimate) mixed reality camera video

masked through chromakey, 3D meshes, and more
mixed reality passthru masked image instability
Most likely caused by perceived movement of MR image inside mask, not mask movement.
3D mesh mask is always placed relative to virtual world, and should not move.
Check "Show Mesh" checkbox on Masking page to see mesh wireframe visualization,
which should not move relative to the virtual world.
MR image movement inside the mask is caused mainly by
position differences between eyes and MR cameras;
compensation options available for this on the MR-General page.
Reprojection - Maps the MR image onto the used 3D mask.
This requires a 3D mask closely corresponding to the physical objects
and precisely positioned to minimize visible artifacts.
Cockpit Distance Compensation - Adjusts displayed stereoscopic MR image
to more precisely fit the specified distance (by default 70cm).
This works for physical cockpit at that distance from user's head,
better stabilizing its MR image,
but physical objects at other distances will have incorrect size/placement.
Virtual Camera Compensation - Adjusts virtual image position
to correspond to the position of the MR cameras (with slider at 100%).
This helps stabilize the MR image, while slightly destabilizing the VR image.
Current bug:  must be reapplied in Settings menu each VR application restart.

AutoHotkey script to toggle full vs masked mixed reality

#Requires AutoHotkey v2.0
#SingleInstance

ToggleFile := EnvGet("TEMP") . "\VR1MODE"
SetTitleMatchMode(1)
if (WinWait("Somnium VR1 Tool")) {
   MinMax := WinGetMinMax()
   if (MinMax == -1)
      WinRestore()
   WinActivate()
   WinGetClientPos(,,&Width)
   Click(460, 25)
   Click(520, 100)
   Click(150, 340)
   Click(Width - 660, 240)
   if (FileExist(ToggleFile)) {
      Send("Overlay{Enter}")
      FileDelete(ToggleFile)
      }
   else {
      Send("3D mesh with chroma key{Enter}")
      FileAppend("x", ToggleFile)
      }
   if (MinMax == -1) {
      sleep(100)
      WinMinimize()
      }
   }

WMR (Windows Mixed Reality - deprecated;  see Oasis)

Assetto Corsa and HP Reverb

enable/disable WMR OpenXR service