The problem we are solving is the "heavy client" requirement for teleop. Most existing solutions are coupled with tethered setups, Unity/Unreal builds, or specific hardware drivers on the operator side.
Teleop_xr is designed to be a decoupled "zero-install" interface. It runs strictly in the headset's browser (Quest 3, Vision Pro) and resolves operator intent into a target robot state before sending it to your control stack.
Architecture (nextjs+webxr frontend on headset <-> Python server on host PC):
Input: WebXR captures 6DOF head/controller states + joystick inputs
Transport: WebRTC for bidirectional streaming (poses out, video in) with sub-100ms latency.
Solver: Uses Pyroki (differentiable Whole-Body IK) to map XR poses to joint states in real-time, handling self-collision and joint limits.
Output: Returns a clean state object for ROS2 or Python integration.
You can map any bimanual robot (humanoids, mobile manipulators) to the interface without custom app builds, we implemented examples for franka (single arm), openarm (bimanual), and unitree h1 (humanoid).
Repo: https://github.com/qrafty-ai/teleop_xr
If you are looking for a clean VR teleoperation solution this might be the one you need :) Any suggestions welcome!