Introduction

Having a high-performance, hardware-intensive XR environment doesn’t mean you have to compromise on hand interaction—especially when combining OctoXR SDK with one of the leading XR streaming platforms: NVIDIA CloudXR. With this integration, headsets and even mobile phones can deliver fully interactive XR experiences while enhancing data security.

The Challenge: Interaction and Immersiveness Without Sacrificing Quality

When building custom solutions with OctoXR, we often hear the same feedback: "If this doesn't feel realistic enough, we can't use it for critical tasks like pilot training, physical rehabilitation, or prototyping." XR content is becoming increasingly sophisticated. While XR headsets are evolving, sometimes even top-tier hardware struggles with rendering performance. This is where cloud-streaming platforms like NVIDIA CloudXR solve the compute challenge.

However, the weakest link in many streaming setups remains input and interaction. Most hand interaction systems aren't optimized for streamed environments.

For many client scenarios—especially in defense and military contexts—another major challenge is data security. Installing apps locally on headsets poses risks, as these applications often contain highly confidential data.

Types of XR Headsets and Platforms

There are three major categories of headsets and platforms for delivering XR content:

Standalone (Untethered) XR Headsets

Examples: Meta Quest, Apple Vision Pro, Pico, HTC Vive XR Elite

• Apps are installed directly on the headset

• Devices run their own OS (e.g., visionOS, Android variants)

• App access via headset app store or sideloading

• Local processing and storage

• Optional sync with mobile or desktop companions

Tethered XR Headsets

Examples: HTC Vive Pro, Valve Index, Varjo XR-4

• Apps run on a connected PC or console

• Headset functions as a display/sensor

• Launched from desktop platforms (e.g., SteamVR, Varjo Base)

• All processing happens on the host machine

Cloud-Streamed XR Platforms

Example: NVIDIA CloudXR

• Apps run on a remote server or edge device

• Streamed to the headset or client device in real time

• Ideal for delivering high-fidelity XR without installing local apps

• Frequently used in enterprise environments for performance and security

NVIDIA CloudXR – The Leader in XR Streaming

NVIDIA CloudXR enables XR applications to be rendered remotely and streamed with ultra-low latency to various devices—including untethered headsets, tethered systems, and even mobile phones. Streaming can occur over Ethernet, Wi-Fi, cellular, or other networks for optimal performance.

A major benefit of this approach is significantly enhanced security. Since no applications are stored locally on the device, all data remains on the server and is delivered via encrypted streaming protocols—ideal for industries dealing with sensitive or classified information.

What OctoXR Adds to the Equation

While streaming platforms solve compute, rendering, and security challenges, interaction is where OctoXR steps in. OctoXR enables:

• Full-featured hand interaction layer: All the capabilities you’d expect in a locally installed XR app—now fully usable in streamed environments.

• Out-of-the-box compatibility: Works with streamed content without additional configuration.

• Physics-based object interaction: Realistic, tactile experiences powered by leading physics.

• Hand gesture support: Supports static and dynamic gestures for intuitive control.

• Intuitive UI components: Includes direct touch, distance pinch, and virtual keyboards.

• Hardware compatibility: Compatible with any hand-tracking device supporting OpenXR.

Use Cases OctoXR Enables with NVIDIA CloudXR

Any use case supported in non-streamed environments can now be enabled through streaming. Examples include:

• Automotive: Virtual prototyping and infotainment UI interaction. [Find out more]

• Aerospace: Inspecting complex engines using pinch, rotate, and grab gestures. [Find out more]

• Healthcare: Physical rehabilitation and medical device training. [Find out more]

• Defense: Mission preparation and tactical coaching using holographic maps and equipment simulations. [Find out more]

Why OctoXR + CloudXR Works

• Low-latency hand interaction engine

• Hardware-agnostic integration

• Fully OpenXR-compatible for broad device support

Conclusion

OctoXR SDK and NVIDIA CloudXR together eliminate the final barrier in cloud-delivered XR: natural interaction. Hand-based control feels intuitive—unlike artificial controller-based methods—making it a true prerequisite for productive, enterprise-grade applications.