Nvidia to Bring Omniverse VR Cloud to Apple Vision Pro

Nvidia announced at its GTC developer conference that it will bring its Omniverse VR Cloud to the Apple Vision Pro headset, enabling enterprise developers to fully render 3D digital twins and stream them on the headset using Omniverse Cloud APIs. This move comes alongside Nvidia's launch of its new Blackwell platform, which includes GB200 super chips for AI acceleration and its NIMs cloud software for enhancing AI inference on Nvidia's hardware, as well as a quantum computing cloud service.

The Omniverse Cloud APIs will allow developers to send OpenUSD scenes from their apps to the Nvidia Graphics Delivery Network, which can then stream content to the Apple Vision Pro. This means that developers will be able to upload their work to the cloud and stream it to an Apple Vision Pro without relying solely on the M2 processor inside the headset. Hybrid rendering, which combines local and remote rendering using technologies from both Apple and Nvidia in the same software, will also be possible.

The collaboration between Nvidia and Apple on the Vision Pro is notable, given Apple's past reluctance to allow Nvidia to have its own drivers on macOS and not enabling CUDA drivers, which had previously led to the two companies not working together on Mac products.

CGI studio Katana demonstrated the capabilities of this collaboration with a demo of an Apple Vision Pro wearer using a car configurator powered by the Nvidia Cloud. The vehicle in the video, called a digital twin, is an object built using real-world stats and metrics to represent in a simulation. Rendering such content requires powerful graphics processors to handle the data on high-resolution displays, which goes beyond the capabilities of typical video game rendering.

Mike Rockwell, vice president of the Vision Products Group at Apple, said, "The breakthrough ultra-high-resolution displays of Apple Vision Pro, combined with photorealistic rendering of OpenUSD content streamed from NVIDIA accelerated computing, unlocks an incredible opportunity for the advancement of immersive experiences. Spatial computing will redefine how designers and developers build captivating digital content, driving a new era of creativity and engagement."

The shift from rendering 3D content on a 2D display to a spatial computing environment opens up new possibilities for interacting with data. Although the Apple Vision Pro may not be powerful enough to run this content natively, Nvidia's RTX Enterprise Cloud Rendering allows digital twins to be brought to spatial environments, highlighting the advanced display technology Apple has incorporated into the Vision Pro.