“We’ve a digital camera passthrough API coming early subsequent yr.”
In a shock announcement at Join 2024 at this time, Meta revealed that it’s going to give builders entry to the passthrough cameras of Quest headsets by way of a brand new API subsequent yr.
“This can allow all types of cutting-edge MR experiences. You’ve got obtained tracked objects, AI purposes, fancy overlays, scene understanding, and a lot extra.”
Whereas blended actuality headsets like Meta Quest 3 and Apple Imaginative and prescient Professional use cameras to allow you to see the true world, at this time solely the system and sure first-party apps truly get uncooked entry to the cameras. Third-party builders can use digital camera passthrough as a background, however their apps do not get to truly see this passthrough. They as a substitute get higher-level knowledge, akin to hand and physique skeletal coordinates, a 3D mesh of your surroundings with bounding bins for furnishings, and restricted object monitoring capabilities. Which means they cannot run their very own laptop imaginative and prescient fashions, which severely limits the augmentation capabilities of those headsets.
Apple Will Give Enterprise Entry To Imaginative and prescient Professional’s Cameras
In visionOS 2, Apple will give enterprise corporations uncooked entry to Imaginative and prescient Professional’s passthrough cameras for private inner apps.
With visionOS 2, Apple is now giving enterprise corporations uncooked entry to Imaginative and prescient Professional’s passthrough cameras for private inner apps, however this requires a particular licence from Apple and is restricted to “in a enterprise setting solely”.
Meta hasn’t but stated whether or not there will be any restrictions to the Passthrough API, however we’ll maintain a detailed eye on the corporate for any additional particulars of this thrilling upcoming functionality, which may unlock totally new use circumstances for Quest headsets.
