Meta’s Progress in Augmented and Virtual Reality

This is an abridged version of Andrew Bosworth’s blog post.

We think 2022 will be remembered as a year when foundational pieces of technology enabling our vision for the future made their way into the hands of developers and users for the first time. This October, Meta shipped several of them in Meta Quest Pro: Mixed reality is a key part of the journey toward full augmented reality devices, and eye and face tracking are essential pieces of the technology stack needed for AR and VR devices to reach their full potential. Quest Pro also brought important advances in display and optics that will move VR hardware forward for the long term.

The long-term value of these technologies will come from how they’re adopted by the ecosystem of developers, creators, and builders that has formed around these devices. That’s why I’m especially proud that we shipped Quest Pro when we did, because the sooner people can begin working with new technology like this, the sooner its full value and potential can be unlocked. We’ll be living with the benefits of this work for decades to come. 

Introducing Meta Reality: Mixed Reality Is More Than Just a Video Feed

Mixed reality is about much more than just displaying a live video feed of your surroundings inside the headset. For it to work properly, the headset needs to understand your room as a 3D space, recognizing the surfaces and objects around you and how they can interact with digital objects. It requires technologies like Spatial Anchors, which allow virtual things to occupy fixed spaces in the physical world, as well as Scene Understanding for reconstructing physical spaces virtually, as well as stereoscopic color  Passthrough, for capturing the physical world and representing it accurately in the headset with a sense of depth for greater comfort. Meta Reality, our system for mixed reality, combines all these technologies and more into a single system for developers to build with. 

Unlocking Advanced Experiences with Eye and Face Tracking

Eye and face tracking have a wealth of possibilities, both today and long into the future. Right now, eye and face tracking are delivering more expressive avatars on Meta Quest Pro, allowing people’s digital selves to more closely match their real-world facial expressions. This is one of many steps we’re taking to create an avatar system that can improve the quality of communication, expression, and connection in virtual spaces — and we’ll have plenty more to share on this in 2023. 

But the longer-term possibilities unlocked by eye and face tracking technology go far beyond avatars. Our vision for true AR glasses will require years of progress making our devices slimmer, lighter, faster, and more powerful, all while consuming way less battery power and generating much less heat. One major efficiency gain will come when devices are smart enough to only render the highest-resolution graphics in the small area where a user is actually looking. This October we demonstrated this capability for the first time thanks to the eye tracking technology on Quest Pro, and it’s going to be driving progress in VR and AR for many years to come. 

Introducing Infinite Display: Measuring Visual Quality in VR

The visual quality of a VR device depends on a range of technologies that go beyond the typical metrics we’ve used when describing the displays on a computer, TV or phone. Along with display resolution, factors like system resolution, sharpness, contrast and color range each play an important part. So does the optical stack itself, where lenses, coatings and films each make a huge difference. We recently shared more detail on how each of these come together in Meta’s Infinite Display system for VR optics, the product of years of investment in research and development for VR and AR technologies. That R&D will continue paying off long into the future, and earlier this year we showcased some of the research prototypes we’ve built to demonstrate these next-generation display technologies. 

Supporting the Virtual World-Building Community

We also saw the beginnings of a new community of creators on our platforms: the virtual world builders. It’s not often you get to witness a whole new way of self-expression and community building emerge on a platform, and the ways creators worked their magic on Horizon Worlds this year was a thing to behold. We’re going to spend 2023 focused on helping this community flourish. 

Meta is far from the only company working on pushing the boundaries here, and we expect to see new competitors joining us in building for AR and VR next year. As new devices hit the market, we believe our industry will enter a new era of growth and competition that will bring enormous benefits to users and the developer community. And Meta will have new devices of its own to share, including the successor to Meta Quest 2, our all-time best selling VR device. You’re going to love it. And you’re definitely going to love the Meta Quest Gaming Showcase in Spring, where we’ll once again show off some of the awesome new games coming in the next year.

Read Andrew Bosworth’s full post.

The post Meta’s Progress in Augmented and Virtual Reality appeared first on Meta.