Meta Quest Solves Digital Persona for Apple Vision Pro - "Codec Avatars"

Meta Quest Solves Digital Persona for Apple Vision Pro - "Codec Avatars"

One of the most notable features of Apple Vision Pro is the realistic avatar that represents you on FaceTime calls. Despite wearing a visor on the call, you can still appear on the phone or laptop as yourself Meta is currently nearing the launch of its own version for Quest.

The "codec avatar" has been developed previously by Meta's research team and is designed to provide a photorealistic version of the user in a fully immersive environment.

A recent study of the Quest firmware by a user called Luna revealed the potential for the deployment of a new realistic avatar system to replace the cartoon-like avatars currently in use.

In March 2023, CTO Andrew Bosworth warned in an Instagram AMA that despite progress, it would "still be years" before it could be launched. The arrival of firmware does not imply that a launch is imminent, but it does suggest a more urgent timeline.

Meta first announced plans for more realistic avatars in 2019, along with the launch of an R&D project aimed at improving remote communication.

Over the past four years, the company has gradually revealed fidelity updates and new features, but so far, user representation in the virtual world has largely been cartoonish or game-like.

Version 62 of Meta Quest's firmware is only available on the public test channel and not as a default option.

The initial release is supposed to work with headshots and shoulder shots, as displayed in FaceTime and Zoom calls. This is similar to how Apple's Vision Pro personas are displayed.

Unlike personas that use front displays and depth sensors, codec avatars are fully photorealistically drawn, rounded virtual characters. They are virtual 3D models encoded using data captured by smartphones and processed using AI models in the cloud.

This allows them to be fully integrated into the virtual world, rather than simply displayed on a virtual screen on a headset or third-party device. Examples of this technology have already been published on Meta Connect.

Bosworth states in a March video: "The challenge is how to get a capture from a cell phone in a mixed lighting situation in a short time. The challenge is how to get the sensor to pick up enough data to animate the avatar. There are a lot of engineering challenges as well as research challenges."

This suggests that codec avatars will likely require better hardware than what is currently available for Quest headsets and may be reserved for a future Quest Pro 2.

Categories