Codec Avatars are a research effort by Meta to produce photorealistic, real-time avatars of people that are learned from capture data and driven by sensors to reproduce expression and appearance.
Semantic Classification
Content
- Codec Avatars are a line of research from Meta aimed at creating photorealistic digital representations of individuals that can be animated in real time. The avatars are built from detailed multi-camera capture and a learned model, then driven by sensors on a headset to mirror the wearer’s facial expression and gaze.
- The approach treats the avatar as an encoder-decoder system, where compact driving signals are decoded into a high-fidelity rendered face, which suits transmission over a network for telepresence. Open challenges include reducing the capture requirements and generalising convincingly to many users.