The combined domain encompassing the creation, animation, and management of digital representations of human beings in virtual environments, including photorealistic digital humans, stylised avatars, and fictional characters. This domain integrates motion capture, AI-driven face and body generation, lip-synchronisation, and avatar portability standards to produce consistent, expressive agents across metaverse platforms.
Semantic Classification
Content
- This section focuses on platforms and research enabling the generation of complete avatars, encompassing both visual representation and underlying technologies.
-
- REPLIKANT: An AI-assisted 3D avatar and animation platform designed for creators.
- Meta Research Paper: A research paper from Meta exploring an unspecified aspect of avatar generation.
- Heygen: A platform for generating and animating realistic avatars from text prompts and images.
- Synthesia: A leading platform for creating AI-powered videos featuring realistic avatars.
- This section explores projects and techniques dedicated to the intricate art of generating digital faces, capturing nuances of expression and identity.
-
- Media2Face: A project focused on generating faces from various media inputs.
- Towards a Simultaneous and Granular Identity-Expression Control in Personalized Face Generation: Research on personalized face generation with fine-grained control over identity and expression.
- HeadStudio: A project utilizing 3D Gaussian Splatting to create animatable head avatars from text.
- Lip Sync: Achieving realistic lip synchronization is crucial for believable avatars. These resources delve into this challenge:
- VASA-1 (Microsoft Research): A project from Microsoft Research exploring lip synchronization techniques.
- wav2lips: An article exploring the use of wav2lip for deepfake audio and lip synchronization.
- Realtime lip-sync API: An API offering real-time lip synchronization capabilities.
- This section examines projects that focus on generating realistic and diverse digital bodies, laying the foundation for virtual humans and characters.
-
- Volumetric Primitives (MVP): A research paper exploring the use of volumetric primitives for avatar representation.
- Gaussian Shell Maps: A research project dedicated to the use of Gaussian Shell Maps for efficient 3D human generation.
- En3D: A project focused on generating 3D human models with realistic detail.
- Character Creator 3 (Reallusion): A powerful tool for creating realistic and stylised 3D characters.
- The human hand, with its intricate movements and expressiveness, poses a unique challenge for digital recreation. This section highlights projects tackling this complexity.
- URHand: A project dedicated to the generation and animation of realistic digital hands.
- This section explores projects that leverage audio input to drive avatar animation, creating a more immersive and responsive experience.
- audio2photoreal (Facebook Research): A project from Facebook Research focused on generating photorealistic avatars driven by audio.
- Realistic skin texture is crucial for believable digital humans. This section explores projects and research dedicated to accurately estimating and recreating human skin textures.
- SMPLitex: A generative model and dataset for estimating 3D human texture from a single image.
- Human texture estimation from a single image: A broader exploration of the field of human texture estimation from single images.
- This section encompasses a variety of projects and resources that contribute to the broader field of digital human and avatar creation.
- StableIdentity: A project exploring the insertion of individuals into new environments.
- What You See Is What You GAN: An NVIDIA research paper on rendering high-fidelity geometry in 3D GANs.
- Generating Consistent Characters (Twitter Thread): A Twitter thread by Manu.Vision discussing methods for generating consistent characters.
- Consistent Characters with Scenario (YouTube Tutorial): A YouTube tutorial on creating consistent characters within a specific scenario.
- MoMask (GitHub Repository): The official implementation of MoMask, a method for generative masked modelling of 3D human motions.
- Character Asset Creator (Reddit Post): A Reddit post discussing a Character Asset Creator for StableDiffusion.
- Mikugg Emotions Script: A script for creating emotions in Mikugg.
- Miku.gg Bot Emotions Documentation: Documentation on creating bot emotions within the Miku.gg platform.
- Consistent Character Maker Workflow (Civitai): A workflow for generating consistent characters using Stable Diffusion.
- Achieving consistency in character appearance across multiple generations is a key challenge. This section highlights resources and techniques addressing this issue.
-
- Consistent Character Creation (OpenArt): A workflow for achieving full-character consistency using the SDXL model with a Lightning 8-step LoRA.
- This section outlines a comprehensive workflow for creating and animating 3D characters, leveraging a variety of tools and techniques.
- LinkedIn Post detailing the workflow: A LinkedIn post outlining the workflow in detail.
- Tools used in the workflow:
- Midjourney: For generating initial character concepts.
- Tripo3d.ai: For creating 3D models from images.
- AI Digimans’ PBR Texture Generation Tool: For generating realistic textures.
- Mixamo: For animating the 3D models.
- Spline.design: For further customisation and enhancement.
- Bringing digital characters to life requires compelling animation. This section explores projects and resources focused on achieving realistic and expressive character movement.
-
- Animatable Gaussians (GitHub Repository): Code for “Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling,” presented at CVPR 2024.
- 3D Gaussian Blendshapes: Exploration of the use of 3D Gaussian Blendshapes for head avatar animation.
- Eggnog: A platform for creating infinite AI videos.
- Efficient Portrait Animation (LivePortrait): A project focused on efficient portrait animation with stitching and retargeting control.
- Note by Daniele: A note on LivePortrait by Daniele.
- ComfyUI Nodes for LivePortrait (GitHub Repository): ComfyUI nodes designed for LivePortrait.
- CLARA2 (GitHub Repository): A 3D-rendered AI agent designed to present PowerPoint presentations.
- Consistent Character API (Replicate): An API for running fofr’s consistent character model.
- Joystick-Controlled Character Manipulation (Twitter): A concept for manipulating character features using a joystick.
- Volucap Authentic Digital Avatars: A company specialising in creating authentic digital avatars.
- Openpose Controlnets (Civitai Article): An article explaining how to use and generate poses with Openpose Controlnets V1.1.
- AI Modelling Agency (Creative Bloq Article): An article discussing the emergence of AI modelling agencies.
- Free VRChat Avatars & 3D Assets (VRCMods): A collection of free VRChat avatars and 3D assets.
- AniTalker: A project focused on animating talking heads.
- VLOGGER: A project related to creating virtual vloggers.
- StoryDiffusion: A project exploring consistent self-attention for long-range image and video generation.
- SoccerNet Game State Reconstruction (GitHub Repository): A project focusing on athlete tracking and identification on a minimap.
- Ukraine’s AI Avatar for Consular Affairs (Reddit Post): A discussion on Ukraine’s use of an AI avatar for consular updates.
- Animating Images with Viggle AI (YouTube Tutorial): A tutorial on animating images using Viggle AI.
- PhysAvatar (Hugging Face Paper): Research on learning the physics of dressed 3D avatars from visual observations.
- Vid2Avatar: A project focused on reconstructing 3D avatars from videos.
- Human Tracking and SLAM Capture (YouTube Video): A demonstration of human tracking and SLAM capture technology.
- ComfyUI Character Turntable with SV3D (Reddit Post): A discussion on creating character turntables using ComfyUI and SV3D.
- Animating Characters for Free (LinkedIn Post): A post highlighting methods for animating characters for free.
- Midjourney Character Reference Feature (Medium Article): An article exploring Midjourney’s Character Reference feature.
- Full-Character Consistency with SDXL (Reddit Post): A discussion on achieving full-character consistency using SDXL.
- Create Bot Emotions (Miku.gg Documentation): Documentation on creating bot emotions within the Miku.gg platform.
- Consistent Emotions on a Character with ComfyUI (Reddit User): A Reddit user’s plans to publish a method for achieving consistent emotions on a character using ComfyUI.
- BakedAvatar: A project focused on avatar creation.
- Dreamtalk (GitHub Repository): The official implementation of Dreamtalk, focusing on expressive talking head generation.
- CharTurnerBeta LoRA (Civitai): A LoRA for multi-direction consistency in Stable Diffusion character generation.
- VividTalk: A project focused on one-shot audio-driven talking head generation.
- Gaussian-Based Avatars (Hugging Face Papers):
- Relightable Gaussian Codec Avatars: Research on using Gaussian codecs for avatar representation.
- Gaussian Head Avatar: Research on creating high-fidelity head avatars using dynamic Gaussians.
- NLW Education Discord Projects (Discord Channel): A Discord channel discussing projects related to ElevenLabs and character/avatar creation.
- D-ID AI Video Mobile App: A mobile app for creating AI videos.
- GAIA (Microsoft): A project from Microsoft exploring advanced avatar technologies. This meticulously curated collection offers a comprehensive overview of the dynamic field of digital human and avatar creation. Explore, learn, and contribute to the ongoing evolution of this exciting frontier! Note: Some links may lead to projects or resources that are still under development or experimental. Remember to review any licensing information before using code or assets from these projects.
-
-