lllyasviel/ControlNet: Let us control diffusion models!
GitHub
lllyasviel/ControlNet: Let us control diffusion models!: Let us control diffusion models! Contribute to lllyasviel/ControlNet development by creating an account on GitHub.
https://github.com/lllyasviel/ControlNet
The ControlNet project provides a way to control diffusion models. It includes a number of features to help with this, including the ability to automatically download annotators and the ability to shift the guess mode to UC disconnect in order to save memory.
Controlnet and character posing reddit post
https://www.reddit.com/r/StableDiffusion/comments/11owo31/something_that_might_help_ppl_with_posing/
Stable Diffusion Art
https://stable-diffusion-art.com/controlnet/
The Twitter conversation includes an AI-generated video created by TomLikesRobots using canny edge detection and EbSynth technology to make a hobbit and Dumbledore contending with Sauron’s will. Many people were impressed with the video and asked questions about the software and hardware used to create it. Finally, TomLikesRobots answered the questions by telling them he was using Windows 11, an RTX 3080 with 10GB VRAM and that the img2img part with ControlNet can be run online, but EbSynth might be trickier. https://twitter.com/TomLikesRobots/status/1627073211656732676
The text is a collection of video tutorials by Albert Bozesan, demonstrating how to use AI to create various types of digital art and design, including movie and game titles, vector graphics, pixel art, 3D assets, fantasy maps, motion graphics assets, and seamless textures. Bozesan uses a software called Stable Diffusion, which is free and runs on NVIDIA GPUs. He also mentions other AI tools such as ControlNet and Dream Textures for specific tasks. The videos show step-by-step instructions on how to install and use the software, as well as practical tips and techniques for achieving different visual effects. In addition, there are videos covering the latest AI research by NVIDIA and Google, as well as challenges with other artists using AI. The text also includes a note about the use of cookies and data on YouTube, as well as options for managing privacy settings. https://m.youtube.com/watch?v=sNcEhR65pw0&feature=youtu.behttps://m.youtube.com/watch?v=sNcEhR65pw0&feature=youtu.be
This is a conversation between TomLikesRobots about an experiment with “Ebsynth and Controlnet Img2img”. TomLikesRobots has tried different noise percentage values and blends. The experiment involves switching out Elizabeth Moss for Penelope Cruz, and despite some difficulties, the team is making progress towards high-detail photorealism. https://twitter.com/TomLikesRobots/status/1628104009146826763?s=20
The paper proposes a novel approach to generate new video content by combining zero-shot text-to-video generation with ControlNet. The method takes multiple sketched frames as input and generates video output that matches the flow of these frames. By incorporating ControlNet to enable additional input conditions, the approach leverages the benefits of both zero-shot text-to-video generation and the robust control provided by ControlNet. Experiments demonstrate that the method excels at producing high-quality and remarkably consistent video content that accurately aligns with the user’s intended motion for the subject within the video. A demo of the approach is also available for users to try out. https://sketchingthefuture.github.io/