-
Seems that everyone expects this to break through this year.
-
Justine Moore from A16Z has compiled 2023 on Twitter.
Closed Source Video id:: 659a922a-1d6b-4ae8-82ad-8d7c2814f25f
Pika Labs
- Current leader: https://twitter.com/martial_artwork/status/1742138390517014918
- Prompt Creativity & Flexibility: Excels in this area, enabling users to directly influence the animation with their prompts.
- Human Motion Animation: Attempts adventurous animations but may result in distortions.
- Camera Motion Options: Offers accurate, straightforward camera motions but lacks the dynamic range of Runway ML.
- Pros: Free version (recently reduced quality), supports multiple aspect ratios, provides tutorials for prompt writing.
- Cons: Creations are visible to other users, potential for idea theft, and traffic issues on Discord server. Expensive to use through Pika Art website $60pcm,
Runway ML
- twitter link to the render loading below https://twitter.com/bennash/status/1746188870679400543
- Basic Animation: Offers cinematic camera movements and more convincing human motion, but faces issues with brightness and image integrity.
- Prompt Creativity & Flexibility: Less flexible in prompt creativity, occasionally disregarding user prompts.
- Human Motion Animation: Produces high-quality animations but sometimes distorts the original image.
- Camera Motion Options: Provides dynamic camera shots, including zooming, panning, and rotating, but may lead to distortion.
- Pros: Web-based platform ensuring privacy, offers 120 free credits, advanced features, and the option to extend video length.
- Cons: Limited to 16:9 aspect ratio, may not be as flexible as Pika Labs in prompt generation.
Mid Journey have said:
- Midjourney Video “will not be like any other AI video products that are currently available out there and will be 10X better.”
- David Holz: “MidJourney video may not be consistently making what you want, but the quality will be consistently good by default.”
- Video Training: The Midjourney team will start to train the video/animation model, which will come before 3D.
- Already have all the data needed to train the model.
- 3D: needs more data to train, so it’s a bit slower than expected.
VideoPoet – Google Research
- Overview: Google’s text to video, linked to Bard, but not yet available.
HeyGen for video avatars
- Overview: HeyGen emphasizes security and ethics in its AI video platform, being SOC 2 compliant and focusing on data protection.
- Notable Features: Known for its user-friendly interface and effectiveness in creating short, engaging videos useful for various departments like HR and training.
- Target Audience: Targets SMEs, offering a range of applications from casual to professional use.
Virtual production
Simulon (Virtual Production)
- Cloud rendered magic: Still early, and I’m not QUITE sure how it works.
https://twitter.com/diveshnaidoo/status/1735006300386336919
My flossverse stuff from 2022
https://twitter.com/flossverse/status/1629601804521537537
Skyglass
- Straight up virtual production on iPhone
https://twitter.com/skyglassapp/status/1712599252575412474
Adobe integrates everything to Premier
Other Notable Research
What’s next: 3D world creation
- Again, midjourney are working on a model. - 🟢 Best I can find is Sudo AII
- https://research.nvidia.com/labs/toronto-ai/AlignYourGaussians/
- Mosaic-SDF for 3D Generative Models (connectedpapers.com)
- https://lioryariv.github.io/msdf/
Voice to CAD like Tony Stark is obviously coming
Metaverse and Telecollaboration
- 🟢 I could go on all day about this, goods and bads. I literally wrote a book on it.
- 🟢 A lot (for me) hinges on OpenUSD the universal scene language. It’s been SO long since we have had something useful.
- Nvidia have a text to 3D pipeline for Omniverse. Will be interesting to see what the use cases are. This is their new Cesium [geo tile integration](https://cesium.com/blog/2024/01/16/now-available-[[NVIDIA Omniverse]]-aeco-demo-pack/) giving global instant models.
https://twitter.com/BlockadeLabs/status/1719818562917761094
- This is a presentation slide and the next slide is Open Generative AI tools