This is a linkpost for https://makeavideo.studio/

Meta AI (Facebook) created a text-to-video model by taking a diffusion text-to-image model, adding temporal convolutional and attention layers, and fine-tuning it with video data (without text). They also use spatial and temporal super-resolution networks. Showing, to the surprise of no one who was paying attention, that our existing mostly homogeneous architectures can be easily extended to understand, to some extent, the structure of everyday reality. It's not the first text-to-video model, but it's much better than what came before.

New Comment
4 comments, sorted by Click to highlight new comments since: Today at 2:39 PM
[-]P.2y41

Emad from Stability AI (the people behind Stable Diffusion) says that they will make a model better than this.

I think this'd be better with a description of what it is and how it's relevant. (Linkposts generally benefit from that, and in this case it almost looks like spam if you're not paying attention)

[-]P.2y21

And here we have another one: https://phenaki.video/

[-]P.2y20

And a 3D one by optimizing a differentiable volumetric representation using 2D diffusion: https://dreamfusionpaper.github.io/