Meta announces Movie Gen AI video and audio generator

Meta has announced its latest AI technology, Meta Movie Gen. Meta Movie Gen can generate video and audio from text prompts. The technology can also customize your own videos and create audio files.

What can Movie Gen do?

Meta Movie Gen is a new family of generative AI models that can create 1080p HD video files from your text prompts. It also offers precise instruction-based video editing and can personalize your existing images and videos. Meta’s research paper on the AI generator explains that the technology is currently limited to generating videos up to 16 seconds long playing back at 16 frames per second. In addition, the AI can create background music and sound effects synchronized to the video content.

Videos from text

Meta says that Movie Gen can produce unique videos from text prompts to create a custom masterpiece. One of the sample videos on the website used the phrase, “A fluffy koala bear surfs. It has a grey and white coat and a round nose. The surfboard is yellow. The koala bear is holding onto the surfboard with its paws. The koala bear’s facial expression is focused. The sun is shining.” It’s a very detailed description and the video appears to incorporate all of the different aspects of the prompt.

Personalized videos

With Movie Gen you can also upload an image of yourself and transform it into a personalized video. Meta says that Movie Gen’s cutting-edge model creates videos that preserve human identity and motion. Again, the text prompts are very detailed such as, “A woman DJ spins records on a rooftop in LA. She is wearing a pink jacket and giant headphones. There is a cheetah next to the woman. The background is a cityscape.”

Transform your videos

Movie Gen can transform existing videos with your text inputs. Meta says that the AI enables precise video editing, from styles and transitions to “fine-grained edits.” One visually striking sample video shows a series of video clips of a man running. Movie Gen then adds blue pom poms to his hands, turns the landscape into a desert and even dresses the man in a dinosaur suit.

Sound effects and music

Movie Gen can generate audio from your videos from video and text inputs. Meta says you can also create and extend sound effects, background music or entire soundtracks. The company has shared several sample videos with music. In addition, one impressive sample shows an AI-generated video of a skateboarder with accurately synchronized sound effects. The audio text prompt was, “Wheels spinning, and a slamming sound as the skateboard lands on concrete.”

What we think

Companies will always share the best videos when advertising a new generative AI technology. So, you don’t get to see all the times it didn’t quite work out. However, Meta’s Movie Gen samples are certainly eye catching. The company also claims that Movie Gen outperforms rival products such as Runway Gen3, LumaLabs, Kling1.5 and OpenAI Sora on subjective metrics including motion naturalness and realness. Meta says it used 100 million videos and one billion images from the internet to train Movie Gen. Unfortunately, the company hasn’t specified if this was licensed or public domain content. As a result, the concerns over the material being used to train generative AI are set to rumble on.

Pricing and availability

Meta says that Movie Gen will be launched on Instagram in 2025.

Pete Tomkies
Pete Tomkies
Pete Tomkies is a freelance cinematographer and camera operator from Manchester, UK. He also produces and directs short films as Duck66 Films. Pete's latest short Once Bitten... won 15 awards and was selected for 105 film festivals around the world.

Related Content