OpenAI: new feature transforms text into 60-second video

Google announced that it will release version 1.5 of Gemini for developers and business users. Not to be outdone, one of Google’s biggest competitors – OpenAI – also made a big announcement about AI. However, this announcement involves a new text-to-video AI model.

OpenAI: new feature transforms text into 60-second video

In a blog post and later on social media, OpenAI revealed a new text-to-video generative AI model called Sora. The ad is accompanied by clips created by the software, ranging from a Chinese Lunar New Year celebration to an animated monster.

Introducing Sora, our text-to-video model.

Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. https://t.co/7j2JN27M3W

Prompt: “Beautiful, snowy… pic.twitter.com/ruTEWn87vf

— OpenAI (@OpenAI) February 15, 2024

OpenAI states that Sora is currently being made available to some teams to “assess critical areas for damage or risk.” These teams include experts in areas such as disinformation, hateful content and bias. In addition to these tests, Sora will also be subjected to the security measures that exist for DALL-E 3. The company adds that it is working on tools to help detect whether a video was generated by Sora.

While others like Pika and Stability AI have surpassed OpenAI when it comes to AI video generation, there are a few things that make Sora stand out. For one, Sora can create up to 60 seconds of video, while competitors only manage around four seconds. Then there is the sharpness, resolution and precision of the surrounding world.

https://t.co/SOUoXiSMBY pic.twitter.com/JB4zOjmbTp

— Sam Altman (@sama) February 15, 2024

There are more than 35 examples that can be consulted on the OpenAI website. Although the results are impressive, the model is far from perfect. In fact, the company admits this.

The current model has weaknesses

May have difficulty accurately simulating the physics of a complex scene and may not understand specific instances of cause and effect. For example, a person may bite into a cookie, but then the cookie may not have a bite mark.

The model may also confuse spatial details of a message, for example, mixing left and right, and may have difficulty with accurate descriptions of events that occur over time, such as following the trajectory of a specific camera.
You can see an example of this in the first video presented on the blog. The video shows a woman walking through Tokyo. If you watch closely, you’ll notice that the woman’s legs shift or stutter occasionally, her feet slide across the floor, and her clothes and hair change near the end.

Although Sora is not available to the general public, Executive Director Sam Altman has been accepting suggestions from X (formerly Twitter) users.