AI Experiments: Using Sora AI to Generate Videos

One of my goals this year is to experiment with AI tools and have fun doing it. This is the first in the series, using Open AI’s Sora text-to-video tool to create a fictional advertisement for Polar Explorers back in the polar heyday:

The sheer number of daily AI announcements and new tools feels overwhelming at the moment.

One way to cut through this noise is to set yourself an AI project and see it through to the end.

Whether you use one tool or 10, and whether it’s the latest this-or-that or just run-of-the-mill ChatGPT, it doesn’t matter. What matters is that you complete something with GenAI. In the process, you’ll learn how to use these tools, what they’re good at, and what they’re not good at. And in the process, you’ll keep yourself relevant in the future.

For this experiment, I wanted to learn about creating videos using GenAI, and specifically, OpenAI’s text-to-video tool: Sora AI.

Background

The inspiration for this project came from this article at the end of last year, about Coca-Cola creating an entire ad using AI tools and how the project was more like developing software than a traditional film shoot.

I was also reading Shackleton’s biography at the time.

In 1914, Sir Ernest Shackleton was recruiting men to join his boldest Antarctic expedition: the Imperial Trans-Antarctic Expedition.

The expedition is famous because the ship Endurance was trapped in the ice and sank, leaving 28 men stranded on the sea ice with no hope of rescue. The men escaped via a perilous small boat crossing of the Southern Ocean, in what has become one of the greatest survival stories of all time.

Shackleton posted this famous, likely apocryphal, newspaper advertisement to recruit sailors and explorers for his expedition and received over 5,000 responses.

Ad mockup generated by The Newspaper Clipping Generator

It’s often touted as an example of good copy, although the original has never been found and it seems the general consensus is that it’s not authentic.

Polar Ad for the TikTok Generation

I decided to try recreating this ad as a video for the TikTok generation, with the help of AI. In other words, imagine if Shackleton had TikTok and YouTube back in the day, what would his ad look like?

Here’s the result:

And, of course, the TikTok generation would most likely watch it in portrait orientation on their phones, so I uploaded to YouTube shorts here (since I’m not on TikTok).

How This Video Was Made

Sora to generate videos

I used OpenAI’s new video tool, Sora, to generate videos based on prompts.

Sora AI text-to-video tool

For example, to get the video of the gentleman reading his paper wearing a bowler hat, I used this prompt:

A gentleman in a finely tailored suit, wearing a polished top hat, strides confidently along New Burlington Street. The street is bustling with horse-drawn carriages, women in Edwardian dresses, and men in similar formal wear. The architecture is early 20th century, with brick buildings lining the street. The scene is lively and captures the essence of London in 1914.

And to get the polar explorers on the ice, I used this prompt:

5 edwardian polar explorers walk slowly across a wind-swept ice cap with snowy mountains in the distance. The 5 men are dragging a heavily laden sledge behind them. We are behind them looking at their backs.

ElevenLabs to generate voices

ElevenLabs is a tool that converts text to speech, with a wide variety of voices to play with.

I inputted the text from the ad:

“Men wanted for hazardous journey. Low wages, bitter cold, long hours of complete darkness. Safe return doubtful. Honour and recognition in event of success.”

And then experimented with various old-fashioned voices to get the tone I was looking for.

Suno to generate background music

To create background music, I used a tool called Suno.

Suno generates music based on a song description and optional settings like whether it’s instrumental or not.

Video Production

I’ve been making course videos for years, so I used the same tools to bring the different Sora clips together into a single video.

Steps:

  • Download the assets from the AI tools above
  • Import into Screenflow
  • Add video and audio clips into timeline and trim as needed
  • I added a vignette and vintage filter to the videos to make them look older
  • Export
  • Using Handbrake to shrink the video file size
  • Upload to YouTube
ScreenFlow workflow

I had some fun creating this and learnt some new skills along the way.

And, although the videos are very obviously AI generated, it’s incredibly impressive that all this was generated from a few paragraphs of text.

Just image where this leads… it’s not hard to picture a world where you ask Netflix to generate a new Jurassic Park movie set in the Caribbean featuring Dwayne Johnson. And you come back half an hour later and it’s ready for you to watch.

We live in wild times!

Leave a Reply

Your email address will not be published. Required fields are marked *