I Let OpenAI Operator Manage My Google Sheet: Here’s What Happened.

OpenAI recently launched their agent tool called Operator. It’s an AI powered tool that can navigate the web and perform tasks for you using its own browser. It can click, type, scroll, and generally interact with web pages the way you or I would.

Naturally, I tested it out with a Google Sheets project to see how it fared. Specifically, I asked it to complete 3 tasks for me:

  1. Create a pivot table
  2. Create a chart
  3. Write a script to email a pdf copy of the Sheet to me

Here’s how it got on:

OpenAI Operator in action

I pasted the URL of my Google Sheet into the Operator chat window and asked it to get to work.

It opened the Sheet in a Linux browser within Operator and then handed control to me to login to the Sheet. Once that was complete I handed control back to Operator to work it’s magic.

It was wild watching the computer take control of my Sheet. I watched it open menus, insert objects, and write code.

The craziest moment was when it made a mistake with the pivot table and self-corrected 🤯.

Well, that, or when it opened my Apps Script editor and started coding!

But it wasn’t all plain sailing.

It got itself completely stuck when it was trying to save the script it wrote. It kept pressing the “Undo” button and, eventually, I had to step in and tell it where the “Save” button was.

I felt like I was like working with a digital assistant.

I’m excited for this to go fully multimodal. Imagine working through this scenario with voice activation instead of using a chatbot.

We overestimate the short-term change, but underestimate the long-term change. It feels like we’re moving pretty darn fast at the moment 😯.

Operator Availability

At the time of writing (27th January 2025) Operator is available to OpenAI Pro users in the U.S..

Can AI Studio teach me how to build a pivot table?

January 2025

Hardly a week passes at the moment without some technology announcement that makes me go “wow!”.

This week it’s Google’s AI Studio that has been impressing me, offering a glimpse into what the future of software education could look like.

What’s different with this tool is that it lets you share your screen and have a verbal conversation with the AI model (in this case Gemini), so it can walk you through a problem.

In my case, I used it to walk me through building a pivot table. And whilst it didn’t get everything correct (yet!), it’s impressive and feels like you’re talking to an assistant, not your computer.

How to use AI Studio to teach you

  • Go to https://aistudio.google.com/
  • Sign in with your Google account. It’s free to use. When you log in, the homepage looks like this:
Google AI Studio homepage
  • Click on the “Stream Realtime” in the left menu (shown by the red arrow).
  • Share your screen and have a conversation with AI Studio! For example, you could say:

    “I want you to walk me through building a pivot table that summarizes the total sales price by property type.”

    (And say it out loud, don’t type it in.)
  • Because the Pivot Table once existed under the data menu, AI Studio initially guided me there first (the only serious mistake it made). I said “I don’t see the pivot table option under the data menu, is it under one of the other menus?”

    So, don’t hesitate to interject if you can see it making a mistake.
  • Once you have a basic pivot table, you could ask for help sorting the data or adding another category.
  • Another great use case is asking it to explain lines of code to you. For example, share your screen showing a code block and try something like this “can you explain what the code on lines 52 – 56 does?”

It’s early days, so it’s far from perfect. It does make mistakes so you can’t rely on it blindly. You still need to cultivate your own knowledge.

But it’s a glimpse into what’s around the corner when we have infinitely patient AI assistants at our beck and call. And I think that’s a bright future, where we can be dramatically more efficient, focused on insights and outcomes, not code syntax or formula issues.

What do you think?

AI Experiments: Using Sora AI to Generate Videos

One of my goals this year is to experiment with AI tools and have fun doing it. This is the first in the series, using Open AI’s Sora text-to-video tool to create a fictional advertisement for Polar Explorers back in the polar heyday:

The sheer number of daily AI announcements and new tools feels overwhelming at the moment.

One way to cut through this noise is to set yourself an AI project and see it through to the end.

Whether you use one tool or 10, and whether it’s the latest this-or-that or just run-of-the-mill ChatGPT, it doesn’t matter. What matters is that you complete something with GenAI. In the process, you’ll learn how to use these tools, what they’re good at, and what they’re not good at. And in the process, you’ll keep yourself relevant in the future.

For this experiment, I wanted to learn about creating videos using GenAI, and specifically, OpenAI’s text-to-video tool: Sora AI.

Background

The inspiration for this project came from this article at the end of last year, about Coca-Cola creating an entire ad using AI tools and how the project was more like developing software than a traditional film shoot.

I was also reading Shackleton’s biography at the time.

In 1914, Sir Ernest Shackleton was recruiting men to join his boldest Antarctic expedition: the Imperial Trans-Antarctic Expedition.

The expedition is famous because the ship Endurance was trapped in the ice and sank, leaving 28 men stranded on the sea ice with no hope of rescue. The men escaped via a perilous small boat crossing of the Southern Ocean, in what has become one of the greatest survival stories of all time.

Shackleton posted this famous, likely apocryphal, newspaper advertisement to recruit sailors and explorers for his expedition and received over 5,000 responses.

Ad mockup generated by The Newspaper Clipping Generator

It’s often touted as an example of good copy, although the original has never been found and it seems the general consensus is that it’s not authentic.

Polar Ad for the TikTok Generation

I decided to try recreating this ad as a video for the TikTok generation, with the help of AI. In other words, imagine if Shackleton had TikTok and YouTube back in the day, what would his ad look like?

Here’s the result:

And, of course, the TikTok generation would most likely watch it in portrait orientation on their phones, so I uploaded to YouTube shorts here (since I’m not on TikTok).

How This Video Was Made

Sora to generate videos

I used OpenAI’s new video tool, Sora, to generate videos based on prompts.

Sora AI text-to-video tool

For example, to get the video of the gentleman reading his paper wearing a bowler hat, I used this prompt:

A gentleman in a finely tailored suit, wearing a polished top hat, strides confidently along New Burlington Street. The street is bustling with horse-drawn carriages, women in Edwardian dresses, and men in similar formal wear. The architecture is early 20th century, with brick buildings lining the street. The scene is lively and captures the essence of London in 1914.

And to get the polar explorers on the ice, I used this prompt:

5 edwardian polar explorers walk slowly across a wind-swept ice cap with snowy mountains in the distance. The 5 men are dragging a heavily laden sledge behind them. We are behind them looking at their backs.

ElevenLabs to generate voices

ElevenLabs is a tool that converts text to speech, with a wide variety of voices to play with.

I inputted the text from the ad:

“Men wanted for hazardous journey. Low wages, bitter cold, long hours of complete darkness. Safe return doubtful. Honour and recognition in event of success.”

And then experimented with various old-fashioned voices to get the tone I was looking for.

Suno to generate background music

To create background music, I used a tool called Suno.

Suno generates music based on a song description and optional settings like whether it’s instrumental or not.

Video Production

I’ve been making course videos for years, so I used the same tools to bring the different Sora clips together into a single video.

Steps:

  • Download the assets from the AI tools above
  • Import into Screenflow
  • Add video and audio clips into timeline and trim as needed
  • I added a vignette and vintage filter to the videos to make them look older
  • Export
  • Using Handbrake to shrink the video file size
  • Upload to YouTube
ScreenFlow workflow

I had some fun creating this and learnt some new skills along the way.

And, although the videos are very obviously AI generated, it’s incredibly impressive that all this was generated from a few paragraphs of text.

Just image where this leads… it’s not hard to picture a world where you ask Netflix to generate a new Jurassic Park movie set in the Caribbean featuring Dwayne Johnson. And you come back half an hour later and it’s ready for you to watch.

We live in wild times!