Animated Data Visualization of Washington D.C.’s warming temperatures

I love animated charts where the animation is central to the storytelling; where it adds value, rather than being simply fancy decoration.

When it’s done well, it can be extremely powerful, mesmerizing and compelling.

One of the best examples I’ve seen is Bloomberg’s animated temperature visualization. I love how powerfully this chart conveys its message, and I’ve been semi-obsessed with it ever since I saw it.

So, over the past few weeks and since my last experiment with the Google Visualization API, I’ve been chipping away at this project to create an animated temperature chart within the Google environment. It’s finally in a state I can share here, however it’s what you might call an MVP still…

Here’s my animated temperature chart built with the Google Visualization API connecting to Washington D.C. data in a Google Sheet:

Animated temperature chart using Google Chart API

You can see this animated chart in all its glory here.

The original Bloomberg temperature chart

Here’s the original animated visualization from Bloomberg – one of my favorite visualizations – that inspired this whole exercise.

I find it mesmerizing to watch and tremendously effective at illustrating the trend of rising temperatures.

I make no claim that my chart is a patch on Bloomberg’s sumptuous, smoothly animated chart with its incredible attention to detail (click through to the live version to see what I mean).

Hopefully though, my version captures some of the same spirit and demonstrates how powerful Google’s Visualization library can be.

Another example: NASA’s animated chart

I saw this chart from NASA show up in my Twitter feed recently and I couldn’t help but smile in appreciation.

Another beautiful, animated chart showing long-term temperature trends. This arrived as a timely dose of inspiration to push me to persevere with my own attempt.

NASA released this last month, showing that “August 2016 was the warmest August in 136 years of modern record-keeping”. Wowser!

I wonder if the authors of this chart were similarly inspired by the Bloomberg chart?

Creating an animated temperature chart with Google Sheets and the Google Chart API

Ok, so is it possible to create something similar with data in a Google Sheet?

Yes, but it involves coding with Google’s Visualization API.

The data is the same that I used for my other Google Visualization API project, which I documented here.

It comes from the University of Dayton: http://academic.udayton.edu/kissock/http/Weather/citylistUS.htm

I’ve put all of the data cleaning/wrangling in my previous post, so I won’t repeat it here.

Suffice to say, I did create a new pivot table with average monthly and average annual temperatures. Here’s the data in a Google Sheet, ready to go. It shows the average monthly temperature for each year from 1995 to 2016, along with an annual average.

Onwards with the Google Visualization API…

After plenty of head-scratching, reading documentation and trying different examples, I finally managed to create an initial, working prototype that cycled through the temperature data in my Google Sheet, comparing a baseline (1995) to each successive year:

First working prototype of the animated temperature chart

It was a start, even if it didn’t show much!

After another bout of head-scratching and staring gormlessly into space, I was a little closer:

Animated temperature chart progress

After another bout of…you get the gist.

I added the year in the top left and removed the legend. Also, you’ll notice in this next GIF that I’ve got the Chrome Dev Tools open so I can see the objects being created at each iteration of the loop. This helped me massively when figuring out how to build the Series object dynamically.

Animated temperature chart progress

Getting much closer now!

The last major hurdle was building the series options object dynamically within the loop, so I could control the colors of each series.

This allowed me to set the most recent monthly data to red, the most recent annual average to grey, and then have everything else fade into the background with muted colors.

After a final few tweaks, I had the chart I was after:

Animated temperature chart using Google Chart API

I’m not going to go through the code step-by-step, but it’s all here on GitHub.

I will however highlight a few interesting parts of the code, which were key to making this work.

Very briefly, the code collects all of the data from the Google Sheet and initially shows only the first two columns of data, the 1995 average line and the 1995 monthly line.

I used the DataView class to show only specific columns of data in the chart, e.g. the initial chart view is created from the data in this snippet:

var data = response.getDataTable();
var chart = new google.visualization.LineChart(document.getElementById('chart_div'));
var view = new google.visualization.DataView(data);

// set columns initially
view.setColumns([0,1,2]);
...

Then I loop through the remaining columns, adding them one at a time to the view and refreshing the chart on each iteration:

function updateChart() {
    if (i < howManyTimes ){
      arr.push(i+2)
    }
    view.setColumns(arr);
...

The color of the lines is controlled by the options object, which I build dynamically within each loop. Modulus arithmetic is used to distinguish odd and even columns (to distinguish between monthly and annual data) and color them accordingly:

for (var j = 0; j < i; j++) {
  var innerObj = {};
  if ((j % 2) == 0) {
    innerObj["color"] = '#e9e9e9';
    innerObj["lineDashStyle"] = [4,1];
  }
  else {
    innerObj["color"] = '#ffdbdb';
    innerObj["lineDashStyle"] = [0,0];
  }
  seriesObj[j] = innerObj;
}
...

The other key feature is a timer that refreshes the chart on every iteration of the loop, with an 80 millisecond buffer:

if( i < howManyTimes ){
    setTimeout( updateChart, 80 );
}

The full code can be seen here on GitHub.

Documentation

Should you want to create something with the Chart API yourself, here's some resources relating to the techniques discussed above:

Google Visualization API reference
DataTable class
DataView class
Line chart examples with the Visualization API
Animation examples with the Visualization API
And finally, the Google Visualization API Forum

Recreating Tufte’s famous weather chart with Google Sheets and Charts API

My Twitter feed is mostly populated with data-related resources and people, so I almost always find something new and shiny going on there to distract me.

Sometimes this can be a good thing though (and sometimes, it can be the best thing ever).

Take last Friday. I found myself procrastinating, browsing my home feed when I saw this tweet for the upcoming Microsoft Data Science summit:

Tufte temp chart tweet

What caught my eye was not the actual tweet, but the sumptuous weather visualization that I recognized as one of Edward Tufte’s famous creations.

I clicked through to the article, which linked to Tufte’s original chart (here) and a fantastic tutorial about re-creating the chart using R, from Brad Boehmke (which is actually the chart shown in the tweet above).

It looks at daily average temperature over a period of time and captures the historic min/max range, the normal range (95% confidence interval), the 2014 temperature line and picks out the highs and lows. It renders all of this information in a single, beautiful column/line combination chart.

Curiousity piqued, I wondered whether this would be possible with Google Sheets?

The short answer is….. Continue reading Recreating Tufte’s famous weather chart with Google Sheets and Charts API

Build time-based triggers in Apps Script projects to automate programs

Ever find yourself needing to repeat an action in your Google Sheets?

For example, on a recent client project I wanted to cycle through my spreadsheet data, compare it to another dataset and apply a matching algorithm. To do this and stay within the 6 minute execution limit for Apps Script, I split my data into “blocks” of 10 rows, counted how many blocks I had and then ran the matching algorithm once for each block until they’d all been processed.

By adding time-based triggers, I setup my Google sheet to do this automatically, rather than having me click a button each time.

Using Apps Script (read my getting started guide here), you can add these time-based triggers programmatically, so they can run and stop automatically.

In this post I’m going to show a skeleton example of this time-based architecture, to which you can add your own functionality.

The entire code file is here on GitHub and the spreadsheet is available here (feel free to make a copy: File > Make a copy… and then you can open the script editor to modify the code).

Time-based triggers with Google Apps Script

In this simple example, I’m going to output a random number in a new line of my spreadsheet every minute until I’ve looped through a set number of times (2 in this case). It’s deliberately basic as the focus is on the trigger/timing architecture.

Here’s a screencast of this simple program in action (speeded up):

Auto trigger loop

Behind the scenes, everything is running on autopilot once I’ve clicked that first Run button.

Continue reading Build time-based triggers in Apps Script projects to automate programs

Creating a d3 chart with data from Google Sheets

In the following image, I have two browser windows open. On the left, a Google Sheet containing a simple table of data. In the right browser window, a web page with a d3 chart that displays the data from the Google Sheet and can be refreshed dynamically when the data changes.

d3 + Google Sheets

Continue reading Creating a d3 chart with data from Google Sheets

Save time with this custom Google Sheets, Slack & Email test scoring bot

You’ve marked your students test scores and recorded it all in a beautiful Google Sheet.

You’ve dotted your i’s and crossed your t’s, checked your spelling and made sure all your scores are ready to go.

Now comes the tedious part, copying and pasting each student’s scores and feedback into an email or Slack message to send back to them.

Wait, there’s a better way!

With a little bit of upfront effort using Google Apps Script, you can build yourself a custom Google Sheets Slack/Email app that can automate that whole final step for you.

Besides being useful, it’s also pretty fun to post stuff from your Google Sheet direct to Slack!

Send data from Google Sheets to Slack
Continue reading Save time with this custom Google Sheets, Slack & Email test scoring bot