Loading the weather dataset

First things first, we learn how to load our dataset and look at the structure.

Project Source Code

Get the project source code below, and follow along with the lesson material.

Download Project Source Code

To set up the project on your local machine, please follow the directions provided in the README.md file. If you run into any issues with running the project source code, then feel free to reach out to the author in the course's Discord channel.

  • |

Lesson Transcript

  • [00:00 - 00:11] So the first step to visualizing any dataset is to understand the structure of the data. So we're going to start by loading up our data file into our JavaScript file.

  • [00:12 - 00:20] You can see right here, our data is all in this MyWeatherData.json file. We can actually just open this up and look at it.

  • [00:21 - 00:36] We can see by the extension that.json extension, it's a JSON file which stands for JavaScript Object Notation. You can see this is a lot like what an array with objects looks like in JavaScript.

  • [00:37 - 00:53] So this first day goes all the way down to here and this is one day worth of data for January 1st, 2018. You can see we actually have a lot of data for every day, but this is pretty hard to look at.

  • [00:54 - 01:05] So let's go ahead and load it into our JavaScript file. So let's create a variable, const data equals, and then we can use d3.json.

  • [01:06 - 01:20] So this is just a method that d3 has that will load up a JSON file. There's also d3.csv for comma separated value formatted file.

  • [01:21 - 01:26] That's just going to be a file where the data is stored in a different format. But this one's JSON.

  • [01:27 - 01:44] So we have this handy shortcut and then we're going to type the name, so MyWeatherData.json. We're not getting errors anymore, so let's log out what is data right now.

  • [01:45 - 01:51] So over here, we can see that we're not seeing our array. Instead, we're seeing this promise.

  • [01:52 - 02:02] And that's because d3.json returns a promise. So we're going to have to add the await keyword here.

  • [02:03 - 02:11] If you're unfamiliar with promises or async await, don't get to bog down right now. We'll add some resources for you down below.

  • [02:12 - 02:34] Basically, all you need to know is await makes our script wait until this JSON file has been parsed and can be used so that on the next line, anytime we use this variable data, we can be sure that it does exist. So over here, we can see all of the objects.

  • [02:35 - 02:42] This is a pretty long array. I think it's a year of data, so this is going to be an array with their Injourn65 objects in it.

  • [02:43 - 02:59] So if we open it up and we look at this first object, we can see all of the different metrics in our data set. So just a quick glance at these other items in the array, we can see that they're all similar.

  • [03:00 - 03:10] So you don't have to worry about normalizing our data set. And if we take a little bit closer look, we can see here's a time variable.

  • [03:11 - 03:17] So this is a JavaScript timestamp. It's just a bunch of numbers.

  • [03:18 - 03:26] And if we parse that using the date class, we're going to be able to get a JavaScript date time. We'll go more into that later.

  • [03:27 - 03:35] We have a summary string. It's mostly cloudy throughout the day on this day, wish it were it is.

  • [03:36 - 03:50] And then we can also see the dark say API has an icon metric, probably for their own internal purposes. For visualizing each day, they probably have an icon set.

  • [03:51 - 03:56] So they have this partly cloudy day icon. Probably won't be of much use to us.

  • [03:57 - 04:05] There's all these different times. So we have sunrise time, sunset time, the temperature, high time, all of these different times.

  • [04:06 - 04:15] And we can see there are all these timestamp numbers. But there are a few interesting metrics in here, like moon phase.

  • [04:16 - 04:26] So we're most of the way through almost to a full moon. I'm assuming precipitation intensity, the probability of precipitation.

  • [04:27 - 04:35] So zero percent chance that it will rain. And that is very not intense.

  • [04:36 - 04:40] Temperature low, dew point, humidity. There's a lot of good stuff in here.

  • [04:41 - 04:53] And then down here we can also see that the date is stored as a string. So instead of one of these timestamps, we have year dash month dash day.

  • [04:54 - 05:02] So this is January 15, 2018. D3 has some methods for parsing something like this.

  • [05:03 - 05:11] We'll just have to tell it what is the format that we're putting in. And we'll go into that a little bit more later.

  • [05:12 - 05:25] So I encourage you to spend a few minutes just going through one of these objects and kind of familiarize yourself with what's here. And maybe write down a few questions of predictions that you might make.

  • [05:26 - 05:43] Like, precipitation intensity will always be zero if the probability is zero or the temperature high correlates with the temperature low. Maybe some more interesting things like maybe does humidity correlate with pressure.

  • [05:44 - 05:50] So if you write down a few questions that you have now, we'll have a chance to dig more into them a little bit later.