Visualizing Climate Change with Python

In a previous edition of The Analyst, we explored the power of Python’s Pandas library for data analysis. Now, we’re diving into the visualization and analysis of climate change trends—a topic of growing importance to financial institutions and the investment industry. This article focuses on transforming and visualizing data to unlock insights during exploratory data analysis (EDA).

Effective EDA uncovers hidden patterns and trends and highlights potential issues within data sets, making it an essential step in any analysis workflow. Additionally, we’ll spotlight the JSON format—a cornerstone for storing and sharing web-based data and a key format for representing climate data.

Visualizing Toronto’s temperature

We’ll start by analyzing Toronto’s temperature trends over time. The first step is identifying a reliable data set or an effective data application programming interface (API). One such option is the Meteostat API, which provides access to average temperature data for specific latitude and longitude coordinates.

Figure 1:

If we are interested in the average temperature (tavg), we note that some datapoints are unavailable (NaN), so start by cleaning the data set. We could drop all entries where tavg is NaN data, but say we want to go as far back as possible. We will instead fill NaN with the average of tmin and tmax (an admittedly imperfect but reasonable enough estimation) and do a quick line plot.

Figure 2:

With Python, we can easily compare temperatures of different regions by visualizing them on a map. These tools can be an invaluable way to understand the structure for highly complex JSON data

Seasonal data

The above plot is complex to decipher but already displays one of the common challenges when analyzing time series data: seasonality. To account for it properly (i.e., to compare apples to apples), we will perform some light data manipulation. These are the steps:

1) Reset index so that the “time” column becomes just another column, then parse the year and month of each date into separate columns.

2) Group by Year, Month pairing. We will average the grouped values.

3) Select only the year, month, and tavg data points. Drop the rest.

Figure 3:

 

Below, for each month, we plot the monthly average over the last 80 years.

Figure 4:

This analysis is clearer and appears to showcase some upward trend for each month. However, the chart looks very busy. With Python, we can easily create subplots and isolate the data series of each month, all with a few lines of code. To make our results more conclusive, we will also add a line of best fit through each subplot with the help of the Numpy library.

Figure 5:

Another popular and compact way to visualize temperature changes is to use a heat map. We will rely on Seaborn, yet another popular library for visualization that is built on top of matplotlib and integrates very well with Pandas dataframes.

Figure 6:

 

While more definite conclusions should be supported by deeper statistical analysis, the above visualizations showcase Toronto’s rising temperature trends over the years.

 

Visualizing data on interactive maps

 Suppose we want to compare Toronto’s temperature to other regions worldwide. We could overlay even more data series onto the above charts, but let us take advantage of another type of visualization: interactive maps.

With Python, we can easily compare temperatures of different regions by visualizing them on a map. Several libraries allow us to do this, namely Folium, Leaflet, and Mapbox. We will focus on Folium, which allows for easy overlay of labels and markets  on a map using Python.

Here, we will start with using the Folium library to overlay the temperature of several major North American cities, once again using Meteostat to pull the underlying temperature data.

Figure 7:

Unlike tabular data, climate data is often stored in GeoJSON format—a structure designed to capture geographic information such as weather patterns, temperature zones, and flood risks tied to specific coordinates. In the final section of this article, we will delve into effectively utilizing GeoJSON for mapping and data analysis.

JSON supports various data types, including strings, numbers, arrays, and nested objects, allowing for the representation of complex data structures.

 

JSON and GeoJson data.

Financial analysts are no strangers to tabular data, which is arguably the most common method of representation (e.g., Excel, SQL databases) and often the first thing that comes to mind when people think of “data”. However, the data is often returned in JSON format when working with web APIs. JSON (JavaScript Object Notation) is a widely adopted format for exchanging data between servers and web applications. It organizes information as “key-value” pairs, making it both human-readable and easily parsed by machines. JSON supports various data types, including strings, numbers, arrays, and nested objects, allowing for the representation of complex data structures. This flexibility makes JSON a go-to format in modern web development and data science workflows.

For location-based data, a standard extension of JSON is GeoJSON, a specialized format designed to store geographic and spatial information. GeoJSON builds upon the JSON structure by introducing standardized keys such as “type”, “geometry”, and “ properties”, which describe spatial features like points, lines, and polygons. These features can represent anything from a single location (e.g., a city) to extensive geographic regions (e.g., countries or natural landscapes). GeoJSON is especially useful for visualizing and analyzing geographic data in mapping tools and Geographic Information System (GIS) applications.

One of GeoJSON’s key advantages is its compatibility with a wide range of tools, including Python libraries such as Folium. By combining the structured nature of JSON with the spatial focus of GeoJSON, we can seamlessly exchange and analyze geographic data across diverse systems.

Suppose we are interested in mapping and comparing global temperatures across countries worldwide. To do this, we first need to overlay country borders, available in GeoJSON format at the URL below.

We can use Python’s Requests library to retrieve the data and return it in JSON format. JSON data can be complex, and navigating through it in Python may initially seem challenging (we’ll demonstrate this below). To help with this, we also include an image of the tree structure generated using an online JSON viewer (https://jsoneditoronline.org/). These tools can be an invaluable way to understand the structure for highly complex JSON data.

Figure 8:

Remember that JSON stores data in a hierarchical, tree-like structure, which can be represented as a nested dictionary in Python. Let’s unpack it to understand the data better.

At the first layer, we see “type” and “features“. This is a common way to conceptualize GeoJSON data, as GeoJSON is structured around a FeatureCollection, which contains multiple features, each representing a spatial object like a point, line, or polygon. Each feature typically has:

  1. Geometry: Describes the shape (e.g., Point, LineString, Polygon) and coordinates.
  2. Properties: Key-value pairs that describe metadata or attributes associated with the geometry (e.g., country names, population).

We get the “features” data set via below and see that it is a Python list of length 241. This aligns with what we saw in the image above.

Figure 9:

Referencing the first value of the list, we see that it is yet another dictionary with three key-value pairs: “type“, “properties“, and “geometry“, just as shown in the image above. We would be particularly interested in the “name” of the country, which is likely under “properties”, and the “geometry” key, as it contains the coordinates.

 

Figure 10:

Referencing “properties” under the variable first_country_data, we see that one of the keys is in fact “name“. Let’s retrieve the associated value. It is Aruba!

Figure 11:

 

 

As for “geometry”, we note that the dictionary retrieval contains the key “coordinates”, which will return a list of coordinates. These coordinates make up the border of Aruba.

 

Figure 12:

 

We have now found the pattern to access all countries’ names and border coordinates in the data set. We want to get the border coordinates of Canada and plot it on the map. Folium’s built-in GeoJSON() method allows us to easily map GeoJSON data onto the interactive map.

Figure 13:

 

Choropleth map

Defining country borders on the map is crucial, as it establishes the framework for layering additional data. With these boundaries in place, we can now create a choropleth map, which uses variations in color or shading to represent data values across geographical areas to visualize and compare temperature changes in countries over time.

Let us first read in global historical temperature. The data set can be viewed on Kaggle.

We will apply some initial transformations to the data, such as dropping NaNs and converting the date type.

Figure 14:

Although the data set goes back to the year 1743, we cannot assume that 1743 data is available for all countries. Let us find the maximum range of dates for which we would have data for all countries.

For each country, we find the most recent data points available.

Figure 15:

 

We also find the earliest start date for which we would have data on all countries.

Figure 16:

From above, we see that for select countries, data is not available until 1948. So, let’s use 1948 as the starting point. To keep it simple, let’s compute the temperature change in August from 1948 to 2013 for each country.

Figure 17:

We are now ready to overlay the data onto the Folium map using the folium.Choropleth() method. The method has many optional arguments, but the key arguments to note in this case are geo_data, data, columns, and key_on, all of which are needed to properly overlay and join the two data sets.

Figure 18:

Folium allows us to overlay various data onto maps, making it easier to identify trends and clusters at a glance. For instance, a financial institution might overlay flood zone data in coastal cities with insurance claims to better inform property insurance pricing.

Python’s extensive ecosystem of libraries, as well as data APIs written for Python, simplifies data procurement, transformation, and visualization. At the same time, Folium enables the creation of interactive maps to visualize trends and patterns. Python’s seamless handling of GeoJSON data, essential for representing geographic information, makes integrating spatial data into analyses straightforward. As such, Python stands out as a versatile and powerful tool for visualizing climate data.