You can run this notebook in a live session or view it on Github.
Calculating Seasonal Averages from Timeseries of Monthly Means¶
Author: Joe Hamman
The data used for this example can be found in the xarray-data repository. You may need to change the path to rasm.nc
below.
Suppose we have a netCDF or xarray.Dataset
of monthly mean data and we want to calculate the seasonal average. To do this properly, we need to calculate the weighted average considering that each month has a different number of days.
[1]:
%matplotlib inline
import numpy as np
import pandas as pd
import xarray as xr
import matplotlib.pyplot as plt
Open the Dataset
¶
[2]:
ds = xr.tutorial.open_dataset('rasm').load()
ds
[2]:
<xarray.Dataset> Dimensions: (time: 36, x: 275, y: 205) Coordinates: * time (time) object 1980-09-16 12:00:00 ... 1983-08-17 00:00:00 xc (y, x) float64 189.2 189.4 189.6 189.7 ... 17.65 17.4 17.15 16.91 yc (y, x) float64 16.53 16.78 17.02 17.27 ... 28.26 28.01 27.76 27.51 Dimensions without coordinates: x, y Data variables: Tair (time, y, x) float64 nan nan nan nan nan ... 29.8 28.66 28.19 28.21 Attributes: title: /workspace/jhamman/processed/R1002RBRxaaa01a/l... institution: U.W. source: RACM R1002RBRxaaa01a output_frequency: daily output_mode: averaged convention: CF-1.4 references: Based on the initial model of Liang et al., 19... comment: Output from the Variable Infiltration Capacity... nco_openmp_thread_number: 1 NCO: netCDF Operators version 4.7.9 (Homepage = htt... history: Fri Aug 7 17:57:38 2020: ncatted -a bounds,,d...
- time: 36
- x: 275
- y: 205
- time(time)object1980-09-16 12:00:00 ... 1983-08-...
- long_name :
- time
- type_preferred :
- int
array([cftime.DatetimeNoLeap(1980, 9, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1980, 10, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1980, 11, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1980, 12, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 1, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 2, 15, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 3, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 4, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 5, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 6, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 7, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 8, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 9, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 10, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 11, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 12, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 1, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 2, 15, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 3, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 4, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 5, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 6, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 7, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 8, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 9, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 10, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 11, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 12, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 1, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 2, 15, 12, 0, 0, 0), cftime.DatetimeNoLeap(1983, 3, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 4, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1983, 5, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 6, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1983, 7, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 8, 17, 0, 0, 0, 0)], dtype=object)
- xc(y, x)float64189.2 189.4 189.6 ... 17.15 16.91
- long_name :
- longitude of grid cell center
- units :
- degrees_east
array([[189.22293223, 189.38990916, 189.55836619, ..., 293.77906088, 294.0279241 , 294.27439931], [188.96836986, 189.13470591, 189.30253733, ..., 294.05584005, 294.30444387, 294.55065969], [188.71234264, 188.87800731, 189.04515208, ..., 294.335053 , 294.58337453, 294.8292928 ], ..., [124.04724025, 123.88362026, 123.71852016, ..., 16.83171831, 16.58436953, 16.33949649], [123.78686428, 123.62254238, 123.45672512, ..., 17.11814486, 16.87043749, 16.62518298], [123.52798366, 123.36295986, 123.1964407 , ..., 17.40209947, 17.1540526 , 16.90845095]])
- yc(y, x)float6416.53 16.78 17.02 ... 27.76 27.51
- long_name :
- latitude of grid cell center
- units :
- degrees_north
array([[16.53498637, 16.7784556 , 17.02222429, ..., 27.36301592, 27.11811045, 26.87289026], [16.69397341, 16.93865381, 17.18364512, ..., 27.5847719 , 27.33821848, 27.0913656 ], [16.85219179, 17.09808909, 17.34430872, ..., 27.80584314, 27.55764558, 27.30915621], ..., [17.31179033, 17.56124674, 17.81104646, ..., 28.4502485 , 28.19718339, 27.94384744], [17.15589701, 17.40414034, 17.65272318, ..., 28.23129632, 27.97989251, 27.72821596], [16.99919497, 17.24622904, 17.49358736, ..., 28.01160028, 27.76185586, 27.51182726]])
- Tair(time, y, x)float64nan nan nan ... 28.66 28.19 28.21
- units :
- C
- long_name :
- Surface air temperature
- type_preferred :
- double
- time_rep :
- instantaneous
array([[[ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], ..., [ nan, nan, nan, ..., 27.03290153, 27.03125761, 27.33531541], [ nan, nan, nan, ..., 27.2784053 , 26.80261869, 27.08603517], [ nan, nan, nan, ..., 27.02344402, 26.56473862, 26.73064933]], [[ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], ... [ nan, nan, nan, ..., 27.8597472 , 27.82928439, 28.09249224], [ nan, nan, nan, ..., 27.89704094, 27.31104941, 27.67387171], [ nan, nan, nan, ..., 27.46837113, 27.0088944 , 27.23017976]], [[ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], ..., [ nan, nan, nan, ..., 28.95929072, 28.87672039, 29.04890862], [ nan, nan, nan, ..., 29.036132 , 28.42273578, 28.68721201], [ nan, nan, nan, ..., 28.66381585, 28.18595533, 28.20753022]]])
- title :
- /workspace/jhamman/processed/R1002RBRxaaa01a/lnd/temp/R1002RBRxaaa01a.vic.ha.1979-09-01.nc
- institution :
- U.W.
- source :
- RACM R1002RBRxaaa01a
- output_frequency :
- daily
- output_mode :
- averaged
- convention :
- CF-1.4
- references :
- Based on the initial model of Liang et al., 1994, JGR, 99, 14,415- 14,429.
- comment :
- Output from the Variable Infiltration Capacity (VIC) model.
- nco_openmp_thread_number :
- 1
- NCO :
- netCDF Operators version 4.7.9 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)
- history :
- Fri Aug 7 17:57:38 2020: ncatted -a bounds,,d,, rasm.nc Tue Dec 27 14:15:22 2016: ncatted -a dimensions,,d,, rasm.nc rasm.nc Tue Dec 27 13:38:40 2016: ncks -3 rasm.nc rasm.nc history deleted for brevity
Now for the heavy lifting:¶
We first have to come up with the weights, - calculate the month lengths for each monthly data record - calculate weights using groupby('time.season')
Finally, we just need to multiply our weights by the Dataset
and sum allong the time dimension. Creating a DataArray
for the month length is as easy as using the days_in_month
accessor on the time coordinate. The calendar type, in this case 'noleap'
, is automatically considered in this operation.
[3]:
month_length = ds.time.dt.days_in_month
month_length
[3]:
<xarray.DataArray 'days_in_month' (time: 36)> array([30, 31, 30, 31, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31, 31, 28, 31, 30, 31, 30, 31, 31]) Coordinates: * time (time) object 1980-09-16 12:00:00 ... 1983-08-17 00:00:00
- time: 36
- 30 31 30 31 31 28 31 30 31 30 31 ... 31 30 31 31 28 31 30 31 30 31 31
array([30, 31, 30, 31, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31, 31, 28, 31, 30, 31, 30, 31, 31])
- time(time)object1980-09-16 12:00:00 ... 1983-08-...
- long_name :
- time
- type_preferred :
- int
array([cftime.DatetimeNoLeap(1980, 9, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1980, 10, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1980, 11, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1980, 12, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 1, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 2, 15, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 3, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 4, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 5, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 6, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 7, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 8, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 9, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 10, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1981, 11, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1981, 12, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 1, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 2, 15, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 3, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 4, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 5, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 6, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 7, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 8, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 9, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 10, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1982, 11, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1982, 12, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 1, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 2, 15, 12, 0, 0, 0), cftime.DatetimeNoLeap(1983, 3, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 4, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1983, 5, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 6, 16, 12, 0, 0, 0), cftime.DatetimeNoLeap(1983, 7, 17, 0, 0, 0, 0), cftime.DatetimeNoLeap(1983, 8, 17, 0, 0, 0, 0)], dtype=object)
[4]:
# Calculate the weights by grouping by 'time.season'.
weights = month_length.groupby('time.season') / month_length.groupby('time.season').sum()
# Test that the sum of the weights for each season is 1.0
np.testing.assert_allclose(weights.groupby('time.season').sum().values, np.ones(4))
# Calculate the weighted average
ds_weighted = (ds * weights).groupby('time.season').sum(dim='time')
[5]:
ds_weighted
[5]:
<xarray.Dataset> Dimensions: (season: 4, x: 275, y: 205) Coordinates: xc (y, x) float64 189.2 189.4 189.6 189.7 ... 17.65 17.4 17.15 16.91 yc (y, x) float64 16.53 16.78 17.02 17.27 ... 28.26 28.01 27.76 27.51 * season (season) object 'DJF' 'JJA' 'MAM' 'SON' Dimensions without coordinates: x, y Data variables: Tair (season, y, x) float64 0.0 0.0 0.0 0.0 ... 23.15 22.08 21.73 21.96
- season: 4
- x: 275
- y: 205
- xc(y, x)float64189.2 189.4 189.6 ... 17.15 16.91
- long_name :
- longitude of grid cell center
- units :
- degrees_east
array([[189.22293223, 189.38990916, 189.55836619, ..., 293.77906088, 294.0279241 , 294.27439931], [188.96836986, 189.13470591, 189.30253733, ..., 294.05584005, 294.30444387, 294.55065969], [188.71234264, 188.87800731, 189.04515208, ..., 294.335053 , 294.58337453, 294.8292928 ], ..., [124.04724025, 123.88362026, 123.71852016, ..., 16.83171831, 16.58436953, 16.33949649], [123.78686428, 123.62254238, 123.45672512, ..., 17.11814486, 16.87043749, 16.62518298], [123.52798366, 123.36295986, 123.1964407 , ..., 17.40209947, 17.1540526 , 16.90845095]])
- yc(y, x)float6416.53 16.78 17.02 ... 27.76 27.51
- long_name :
- latitude of grid cell center
- units :
- degrees_north
array([[16.53498637, 16.7784556 , 17.02222429, ..., 27.36301592, 27.11811045, 26.87289026], [16.69397341, 16.93865381, 17.18364512, ..., 27.5847719 , 27.33821848, 27.0913656 ], [16.85219179, 17.09808909, 17.34430872, ..., 27.80584314, 27.55764558, 27.30915621], ..., [17.31179033, 17.56124674, 17.81104646, ..., 28.4502485 , 28.19718339, 27.94384744], [17.15589701, 17.40414034, 17.65272318, ..., 28.23129632, 27.97989251, 27.72821596], [16.99919497, 17.24622904, 17.49358736, ..., 28.01160028, 27.76185586, 27.51182726]])
- season(season)object'DJF' 'JJA' 'MAM' 'SON'
array(['DJF', 'JJA', 'MAM', 'SON'], dtype=object)
- Tair(season, y, x)float640.0 0.0 0.0 ... 22.08 21.73 21.96
array([[[ 0. , 0. , 0. , ..., 0. , 0. , 0. ], [ 0. , 0. , 0. , ..., 0. , 0. , 0. ], [ 0. , 0. , 0. , ..., 0. , 0. , 0. ], ..., [ 0. , 0. , 0. , ..., 11.55366633, 11.2195327 , 11.31903364], [ 0. , 0. , 0. , ..., 11.92058204, 11.1181437 , 11.30690884], [ 0. , 0. , 0. , ..., 11.66125566, 11.37199549, 11.57434832]], [[ 0. , 0. , 0. , ..., 0. , 0. , 0. ], [ 0. , 0. , 0. , ..., 0. , 0. , 0. ], [ 0. , 0. , 0. , ..., 0. , 0. , 0. ], ... [ 0. , 0. , 0. , ..., 21.36989633, 21.33322482, 21.59696141], [ 0. , 0. , 0. , ..., 21.60755533, 21.12420209, 21.4197585 ], [ 0. , 0. , 0. , ..., 21.31209877, 21.00573541, 21.16441974]], [[ 0. , 0. , 0. , ..., 0. , 0. , 0. ], [ 0. , 0. , 0. , ..., 0. , 0. , 0. ], [ 0. , 0. , 0. , ..., 0. , 0. , 0. ], ..., [ 0. , 0. , 0. , ..., 22.13996755, 21.97305741, 22.16620228], [ 0. , 0. , 0. , ..., 22.3924211 , 21.72999901, 21.98601135], [ 0. , 0. , 0. , ..., 22.08083763, 21.73405454, 21.95897774]]])
[6]:
# only used for comparisons
ds_unweighted = ds.groupby('time.season').mean('time')
ds_diff = ds_weighted - ds_unweighted
[7]:
# Quick plot to show the results
notnull = pd.notnull(ds_unweighted['Tair'][0])
fig, axes = plt.subplots(nrows=4, ncols=3, figsize=(14,12))
for i, season in enumerate(('DJF', 'MAM', 'JJA', 'SON')):
ds_weighted['Tair'].sel(season=season).where(notnull).plot.pcolormesh(
ax=axes[i, 0], vmin=-30, vmax=30, cmap='Spectral_r',
add_colorbar=True, extend='both')
ds_unweighted['Tair'].sel(season=season).where(notnull).plot.pcolormesh(
ax=axes[i, 1], vmin=-30, vmax=30, cmap='Spectral_r',
add_colorbar=True, extend='both')
ds_diff['Tair'].sel(season=season).where(notnull).plot.pcolormesh(
ax=axes[i, 2], vmin=-0.1, vmax=.1, cmap='RdBu_r',
add_colorbar=True, extend='both')
axes[i, 0].set_ylabel(season)
axes[i, 1].set_ylabel('')
axes[i, 2].set_ylabel('')
for ax in axes.flat:
ax.axes.get_xaxis().set_ticklabels([])
ax.axes.get_yaxis().set_ticklabels([])
ax.axes.axis('tight')
ax.set_xlabel('')
axes[0, 0].set_title('Weighted by DPM')
axes[0, 1].set_title('Equal Weighting')
axes[0, 2].set_title('Difference')
plt.tight_layout()
fig.suptitle('Seasonal Surface Air Temperature', fontsize=16, y=1.02)
[7]:
Text(0.5, 1.02, 'Seasonal Surface Air Temperature')
[8]:
# Wrap it into a simple function
def season_mean(ds, calendar='standard'):
# Make a DataArray with the number of days in each month, size = len(time)
month_length = ds.time.dt.days_in_month
# Calculate the weights by grouping by 'time.season'
weights = month_length.groupby('time.season') / month_length.groupby('time.season').sum()
# Test that the sum of the weights for each season is 1.0
np.testing.assert_allclose(weights.groupby('time.season').sum().values, np.ones(4))
# Calculate the weighted average
return (ds * weights).groupby('time.season').sum(dim='time')