Quantifying the relationship between weather and vegetation on a regional scale is a statistical problem which requires long time-series of NDVI. GIMMS is a 25 year (1981-2006) NDVI dataset based on inputs from a sequence of NOAA weather satellites carrying AVHRR radiometers. The GIMMS team removed satellite drift, calibration effects etc.[1] Of course NDVI has a complex spatial variation which is related to variations in vegetation cover type, not to weather variations. To isolate the impact of weather variations, it makes sense to “standardize” NDVI. Vegetation Condition Index (VCI) is:

For example, NDVI_min might refer to the minimum local May NDVI value observed during 1981-2006. A tropical rainforest and a semi-arid region have very different NDVI, but both may have VCI close to 1 during a favourable period.

The maps below show the correlation between VCI and 3-month Standardized Precipitation Index (SPI3) computed for Africa by calendar month. GIMMS 8km NDVI was upscaled to 200km to match the scale of gridded monthly climate datasets for precipitation (GPCP).[2] The red areas correspond to areas of high correlation between VCI and SPI3. Vegetation in these areas responsive to rainfall variations on a 3-month timescale. A striking feature  is the line high correlation in the Sahel region. Vegetation in the Sahel is highly sensitive to drought. Southern Africa is also highly rainfall sensitive during Dec-May. It is impressive that the combination of datasets of very different origins (GPCP and GIMMS) produces spatially and temporally consistent information.

Sensitivity of Vegetation to Precipitation 1981-2006

R code snippets

Most of the effort is in pre-processing the climate and NDVI data so that they share the same projection, grid, time coordinate etc. GDAL and the interp() function from the akima package were used for this purpose. Eventually we end up with SPI3 and VCI data represented by N × T matrices where N = Nx ×Ny is the dimension of the spatial maps and T is the number of observations. From there,  SPI3 and VCI maps were organized by calendar month,

vci.m <- lapply(seq(1:12),function(m) vci[,seq(from=m, to=T, by=12)])
spi3.m <- lapply(seq(1:12), function(m) spi3[,seq(from=m, to=T, by=12)])

Correlation maps between VCI and SPI3  for each calendar month are obtained from,

corr.maps <- lapply(seq(1:12),function(m) sapply(seq(1:N), function(i) cor( vci.m[[m]][i,],spi3.m[[m]][i,]) ))

Finally the above map table was produced using spplot() from the sp package with overlays from the maps package.

[1]  Tucker, C.J., J.E. Pinzon, and M.E. Brown (2004), Global Inventory Modeling and Mapping Studies, NA94apr15b.n11-VIg, 2.0, Global Land Cover Facility, University of Maryland, College Park, Maryland, 04/15/1994.

[2] SG: Adler, R.F., G.J. Huffman, A. Chang, R. Ferraro, P. Xie, J. Janowiak, B. Rudolf, U. Schneider, S. Curtis, D. Bolvin, A. Gruber, J. Susskind, P. Arkin, 2003: The Version 2 Global Precipitation Climatology Project (GPCP) Monthly Precipitation Analysis (1979-Present). J. Hydrometeor., 4,1147-1167.

The above wind speed maps are based on NCEP GFS analysis at 300mb pressure level (equivalent to ≈ 10km). The speed scale is in m/s.

There are many interesting things to notice about these maps. Firstly, there are several (5 or 6) planetary scale meanders of the jet stream. The meanders are called planetary or Rossby waves. Secondly, the enclosed area is larger and wind speeds higher in the northern hemisphere. This situation is reversed during the southern winter. Notice the closed loop of clockwise circulating air close to New Zealand. In this case a jet stream meander has grown large, become unstable and broken off. The loop encloses a pocket of cold air. Such detached loops can persist and remain in the same location for days.

The graphic below shows the GFS 300mb analysis wind speeds for ooUTC 19 Jan 2010 and the forecast wind speeds for 00UTC 20 & 21 January.  Rossby wave propogation can be seen clearly (the ridges and troughs advance anticlockwise in the Northern hemisphere, clockwise in the Southern Hemisphere)

code

300mb wind velocities (zonal wind component Uvel and meridonal wind component Vvel) at 300mb were extracted from 0.5° GFS grib files. In an earlier post complete GFS forecasts in grib2 format were downloaded and relevant fields extracted using the wgrib2 utility. In fact, it is possible to download only the required fields. This much faster partial-http option uses cURL and an easy to use Perl script called get_gfs.pl from NCEP.

jet.R is the script which produced the above graphics. Here Roger Bivand’s sp and rgdal packages are used to transform the latitude-longitude GFS projection to  a polar projection. For example, wind speeds in the northern hemisphere are contained in the SpatialGridDataFrame object sg_north. It is transformed into Universal Polar Stereographic (ups in the Proj.4 library) using
sg_north <- spTransform(sg, CRS("+proj=ups +north"))
sg_north@bbox <- polar_north@bbox
Unfortunately spTransform() produces a SpatialPoints object, because the grid is non-uniform after transformation. To recover a SpatialGrid object, the interp() function from package akima was used to resample back onto a regular grid. This is the slowest part of jet.R.

The output from the U.S. National Centers for Environmental Prediction (NCEP) Global  Forecast System (GFS) is freely available. The surface resolution of the model is ≈ 0.3º× 0.3°. The model runs every 6 hours, producing forecasts at 3-hourly intervals extending out to 16 days. As an example of output from GFS, the map (below) shows the predicted average  temperature at 2 metres over the entire globe for the next 24 hr (date of this post). The map shows predicted cold conditions in Europe, and the continuing heatwave in Australia.

How the map was made

GFS forecasts are in a format called GRIB2. According to Wikipedia, “GRIB (GRIdded Binary) is a mathematically concise data format commonly used in meteorology to store historical and forecast weather data.” GRIB files contain physical fields such as temperature, humidity etc defined on a spatial grid, as well as boundary conditions such as vegetation type and elevation. The data might be assimilated from observations, or output from a forecast model.

The first step is to translate the GRIB into a raster format such as netcdf which can be read in R. For example, the GRIB2 file gfs.2009121700/gfs.t00z.sfluxgrbf03.grib2 contains the 3-hr forecast surface data on 17 Dec 2009  produced at 00 UTC (midnight universal time). An inventory of the data contained in this file can be seen here. Download this forecast as temp.grb

loc=file.path("ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.2009121700/gfs.t00z.sfluxgrbf03.grib2")
download.file(loc,"temp.grb",mode="wb")

To read temp.grb a utility called wgrib2 needs to be installed on your system. Then data such as land fraction can extracted into a netcdf file LAND.nc using the R shell command

shell("wgrib2 -s temp03.grb | grep :LAND: | wgrib2 -i temp00.grb -netcdf LAND.nc",intern=T)

The ncdf package can now be used to read the contents of LAND.nc.

library(ncdf)
landFrac <-open.ncdf("LAND.nc")
land <- get.var.ncdf(landFrac,"LAND_surface")
x <- get.var.ncdf(landFrac,"longitude")
y <- get.var.ncdf(landFrac,"latitude")

The 1152×576 matrix land takes values 1 for land and 0 for water (sea-ice is 1). x and y are the longitude and latitude of the non-uniform GFS grid.

2m temperature data can be read in the same way. The average of the first 8  forecasts was called t2m.mean and plotted using image.plot() from the fields package:

library(fields)
rgb.palette <- colorRampPalette(c("snow1","snow2","snow3","seagreen","orange","firebrick"), space = "rgb")#colors
image.plot(x,y,t2m.mean,col=rgb.palette(200),axes=F,main=as.expression(paste("GFS 24hr Average 2M Temperature",day,"00 UTC",sep="")),axes=F,legend.lab="o C")
contour(x,y,land,add=TRUE,lwd=1,levels=0.99,drawlabels=FALSE,col="grey30") #add land outline

warming in the satellite era

Data from satellites have been used to construct a global temperature record beginning in December 1978. This covers the recent warming period. The University of Alabama at Huntsville (UAH) 2LT product gives average temperature of the lower atmosphere derived from microwave radiances. Planck’s law says that microwave radiances are related to temperature.

The plot below shows UAH monthly time-series and a least-squares regression fit. The slope of the regression line is 0.013ºC/year (equivalent to 0.13°C/decade or 1.3°C/century). Also shown is the Gistemp global temperature time-series over the same time interval. Gistemp is based on surface weather stations. The trend in the Gistemp series is higher, 0.18°C/decade.

uncertainty in trend

A linear regression fit to global temperatures is a statistical model in which strong residual variations are superposed on a linear warming trend. The residual variations reflect “noise” in the climate system. ENSO, random events such as volcanos, and many other modes of variability contribute to the “noise”. The “uncertainty” in the global warming trend can be estimated in the context of this statistical model.

From this point of view, the UAH monthly data are one realisation of a random sample of length N. In reality we only have one sample, the UAH data from the real world. However, if repeated sampling and slope measurement were possible, it would yield a range of outcomes due to noise. The width of the resulting slope probability distribution reflects the uncertainty in the global warming trend. It is evident that there is strong serial correlation in the residuals. This enhances the uncertainty, because it reduces the effective number of independent degrees of freedom in the data. Fewer degrees of freedom means more uncertainty.

The UAH residuals are fit to an auto-regressive model to account for serial auto-correlation. The Akaike information criterion to choose an optimal order n. For UAH it  turns out that n=2, with coefficients 0.58  and 0.24. To find the slope probability distribution 10,000 time-series were used in a Monte Carlo simulation. Each simulated time-series assumes a UAH trend-line of 0.13°C/decade and an AR(2) noise term parameterized using the AR(2) model fit to the UAH residuals. Linear regression fit on each of these simulated time-series yields the slope probablity distribution.

A typical result is shown above along with a fit to the normal distribution. By construction the mean is 0.13°C/decade as expected. The standard error is 0.036°C/decade. This is 3.4 times greater than the standard error with independent and identically distributed (iid) residuals.

The 2σ confidence interval for UAH global temperature trend is 0.05 to 0.2°C/decade. Unfortunately, this is quite a wide window. Would another 30 years of data help? Assuming that the properties of the UAH time-series do not change, and extending the above simulation for another 30 years, the standard error is reduced from 0.035°C/decade to 0.026°C/decade. Better, but still a surprising degree of uncertainty.

R code

The above analysis was carried out using R functions ar() and arima.sim()

fitLin <-lm(GLOBAL~dates) # linear regression fit to GLOBAL= UAH global temperature, dates = UAH dates
ar.fit <- ar(residuals(fitLin)) # AR model for residuals
N=10000; # number of simulated UAH time-series
fits <- cbind(rep(0,N),rep(0,N))
trend <- coefficients(fitLin)[1] + dates*coefficients(fitLin)[2] # UAH trend line
#Monte Carlo
for( i in 1:N){
warming <- trend+arima.sim(n = NData, list(ar = c(ar.fit$ar[1], ar.fit$ar[2])), sd=sqrt(ar.fit$var.pred)) #generate sample
fit.warm <- lm(warming~dates) # linear fit
fits[i,1] <- coefficients(fit.warm)[1];
fits[i,2] <- coefficients(fit.warm)[2]; # slope of simulated data
}
library(MASS) # MASS library must be installed
gf <- fitdistr(fits[,2],"normal") # fit a normal distribution to the slopes
mn <-gf$estimate[1] #mean
sig <- gf$estimate[2] #standard error


West of Ireland oak woodland - October 11

Flux tower eddy covariance instruments measure CO2 flux between an ecosystem and the atmosphere. Observed CO2 fluxes (also called NEE, Net Ecosystem Exchange) can be correlated with locally measured solar intensity, temperature, humidity etc. With enough data, a detailed picture of the local response of the ecosystem to environmental variables to be built up. Flux towers are one of the important tools for quantitative understanding the sensitivity of vegetation to climatic variability.

CO2 flux data  are noisy, to say the least. If you want to understand ecosystem reponse, you need a statistical model.  This post gives some simple ideas and R code showing how such a model can be developed. The example used is Harvard Forest [1], Massachussetts, a temperate deciduous broadleaf forest for which a large amount of flux tower data (collected by Munger & Wofsy) is available. It could equally be applied to tropical forest, grassland, cropland etc at one of the 200+ fluxnet sites worldwide. The model falls short of the full scientific rigour given in the references, but it can easily be developed into a more complete tool.

The main interest here is ecosystem response to sunlight i.e. photosynthesis. The light dependent part of the NEE is GEP (Gross Ecosystem Production), while the light independent part is RE (ecosystem respiration).

Pre-processing

As always, the first steps are about getting the data into the right form. We use half-hourly data for 1991-2007  for Harvard Forest  i.e Ameriflux Level 2, site code USHa-1. This raw data has not been subject to any gap-filling statistical treatment. Here are the pre-processing steps:

1. download the annual data files and create a dataframe harvard using the rbind() combining all of the data
2. replace missing data (e.g. -9999) with “NA”
3. add a new fornight index, indicating to which fortnight of the year each row of data belongs
4. select a subset of the Ameriflux variables of interest. e.g. column headings c(”YEAR”,”DTIME”,”FN”,”TA”,”NEE”,”TS1″,”VPD”,”PAR_in”)
5. drop rows of data when one or more variables are undefined
6. group the data according to fortnights index i.e. create a list of 26 elements where each element is a dataframe containing all data sharing the same fortnight index.
7. drop “dark” data i.e. rows of data where PAR < 30.

The environmental variables retained at step 4 are TA (atmospheric temperature), TS1  (soil temperature), VPD (vapour pressure deficit) and PAR (photosynthetically active radiation). PAR is solar radiation in the range 0.4-0.7 microns which is energetic enough to contribute to photosynthesis. It has units μ mols s^-1 m^-2. (1 μ mole = 6 x 10¹⁷.)

Here is what the R code looks like:
#download the annual data files and create a dataframe harvard
loc=file.path("ftp://cdiac.ornl.gov/pub/ameriflux/data/Level2/Sites_ByID/US-Ha1/with_gaps/usmaharv_1991_L2.csv"); # Ameriflux US_Ha1 download.file(loc,"temp.dat");
ha_1991=read.csv("temp.dat", header=FALSE,skip=20); .........................
loc=file.path("ftp://cdiac.ornl.gov/pub/ameriflux/data/Level2/Sites_ByID/US-Ha1/with_gaps/usmaharv_2007_L2.csv"); # Ameriflux US_Ha1 download.file(loc,"temp.dat");
ha_2007=read.csv("temp.dat", header=FALSE,skip=20); harvard <- rbind(ha_1991,ha_1992,ha_....._2006,ha_2007); # merged dataframes
#replace missing data with "NA"
temp <- (harvard[,] == -9999 | harvard[,] == -6999); #replace -9999 or -6999 with NAs
harvard[temp]=NA;
#add a new fortnight index
lims <- seq(from=1,to=365,by=14); # fortnightly data
lims[27] <- 367; # bound adjust, last fortnight has extra days fortnight
<- function(d) {sum(ifelse(d<lims,0,1))}  # convert from DOY to fortnights
harvard <- data.frame( "FN" = sapply(harvard$DOY,fortnight),harvard) # add fortnight labels 1..26 column
#select a subset of the Ameriflux variables of interest
sub<- c("YEAR","DTIME","FN","TA","NEE","TS1","VPD","PAR_in")
harvardS <- harvard[,colnames(harvard) %in% sub, drop=FALSE] # subset dataframe
# drop data when U* < 0.2 m/s
u_cut <- (harvard$UST < 0.2|is.na(harvard$UST));
#drop every row of data when one or more variables are undefined
x <- is.na(harvardS$TA);
y <- is.na(harvardS$NEE);
z <- is.na(harvardS$TS1);
u <- is.na(harvardS$VPD);
v <- is.na(harvardS$PAR_in);
w <- !(x|y|z|u|v|u_cut); #boolean. w is true when all variables are defined and U* > 0.2 m/s
harvardS <- data.frame("YEAR"=harvard$YEAR[w],"FN"=harvard$FN[w],"TA"=harvard$TA[w],"NEE" = harvard$NEE[w], "TS1" = harvard$TS1[w], "VPD" = harvard$VPD[w],"PAR" = harvard$PAR_in[w])
#group the data according to fortnights index
getF <- function(i) {subset(harvardS,FN==i)}
# list of dataframes according to fortnight index
harvardF <- lapply(seq(1:26),getF) # generate list
#drop "dark" data i.e. rows of data where PAR < 30.
harvardS.light <- subset(harvardS,PAR>30)#day only
getF.light <- function(i) {subset(harvardS.light,FN==i)}
# list of dataframes according to fortnight value
harvardF.light <- lapply(seq(1:26),getF.light) # generate list

harvardF.light contains the required clean daytime half-hourly data grouped by fortnight index.

Model-Building

The plots below show NEE versus PAR. Fortnights 8 & 22 (mid April & late October) are winter months with positive NEE and little evidence of photosynthetic response. Fortnights 14 & 18 (early July and late August) show photosynthesis (-ve NEE) and a strong response to PAR.

The NEE-PAR curves during summer months saturate at high light levels. This is not too surprising. Photosynthesis is limited by water and CO2 availability and so cannot increase indefinitely with increasing PAR. A simple light saturation model is [2]

The second term on the right-hand-side represents GEP.  At low light levels GEP ≈ αPAR. The parameter α is called the “photosynthetic efficiency”. At high light levels,  GEP ≈ β and so β is the light saturation GEP.

This 3-parameter simple light saturation model can be fit to extract parameters for each fortnightly period. However, a better model recognises that photosynthesis responds to changes in moisture availability and temperature. This effect can be introduced by replacing in the numerator of GEP, where ε depends on atmospheric temperature TA and vapour pressure deficit VPD. ε=1 under optimal temperature and humidity conditions, otherwise ε<1. We need to input a reasonable model for how quickly GEP falls away about the optimal values of TA (at about 25C) and VPD (at zero  kPa). After some experimenting, the following model was found to be reasonable[3]

and are two additional parameters which must be determined. This 5-parameter model can be fit for each of the photosynthetically active periods 11-20.

The nls() function in R carries out a numerical maximum likelihood fit using least squares. Unlike ordinary linear least squares, it is not guaranteed to converge. The model and initial parameter values have to be chosen carefully. A good approach is to use parameters from the simple light saturation model as initial values in nls(). Here is the code needed to do the 5-parameter model fits


#fits.mid is a list of fitted R objects
for(i in start_period:end_period) {
fits.mid[[i]] <- nls(NEE~ gamma - alpha*beta*(lambda^2/((TA-25)^2+lambda^2))*mu^2/(VPD^2+mu^2)*PAR/(beta+alpha*PAR), data = harvardF.light[[i]], start=list(gamma=1,alpha=0.02,beta=40,lambda=40,mu=1),trace=F);

In fact, λ and μ are not expected to vary much during the summer period, since they depend on leaf properties at the cellular level.  Indeed they turn out to be roughly constant for fortnight periods 11-20. It is reasonable to replace them by their average values μ_av = 2.1kPa and λ_av=34 μ mols s^-1 m^-2. With ε determined in this way, a final 3-parameter model fit is carried out. Once again the three parameters are RE, α and β but this time the model takes better account of temperature and moisture sensitivity, via ε.

Results

The model suggests that VPD is an important factor limiting GEP. e.g. vapour pressure deficit of 1kPa reduces GEP by 20% when kPa. This effect is seen in the plots below.

The implication of this is that about 30% of the apparent “light saturation” of the NEE PAR response is due to water stress (VPD). The remainder is due to CO2 availability and possibly other constraints.

The plot below shows the variation of parameters R_E , β and α with fortnight index. Only β shows a strong variation during the summer months, with a peak in early June

corresponds to the number of  photosynthetically active photons  arriving at the forest canopy for every CO2 molecule trapped by photosynthesis. This number is about 14.

A common feature of future climate scenarios is competition between increased CO2 availability and increased water stress layed out on a global scale. Clearly flux tower data are good place to look for these effects.

The R script for the model and plots can be found here.

References

[1] Factors controlling CO₂ exchange on timescales from hourly to decadal at Harvard Forest, Urbanski, S., C. Barford, S. Wofsy, C. Kucharik, E. Pyle, J. Budney, K.

McKain, D. Fitzjarrald, M. Czikowsky, J. W. Munger http://www.agu.org/pubs/crossref/2007…/2006JG000293.shtml

[2] See for example, An evaluation of models for partitioning eddy covariance-measured
net ecosystem exchange into photosynthesis and respiration P. Stoy et al, http://www.geos.ed.ac.uk/homes/pstoy/Stoy_06b.pdf

[3] More detailed models are give in An empirical model simulating diurnal and seasonal CO₂ flux for diverse vegetation types and climate conditions, M. Saito, S. Maksyutov, R. Hirata, and A. D. Richardson. http //www.biogeosciences.net/6/585/2009/bg-6-585-2009.html & references.

Sea ice extent has been monitored by satellites in polar orbit since 1978. The remote sensing technique employed (passive microwaves) can discriminate between open water and sea ice even during the dark polar winter or cloudy conditions. Microwave emissions are an important tool in monitoring water in all forms – atmospheric water vapour, sea temperatures, soil moisture, snow cover …

The plot below shows Arctic sea ice extent (SIE) derived from the AMSR-E instrument (Advanced Microwave Scanning Radiometer[1]). AMSR-E was built by the Japanese Aerospace Exploration Agency (JAXA). The plot below is available from the International Arctic Research Center (IARC)-JAXA information site.

Sea ice extent time-series

Sea ice has been monitored using microwaves since 1978. AMSR-E is the newest instrument with the highest resolution (6km, 89GHz).  It is therefore viewed as the most reliable, but it has only been in operation since 2002. Comiso and Nishio show that AMSR-E and earlier passive microwave data  are compatible after minor re-calibration. Differences in derived SIE are accounted for by lower spatial resolution of the earlier data.

The plot on the left shows SIE in the 2002-2008 overlap period of AMSR-E and older data (”SMM/I“) obtained from National Snow and Ice Data Center (NSIDC).  Plotting AMSR-E against SMM/I derived SIE (plot on right) shows a close linear relation. An AMSR-E compatible time-series can therefore be generated back to 1978. The attached R script also interpolates missing data to produce a continuous daily time-series (called final.ts).

The daily time-series final.ts looks like:

What is the trend in this time-series and how unusual are recent years? A standard method is used to remove the seasonal signal from final.ts, leaving the “trend” and “volatility”. Here is the short R code required:

seasonal_analysis <- stl(final.ts,s.window="periodic",robust=TRUE)$time.series
anomaly <- seasonal_analysis[,2]+seasonal_analysis[,3];
anom.fit <- lm(anomaly~time(final.ts));
coefs <- coefficients(anom.fit);
slope <- as.numeric(round(100*coefs[2]*10/(coefs[1]+coefs[2]*1985),1)); # slope of fit
plot(anomaly,lwd=2,main=paste("Arctic Sea Ice Extent Anomaly to",end_date),ylim=c(0,13),ylab="10^6 km2",xlab="",font.lab=2,font.axis=2);
abline(anom.fit,col=2);
legend("bottomleft",legend=c("Arctic Sea Ice Extent (seasonal signal removed)",paste("trend=",as.expression(slope),"% per decade")),col=1:2,lty=1,lwd=2:1);
text(1995,4, "data from www.ijis.iarc.uaf.edu & www.nsidc.org");

The result is:

Conclusions

Arctic sea ice extent has been shrinking by nearly 4% per decade since 1978. Volatility about this trend has clearly picked up in the past few years. The past two summers (2007 and 2008) have been unusual, with deep below trend summer minima.

Ice has recovered somewhat so far this year, and it will be very interesting to see the extent of the melt next September.

[1] The “E” in AMSR-E indicates that the instrument is carried on board one of Nasa’s Earth Observing System satellites (EOS),  Aqua.

The World Data Center for Paleoclimate (NOAA) maintains a  tree ring database. As well as raw data on growth rings for individual trees, the database contains “chronology” files labelled  ***.crn. These files contain annual growth data averaged over small stands of trees, and indexed relative to the mean growth. Thus an index value of 1000 is average growth, index lower than 1000 is below average growth etc. Here is a simple R script which uploads the chronology file  “ca535.crn” file into a time-series treeRing.ts. ca535.crn describes annual growth in an ancient Bristlecone Pine forest in the White Mountains of California from 6000 BC to 1979 AD. The world’s oldest known (non-clonal) tree  is a member of the stand of Bristlecone Pines making up this index. The area is known as Methuselah Grove.

You can click on the plot below to see a full-screen version.

Most of the volatility in the growth of Bristlecone Pines in the White Mountains is due to drought. The investigator Edmund Schulman wrote “There is something a little fantastic in the persistent ability of a 4,000-year-old tree to shut up shop almost everywhere throughout its stem in a very dry year, and faithfully to reawaken to add many new cells in a favorable year.” (1958). With so much high quality data (7980 points), many interesting questions can be asked about climatic variability at this location. For example, a statistical model can be built relating recent tree ring growth to observed precipitation and temperature, say. Such models are used to extract information about historical climate.

The following R commands produce a histogram of tree ring growth values (probability distribution) as well as a smooth curve fit:

library(MASS)
truehist(treeRing.ts,nbins=50,main="Distribution of Growth Index at Methuselah Grove",font.main=2,font.axis=2,font.lab=3,xlab="Growth Index",ylab="Density");
lines(density(treeRing.ts),lwd=2)

The distribution approximates a bell-shaped curve, but there is a pronounced “fat-tail” on the low growth side. For example, there  is only one year with growth index > 1900, but there are 44 years with growth < 100.

If you only ever read one book about statistics, make sure it is Nassim Taleb’s book The Black Swan. Fat-tailed distributions (such as drought impact on Bristlecone Pines) are the norm in complex systems. Many of Taleb’s examples are from finance and economics. However geophysical time-series, such as precipitation levels, often have this property. When making inferences from data, it is important to decide in advance what kind of process you are likely to be dealing with. A process where there is no clear constraint on the outcome (such as a physical conservation law) is likely to be subject to fat-tail events.

The asymmetry in growth distribution in White Mountain Bristlecone Pines is due to the fact that higher than normal precipitation levels cannot boost growth beyond the limits set by available temperature, sunlight and plant physiology. Drought years, however, virtually eliminate growth.

Traditional farmers in drought-prone parts of the world are acutely aware of the fat-tailed distribution. Farming practices which appear sub-optimal to some, may in fact be adapted to the existence of fat-tails. Nassim Taleb points out that risk managers in banks tend to behave as though Black Swans do not exist.

In that sense, bankers know less about real-world statistics than subsistence farmers.

The U.S. National Climate Data Center (NCDC) maintain a large dataset called the Global Historical Climate Network (ftp site GHCN) – temperature and other data collected at more than 40,000 weather stations. Unfortunately, individual weather stations come and go and there are many gaps in the data. A much smaller database of about 1,000 stations is maintained by the Global Climate Observing System (GCOS) sponsored by World Meteorological Organisation (WMO) and other organisations. We focus on this subset because these stations are up-to-date data and reliable.

The locations of GCOS Global Surface Network (GSN) stations are shown in Google Earth/Maps below.

To see more stations, zoom in to a region of interest and switch to Satellite view. Click on the GCOS logo placemark to see GHCN station ID and other station metadata. If  Google Earth is installed on your system, it may be better to copy the text file gcosgsn, paste into notepad and save as gcosgsn.kml. Clicking on this .kml file opens GE on your system with all GSN stations visible.

gscosgsn was generated using an excel spreadsheet. Improved GSN geo-location data were obtained by cross-referencing the GCOS list with NOAA’s GSOD (0.001^o accuracy).  Other metadata, such as station start year, can easily be added.

How can the climatic variability at a particular GSN station be examined? To do this R will need to be installed on your system. Install it from CRAN if required.

In what follows, a monthly time-series analysis is carried out on a user-selected station. Daily GHCN station data are read into R courtesy of a  data-scraping script from Climate Audit. anything.ts indicates an R time-series object by the way.

1. Start R. You will see the friendly command line prompt in the R console.

>

2. Enter the following command to load required R script.

source("http://joewheatley.net/wp-content/uploads/2009/05/ghcntor.txt")

3. Select a station of interest in GE/GE Plugin. Copy the GHCN station ID from the placemark balloon and enter the following command to load the GHCN data file into R gsn1 <-read.ghcnd("paste_id_here")
4. Plot the mean monthly temperature by copying and paste the following into R
   gsn1.ts <- getTMean.ts(gsn1);
plot(gsn1.ts, main="Mean Monthly Temperature UCCLE Belgium", ylab="o C",xlab="Year",font.axis=2,font.lab=2)
lines(getTrend.ts(gsn1.ts),col=2,lwd=2);
maxTrend <- max(getTrend.ts(gsn1.ts));
maxLine.ts <- ts( rep(maxTrend,length(gsn1.ts)), start= start(gsn1.ts), deltat=1/12)
lines(maxLine.ts, col=2,lty=2);
5. To see the climatic variability
   plot(getResidual.ts(gsn1.ts), main="Monthly Temperature Anomaly UCCLE Belgium", ylab="o C",xlab="Year",font.axis=2,font.lab=2,col=3);

For example, station ID BE000006447 (UCCLE Belgium) generates the plots below.

The periodic black line in the first plot is the monthly average station temperature. Data continuity is from 1831 in this instance. The red line is the long-term trend when seasonal variations and residual random variations are extracted by means of the R stl function. As you can see it indicates that mean temperatures at UCCLE are the highest they have been since 1831.  Compared to the average temperature for 1831-2009, this anomaly is about 1.36^o. A prime candidate for this is global warming, but local factors such as land use change or increased urbanisation also affect individual station data.

The temperature anomaly plot  (green line) shows the residual temperature variations at UCCLE. Of course these have a  larger year-to-year influence than the trend line variations.

If you have followed the above steps you have done something at once both cool and complex. You have combined the most reliable climate data with the best available statistical resource.