The approach taken by organisations such as ECMWF or NCEP is to re-run numerical forecast models with a range of carefully chosen initial conditions. The collection of runs is called the ensemble. Ensemble prediction systems (EPS) give probabilistic forecasts for variables such as rainfall, temperature etc. Current operational EPS have 20 (GFS)  or 51 (ECMWF) ensemble members from which the probability distributions are derived. ECMWF give an overview of their system here. The probability distributions capture part of the intrinsic uncertainty in weather or climate.

The graph below shows histograms of 20 ensemble member temperatures near some major cities. The data were extracted from NCEP GENS 16-day 2m temperature forecast produced at 00UTC 2 Feb 2010 (i.e GFS forecasts for 18 Feb).

The maps below show some corresponding ensemble statistics for the entire globe (1° resolution, equal area cylindrical projection).

The upper map indicates that forecast uncertainty (standard error) is high between 40° and 60° in both hemispheres (related to the chaotic behaviour of  jet streams.) Currently, 16 day temperatures north of Lake Baikal in Siberia are very uncertain, for example. The contours indicate ensemble median temperatures.

Skewness in ensemble temperatures is shown in the lower map. For example, large negative skewness is found in north central US, eastern mediterranean, and Paraguay/Mato Grosso. This suggests tail risk of low temperatures relative to ensemble mean in these areas.

EPS is the future of weather and climate forecasting. These systems produce huge amounts of data. Building useful applications of EPS is both a challenge and an opportunity.

For anyone interested, the R code used to produce these graphs is given here.

The above wind speed maps are based on NCEP GFS analysis at 300mb pressure level (equivalent to ≈ 10km). The speed scale is in m/s.

There are many interesting things to notice about these maps. Firstly, there are several (5 or 6) planetary scale meanders of the jet stream. The meanders are called planetary or Rossby waves. Secondly, the enclosed area is larger and wind speeds higher in the northern hemisphere. This situation is reversed during the southern winter. Notice the closed loop of clockwise circulating air close to New Zealand. In this case a jet stream meander has grown large, become unstable and broken off. The loop encloses a pocket of cold air. Such detached loops can persist and remain in the same location for days.

The graphic below shows the GFS 300mb analysis wind speeds for ooUTC 19 Jan 2010 and the forecast wind speeds for 00UTC 20 & 21 January.  Rossby wave propogation can be seen clearly (the ridges and troughs advance anticlockwise in the Northern hemisphere, clockwise in the Southern Hemisphere)

code

300mb wind velocities (zonal wind component Uvel and meridonal wind component Vvel) at 300mb were extracted from 0.5° GFS grib files. In an earlier post complete GFS forecasts in grib2 format were downloaded and relevant fields extracted using the wgrib2 utility. In fact, it is possible to download only the required fields. This much faster partial-http option uses cURL and an easy to use Perl script called get_gfs.pl from NCEP.

jet.R is the script which produced the above graphics. Here Roger Bivand’s sp and rgdal packages are used to transform the latitude-longitude GFS projection to  a polar projection. For example, wind speeds in the northern hemisphere are contained in the SpatialGridDataFrame object sg_north. It is transformed into Universal Polar Stereographic (ups in the Proj.4 library) using
sg_north <- spTransform(sg, CRS("+proj=ups +north"))
sg_north@bbox <- polar_north@bbox
Unfortunately spTransform() produces a SpatialPoints object, because the grid is non-uniform after transformation. To recover a SpatialGrid object, the interp() function from package akima was used to resample back onto a regular grid. This is the slowest part of jet.R.

The output from the U.S. National Centers for Environmental Prediction (NCEP) Global  Forecast System (GFS) is freely available. The surface resolution of the model is ≈ 0.3º× 0.3°. The model runs every 6 hours, producing forecasts at 3-hourly intervals extending out to 16 days. As an example of output from GFS, the map (below) shows the predicted average  temperature at 2 metres over the entire globe for the next 24 hr (date of this post). The map shows predicted cold conditions in Europe, and the continuing heatwave in Australia.

How the map was made

GFS forecasts are in a format called GRIB2. According to Wikipedia, “GRIB (GRIdded Binary) is a mathematically concise data format commonly used in meteorology to store historical and forecast weather data.” GRIB files contain physical fields such as temperature, humidity etc defined on a spatial grid, as well as boundary conditions such as vegetation type and elevation. The data might be assimilated from observations, or output from a forecast model.

The first step is to translate the GRIB into a raster format such as netcdf which can be read in R. For example, the GRIB2 file gfs.2009121700/gfs.t00z.sfluxgrbf03.grib2 contains the 3-hr forecast surface data on 17 Dec 2009  produced at 00 UTC (midnight universal time). An inventory of the data contained in this file can be seen here. Download this forecast as temp.grb

loc=file.path("ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.2009121700/gfs.t00z.sfluxgrbf03.grib2")
download.file(loc,"temp.grb",mode="wb")

To read temp.grb a utility called wgrib2 needs to be installed on your system. Then data such as land fraction can extracted into a netcdf file LAND.nc using the R shell command

shell("wgrib2 -s temp03.grb | grep :LAND: | wgrib2 -i temp00.grb -netcdf LAND.nc",intern=T)

The ncdf package can now be used to read the contents of LAND.nc.

library(ncdf)
landFrac <-open.ncdf("LAND.nc")
land <- get.var.ncdf(landFrac,"LAND_surface")
x <- get.var.ncdf(landFrac,"longitude")
y <- get.var.ncdf(landFrac,"latitude")

The 1152×576 matrix land takes values 1 for land and 0 for water (sea-ice is 1). x and y are the longitude and latitude of the non-uniform GFS grid.

2m temperature data can be read in the same way. The average of the first 8  forecasts was called t2m.mean and plotted using image.plot() from the fields package:

library(fields)
rgb.palette <- colorRampPalette(c("snow1","snow2","snow3","seagreen","orange","firebrick"), space = "rgb")#colors
image.plot(x,y,t2m.mean,col=rgb.palette(200),axes=F,main=as.expression(paste("GFS 24hr Average 2M Temperature",day,"00 UTC",sep="")),axes=F,legend.lab="o C")
contour(x,y,land,add=TRUE,lwd=1,levels=0.99,drawlabels=FALSE,col="grey30") #add land outline