Skip to contents

Workflow Example 1

Peter Levy

2026-07-21

workflow_example.Rmd

This vignette illustrates the typical workflow for processing met data with metamet. The basic steps are to:

  • read the observation data
  • create a corresponding metadata table dt_meta for the observed variables
  • create a metadata table dt_site for each site with observations
  • combine these into a single metamet object

Example data

To illustrate, we will use some publicy available data from Whim Moss, available from the EIDC data centre. Metadata are available in the supporting documentation, but not in a machine-readable format, so we have to extract it manually the first time. Thereafter it is available within the metamet object for all future use.

Firstly, we load the metamet library, as well as the here library which simplifies specifying file paths.

here::i_am("vignettes/workflow_example.Rmd")
library(metamet)
library(here)

Observation data dt

Having downloaded the observation data, we can read it from a file and display a few rows.

fname <- here("inst/extdata/UK-WHM/historical/eidc/whim_met_2002_2023.csv")

dt <- data.table::fread(fname)
dim(dt)
#> [1] 736321     14
dt
#>                Timestamp  Rain     LWS  AirT    RH   PAR Total_solar Net_rad
#>                   <char> <num>   <num> <num> <num> <num>       <num>   <num>
#>      1: 01/01/2003 00:00     0 6999.00 1.534  93.9    NA           0 -23.820
#>      2: 01/01/2003 00:15     0 6999.00 1.509  93.8    NA           0 -33.320
#>      3: 01/01/2003 00:30     0 6999.00 1.354  94.3    NA           0 -37.650
#>      4: 01/01/2003 00:45     0 6999.00 1.359  94.5    NA           0 -25.340
#>      5: 01/01/2003 01:00     0 6999.00 1.350  94.5    NA           0 -26.670
#>     ---                                                                     
#> 736317: 31/12/2023 23:00     0   18.96 2.796    NA     0           0  -1.570
#> 736318: 31/12/2023 23:15     0   18.66 2.848    NA     0           0  -3.031
#> 736319: 31/12/2023 23:30     0   18.51 2.845    NA     0           0  -4.105
#> 736320: 31/12/2023 23:45     0   18.32 2.902    NA     0           0  -5.239
#> 736321: 01/01/2024 00:00     0   17.85 2.959    NA     0           0  -2.587
#>            WS    WD Soil_VWC Soil_T1 Soil_T2   WTD
#>         <num> <num>    <num>   <num>   <num> <num>
#>      1: 5.605 153.8       NA   3.979   4.441    NA
#>      2: 5.713 160.8       NA   3.984   4.438    NA
#>      3: 5.388 157.7       NA   3.967   4.430    NA
#>      4: 5.675 157.1       NA   3.960   4.426    NA
#>      5: 5.320 155.8       NA   3.946   4.418    NA
#>     ---                                           
#> 736317: 1.237 218.6     91.2   5.090   4.911  4.48
#> 736318: 0.726 226.5     91.2   5.084   4.909  4.49
#> 736319: 0.738 212.5     91.2   5.085   4.907  4.50
#> 736320: 1.286 239.3     91.2   5.086   4.906  4.50
#> 736321: 0.859 253.6     91.2   5.086   4.912  4.53

The data consist of a timestamp column, and 13 variables observed every 15 minutes from 2003 until 2023, giving 736321 rows and 14 columns.

Observation metadata dt_meta 

knitr::kable(dt_meta[site == "UK-WHM", ..v_col], format = "html")
site name_dt name_local units_local type time_char_format range_min range_max name_era5 units_era5 imputation_method
UK-WHM site site site NA NA site era5
UK-WHM Timestamp Timestamp time %d/%m/%Y %H:%M NA NA time
UK-WHM Rain Rain mm precipitation 0 50 tp mm era5
UK-WHM LWS LWS 1 arbitrary -99999 99999 rh 1 era5
UK-WHM AirT AirT degC temperature -40 50 t2m degC era5
UK-WHM RH RH % humidity 30 120 rh % era5
UK-WHM PAR PAR umol/m^2/s energy flux 0 2200 ssrd umol/m^2/s era5
UK-WHM Total_solar Total_solar W/m^2 energy flux 0 1200 ssrd W/m^2 era5
UK-WHM Net_rad Net_rad W/m^2 energy flux -500 1200 rn W/m^2 era5
UK-WHM WS WS m/s wind speed 0 30 ws m/s era5
UK-WHM WD WD degree wind direction 0 360 wd degree era5
UK-WHM Soil_VWC Soil_VWC % soil moisture 0 100 swvl1 % era5
UK-WHM Soil_T1 Soil_T1 degC temperature -20 50 stl1 degC era5
UK-WHM Soil_T2 Soil_T2 degC temperature -20 50 stl1 degC era5
UK-WHM WTD WTD cm height -10 10 swvl1 m era5

Site data dt_site

The site metadata required is minimal. Any additional variables can be added, but at a minimum we require the site name, a uniquely identifying code site, longitude, latitude (in degrees with a decimal fraction) and elevation (in m). These are easily gleaned from the supporting documentation and could be entered in excel, read in as a .csv text file, or entered directly in R as below.

dt_site <- data.table::data.table(
  site = "UK-WHM",
  long_name = "Whim Moss",
  lon = -3.27155,
  lat = -55.76566,
  elev = 316
)
dt_site
#>      site long_name      lon       lat  elev
#>    <char>    <char>    <num>     <num> <num>
#> 1: UK-WHM Whim Moss -3.27155 -55.76566   316

In practice, we are likely to have multiple sites as in the table below, and it is easiest to append rows to a .csv or excel file as new sites are added.

fname <- here("inst/extdata/dt_site.csv")
dt_site <- data.table::fread(fname)
knitr::kable(dt_site, format = "html")
site long_name lon lat elev
UK-AMO Auchencorth Moss -3.243000 55.79230 120
UK-EBU Easter Bush -3.207100 55.86740 119
UK-WHM Whim Moss -3.271550 55.76566 316
UK-BUC Bush Cabin -3.205767 55.86227 189

Create metamet object

TBC