2026 Week 28

Metadata

• Continued improvements to the metadata structure, with great discussions around how we define and model a site vs a station.
• Unique labels for every timeseries now available for the UI to use, and leave behind the duplication days

Public UI - Landing Page

Landing page has been redesigned based on designed by David Green, looking closer to being ready for our public launch in september. See it here

Ecolog data is live in our UI!

We now have ecolog data in our ui, see it here.

This makes the following diagram, now fully green (we still have a little tidy up to do).

Sensor integration

We had a successful test of switching a Campbell datalogger between AWS accounts (from the original test account, to the long-term production account).

This was done fully remotely via MQTT using the open source Campbell Control library developed as part of FDRI. Thanks to Nathan for curating the infrastructure so that this could be done without physical access, and to Jasmine in the field team for being our guinea pig, remotely changing settings and rebooting from the command line :)

Flux

After some configuration teething troubles we how have flux data flowing in from Tring Grange to the production account - in hourly chunks in Campbell tob3 format, about two hours behind when it was captured. Thanks to Andrew Sun in IT for patient testing of this! Starting the process of automating download of the historic data captured at the site, with files currently going back to late February - over a slow connection, could take a while.

The data is then mirrored back to the staging account for processing.

Ingestion

With the flux data going into staging we have now managed to deploy the ingestion service and can see flux data coming into the ingested bucket. After a few days monitoring, the ingestion service will be deployed into the production account. Then - processing!

🥳 A Gridded Data Bonanza 🥳

Tbh I just wanted to use the word “bonanza” as it was in my head for some reason, but there has genuinely been a lot of stuff happening in gridded data land these past few weeks:

• New Datasets: Both Hydro-PET and HadUK-Grid Rainfall have been converted and uploaded (both chunking versions)
• The GEAR-1hr dataset has been rechunked to allow more performant access depending on what type of analysis you’re doing
• New Notebooks: Notebooks showing how to access and do basic analysis are ready for Hydro-PET and HadUK-Grid Rainfall, in addition to the existing notebooks for GEAR-1hr
• The R versions of these notebooks have been adapted to use stars instead of R’s basic zarr package to improve the performance here as well
• A simple comparison has been done of our previous ‘one-size-fits-all’ chunking strategy, and our new chunking strategy that has two versions of the dataset chunked to be performant for their respective analysis types.