FDRI Weeknotes

2025 Week 39

Lots happening, this sat in my drafts for toooo long.

NRFA

The NRFA work has been taking over the last few weeks, lots of progress across different areas:

Backups

Pasted image 20250926170614

We had an unfortunate cost explosion when setting up the backups for our raw timeseries sensor data. The backup was set to run every hour, with the initial run estimated to cost $10-15 and future incremental backups to cost a few pennies. Unfortunately our backups were failing for a week which cause the costs to balloon since it was costing us $10-15 per hour! We fixed stopped this promptly once noticed and configured cost alerts that get sent to people at CEH so we can respond quicker if a cost explosion was to happen again.

We’ve fixed the backups now and our fdri raw data is being backup to another s3 bucket using the glacier tier.

Medium Term Timeseries Plan

This week we collectively came up with a medium level plan for the work we are currently doing and were we think we will be in 3 months.

Workstreams with 3 month plan here: https://github.com/NERC-CEH/fdri_words_private/blob/main/2025-09-25-workstreams.md

Metadata Site IDs

Ongoing discussion on getting a complete list of current site ids and how our processes should look.

Phenocam ingestion

The phenocam ingestion process is now indexed in postgres for quicker lookup and searches across the images. This was the first time we are connecting to the database from our k8s cluster, this required some linked between AWS IAM Roles -> K8s Roles -> Postgres Roles. A write up of this is available here: https://github.com/NERC-CEH/fdri_words/blob/main/timeseries/RDS-Database-Access.md

NRFA ingester docs

We are ingesting NRFA data from SEPA and EA, a write of this is here: https://github.com/NERC-CEH/fdri_words/tree/main/timeseries/tools/ingester/nrfa

RDS postgres roles <-> k8s service accounts <-> AWS IAM role mappings

We worked out some tricky permissions mappings, detailed here: https://github.com/NERC-CEH/fdri_words/blob/main/timeseries/RDS-Database-Access.md

This now allows applications to connect to our main database with restricted permissions to only access read and/or write certain schemas.

Debugging inside a k8s cluster

Related to the above, debugging permissions (it’s always a typo) can be tricky, write up here on how we create debug pods to help us step inside the cluster to help with debugging.

https://github.com/NERC-CEH/fdri_words/blob/main/timeseries/Debugging-inside-K8s-Cluster.md