2025 Week 13
Another week has gone by, more updates from FDRI digital developments.
🖼️ Live FDRI Data Viewer
We managed to get the FDRI data table live this week, select “QSF5-STKX-4YVB” on the right. The other options have just come in (things are moving fast) and I see now as writing this they aren’t quite working in our ui. This is showing live data from the FDRI sensor in the labs at wallingford with only a few seconds delay.
Brief tech details
- Uses the existing tech stack we built for COSMOS to ingest -> store -> api -> ui
- Displaying thousands of rows performantly is done using react-window which only renders the data you can see on the screen
🤘More metadata integration
Cosmos explorer ui is now using some metadata to display variable labels and units.
Lots more metadata discussions are on going in the fdri-discovery repo. With an aim to start using it more in our processing pipeline.
As the metadata systems are being developed now that we have started using them we have started seeing some things break as the metadata is improved/changed. This is natural at this early stage of development, but we are starting to think how do we manage dependant systems how do we best communicate breaking changes across teams and how do we build resiliancy into our systems. Anybody with any thoughts please reach out!
🌊NOC Gridded Data and Data Science Platform
Many people in the team were in a cross org session at NOC in liverpool this week. This was mostly focused on
- UKCEH Datalabs and NOC’s similiar Data Science Platform
- NOC and UKCEH work on Gridded Data. Converting netcdf -> Analysis-Ready, Cloud Optimized (ARCO) Zarr and best practices and experiences of processing and visualling this data
This was led by Matt Brown, please do reach out to him to hear more!
🚛Asset Management Software Workshop
There was an early stage Asset Management Software with Arup (external company) this week. Working through the requirements from WP2 on what an asset management software solution would be capable of.
Asset Management Software in this context is the piece that tracks the which sensors are deployed where, e.g a field engineers places a sensor in a field and mark in the asset management software, sensor X was setup at datetime Y, running firmware Z, etc.
This is a critial piece to everything we are building and it’s important we get the digital requirements in at this stage.
📷 Phenocam
We managed to get our basic phenocam ui live! This should show the most recent phenocam image a few seconds after we receive them. This week we’ve also made a few tweaks to the api that should make it support our use cases a bit better.
We still have a few bugs to fix and improvements we want to do on the above, but our original plan is all green!
For most of the phenocam work we have followed how the timeseries work was implemented, ongoing is thinking about the use cases and how they differ, for example the access patterns can be quite different which requires though about how we store our data.