2025 Week 10
A bit late this week since I was on leave on friday, but still lots happening. Hope to style this site a bit better with some FDRI theming, better navigation and contact details etc.
❇️UI
This week we added some improvements to the data explorer chart. You can now select multiple variables at a time to plot on the same chart. We are about to expand this to allow plotting any variable from any collection and site with any other, watch this space!
We also had some questions about our choice of using plotly.js. Some suggested alternative libraries were recharts and nivo charts. To quickly evaluate these libraries we were able to ask cursor to rewrite our codebase with them, which it did a pretty good job of! We aren’t planning on switching charting libraries finding plotly to still have the most out the box features, but it was really nice to be able to quickly compare charting libraries like this.
🚒Grafana Alerting
We have been using grafana for a while to monitor our systems health. This week we set up alerting so we receive a message on a dedicated teams channel when something goes wrong. We’ve a few cases were things have broken, look very broken in our grafana dashboards, but nobody was looking. This should force us to know when something goes wrong.
🥧WP1 Raspberry Pi devices
Work Package 1 have setup some raspberry pi devices that they are trialling out, including some cameras. We are still figuring out how these would fit in with what we have so far but at the moment we are thinking
- Custom MQTT client setup on the raspberry pis for device management
- Direct to s3 photo upload via python scripts for data
🔍IOT raw message browsing/searching
Currently when messages come in via MQTT we write them directly to s3, this is done by a simple configuration rule and is very reliable. However it means this raw data isn’t easy to browse, each message needs to be downloaded individually. We started looking at setting up AWS opensearch to also forward the messages into. We hope this provides a simple UI and easy way to browse this raw data. If it works well we plan to keep around 30 days of data searchable in this way. We made some good progress on setting this up this week, but it’s not quite finished.
📸 FDRI Phenocam
After getting the images streaming into our lin2 server the week before, we were briefly blocked when they got switched to read only, so we could see the images existed but couldn’t open them. IT were able to fix this for us quite quickly and we have images streaming in to s3.
We are now starting to think about what the plan should be after the end of march, we have a few options on what we could focus on
- Extend the phenocam UI, use login, tagging, mask management
- Start building an image processing pipeline, extracting RGB values into a derived dataset
- Metadata integration
Most likely we will start piecing together the image processing pipeline to create the RGB derived dataset.
🧊 Kubecon 2025
https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/
Kubecon is in london this year, Josh Holland and myself (dom ginger) are planning on attending, please reach out if interested or if you are attending :)
📚 Dev Book Club
Book club started today! (10.03.2025) Thanks for everybody that came, I found it very useful and I think we had some great discussions.