How Agri-EPI Centre is optimizing agri-tech with FME

Revolutionizing farming with data-driven solutions at Agri-EPI Centre

Navigating the complexity of data-driven agriculture, Agri-EPI Centre stands at the forefront of revolutionizing farming practices through cutting-edge technology. Born out of a need to promote sustainable and profitable food production, Agri-EPI utilizes innovative technologies to help mitigate climate change effects, including efficient resource management, digital tools and data analytics, soil monitoring, precision farming, harvest management, and multi-spectral imaging. They work with various collaborators, from farmers, food suppliers, engineers, scientists, start-ups, and the government to promote sustainable food production practices.

The challenge: Integrating a diverse data ecosystem

The data ecosystem that underpins their operations and their need to integrate data from various sources, such as IoT sensors and Farm Management Information Systems (FMIS), into a coherent, actionable catalogue presented a significant challenge.

Manual data collection, storage, and processing were labour-intensive, error-prone, and time and resource-consuming. There was a pressing need for Agri-EPI to create a range of automated data workflows to expedite data collection from disparate sources, validate and ensure robust data quality, and prevent data loss through centralizing the data storage and creating a robust backup solution.

The solution: Making agriculture smarter with the FME platform

Recognizing the need for a platform that could handle data diversity efficiently and flexibly, Agri-EPI reviewed the functionality of several tools. The preferred solution, FME, quickly emerged because it offered combined capabilities from many of the other assessed solutions under one offering, robust support for large-scale data processing, and flexible deployment for cost-effectiveness. FME’s capabilities extend beyond spatial data, making it a powerful tool for various data challenges in the agri-tech sector.

The team is using FME Form, Flow, and Flow Apps that can be made accessible to agricultural partners who wish to access specific data or can be used to upload content for inclusion in the database for analysis, all made possible with no code required.

Tensing orchestrates FME data integration

Agri-EPI chose four initial data sources for Tensing to develop FME workflows to extract data from and load to an open-source PostgreSQL database. Following their development, Tensing configured and deployed FME to orchestrate processing transactions. With the new system, FME Flow integrates and centralizes data on a daily cadence into a data warehouse from the following data sources:

• Richies Beef Monitor: Used to integrate cattle weight data, allowing for better livestock management. This web dashboard does not offer an API interface, so FME authenticates and then ‘scrapes’ the data directly from the web page. The solution is flexible enough to iterate through a range of URLs to extract the full dataset.

The Ritchie Beef Monitor is a smart water trough that weighs cattle via tag scanning, relaying data to an online platform.

• Fullwood: Real-time dairy milking data is integrated into the system, enhancing dairy production efficiency and sustainability. The system produces CSV output files regularly, and FME Flow retrieves these files and loads only the incremental changes to the database.
• UNIFORM Agri: The integration of herd health and treatment data allows for better monitoring of animal health and improved treatment plans, enhancing the overall welfare of livestock. FME connects directly to the associated database and synchronizes any changes.
• IrriMAX: A network of 70 high-precision soil sensors across 20 farms provides crucial data on soil moisture, salinity, and temperature, enabling real-time monitoring for precision irrigation. IrriMAX Live exposes the sensor data via an API, and FME retrieves the data.

An IrriMAX sensor installed in a potato field provides real-time soil data via an API.

Results: Efficient data management and a strong foundation

Every day, FME gathers thousands of new data points from the four systems. This data is centralized in a database for analysis and can be easily filtered, queried and exported in the desired format. The system has proven its robustness already as it collected all missed data following a recent server downtime period due to relocation. 

The updated process has saved the team countless hours each month in data collation and has bridged any potential gaps in data recording. Much of the data available in the past was ‘dark data,’ as due to the sheer volume, it was impossible for Agri-EPI to analyze. FME-facilitated dashboards provided the first look at their data in a processed form, enabling them to analyze data in a meaningful way without hours of manual work. FME has also helped them avoid potential data duplication and ensured the most recent data is readily available to the team.

Panagis Tzivras, GIS Software Engineer from Agri-EPI Centre, notes “effective data management is critical to our mission at Agri-EPI. Through our partnership with Tensing and use of the FME platform, we’ve transformed our data workflows and improved the accessibility of our data”.

The in-house technical team has begun adapting the workspaces to compile data from additional sources, with FME smoothing the transition. With the first steps setting the foundation for preparing for future data pipelines and delivery formats and methods, the initial integration of different systems and the automation of data collection is a useful framework to evolve and build more complex data workflows and automations moving forward.

Preparing for future growth with FME’s LiDAR support

Agri-EPI has recently started collecting multi-spectral imaging and Light Detection and Ranging (LiDAR) data from their farms, using Unmanned aerial vehicles (UAVs). Until recently, this was only possible for large-scale mapping projects, but with the accessibility of UAVs, undertaking analysis of orchards and vineyards is now possible, as FME is also capable of managing and transforming point cloud LiDAR and raster data efficiently. Agri-EPI Centre and Tensing will continue to use FME to solve new data integration challenges and create actionable data-driven insights for Agri-EPI and their diverse collaborators.

Learn more about how FME can be used to build resilience and mitigate the impacts of climate change from our webinar with Tensing and Agri-EPI.