Rothamsted Research, one of the world’s leading agricultural research institutes, has worked with data consultants, Tessella, to make data generated from one of its National Capability openly available to the world’s researchers. The institute conducts research into sustainable farming that strives to address global challenges such as food security, health, and the environmental impact of agriculture.
Tessella’s ability to quickly understand agricultural data on the same level as our researchers was key to their being able to build the right software and propose the IT infrastructure to manage it.
Dr Paul Harris
Project Manager at Rothamsted
Rothamsted has a long history of world leading agricultural science, and through this has made significant contributions to crop, grazing livestock and soil science, but also to the development of statistical and mathematical methods used in agricultural research. It is home to one of the longest running continuously monitored crop and soil experiments in the world, and is considered by some to be one of the birthplaces of modern statistical theory and practice.
Rothamsted’s remit includes ensuring the world’s agricultural researchers can benefit from its insight and capabilities. A vital part of this is ensuring its data are accessible and usable to all. The collaboration between Tessella and Rothamsted Research continues this remit, where livestock and grasslands data from Rothamsted’s North Wyke Farm Platform have been made publically available via a web data portal design by Tessella.
The Benefit of Open Research Data
Professor Chris Rawlings, Head of Department of Computational and Systems Biology at Rothamsted, says: “Opening up data so that the world can benefit from it is a major trend in publically funded research – by sharing data we have a better chance to understand and address today’s big global challenges.”
“The practicalities of actually taking that data, cleaning it up, making it intelligently searchable, presenting it in a usable format, and designing the IT systems to provide easy access, however, is more complex. This project bought together complementary skills in agricultural research, data processing, software design and IT from both Tessella and Rothamsted to develop a system that will help Rothamsted to drive forward its own research agenda and also contribute to future advances in global agricultural research.”
The Data Underpinning Agricultural Research
Rothamsted’s North Wyke Farm Platform in Devon, funded by the BBSRC as a UK National Capability is used for studies of grassland livestock systems. The experimental platform comprises of three livestock farming systems, each using alternative approaches to their management.
Extensive and detailed measurements are taken, including water runoff and water chemistry, soil moisture, rainfall, grass species, soil nutrients, and greenhouse gas emissions all of which are related to the farm management data and the farm’s productivity (e.g. livestock weight gain). Trade-offs between ecosystem losses to productivity gains are core objectives of the research.
This vast amount of data are invaluable to agricultural researchers, both within and outside of Rothamsted Research, who can use it to understand and model different approaches to grassland farming and how they can improve the quality of produce, reduce costs, limit water/soil pollutants, and limit greenhouse gas emissions. The platform not only generates its core experimental data, but also provides a valuable resource for researchers from all over the world to conduct their own specific research projects on the platform (with a proviso, it does not impinge on the platform’s long-term integrity).
Making Data Open and Accessible
Rothamsted Research worked with Tessella to ensure the vast quantities of data collected from the North Wyke Farm Platform, were processed, quality controlled, made searchable, and presented through an easy to use online portal to anyone wishing to use it for their own research.
Dr Paul Harris, project manager at Rothamsted for the Farm Platform Data Portal said: “Tessella’s ability to quickly understand agricultural data on the same level as our researchers was key to their being able to build the right software and propose the IT infrastructure to manage it. This is an important distinction between working with Tessella, with their scientific background and approach, and organisations who are first and foremost software developers.”
“Designing a system to collate and present data, requires an appreciation of the data and how it will be used. Tessella’s business model allowed us fast, flexible and scalable access to their teams. We could draw on a wide range of skills at short notice without long term commitments, and solve a complex problem in a short space of time with limited resources.”
The back end involved designing a management interface and data loading process to upload complex data sets into bespoke databases. Data are automatically tagged to make it easily searchable and then fed into the public portal.
The bulk of the project was creating the front end where researchers access the data. This was designed to be visually engaging, with an easy-to use user interface with rich functionality. Users can quickly get a very good sense of what’s available, search by different terms and quickly and flexibly select, query and download data sets relevant to them. The portal was designed to allow researchers to browse data, select data of interest, preview, and download files with their selected data.
The portal was developed using an agile approach – employing an iterative process of trying, testing and refining. It adopted the Scrum methodology – involving two-week development sprints followed by detailed reviews between Tessella and Rothamsted.
Rawlings says: “When you’re working with huge complex data sets, it’s not easy to visualise the best way to present them. The agile approach meant we were engaged in the process, and it allowed us to make changes and iron out problems collaboratively as we progressed to ensure we reached a result we were all happy with.”