Standardising Methods in Climate Change Experiments: A Community Effort

Post provided by AUD HALBRITTER

CHINESE TRANSLATION PROVIDED BY HUI TANG

這篇博客文章也有中文版

Climate change is threatening biodiversity and ecosystems around the world. We urgently need to better understand how species and ecosystems respond to these changes. There are already thousands of climate change experiments and observational studies out there that could be used to synthesise findings across systems and regions. But it turns out that making meaningful syntheses isn‘t always so straightforward!

The Need for Standardised Methods and Reporting

There are two major challenges (and some minor ones too) for synthesising data across different experiments. First, the data are not always available. This problem arises because key study information – such as metadata, covariates or methodological details – are often not adequately or consistently reported across studies.

The second problem is that scientists use different protocols. This leads to a diversity of ways of measuring and quantifying the same variables. Different protocols may measure or report the same variables in slightly different ways, so the data are not compatible. Consistency in measurements and protocols is one reason why working in large networks – such as ITEX, Herbivory, or NutNet – to name only a few, is so powerful. In these networks, experiments and observations are repeated across large regions or worldwide using strict protocols for experimental design and measurements.

But what about those of us who aren‘t part of those networks? The answer is simple… in theory. If we all used the same standard methods and protocols across the community, the data from these studies would be re-usable and comparable. The reality is quite different though, and the big question is, “how can we get there?”.

From a  Problem to a Handbook

At a ClimMani COST Action Network meeting in Copenhagen in 2015, this issue was discussed. Vigdis Vandvik and Hans DeBoeck led a discussion group on ‘facilitating synthesis by standardising methods across experiments’. Together with a small group of participants (Nelson Abrantes, Thomas Wohlgemuth, György Kröel-Dulay, Sara Vicca, Philip Ineson, Jürgen Kreyling, and Håkan Wallander), they decided that the solution was really quite simple: let’s write a handbook!

They came up with the title, “Handbook for field measurements in ecosystem research”, and (because they realised this would be a lot of work) decided it was best to involve a lot of people. So they set out to contact all the experts they knew from across the community “to make sure that enough people are willing to invest time” as they wrote in the report from the meeting.

Vigdis and a group of students, PhDs and postdocs (including myself) at the University of Bergen started collecting all protocols from climate change experiments that we could find, while eating pizza sponsored by ClimMani (thanks to Linn Vassvik, Christine Pötsch, William Fernando Erazo Garcia, Inge Althuizen, Francesca Jaroszynska, Ragnhild Gya, Amy Eycott, and Siri V. Haugum for helping with this boring but crucial task).

“Why involve students?” you might ask. Well, students are one of the main target audiences for the handbook. They‘re also really used to reading up on new methods to find out which ones to use and how. Also, maybe because of this, the students really saw the point of the handbook and were keen to contribute! So keen, in fact, that several of them have now ended up on the main contributors part of the author list – a special thanks to Bernd Berauer and Ragnhild Gya.

In March 2017, we organised a workshop at Finse in the mountains in southern Norway (where the scenes from the planet Hoth in ‘The Empire Strikes Back’ were filmed). The  group had now grown to 26 ClimMani members. We discussed the goal, scope, and structure and identified five main groups (chapters) of responses that we generally measure in climate change experiments. We decided we would not only write the protocols, but also organise an internal review process.

In this review process experts who were not involved in writing a protocol would give feedback to the author(s) of each chapter and protocol. Lead authors were appointed to organise the writing and reviewing of the different chapters. I took on the challenging task of helping Vigdis out with overseeing the process, communicating with the authors, and making sure that everybody met the deadlines. The workshop also included some skiing and sauna, and it was documented extensively.

We quickly figured out that we needed even more help and expertise to write the handbook. Colleagues and other experts from the community were identified and the response from the community was extremely positive. It turned out that everyone we asked was keen to join what was by now known amongst the authors as ‘the most boring paper we ever wrote’ (repeatedly writing the methods section of a paper)! [Editor’s Note: At Methods in Ecology and Evolution, we think that methods sections are super fun and cool.] A huge amount of work lay before us, but it seemed like a more manageable task now with over 100 experts contributing. And they did contribute and deliver protocols without needing excessive amounts of nagging.

The protocols were finalised by the end of 2017. A second workshop was organised in January 2018 in Bergen to bring together all protocols and agree on data presentation. Gaps were identified and a few protocols were added and some expanded. Once the handbook was finished, we wrote ‘The handbook for standardised field and laboratory measurements in terrestrial climate‐change experiments and observational studies (ClimEx)‘, which is now published in Methods in Ecology and Evolution and is really just a motivation for and pointer to the actual handbook (appendix of the paper).

The ClimEx Handbook: A Community Effort by 115 Experts

The handbook of standardised methods in climate change studies (the ClimEx Handbook) provides detailed descriptions of standardised methods for measuring a broad range of response variables in terrestrial ecosystems. It focuses on plant communities and the organisms they interact with. The 66 protocols (539 pages) are divided into 5 chapters, which are presented as appendices to the MEE paper:

  • site characteristics and data management
  • carbon and nutrient cycling
  • water cycling
  • species and interactions
  • stress physiology

In each protocol, we describe which response variables should be measured broadly across studies to facilitate data re-use, synthesis, and upscaling. All of them have a Gold standard and, if applicable, we offer minimal requirement Bronze standards. We discuss special cases, emerging issues and challenges that address adaptations needed for specific systems or situations as well as new methods under development. Finally, we give guidance on additional variables that should be measured more widely to improve inter-site and inter-study comparisons and model-experiment interactions.

The protocol writing team consisted of 85 authors from all career stages – from master students to retired professors. The review round was conducted by 50 experts. In total, 115 scientists, of which 45 were women (39%), from 21 countries on four continents were involved.

Extending the Shelf Life of The ClimEx Handbook

The ClimEx Handbook contains many established protocols, that are relatively constant and will not change much over time. Others, and especially the ones that rely heavily on developing technology, will by nature be outdated in the not-too-distant future. We would like the ClimEx Handbook to remain a useful resource for the community in the future. So, we‘ve made the handbook available on the ClimEx Handbook webpage.

We encourage scientists from the climate change research community to give us feedback and make suggestions for updates as needed to specific protocols. These comments will be assessed by the authors of ‘The handbook for standardised field and laboratory measurements in terrestrial climate‐change experiments and observational studies (ClimEx)‘. Any accepted updates will be time-stamped and visible in the online version. Hopefully this is a way to amend the protocols and extend the shelf life of the ClimEx Handbook.

Paying it Forward: Continuing the Community Effort

In hindsight, the process of developing and writing the ClimEx Handbook has been very rewarding. The discussions – both over which protocols to include and recommend and how these protocols map onto different types of synthesis and reuse – have been challenging, educational, and interesting. And the good news is that it’s not too late to get involved!

We hope that the broader climate change community will get engaged in the ClimEx Handbook to help us develop and improve it, that you will use it as a learning resource, and spread the word to increase its reach and usage. This will ensure that it will still be a useful resource for the community in the future. We believe that using and developing the handbook through a community effort will facilitate awareness by next generation (and older generation) climate change scientists of the importance of using standardised methods to promote data availability, compatibility, and transparency.

These improved research practices should not only result in better returns on the time and money invested in individual climate change research projects, but also facilitate second-order research outputs across projects and scientific disciplines. The ultimate goal is to optimise the quality and impact of our science to fulfil society’s needs in a changing world.

To find out more about the Climex Handbook, read our Open Access Methods in Ecology and Evolution article ‘The handbook for standardised field and laboratory measurements in terrestrial climate‐change experiments and observational studies (ClimEx)‘. (No Subscription Required)

1 thought on “Standardising Methods in Climate Change Experiments: A Community Effort

  1. Pingback: Issue 11.1: Climate Change, Genomic Divergence, Bayesian Modelling and More | methods.blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s