Post provided by William J Sutherland, Erica Fleishman, Michael Mascia, Jules Pretty and Murray Rudd

To celebrate the 10th Anniversary of the launch of Methods in Ecology and Evolution, we are highlighting an article from each volume to feature in the Methods.blog. For Volume 2, we have selected ‘Methods for Collaboratively Identifying Research Priorities and Emerging Issues in Science and Policy’ by Sutherland et al. (2011).  In this post, the authors discuss the background and key concepts of the article, and changes in the relation between science and policy since the paper was published.

 

Between the late 1990s and early 2000s, recognition of the value of scientific evidence to government decision-making grew. As interest in projecting future issues to inform policy decisions increased, we recognised that ecologists did not have the methods to conduct this type of work effectively. In the United Kingdom, the Government Office for Science established the Foresight programme to support policy making; scientific advisory committees became common, and every Ministry appointed a Chief Scientist. Given this context, we explored the use of horizon scans to assess the future and better understand uncertainties.

In the United States during the 1990s, the notion of researchers and practitioners collaborating to establish priorities and research needs, and to understand better the opportunities and constraints under which the different entities work, became much more widespread. Numerous government-funded partnership networks are now in operation, and it has become common for some agencies to expect responses to calls for proposals to indicate plans for co-production, stakeholder engagement, and development of actionable science.

Considering the current global COVID-19 pandemic, the relevance of science and its translation to decision-making by governments worldwide is evident.  Many governments failed to recognise the political and economic threat of a pandemic although pandemics had been identified as a significant risk by a considerable number of countries, such as in the United Kingdom’s National Risk Register.

Horizon Scanning to Identify and Prioritise Research Topics

Lead author William J Sutherland (hereafter WJS) was aware that ecological research often did not appear to inform policy and management, perhaps because ecologists rarely asked policy makers and practitioners what research they would find useful and applicable. Accordingly, he convened a workshop with that objective. The resulting paper, published in Journal of Applied Ecology, was the third most downloaded paper in any of Blackwell’s 850 journals that year.

WJS was struck by the fact that George W. Bush’s move to biofuels in 2006, followed swiftly by similar action by the European Union, surprised many ecologists, and little research existed on the potential environmental effects of biofuel use. His colleague Andrew Watkinson had mentioned horizon scanning as a sensible method for identifying potential future trends, and WJS decided to investigate this method further. In 2007, he brought together 35 individuals from diverse sectors for a horizon scan of threats and opportunities for conservation of biological diversity in the United Kingdom.

WJS’ early work motivated identification and prioritization of research topics with practical application to multiple disciplines at the global level and in individual countries, such as the United States and Canada. It also led to additional horizon scans at different geographic levels. As such initiatives began to proliferate, and their leaders sought advice on methods, we decided to share our experiences and offer guidance on these collaborative efforts.

 

Article Overview

The aim of our article was to describe the novel process we developed to identify high-priority information needs that can be met by new research in both the social and natural sciences. We wanted to suggest processes for conducting similar exercises. As stated in the original paper, these methods are based on principles of inclusivity, openness and democracy. Methods to maximize inclusiveness and rigour in such exercises include solicitation of questions and priorities from an extensive community, online collation of material, repeated voting and engagement with policy networks to foster uptake and application of the results.

As we stated then, these methods are transferable to a wide range of policy or research areas within and beyond the conservation sciences.

Development of Methods Over Time

Over time, we improved the rigour of these exercises. For example, during the first, United Kingdom based 100 questions exercise, we attached printed questions to the walls and asked participants to indicate moderate or strong support for each with silver or gold stars, respectively. Participants largely used gold stars, breaking numerous guidelines for avoiding bias. 

In our horizon scans, we increasingly have sought to reduce the impact of individual biases or preferences on the collaborative outcome. For example, for each horizon topic that is retained through the first round of culling, we assign two individuals who did not suggest the topic to research it before group discussion, and we use median ranks to determine which topics will be  included in the final list (thereby excluding outliers). Rudd and Fleishman (2014) asked participants to apply best-worst scaling to prioritize 40 questions identified in a previous exercise, minimizing the potential for anchoring biases and survey inattentiveness.  

Following completion of our tenth annual horizon scan of topics applicable to global conservation of biological diversity, we reviewed the 15 topics identified a decade ago and assessed their development in the scientific literature and news media. Five topics appeared to have had widespread salience and effects. The effects of six topics were moderate, three had not emerged, and the effects of one topic were low. We noted that an increase in interest and research following the identification of an issue may not always be desirable. Simply highlighting risks and opportunities may motivate rapid behaviour change that leads to outcomes favourable for society or its values.

One key message of our paper, that it is sensible to look for issues that might become prominent in the future and to create agendas for research, are reasonably well accepted. Another key message is that the manner in which decisions are made is critical. Simply consulting with advisers or maintaining a standing group of experts is a weak way of informing decisions. Instead, it is worthwhile to consider how experts contribute to the decision-making process. For example, wider use of versions of the Delphi Technique may be useful. Examining the evidence, voting, anonymous decisions, and iterating the decision-making process all improve the application of science. As discussions of COVID-19 highlight, clear and rigorous means of making decisions are essential. 

There is potential to use the questions and issues raised in exercises such as ours as a basis for further research on the science-policy interface itself, increasing understanding of how, when, and why scientists might be willing to engage collaboratively with policy-makers, and on differing perceptions about the credibility of scientific evidence.

There often remain gaps between scientific consensus or understanding at a given time and the political actions that are most likely to maximise environmental and social goods. It is difficult to change fixed attitudes and the reluctance of some to use evidence. But clarity on processes by which scientists deliberate and identify consensus priorities can strengthen the weight of facts and evidence. It can identify potential future surprises.

To read the selected Volume 2 Methods in Ecology and Evolution article in full, visit ‘Methods for Collaboratively Identifying Research Priorities and Emerging Issues in Science and Policy‘.

To find out about the selected Methods in Ecology and Evolution article for Volume 1, visit 10th Anniversary Volume 1: The Art of Modelling Range-Shifting Species.