Post provided by Katharina Gerstner

Quantitative syntheses of primary research studies (meta-analysis) are being used more and more in ecological and evolutionary research. So knowing the basics of how meta-analysis works is important for every researcher. Meta-analytical thinking also encourages us scientists to see each single primary research study as a substantial contribution to a larger picture.

To be included in a meta-analysis, relevant primary research studies must be easy to find and basic information about the methods and results must be thoroughly, clearly and transparently reported. Moreover, papers with accessible data are the most useful for meta-analyses. Many published papers provide this information, but it’s not unusual for essential data to be omitted. Studies that are missing these details can’t be used in meta-analyses, which limits their reach.

Getting Found in a Literature Search

People usually begin meta-analyses with a systematic literature search aiming to get a representative sample of existing primary studies. Identifying a search strategy is a first – and crucial – step. In this step, meta-analysts choose their data sources, some popular ones are search engines (e.g. Web of Science, SCOPUS, Google Scholar) and grey literature. They also decide on the keywords used for the literature search. To make your research more likely to be identified in a literature search, you should carefully consider your choice of title, abstract content and keywords. Ideally, you want to find a balance between being broad enough to be found through a keyword search and being specific enough to be identified as relevant.

This may seem obvious, but primary studies must be accessible to the meta-analyst to be included in a meta-analysis. There are ongoing debates about the pros and cons of open access, publishing through this model makes things much easier for meta-analysts though. If a publication cannot be made open access (and this often isn’t an option), many journals allow primary studies to be posted online in an unedited form (usually the pre-peer reviewed form) and this can substantially reduce the effort for meta-analysts to gain access to primary studies. Always make sure to check your license agreement before posting any accepted articles though.

Reporting Usable Outcomes of Primary Research to Enable Calculation of Effect Sizes

To be jointly analysed in a meta-analysis, the outcome of each included study must be expressed on a common scale. This measure of outcome, called ‘effect size’, includes information on the direction and magnitude of an effect of interest from each study  The most commonly used effect sizes in ecology are standardised mean differences, response ratios and correlation coefficients (using Fisher’s Z transformation). Accurate data extraction from primary research studies is essential to calculating effect sizes and it’s one of the most time-consuming parts of a meta-analysis.

The success and legitimacy of a meta-analysis depends on accurate and complete reporting of study outcomes. Reports of primary research outcomes should generally include basic information on means, sample sizes and measures of variation to be useful in a meta-analysis. If correlation coefficients are reported or calculated, the sample sizes should be provided.  It’s best to have access to the raw data as well wherever possible.

A common goal of ecological meta-analyses is the analysis of effect sizes and causes of variation in study outcomes. So another crucial step alongside the data extraction for calculation of effect sizes is the data extraction of relevant covariates (moderators). Variation in study outcomes may be due to biologically meaningful and important covariates (organism traits, climate, population density, etc.) or to methodological covariates (study duration, experimental conditions, source of material used, etc.). It’s important for us to know which of these covariates had the greatest impact on the study, so detailed descriptions of experimental methods, study design, and the study area are essential. Further information about the environmental context such as climate, soil, elevation, the specific type of ecosystem and other contextual features are useful to our interpretation of results and help to make them comparable with other sites or studies.

Increasing Data Accessibility

Although complete access to all data in published studies has been a much-debated issue, for meta-analyses it’s extraordinarily useful. Published raw data along with sufficient metadata can reduce effort, ambiguity and error in subsequent analysis steps.

To encourage the use of primary research studies in meta-analysis studies our recent commentary in Methods in Ecology and Evolution, ‘Will your paper be used in a meta‐analysis? Make the reach of your research broader and longer lasting’, lists core issues, action-items and reasoning to be considered when publishing primary studies. It includes examples of common errors in descriptions of methodology and reported results. We provide some suggestions of how study details should be correctly reported to enable calculation and analysis of effect sizes.

Whether the results from primary research studies are included in subsequent meta-analyses or not, these guidelines are important for reporting research results in general. This is because they‘re necessary for the interpretation and assessment of study outcomes. Increased implementation of these guidelines by all of us – authors, editors and publishers – and reinforcement by funders, will lead to higher quality and more inclusive syntheses, further the goals of transparency and reproducibility in science, and improve the quality and value of primary research studies.

To find out more about how to make your research usable in meta-analyses, read our Open Access Methods in Ecology and Evolution article ‘Will your paper be used in a meta‐analysis? Make the reach of your research broader and longer lasting’.

This article was shortlisted for the 2017 Robert May Early Career Researcher Prize. Check back here each day this week to find out about more shortlisted articles and find out the winner on Monday 16 April.