Post provided by Aaron O’Brien · Centro de Biotecnología de Sistemas, Universidad Andrés Bello

When a bioinformatics pipeline crashes, at least you know about it. The failures that worry me are the quiet ones: the run finishes, the species table looks plausible, but there are errors hiding inside the results.

I’m Aaron O’Brien, a bioinformatician and software engineer at the Centro de Biotecnología de Sistemas (CSB), Universidad Andrés Bello, in Santiago, Chile. Much of my work involves turning Oxford Nanopore sequencing runs into pictures of microbial communities, for projects ranging from vineyard soils to compost and bioleaching reactors. I also build open-source software to guard against those quiet errors. This post tells the story behind EMITS and its companion tool, ITSxRust, both in Methods in Ecology and Evolution.

Long reads, old tools

To identify fungi in environmental samples, most studies sequence the internal transcribed spacer (ITS), the standard DNA barcode for fungi. The region has three parts: ITS1, the short 5.8S gene and ITS2. Short-read studies usually sequence only ITS1 or ITS2. Long-read platforms from Oxford Nanopore and PacBio read the whole region in one go, which opens the door to species-level identification.

The catch is that much of our analysis software was built for short reads. On long reads it can fail quietly at two steps in particular, and ITSxRust and EMITS each tackle one.

I came to this the slow way, through a species table I couldn’t quite believe. The run had worked, the numbers added up, and the list confidently named relatives I had no reason to expect in that sample. Nothing had failed loudly enough to be worth investigating, so the only way to find out what had gone wrong was to take the pipeline apart a step at a time. What I found were two places where reasonable-looking software quietly makes a guess on your behalf.

Step one: finding the fungal barcode with ITSxRust

Each long read carries the ITS region plus pieces of the ribosomal genes on either side, so the first job is trimming it down to the barcode. That means finding four landmarks: the end of the gene before ITS1, both ends of 5.8S and the start of the gene after ITS2. With noisy reads, or primers that clip a flanking gene, a landmark is often missing. A tool can then discard the read or cut at the read’s edge instead. Cutting at the edge quietly returns a ‘barcode’ that still carries part of the neighbouring gene, typically about 30 bases in our data.

ITSxRust links the four landmarks into a chain. When one or two are missing, it falls back to the pairs it can find, so every region it returns sits between two landmarks it actually detected. In both our test datasets, this rescued thousands of reads that would otherwise have been thrown away. ITSxRust also records why each rejected read failed, so you can tell whether the problem lies in your primers or your settings. It’s available on Bioconda and in the nf-core/ampliseq pipeline.

Step two: counting species with EMITS

Next, you compare each trimmed read with a reference database such as UNITE, and the usual approach credits it to its single best match. That breaks down when close relatives share near-identical barcodes. Add sequencing noise, and the top-scoring match isn’t always the right species. Databases also hold many entries for the same species, which scatters one species’ reads across many records. This matters because species in the same genus can lead very different lives: Penicillium includes both the mould that gives blue cheese its veins and moulds that rot stored fruit.

EMITS uses a statistical technique called expectation–maximisation (EM). Imagine hearing a voice at a party that could belong to either of two identical twins. On its own, the voice can’t tell you which twin spoke. But if you’ve seen one twin all evening and the other not at all, you’d bet on the first.

EMITS makes that bet formally. It first estimates who is in the community. It then splits each ambiguous read among its candidates, according to how well the read matches each one and how common each seems to be. Finally, it updates the estimate and repeats until the answer stops changing. It also pools entries for the same species into one count.

EM had already shown its value for bacteria with Emu. Using Emu for fungi, though, meant building a custom reference database and relying on settings tuned for bacterial 16S data. EMITS works directly with UNITE and EUKARYOME, with presets for current nanopore and PacBio HiFi chemistries.

Putting EMITS to the test

We tested EMITS three ways:

• simulations in which we could dial sequencing noise up and down;
• a nanopore-sequenced ATCC mock community, a mix of 10 fungal species in known proportions;
• a synthetic 21-species community built from reference sequences.

In the simulations, best-hit error climbed as noise rose while EM error stayed essentially flat: under realistic noise it was 80–92% lower. In the mock community, EMITS separated close relatives in genera such as Trichophyton and Penicillium that best-hit counting mixed up, and it held that resolution where Emu, run against a UNITE-formatted database, spread abundance across the wrong species. In the synthetic community, it cut the abundance assigned to species that weren’t there by just over half. During revision we added PacBio HiFi benchmarks and EUKARYOME support, and the tool is better for it.

What’s next

The two tools chain together. ITSxRust trims reads to the barcode, minimap2 aligns them to a reference database, and EMITS turns the alignments into a species table. Both are open source on GitHub (ITSxRust, EMITS).

EMITS answers the question ‘which known species did this read come from?’ The harder question is what to do when the honest answer is ‘none of them’. Most fungal species are still undescribed, and a read from an unknown species will often still find some best match in the database. Flagging those reads, rather than quietly giving them a name, is what I’m working on next.

EMITS and ITSxRust were a team effort with my co-authors at CSB: Catalina Lagos, Kiara Fernández, Bárbara Ojeda and Pilar Parada. If you try them on your own data, I’d love to hear how they get on!

Read the paper here.