This guide will cover the download, installation, and local use of the fastp 1.3.2 (Chen et al. 2018) for the pre-processing of fastq reads for adapter removal and length and quality filtering of paired-end reads.

A Linux based system with Miniconda will be required for this guide and some high computing power is suggested, such a system might exist for your institution or can be purchased through an online cloud computing provide. Follow my guide on cloud based VM setup here: https://scottc-bio.github.io/guides/Virtual-machines-for-bioinformatics.html

A basic understanding of LINUX such as creating and moving directories etc. is assumed for this guide.

1 Installing fastp

fastp is easily installable with conda so create and activate and environment for it.

conda create -n fastp_env fastp -y
conda activate fastp_env

Can check the install by checking the version.

fastp --version

This should return the version number, if it does then it has installed correctly.

2 Preparing output directories

I like to make separate output directories for the trimmed reads outputted from fastp and another one for the reports.

mkdir trimmed_reads
mkdir fastp_reports

3 Running fastp

A single command can be used to perform adapter removal and length filtering.

fastp \
-i sample_R1.fastq.gz \
-I sample_R2.fastq.gz \
-o trimmed_reads/sample_trimmed_R1.fastq.gz \
-O trimmed_reads/sample_trimmed_R2.fastq.gz \
--length_required 30 \
--detect_adapter_for_pe \
-h fastp_reports/sample_fastp.html \
-j fastp_reprts/sample_fastp.json

The path to the raw read fastq files can be changed as well as the paths to the output directories.

If your samples are not paired-end, then just remove the ‘-I’ and ‘-O’ arguments for the reverse reads.

The lenght filter can be changed but 30 is a sort of default.

Additionally, a base quality filter can be added using the ‘-q’ argument e.g. ‘-q 20’. This will remove bases from the ends of reads that fall below this threshold until bases above this quality are reached. However, usually Illumina short read sequences are super high quality and this might not be necessary. It is a good idea to use fastqc (Andrews 2010) to check the quality before and after using fastp, and my guide on how to use fastqc can be found here: https://scottc-bio.github.io/guides/Metabarcoding-quality-control-with-FastQC.html

3.1 Handling multiple samples

To automate this for multiple samples, making a .sh bash script with a loop is the easiest way. To do this, first open a new .sh file editor in Linux:

nano run_fastp.sh

In the text editor that open, place the following code:

#!/bin/bash

for sample in path/to/raw/reads/*_1.fastq.gz
do
    base=$(basename "${sample}" _1.fastq.gz)

    echo "Processing ${base}"

    fastp \
        -i "${sample}" \
        -I "path/to/raw/reads/${base}_2.fastq.gz" \
        -o "trimmed_reads/${base}_trimmed_1.fastq.gz" \
        -O "trimmed_reads/${base}_trimmed_2.fastq.gz" \
        --detect_adapter_for_pe \
        --length_required 30 \
        --thread 8 \
        -h "fastp_reports/${base}_fastp.html" \
        -j "fastp_reports/${base}_fastp.json"

done

Update the paths to the raw read files. These will need to be the relative paths from the location of the .sh file.

I also added the threads argument as fastp will default to 1, so using more will speed things up.

To save this file, press ‘Ctrl + O’, ‘Enter’, ‘Ctrl + X’.

Then make the file executable.

chmod +x run_fastp.sh

Then run.

./run_fastp.sh

3.2 Running in the background

For a lot of samples, this can take maybe an hour or more. It is ultrafast but still takes some time and scales with the number of samples and reads. So it is useful to be able to run this in the background.

The best way to do this is to use tmux.

Create a new tmux window.

tmux new -s fastp_run

Activate the fastp env in this new session.

Then run fastp as before using the .sh file.

You can then detach from this session and any processes will continue to run in the background. Detach by pressing ‘Ctrl + b’ followed by ‘d’.

Then can re-attach to check progress with:

tmux attach -t fastp_run

4 Understanding the output

Fastp will output:

  • Trimmed reads fastq.gz - These are the processed reads from the raw input files
  • .html reports - This can be opened in browser and for each sample is an interactive report on read quality before and after processing with fastp. THe plots are pretty easy to understand but you just want to make sure there is nothing strange about the quality plots or sequence length. But fastp should have handled these issues.
  • .json reports - These contain the same information as the .html reports but are used by software. MultiQC (install and use instructions found here: https://scottc-bio.github.io/guides/Metabarcoding-quality-control-with-FastQC.html) can be used to assemble all the reports into one interactive report:
multiqc fastp_reports -o multiqc_out

References

Andrews, Simon. 2010. “FastQC: A Quality Control Tool for High Throughput Sequence Data – ScienceOpen.” https://www.scienceopen.com/document?vid=de674375-ab83-4595-afa9-4c8aa9e4e736.
Chen, Shifu, Yanqing Zhou, Yaru Chen, and Jia Gu. 2018. “Fastp: An Ultra-Fast All-in-One FASTQ Preprocessor.” Bioinformatics 34 (17): i884–90. doi:10.1093/bioinformatics/bty560.