reproducible-toolchain
Anatomy of a modular toolchain for reproducible bioinformatics workflows
Template Repository: https://github.com/CCBR/CCBR_NextflowTemplate
Presentations
This project has been presented in a few forums:
Abstract
The reproducibility and replicability of data-intensive biological studies depend on the quality of the software used during data analysis. Here we describe a toolchain for the development of reproducible bioinformatics workflows that follow established guidelines for high-quality scientific software. The central component is a template repository for generating new pipeline repositories, ensuring consistent structure across the whole suite of pipelines we have developed. Each pipeline step points to a container to define dependencies, with Docker recipes stored in a central repository for reuse. A Python-based command-line interface (CLI) provides user-friendly commands for initializing projects and launching the pipeline on HPC systems. The CLI uses functions from a shared package to reduce code duplication across template deployments and to streamline ongoing maintenance. Because the interface is consistent, once a biologist uses one of our pipelines, they can easily use others, regardless of which workflow language is used by the developers. The template also contains continuous integration workflows via GitHub Actions for testing, enforcing code format and quality, deploying documentation, and preparing releases. Custom GitHub Actions are defined in a standalone repository for reuse in CI workflows across our organization. Using this toolchain, we have developed multiple Nextflow pipelines for analyzing data from whole-genome sequencing, ChIP-seq, single-cell RNA-seq, and CRISPR experiments, with additional workflows in development. This approach has streamlined our development and maintenance processes by enabling faster updates and consistent improvements across pipelines while following best practices including modular design, code reuse, automated testing, clear documentation, and standardized release practices.
The Toolchain
License
This software is licensed under the MIT license. Text and images included in this repository are licensed under the CC BY-SA 4.0 license.
Citation
View the citation file for the citation information for this project.