Using singularity recipes

Creating a Singularity Definition File

Via Docker bootstrap

A Singularity Definition File is a text file that contains a series of statements that are used to create a container image. In line with the configuration as code approach mentioned above, the definition file can be stored in your code repository alongside your application code and used to create a reproducible image. This means that for a given commit in your repository, the version of the definition file present at that commit can be used to reproduce a container with a known state. It was pointed out earlier in the course, when covering Docker, that this property also applies for Dockerfiles.

We’ll now look at a very simple example of a definition file:

Bootstrap: docker
From: ubuntu:20.04

%post
    apt-get -y update && apt-get install -y python

%runscript
    python -c 'print("Hello World! Hello from our custom Singularity image!")'

A definition file has a number of optional sections, specified using the % prefix, that are used to define or undertake different configuration during different stages of the image build process. You can find full details in Singularity’s Definition Files documentation. In our very simple example here, we only use the %post and %runscript sections.

Let’s step through this definition file and look at the lines in more detail:

Bootstrap: docker
From: ubuntu:20.04

These first two lines define where to bootstrap our image from. Why can’t we just put some application binaries into a blank image? Any applications or tools that we want to run will need to interact with standard system libraries and potentially a wide range of other libraries and tools. These need to be available within the image and we therefore need some sort of operating system as the basis for our image. The most straightforward way to achieve this is to start from an existing base image containing an operating system. In this case, we’re going to start from a minimal Ubuntu 20.04 Linux Docker image. Note that we’re using a Docker image as the basis for creating a Singularity image. This demonstrates the flexibility in being able to start from different types of images when creating a new Singularity image.

The Bootstrap: docker line is similar to prefixing an image path with docker:// when using, for example, the singularity pull command. A range of different bootstrap options are supported. From: ubuntu:20.04 says that we want to use the ubuntu image with the tag 20.04 from Docker Hub.

Next we have the %post section of the definition file:

%post
    apt-get -y update && apt-get install -y python3

In this section of the file we can do tasks such as package installation, pulling data files from remote locations and undertaking local configuration within the image. The commands that appear in this section are standard shell commands and they are run within the context of our new container image. So, in the case of this example, these commands are being run within the context of a minimal Ubuntu 20.04 image that initially has only a very small set of core packages installed.

Here we use Ubuntu’s package manager to update our package indexes and then install the python3 package along with any required dependencies. The -y switches are used to accept, by default, interactive prompts that might appear asking you to confirm package updates or installation. This is required because our definition file should be able to run in an unattended, non-interactive environment.

Finally we have the %runscript section:

%runscript
    python3 -c 'print("Hello World! Hello from our custom Singularity image!")'

This section is used to define a script that should be run when a container is started based on this image using the singularity run command. In this simple example we use python3 to print out some text to the console.

We can now save the contents of the simple defintion file shown above to a file and build an image based on it. In the case of this example, the definition file has been named my_test_image.def. (Note that the instructions here assume you’ve bound the image output directory you created to the /home/singularity directory in your Docker Singularity container, as explained in the “Getting started with the Docker Singularity image” section above.):

$ singularity build /home/singularity/my_test_image.sif /home/singularity/my_test_image.def

Examples from Bioinformatics

BootStrap: docker
From: biocontainers/fastqc:v0.11.9_cv6

%runscript
    echo "Welcome to FastQC Image"
    fastqc --version

%post
    echo "Image built"

sudo singularity build fastqc.sif docker.singularity

Via Debian bootstrap

BootStrap: debootstrap
OSVersion: bionic
MirrorURL:  http://fr.archive.ubuntu.com/ubuntu/
Include: build-essential curl python python-dev openjdk-11-jdk bzip2 zip unzip

%runscript
    echo "Welcome to my Singularity Image"
    fastqc --version
    multiqc --version
    bowtie --version

%post

    FASTQC_VERSION=0.11.9
    MULTIQC_VERSION=1.9
    BOWTIE_VERSION=1.3.0

    cd /usr/local; curl -k -L https://www.bioinformatics.babraham.ac.uk/projects/fastqc/fastqc_v${FASTQC_VERSION}.zip > fastqc.zip
    cd /usr/local; unzip fastqc.zip; rm fastqc.zip; chmod 775 FastQC/fastqc; ln -s /usr/local/FastQC/fastqc /usr/local/bin/fastqc

    cd /usr/local; curl --fail --silent --show-error --location --remote-name https://github.com/BenLangmead/bowtie/releases/download/v$BOWTIE_VERSION/bowtie-${BOWTIE_VERSION}-linux-x86_64.zip
    cd /usr/local; unzip -d /usr/local bowtie-${BOWTIE_VERSION}-linux-x86_64.zip
    cd /usr/local; rm bowtie-${BOWTIE_VERSION}-linux-x86_64.zip
    cd /usr/local/bin; ln -s ../bowtie-${BOWTIE_VERSION}-linux-x86_64/bowtie* .

    curl --fail --silent --show-error --location --remote-name  https://bootstrap.pypa.io/get-pip.py
    python get-pip.py

    pip install numpy matplotlib
    pip install -I multiqc==${MULTIQC_VERSION}

    echo "Biocore image built"

%labels
    Maintainer Biocorecrg
Version 0.1.0

In order to run the command shown below, you have to install debootstrap before e.g. sudo apt-get install debootstrap.

sudo singularity build fastqc-multi-bowtie.sif debootstrap.singularity