Build Recipe

Apptainer uses a Definition File (def file) to create a build recipe to explain how to build a custom container. It has the .def extension. It contains specifics about the base OS to build or the base container to start from, software to install, environment variables to set at runtime, files to add from the host system, and container metadata. For more information, check the Apptainer Definition Files documentation.

Content

This section explains the main content of the def file. For that, an example def file shown below is used.

Bootstrap: docker
From: debian:9

%labels
    Author HPC Team
    Maintainer  hpc@nmsu
    URL https://hpc.nmsu.edu/home/

%files
    ./hello_world.c /opt

%environment
    export PATH="$PATH:/opt"

%post
    echo "Installing required packages..."
    apt-get update && apt-get -y upgrade
    apt-get install -y gcc
    apt-get clean
    mkdir -p /opt
    cd /opt && gcc -o hello_world.out hello_world.c

%runscript
    echo "The output of your program is:"
    /opt/hello_world.out

%test
    /opt/hello_world.out

%help
    To run the container:
	apptainer run <container_name.sif>

    To shell into the container:
	apptainer shell <container_name.sif>

    To execute a custom command or app:
	apptainer exec <container_name.sif> <command or app_name>

    To view the container's metadata:
	apptainer inspect <container_name.sif>

An Apptainer Definition file is divided into two parts:

  • Header: to describe the core operating system to build within the container.

  • Sections: each section is defined by a % character followed by the name of the particular section. All sections are optional, and a def file may contain more than one instance of a given section.

Header

Each def file should contain a header to tell Apptainer about the base OS to use to build the container. It starts with the Boostrap keyword to decide which bootstrap agent that will be used to create the base operating system you want to use. For example, the docker bootstrap agent will pull docker layers from Docker Hub as a base OS to start your image as shown above.

Sections

The following shows the sections that you may add to a def file.

%label

The %lable section is used to define metadata to your container. For example, the section below, %labels, adds the labels Author, Maintainer, and URL to the container during the building process.

%labels
    Author HPC Team
    Maintainer  hpc@nmsu.edu
    URL https://hpc.nmsu.edu/home/

Such metadata can be viewed after the build completes using the command below:

apptainer inspect recipe.sif

%files

The %files section is used to copy files from the building system into your container during the building process. Each line in this section has a pair consists a filename including a path to its location and the copying destination. For example, the section below, states that a C program called hello_world.c located in the current directory must be copied to the folder opt in the container.

%files
    ./hello_world.c /opt

%environment

This section is used to define the environment variables for the container at runtime. For example, the section below sets the environment variable $PATH to the directory opt. However, such enenvironment variables won’t be available until the building process of the container completes, and the container runs.

%environment
    export PATH="$PATH:/opt"

%post

The %post section is used to define commands that customize the building process of your container. For example, the section below lists commands that update your system and install packages within your container during the building process.

%post
    echo "Installing required packages..."
    apt-get update && apt-get -y upgrade
    apt-get install -y gcc
    apt-get clean
    mkdir -p /opt
    cd /opt && gcc -o hello_world.out hello_world.c

%test

The %test section is used to validate the container after the building process completes. For example, the section below runs the executable hello_world.out to validate it.

%test
    hello_world.out

%runscript

The %runscript is used to list a set of commands that would be executed when the container invoked with the apptainer run command. For example, the section below runs the command echo that displays a message to the user and runs a C executable file.

%runscript
    echo "The output of your program is:"
    /opt/hello_world.out

%help

The %help section is used to create a textual metadata file in the container during the build process, which can be displayed by invoking the container with the run-help command.

%help
    To run the container:
	apptainer run <container_name.sif>

    To shell into the container:
	apptainer shell <container_name.sif>

    To execute a custom command or app:
	apptainer exec <container_name.sif> <command or app_name>

    To view the container's metadata:
	apptainer inspect <container_name.sif>

Multi-Stage Builds

Apptainer allows multi-stage builds where one stage can be used for compilation and the other stages are used to copy the resulting binary. This allows a slimmer final image that don’t require the entire development stack.

The following def file shows an example of mult-stage def file.

Bootstrap: docker
From: debian:9
Stage: first

%post
echo "Installing required packages..."
apt-get update && apt-get -y upgrade
apt-get install -y gcc
apt-get clean
export HOME="/root"
cd /root

cat <<-EOF >> hello.c
#include <stdio.h>
    int main() {
        printf("Hello, World!\n");
        return 0;
    }
EOF

gcc -o hello.out hello.c

Bootstrap: docker
From: debian:9
Stage: second

%files from first
/root/hello.out /bin/hello.out

This sections will be executed in the same order as described for a single stage build except for the files from the previous stage being copied before %setup section of the next stage. You can only copy files from the previous stages to the current stages. You can’t copy files from the next stage to the current stage. For example, it’s prohibited to copy files from the second stage to the first stage.

Local Image Builds

You can use an existing container image as your "base" image, and then add customizations to it. This allows you to build multiple images from the same base image. For example, you may want to build several containers with the same custom installation. Instead of customizing these installations from scratch each time, you could start with the appropriate local base container and then customize the new container in %post, %environment, %runscript, and so on.

Example

Some R container images are available on Discovery through the module environment. You can use those images as base images in your container. For more information on how to search and use container images on Discovery, see Discovery Containers. After you find your container image, you can use it as a base container. For example, to use 4.2.0-RStudio_Server_2022.02.2.485-SIF/R.sif as the base image in your def file, you need to load its module r/4.2.0-RStudio_Server_2022.02.2.485-SIF. Then, you be able to find the path to the image using an environmental variable named SIF:

$ echo $SIF
/fs1/software/sstack/rhel_8/stacks/custom/2022a/packages/r/4.2.0-RStudio_Server_2022.02.2.485-SIF/R.sif

To create a container that uses the above image as a base R container and installs some extra packages in it, create a def file with the following content:

Bootstrap: localimage
From: /fs1/software/sstack/rhel_8/stacks/custom/2022a/packages/r/4.2.0-RStudio_Server_2022.02.2.485-SIF/R.sif

%post
# install r package
R -e "install.packages('reader')"

Where the localimage module allows you to build a container from an existing Apptainer container on your host system. And, From specified the path to the base image. In the %post section, R package name reader will be installed.

To build the def file and convert it to SIF image, run:

apptainer build NewR.sif RContainer.def

Instead of installing R from scratch, it’s already installed in the base image. You only install your intended R packages.