Building Containers

Apptainer allows users to build containers from a definition file or external resources like Docker Hub.

Building Containers From a Definition File

The definition file below, debian.def, customize the container building process. First, it defines the type of operating system to run the container, which is the Ubuntu Linux distribution. Next, it lists a set of commands that updates the container and installs the python library and its package manager pip. Finally, it creates the directory opt and copies the python program, sort.py, from the host system to the directory. The python program sort.py applies the sort algorithm insertion sort to sort an array of integer numbers in ascending order.

Create a def file ubuntu.def with the following content.

Bootstrap: docker
From: ubuntu:18.04


%labels
    Author HPC Team
    Maintainer  hpc@nmsu.edu
    URL https://hpc.nmsu.edu/home/

%files
    sort.py /opt

%post
    echo "Installing required packages..."
    apt-get update && apt-get -y upgrade
    apt-get -y install python python-pip
    apt-get clean

    mkdir -p /opt
    cd /opt
%runscript
    python /opt/sort.py

Then, create a python script sort.py with the following content.

class sort():
    def __init__(self):
        pass

    def _insertion_sort(self, a):
        for i in range(1, len(a)):
            key = a[i]
            j = i - 1
            while j >= 0 and key < a[j]:
                a[j+1] = a[j]
                j -= 1

            a[j+1] = key
        print(a)


def main():
    a = [0, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
    print(a)
    s = sort()
    s._insertion_sort(a)

if __name__ == "__main__":
    main()
To build containers on Discovery, you shouldn’t use the login node. You must build containers on a compute node. This can be done by using either OnDemand Interactive Shell(Tmux) or Interactive Shell using command line.

To build a container from a def file (ubuntu.def) and convert it to a SIF file (ubuntu.sif), run:

apptainer build ubuntu.sif ubuntu.def

Downloading Container from the Docker Hub

You can download or build a container from the Docker Hub using the pull or build commands.

For example, to download alpine from the Docker Hub, run:

apptainer pull docker://alpine

This will download the container and convert it to a SIF file.

You can also use the build command to download layers from Docker Hub and assemble them into Apptainer containers.

apptainer build alpine.sif docker://alpine

Example: The Rocker Project

The Rocker project was launched to provide high-quality Docker images containing the R environment. For example, you can use Docker images for machine learning and GPU-based computation in R. Also, you can find some Docker-based geospatial containers for R.

To pull the rocker/ml image, you run:

apptainer pull docker://rocker/ml

This will create a SIF file that you can run using a command line, or customize into a def file.

Building a Sandbox Directory

Apptainer allows building a container within a writable directory called sandbox directory. Such a container operates the same way as a SIF container. To build a sandbox directory, please execute the command below in your Linux terminal. The command builds a sandbox directory called SBDebian that runs on a Debian strach Linux distribution.

apptainer build --sandbox SBDebian/ docker:Debian:9
Sandbox containers are not supported on Discovery. You can only use the SIF containers.

For more information on sandbox containers, please refer to the Apptainer documentation

Cache Folders

When you generate a SIF image from remote sources, Apptainer will cache the image. By default, the cache folder will be created in HOME environmental variable ($HOME/.apptainer/cache). The location of the cache can be changed by setting the APPTAINER_CACHEDIR environment variable as shown below.

For more information, see Build Environment

export APPTAINER_CACHEDIR=/path/to/temporary-cache
When you change the value of APPTAINER_CACHEDIR make sure to select a path that’s unique to you and on a file system with enough space.

Cache Command

apptainer cache command allows you to list and clean up your cache.

Listing Cache

You can use the apptainer cache list to print a summary of cache the usage.

$ apptainer cache list
There are 1 container file(s) using 4.96 GiB and 22 oci blob file(s) using 5.03 GiB of space
Total space used: 9.99 GiB

To get more information about the cache usage, use the -v flag.

$ apptainer cache list -v
NAME                     DATE CREATED           SIZE             TYPE
0e2745d360314675471684   2022-12-01 22:33:44    1.37 GiB         blob
1d5b8676e280f1b603b2e6   2022-12-01 22:33:10    181.25 MiB       blob
301a8b74f71f85f3a31e9c   2022-12-01 22:33:00    29.02 MiB        blob
35985d37d899c4aad2ea6b   2022-12-01 22:33:01    4.39 MiB         blob
39930a67f1304239b24ab6   2022-12-01 22:33:49    3.32 KiB         blob
3c7261a1b48646bb45e688   2022-12-01 22:33:11    52.82 KiB        blob
461ecdf77fd75bff69f59c   2022-12-01 22:33:02    19.29 KiB        blob
5b7513e7876e9c851ae994   2022-12-01 22:33:03    53.62 MiB        blob
6baa2ca0c5f8b18078b26d   2022-12-01 22:33:08    0.54 KiB         blob

Cleaning the Cache

To free the space used by the Apptainer cache, run:

apptainer cache clean

It will ask you to confirm before removing cache entries.

this will delete everything in your cache (containers from all sources and OCI blobs).
Hint: You can see exactly what would be deleted by canceling and using the --dry-run option.
Do you want to continue? [N/y]

Flags:

  1. --dry-run or -n: to see the files that would be deleted.

  2. --force or -f: to clean the cache without asking for confirmation.

  3. --days or -d: remove all cache entries older than specified number of days.

    For example, to remove images older than 10 days, run:

apptainer cache clean --days 10

  1. --type or -T: to remove images of a specific type.

    For example, to remove images of type oci-tmp, run:

 apptainer cache clean -T oci-tmp

See Cache Commands Documentation