How to Use SStack

SStack tool is used to install software stacks on HPC clusters. It enables ease of use by providing simple commands such as install, remove, and update. The following section shows the main SStack commands and their usage with examples.

Getting Started

To get started with SStack, on the Discovery Cluster, run the below module command.

$ module load sstack
$ sstack --help
...
...

Stacks installed in your home directory, on the default path, will automatically show up in your module path. Anything installed to a scratch or project directory will require users to manually adjust their module path.

$ sstack install --type custom --name prj --path /fs1/project/myprj/sstack_rhel8
...
...
$ module use /fs1/project/myprj/sstack_rhel8/modules

SStack Commands

SStack includes an extensive help system. Add --help to any command to see possible options, arguments, or subcommands.

$ sstack --help
Usage: sstack [OPTIONS] COMMAND [ARGS]...

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  install  Install new software stack.
  modules  Manage installed module files.
  remove   Remove existing software stack.
  show     List installed stacks, available types, etc..
  update   Update software stack and regenerate modulefile.
$ sstack show --help
Usage: sstack show [OPTIONS] COMMAND [ARGS]...

  List installed stacks, available types, etc..

Options:
  --help  Show this message and exit.

Commands:
  modules  Show stack modules.
  stacks   Show installed software stacks.
  types    Show available stack types.
$ sstack show stacks --help
Usage: sstack show stacks [OPTIONS]

  Show installed software stacks.

Options:
  -p, --path DIRECTORY  Location of stacks and module files.  [default:
                        /home/username/sstack]
  --help                Show this message and exit.

Install

The install command installs a new software stack. It also creates a module file to make the software available through environment modules.

Syntax

sstack install [OPTIONS]
  • OPTIONS:

    • Required:

      • -t or --type: Type of stack (Spack / conda / etc..)

      • -n or --name: Name of stack (myspack / 2022A / etc..).

    • Optional:

      • -v or --version: Version of stack type.

        • If not specified the default version is used. See sstack show types

      • -p or --path: Location of stacks and module files.

        • If not specified the stack will be installed in the default path, in the users home directory.

For example, to install spack with the name myspack:

sstack install -n myspack -t spack

The messages Stack Successfully Installed! means that the stack is installed successfully. This will create a module file which you can load into your environment to install software.

module load spack/myspack

To install EasyBuild named myeb in a specified directory in this case project space (/fs1/project/project1), run command:

sstack install -n myeb -t easybuild -p /fs1/project/project1/sstack_rhel8

To show the help message, you can use --help option as follows:

sstack install --help

Remove

The remove command uninstalls an existing software stack and removes its module.

Syntax

sstack remove [OPTIONS]
  • OPTIONS:

    • Required:

      • -t or --type: Type of stack (Spack / conda / etc..).

      • -n or --name: Name of stack (myspack / 2022A / etc..).

    • Optional:

    • -p or --path: Location of stacks and module files.

For example, to uninstall the stack myspack Spack, which was installed in the users home directory, run:

sstack remove -n myspack -t spack
Once you uninstall a stack, all previously installed software within that stack will be removed, and you won’t be able to use them anymore.

To show the help message, you can use --help option as follows:

sstack remove --help

Show

Show command lists installed stacks, available types, and other usefull information.

Syntax

sstack show [OPTIONS] COMMAND [ARGS]

COMMAND

  1. types: Show available stack types.

    Syntax

    sstack show types [OPTIONS]
    • OPTIONS:

      • --help: show help message. For example, to show all available stack types, run command:

    sstack show types

    Output

    +-------------+-------------------------------+--------------------------------+
    | Type (Name) | Versions                      | Description                    |
    +-------------+-------------------------------+--------------------------------+
    |             |                               | Spack is a package manager for |
    |             |                               | supercomputers, Linux, and     |
    |             |                               | macOS. It makes installing     |
    |             |                               | scientific software easy.      |
    |             |                               | Spack isnt tied to a          |
    | spack       | ['v0.18', 'v0.17', 'develop'] | particular language; you can   |
    |             |                               | build a software stack in      |
    |             |                               | Python or R, link to libraries |
    |             |                               | written in C, C++, or Fortran, |
    |             |                               | and easily swap compilers or   |
    |             |                               | target specific                |
    |             |                               | microarchitectures.            |
    +-------------+-------------------------------+--------------------------------+
    |             |                               | Conda is an open source        |
    |             |                               | package management and         |
    |             |                               | environment management system. |
    |             |                               | This stack delivers a custom   |
    | conda       | ['Latest']                    | anaconda/miniconda type        |
    |             |                               | deployment based on the        |
    |             |                               | communities 'conda-forge'      |
    |             |                               | channel with 'conda'/'mamba'   |
    |             |                               | as the package manager.        |
    +-------------+-------------------------------+--------------------------------+
    |             |                               | Micromamba is a standalone     |
    |             |                               | version of Mamba which is an   |
    |             |                               | alternative to Conda. This     |
    |             |                               | stack delivers a custom        |
    | micromamba  | ['Latest']                    | anaconda/miniconda type        |
    |             |                               | deployment based on the        |
    |             |                               | communities 'conda-forge'      |
    |             |                               | channel with 'micromamba' as   |
    |             |                               | the package manager.           |
    +-------------+-------------------------------+--------------------------------+
    | custom      | ['N/A']                       | Custom Hand Build Packages.    |
    +-------------+-------------------------------+--------------------------------+
    |             |                               | sstack is a tool to install    |
    |             |                               | multiple software stacks, such |
    |             |                               | as Spack, EasyBuild, Anaconda, |
    | sstack      | ['Latest']                    | Micromamba, etc.               |
    |             |                               | These stacks are then linked   |
    |             |                               | together, using lmod module    |
    |             |                               | files, to easily integrate     |
    |             |                               | with most HPC environments.    |
    +-------------+-------------------------------+--------------------------------+
    |             |                               | EasyBuild is a software build  |
    |             |                               | and installation framework     |
    | easybuild   | ['Latest']                    | that allows you to manage      |
    |             |                               | (scientific) software on High  |
    |             |                               | Performance Computing (HPC)    |
    |             |                               | systems in an efficient way.   |
    +-------------+-------------------------------+--------------------------------+
    |             |                               | Singularity Registry HPC       |
    | shpc        | ['Latest']                    | (shpc) allows you to install   |
    |             |                               | containers as modules.         |
    +-------------+-------------------------------+--------------------------------+

    To show the help message, you can use --help option as follows:

    sstack show types --help
  2. stacks: Show installed software stacks.

    Syntax

    sstack show stacks [OPTIONS]
    • OPTIONS:

      • -p or --path: Location of stacks and module files.

      • --help: show help message. For example, if you want to show all installed stacks in the home directory, run command:

    sstack show stacks

    Output

    +----------------+------------+---------+-----------------------------------------------------+
    | Name           | Type       | Version | Path                                                |
    +----------------+------------+---------+-----------------------------------------------------+
    | 2022A          | conda      | 4.13.0  | /home/tahat/sstack/stacks/conda/2022A               |
    +----------------+------------+---------+-----------------------------------------------------+
    | conda-el7      | conda      | 4.13.0  | /home/tahat/sstack/stacks/conda/conda-el7           |
    +----------------+------------+---------+-----------------------------------------------------+
    | easybuild-l7   | easybuild  | 4.5.5   | /home/tahat/sstack/stacks/easybuild/easybuild-l7    |
    +----------------+------------+---------+-----------------------------------------------------+
    | micro1         | micromamba | 0.24.0  | /home/tahat/sstack/stacks/micromamba/micro1         |
    +----------------+------------+---------+-----------------------------------------------------+
    | micromamba-el7 | micromamba | 0.24.0  | /home/tahat/sstack/stacks/micromamba/micromamba-el7 |
    +----------------+------------+---------+-----------------------------------------------------+
    | shpc-el7       | shpc       | 0.0.55  | /home/tahat/sstack/stacks/shpc/shpc-el7             |
    +----------------+------------+---------+-----------------------------------------------------+
    | shpc1          | shpc       | 0.0.53  | /home/tahat/sstack/stacks/shpc/shpc1                |
    +----------------+------------+---------+-----------------------------------------------------+
    | 2022a          | spack      | 0.17.2  | /home/tahat/sstack/stacks/spack/2022a               |
    +----------------+------------+---------+-----------------------------------------------------+
    | spack/2022     | spack      | 0.17.2  | /home/tahat/sstack/stacks/spack/spack/2022          |
    +----------------+------------+---------+-----------------------------------------------------+

    To show the help message, you can use --help option as follows:

    sstack show stacks --help

Update

Update command updates software stack to the latest version available and regenerates module file.

Syntax

sstack update [OPTIONS]
  • OPTIONS:

    • Required:

      • -t or --type: Type of stack (Spack / conda / etc..).

      • -n or --name: Name of stack (myspack / 2022A / etc..).

    • Optional:

      • -p or --path: Location of stacks and module files.

      • --du Don’t Update Stack Software.

      • --dm Don’t Regenerate ModuleFile.

For example, to update the Spack stack myspack without regenerating module file run command:

sstack update -n 2022a -t spack --dm

Output

Already up-to-date.

Stack "2022a" Sucessfully Updated!
+-------+-------+---------+---------------------------------------+
| Name  | Type  | Version | Path                                  |
+-------+-------+---------+---------------------------------------+
| 2022a | spack | 0.17.2  | /home/tahat/sstack/stacks/spack/2022a |
+-------+-------+---------+---------------------------------------+

Module Setup/Load Commands:

module use "/home/tahat/sstack/modules"
module load spack/2022a

To show the help message, you can use --help option as follows:

sstack update --help

Modules

Modules command allows management of installed module files.

Syntax

sstack modules [OPTIONS] COMMAND [ARGS]

COMMAND

  1. default: To manage default module for each stack type.

    Syntax

    sstack modules default [OPTIONS] COMMAND [ARGS]
    • OPTIONS:

      • --help: show help message.

    COMMAND

    1. reset: remove the symlink default module override. Syntax

    sstack modules default reset [OPTIONS]
    • OPTIONS:

      • -p or --path: Data Root (Path of stacks/modules) [default:/home/user/sstack]

      • -t or --type: Stack Type [required]

      • --help: show the help message

    1. set: set default module for a specific stack type.

    Syntax

    sstack modules default set [OPTIONS]
    • OPTIONS:

      • -p or --path: data root (path of stacks/modules) [default:/home/user/sstack]

      • -t or --type: stack type [required]

      • -n or --name: stack name [required]

      • --help: show the help message

    For example, to select spack/2020A-el8 as a default module Spack stacks, run command:

    sstack modules default set -t spack -n 2020A-el8
  1. show: show stack modules.

    Syntax

    sstack modules show [OPTIONS]
    • OPTIONS:

      • -p or --path: data root (path of stacks/modules) [default:/home/user/sstack]

      • --help: show the help message

    For example, to display all stack modules in the users home directory, run command:

    sstack modules show

    Output

    Module Tree (/home/tahat/sstack/modules):
    
    modules/
    |-- easybuild/
    |   |-- easybuildtest2.lua
    |   |-- easybuild-l7.lua
    |-- spack/
    |   |-- 2022a.lua
    |   |-- spack/
    |   |   |-- 2022.lua
    |-- micromamba/
    |   |-- micromamba-el7.lua
    |   |-- micro1.lua
    |-- shpc/
    |   |-- shpc1.lua
    |   |-- shpc-el7.lua
    |-- custom/
    |   |-- 2020A-el8.lua
    |   |-- 2020B-el8.lua
    |   |-- 2020B-el9.lua
    |-- conda/
    |   |-- conda-el7.lua
    |   |-- 2022A.lua
    
    Notes:
      - "default ->" represent default module overrides using symlinks.
      - Destination becomes default module for that stack type.
      - To modify overrides check "sstack modules default --help".
  1. regen: regenerate stack module file.

    Syntax

    sstack modules regen [OPTIONS]
    • OPTIONS:

      • -p or --path: data root (path of stacks/modules) [default:/home/user/sstack]

      • -t or --type: stack type [required]

      • -n or --name: stack name [required]

      • --help: show the help message

    For example, to regenerate module file of stack conda-el7 of conda type installed in the users home directory, run command:

    sstack modules regen -t spack -n 2022a

    Output

    spack/2022a: Sucessfully Regenerated Module File!
    
    +-------+-------+---------+---------------------------------------+
    | Name  | Type  | Version | Path                                  |
    +-------+-------+---------+---------------------------------------+
    | 2022a | spack | 0.17.2  | /home/tahat/sstack/stacks/spack/2022a |
    +-------+-------+---------+---------------------------------------+
    
    Module Setup/Load Commands:
    
    module use "/home/tahat/sstack/modules"
    module load spack/2022a

Environment Variables

A few environmental variables are supported that can be used in-place of options such as --path. The order of precedence is command option, environment variable, and then the default value (should there be a default).

  1. -p or --path = SSTACK_PATH

  2. -t or --type = SSTACK_TYPE

  3. -n or --name = SSTACK_NAME

Below is an example of doing the same set of commands 3 different ways to achieve an identical effect.

# No Env
sstack install -p /software/sstack -t spack -n 2022A-el7
sstack show stacks -p /software/sstack

# In-Line Env
SSTACK_PATH=/software/sstack sstack install -t spack -n 2022A-el7
SSTACK_PATH=/software/sstack sstack show stacks

# Multi-Line Env
export SSTACK_PATH="/software/sstack"
sstack install -t spack -n 2022A-el7
sstack show stacks
When using the 'multi-line' method be sure to export the bash/shell variable or sstack will not be able to automatically use it.

Marking a Stack Version as Default

Order of Marking a Stack Version Default

If there is no marked default then Lmod selects the “Highest” version across all directories. It sorts all modules of the same type based on their names, and selects the highest one.

For example, if you have two different stacks of Spack types spack/2020A-el8 and spack/2021A-el7, then spack/2021A-el7 is selected as a default module for Spack. This means that when a user runs module load spack, spack/2021A-el7 will load because it’s a default.

Selecting a Specific Module as Default

SStack tool allows users to select a specific module from modules of the same stack type as a default module. For example, if you need to specify a module as default to make sure that anytime you load the stack type you load the appropriate module. To find all available modules of Spack, run command:

module avail spack

Output

------------------------------------------------------------------ /home/tahat/sstack/modules ------------------------------------------------------------------
   spack/2022A-el7    spack/2022B-el7    spack/2023A-el8 (D)

  Where:
   D:  Default Module
...

The above output shows that the module 2023A-el8 has been set as the default module. This is because it has the highest order based on the sorting criterion(2023A is greater than 2022B-el7 and 2022A-el7). To make 2022B-el7 as Spack default module, run command:

sstack modules default set -t spack -n 2022B-el7

To check if the module set as default, run command:

module avail spack

Output

------------------------------------------------------------------ /home/tahat/sstack/modules ------------------------------------------------------------------
   spack/2022A-el7    spack/2022B-el7 (D)   spack/2023A-el8

  Where:
   D:  Default Module
...

Reset the Default Module of a Stack Type

If you made a module as default of a stack type using the sstack modules default set command, SStack allows you to reset the default module. This means that the default will be selected based on the sorting criterion.

For example to reset the default stack of Spack, run command:

sstack modules default reset -t spack

Now check the default module of Spack.

module avail spack

Output

------------------------------------------------------------------ /home/tahat/sstack/modules ------------------------------------------------------------------
   spack/2022A-el7    spack/2022B-el7    spack/2023A-el8 (D)

  Where:
   D:  Default Module
...