Spack
Spack is a package manager for supercomputers, Linux, and macOS. It makes installing scientific software easy. Spack isn’t tied to a particular language; you can build a software stack in Python or R, link to libraries written in C, C++, or Fortran, and easily swap compilers or target specific microarchitectures. For more details visit Spack documentation.
How to Install Spack?
SStack tool allows you to install Spack, and use it to install software in your desired location.
For example, to install a Spack named spack-el7 with the default version in the project space /project/hpcteam/, run command:
sstack install -t spack -n spack-el7 -p /project/hpcteam/To install a specific Spack version, use the option -v or --version. For example, to install the develop version, run command:
sstack install -t spack -n spack-el7 -v developThis creates a module file that you can load in your environment to be able to use the stack to install software.
For example, to use spack-el7, you need to load its module as follows:
module use "/project/hpcteam/modules"
module load spack/spack-el7To check if Spack is now available and you can use it, run the following command:
spack --versionOutput
0.17.2How to Use Spack to Manage Software?
Show Available Packages
The spack list command shows available packages.
To show all available packages that you can install using Spack, run command:
spack listOutput
...
cquery nfft py-parso r-rminer
cracklib nfs-ganesha py-partd r-rmpfr
cradl nfs-utils py-particle r-rmpi
cram nghttp2 py-path-py r-rms
cray-fftw nginx py-pathlib r-rmutil
cray-libsci ngmerge py-pathlib2 r-rmysql
cray-mpich ngmlr py-pathos r-rnaseqmap
cray-mvapich2 ngspice py-pathspec r-rncl
creduce nicstat py-pathtools r-rnexml
cromwell nim py-patool r-rngtools
cromwell-womtool ninja py-patsy r-rnoaa
cronie ninja-fortran py-pauvre r-robust
...You can use the spack list command to query a package. Spack automatically adds wildcards to both ends of the string, or you can add your own wildcards. For example, to show all available Python packages, use py-* which means packages the start with py- and end with anything, as shown below:
spack list 'py-*'Output
py-3to2 py-docopt py-moviepy py-pythran
py-4suite-xml py-docutils py-mpi4py py-pytimeparse
py-abcpy py-docutils-stubs py-mpld3 py-pytools
py-abipy py-dominate py-mpmath py-pytorch-gradual-warmup-lr
py-absl-py py-dotnetcore2 py-mrcfile py-pytorch-lightning
py-accimage py-doxypy py-msal py-pytorch-sphinx-theme
py-acme-tiny py-doxypypy py-msal-extensions py-pytweening
py-adal py-dp-gp-cluster py-msgpack py-pytz
py-adb-enhanced py-dpath py-msgpack-numpy py-pyudev
py-addict py-drmaa py-msrest py-pyugrid
py-adios py-dryscrape py-msrestazure py-pyusb
...Installing Packages
Installing a package with Spack is very simple. Syntax
spack install <package_name>For example, to install the tmux package, run command:
spack install tmuxOnce the package is installed successfully, a module is created where you can load in your envrunment to use the package.
To find the correct module name to load, run command:
module spider tmuxOutput
------------------------------------------------------------------------------------------------------------------------------------------------------------
tmux: tmux/3.2a-kvccbsg
------------------------------------------------------------------------------------------------------------------------------------------------------------
You will need to load all module(s) on any one of the lines below before the "tcl/8.6.12-ukuy5os" module is available to load.
spack/spack-el7
...To load the tmux module run command:
module load tmux/3.2a-kvccbsglkFor more details visit Spack documentation.
Create Spack Environment
To create a Spack environment, run the command:
spack env create <env-name>For example, to create an environment named my_env, run:
spack env create my_envYou can check all available environmnets by running:
spack env listOutput
==> 3 environments
my_env my_spack_env test_envCreate a Module for Spack Environment
Once you create the environmnet, you can create a module file for it. This will add the environment to the module system where you can load it in your enviroument.
You can create module files by updating your SStack Conda with a --du flag. In this case, modulefiles will be created automatically.
Syntax
sstack update [OPTIONS]-
OPTIONS:
-
Required:
-
-tor--type: Type of stack (here it should beconda). -
-nor--name: Name of stack (test-conda/ my-SStack-conda / etc..). -
--duDon’t Update Stack Software.
-
-
For example:
sstack update -t spack -n test-spack --duOutput
spack/test-spack: Sucessfully Regenerated Module File!
+------------+-------+---------+------------------------------------------+
| Name | Type | Version | Path |
+------------+-------+---------+------------------------------------------+
| test-spack | spack | 0.22.1 | /fs1/home/yuxiwang/sstack/rhel_8/stacks/ |
| | | | spack/test-spack |
+------------+-------+---------+------------------------------------------+
Module Setup/Load Commands:
module use "/fs1/home/yuxiwang/sstack/rhel_8/modules"
module load spack/test-spackAfter that, you can check if the modules are ready by:
module availTo see the module for conda environmnet, conda needs to be loaded.
In my example, conda/test-conda is loaded before using module avail.
Output
-------------------------------------- /fs1/home/yuxiwang/sstack/rhel_8/stacks/spack/test-spack/etc/spack/env-modules --------------------------------------
my_env my_spack_env test_env
-------------------------------------- /fs1/home/yuxiwang/sstack/rhel_8/stacks/spack/test-spack/share/spack/lmod/linux-rhel8-x86_64/Core --------------------------------------
bison/3.0.4-v5otd54 glibc/2.28-rf4i54i libevent/2.1.12-4mbjepx openssl/1.1.1k-fa4toej tmux/3.4-y3jwarg
gcc-runtime/8.5.0-5vxko6y gmake/4.2.1-o453s2n ncurses/6.5-jzb4s3c pkgconf/1.4.2-gnnzjgn
-------------------------------------- /fs1/home/yuxiwang/sstack/rhel_8/modules --------------------------------------
conda/test-conda micromamba/test-mamba spack/test-spack (L)
-------------------------------------- /fs1/software/sstack/rhel_8/modules --------------------------------------
conda/2022a conda/2023a conda/2024a (D) custom/2022a custom/2023a custom/2024a (D) spack/2022a spack/2023a (D) sstack/main
-------------------------------------- /etc/modulefiles --------------------------------------
pmix/2.2.5 pmix/3.2.3 pmix/4.1.2 (D)
-------------------------------------- /fs1/software/sstack/modules --------------------------------------
os/centos_7_test (S) os/centos_7 (S) os/rhel_8_test (S) os/rhel_8 (S,L,D)
-------------------------------------- /usr/share/lmod/lmod/modulefiles/Core --------------------------------------
lmod settarg
Where:
S: Module is Sticky, requires --force to unload or purge
L: Module is loaded
D: Default Module
If the avail list is too long consider trying:
"module --default avail" or "ml -d av" to just list the default modules.
"module overview" or "ml ov" to display the number of modules for each name.
Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".