Module Environments and Commands
What are Modules?
Modules are used to connect to different software packages to run your scripts. Different versions of the software can be switched quickly and easily using the module commands. A variety of popular software like R, Python, MatLab, etc are available in the Discovery.
For more details see environment modules documentation
What if the Module you are Looking for isn’t Available?
If the modules you are looking for isn’t available in the discovery, then you can submit the software request by filling the form in this link → (HPC software request). After the module is updated into the discovery, you will be notified via email.
When to Use Modules?
For example, if you have a perl script that needs to be run, you first need to load the perl module to your environment. You can choose the desired version of the perl module from the list of available modules. You can load as many modules as you wish to.
But remember that when loading multiple versions of the same compiler, the module loaded last will always have precedence. The best way to avoid this confusion is to unload the module when it’s no longer needed. |
Module Commands
The below table shows some useful module commands in discovery.
Commands | Syntax | Description |
---|---|---|
module avail |
|
Shows all the available modules |
module spider |
|
Retrieves the specific module |
module load |
|
Loads the specific module to your environment |
module list |
|
Lists all the loaded modules |
module swap |
|
Unloads module m1 and loads module m2 |
module unload |
|
Gets rid of the specific module |
module purge |
|
Removes all the currently loaded modules |
module help |
|
Prints the help information. |
Module Avail
To view the list of available modules in discovery, run the following command.
Syntax: module avail
module avail
Output
------------------------------------------------------------------------------------- /etc/modulefiles --------------------------------------------------------------------------------------
mpi/mpich-x86_64 mpi/mpich-3.0-x86_64 mpi/openmpi-x86_64 (D)
-------------------------------------------------------------------------------- /software/anaconda/modules ---------------------------------------------------------------------------------
alfalfa_gbs anaconda anaconda3 pbbioconda qiime2-2019.10 soilsystems tensorflow-gpu/2.0.0 (D) tensorflow/2.0.0 (D)
amptk/1.4.2 anaconda2 hsc_prediction pytorch redbiom tensorflow-gpu/1.15.0 tensorflow/1.15.0
----------------------------------------------------------------------------- /software/custom/el7/modulefiles ------------------------------------------------------------------------------
bayescan/2.1 fastsimcoal2/2.6.0.3 gcc/4.9.4 openmpi/1.5.5-transat petsc/3.2-p7-transat transat/5.7.1
------------------------------------------------------------- /fs1/software/spack/0.13/share/spack/modules/linux-centos7-x86_64 -------------------------------------------------------------
adios2/2.5.0-gcc-9.2.0-openmpi-24g24ah netcdf-c/4.7.2-gcc-9.2.0-openmpi-vjoppci r-data-table/1.12.2-gcc-9.2.0-tm3afks (D)
ant/1.10.0-gcc-9.2.0-bvkawaf (D) netcdf-fortran/4.5.2-gcc-9.2.0-openmpi-p2elujv (D) r-dbi/1.0.0-gcc-9.2.0-v5d3azu (D)
argtable/2-13-gcc-9.2.0-3lwtbg7 (D) netcdf/4.7.2-gcc-9.2.0-openmpi-j22xar2 (D) r-deldir/0.1-23-gcc-9.2.0-hlmpvdb
arpack-ng/3.7.0-gcc-9.2.0-openblas-openmpi-fp7yot3 (D) netlib-lapack/3.8.0-gcc-9.2.0-hsgmmfb r-desc/1.2.0-gcc-9.2.0-rc3oxh6
at-spi2-atk/2.26.2-gcc-9.2.0-ne7dldi (D) netlib-lapack/3.8.0-gcc-9.2.0-7lemu3c r-devtools/2.1.0-gcc-9.2.0-mxzwt22
at-spi2-core/2.28.0-gcc-9.2.0-ikgtwxh (D) netlib-scalapack/2.0.2-gcc-9.2.0-openblas-openmpi-tvlazvq (D) r-digest/0.6.20-gcc-9.2.0-ik2krtt (D)
atk/2.30.0-gcc-9.2.0-d7uvfiw (D) netlib-xblas/1.0.248-gcc-9.2.0-zjfxgwl r-doparallel/1.0.15-gcc-9.2.0-tsfoz7q
augustus/3.3.2-gcc-9.2.0-6piwhgu (D) nettle/3.4.1-gcc-9.2.0-s5j4zqp (D) r-dosnow/1.0.18-gcc-9.2.0-nnq3aut
autoconf-archive/2019.01.06-gcc-9.2.0-43fyr6f ngsepcore/4.0.0-gcc-9.2.0-474wh3n r-dplyr/0.8.3-gcc-9.2.0-lzxcfzh
autoconf/2.69-gcc-9.2.0-ytaohc4 ninja/1.9.0-gcc-9.2.0-2qvepgo (D) r-dplyr/0.8.3-gcc-9.2.0-2wfidt7 (D)
automake/1.16.1-gcc-9.2.0-4h2g7c4 numactl/2.0.12-gcc-9.2.0-aj6bh6t (D) r-e1071/1.7-2-gcc-9.2.0-lrbslzc
bamtools/2.5.1-gcc-9.2.0-egeykzb (D) nwchem/6.8.1-gcc-9.2.0-openblas-openmpi-ccgzl5e (D) r-ellipsis/0.2.0.1-gcc-9.2.0-nkwlj4t
bcftools/1.9-gcc-9.2.0-oz7iegl (D) oniguruma/6.1.3-gcc-9.2.0-fwjgg3u (D) r-ellipsis/0.2.0.1-gcc-9.2.0-orbbzn3 (D)
bdftopcf/1.0.5-gcc-9.2.0-zsphr3k opari2/2.0.5-gcc-9.2.0-fobgwfp r-evaluate/0.14-gcc-9.2.0-gyvdolg
bdw-gc/8.0.0-gcc-9.2.0-p6hd7qd openbabel/2.4.1-gcc-9.2.0-m2yuj3d r-expm/0.999-4-gcc-9.2.0-7rjk2yu
binutils/2.32-gcc-9.2.0-tzlq2cl (D) openblas/0.3.7-gcc-9.2.0-epeitvj (D) r-fansi/0.4.0-gcc-9.2.0-vxps4xj
bison/3.4.2-gcc-9.2.0-xwnwxvw (D) openfoam-org/7-gcc-9.2.0-openmpi-xfwbnsq r-feddata/2.5.7-gcc-9.2.0-openmpi-at4kcc7
blast-plus/2.9.0-gcc-9.2.0-5bhhtpv (D) openfoam/1912-gcc-9.2.0-openmpi-vmdetr2 r-foreach/1.4.7-gcc-9.2.0-qwslptj
boost/1.70.0-gcc-9.2.0-ndry3ar (D) openjdk/1.8.0_202-b08-gcc-9.2.0-vn2k6b7 r-foreign/0.8-72-gcc-9.2.0-epz7bi3
bowtie2/2.3.2-gcc-4.8.5-dp2quvb openjdk/11.0.2-gcc-9.2.0-iuj2qln (D) r-formatr/1.7-gcc-9.2.0-kmhhqhk
bowtie2/2.3.5.1-gcc-9.2.0-4iv5wso (D) openmpi/3.1.4-gcc-9.2.0-r6y3ifc r-formula/1.2-3-gcc-9.2.0-gmhcjnh
bpp-core/2.2.0-gcc-9.2.0-qrovxx3 (D) openmpi/3.1.4-gcc-9.2.0-7chkypj r-fs/1.3.1-gcc-9.2.0-moko76e
bpp-phyl/2.2.0-gcc-9.2.0-qf3itbg (D) openmpi/3.1.6-gcc-9.2.0-skcle4f r-gdata/2.18.0-gcc-9.2.0-utmgdvb
bpp-seq/2.2.0-gcc-9.2.0-ipxokok (D) openmpi/4.0.3-gcc-9.2.0-gl6vlx7 (D) r-gdistance/1.3-1-gcc-9.2.0-e7ceeyl
busco/3.0.1-gcc-9.2.0-openmpi-65wij7u openssl/1.1.1d-gcc-9.2.0-c3qm3sp r-ggmap/3.0.0-gcc-9.2.0-lorh5ro (D)
bwa/0.7.17-gcc-9.2.0-lzcib6y orca/4.2.1-gcc-9.2.0-openmpi-lqmvcro (D) r-ggplot2/3.2.0-gcc-9.2.0-sxcv4yi
bzip2/1.0.8-gcc-9.2.0-p6q2can (D) pandoc/2.7.3-gcc-9.2.0-7hxzwvt r-ggplot2/3.2.0-gcc-9.2.0-6pc4fx6 (D)
c-blosc/1.17.0-gcc-9.2.0-2xxpn5s pango/1.41.0-gcc-9.2.0-ab67i5l r-gh/1.0.1-gcc-9.2.0-2ax2ets
...
The above output shows the list of all the available modules. If you want to search for a specific module using module avail, the following command can be used.
Syntax: module -r avail <regex-epression>
module -r avail '^gcc'
Output:
------------------------------------------------------------- /software/custom/el7/modulefiles --------------------------------------------------------------
gcc/4.9.4
--------------------------------------------- /fs1/software/spack/0.13/share/spack/modules/linux-centos7-x86_64 ---------------------------------------------
gcc/8.3.0-gcc-9.2.0-czqf5ch gcc/9.2.0-gcc-4.8.5-wqdecm4
------------------------------------------- /fs1/software/packages/spack/share/spack/modules/linux-centos7-x86_64 -------------------------------------------
gcc/6.3.0-gcc-8.2.0-ceghynm gcc/7.3.0-gcc-8.2.0-v7yiaer gcc/8.2.0-gcc-4.8.5-sxbf4jq gcc/8.2.0-gcc-8.2.0-7cqfkrl
--------------------------------------------------------- /fs1/software/packages/custom/modulefiles ---------------------------------------------------------
gcc/latest gcc/520 gcc/530 (D)
Where:
D: Default Module
Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
Output shows the list of available modules for gcc which starts with the string 'gcc' .
Module Spider
Module spider can also be used to search for any specific module. For example, if you want to search for any particular module like R and only its associated versions, use the below command
Syntax: module spider <software-name>
$ module spider r
r:
Versions:
r/4.1.2-RStudio_Server_2022.02.2.485-SIF
r/4.1.3-RStudio_Server_2022.02.2.485-SIF
r/4.2.0-RStudio_Server_2022.02.2.485-SIF
r/4.2.0-2022a-gcc_12.1.0-dd4egyh
To find other possible module matches execute:
$ module -r spider '.*r.*'
For detailed information about a specific "r" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider r/4.2.0-2022a-gcc_12.1.0-dd4egyh
Syntax:: module spider <full-module-name>
$ module spider r/4.2.0-RStudio_Server_2022.02.2.485-SIF
r: r/4.2.0-RStudio_Server_2022.02.2.485-SIF
You will need to load all module(s) on any one of the lines below before the "r/4.2.0-RStudio_Server_2022.02.2.485-SIF" module is available to load.
custom/2022a
Help:
For help contact your site administrators.
It displays the modules associated with the different versions of R and now you can load the appropriate version by loading the prerequisite modules first.
Module Load
To load a specific version of the perl from the list of available modules, you can do it by running the module load command. The syntax for the module load command is the following.
Syntax: module load <software-name>
module load perl/5.26.2
You can also load several modules by typing the module names one by one in a single line. Run the below single-line command to load multiple modules.
Syntax: module load <sofware-name1>…<software-n>
module load perl julia ruby
The above command loads the latest version for perl
, julia
and ruby
because the version isn’t specified. To check the list of loaded modules, module list
command is used which is discussed in the next section.
Press the tab key in your keyboard to complete the software names and versions. The names sometimes contain special characters, spaces and rather than typing the complicated filename, you can start typing the beginning of the name, and then hit tab key afterward. |
Module List
The module list
command helps to list all the currently loaded modules in your environment.
Syntax: module list
module list
Output:
Currently Loaded Modules:
1) perl/5.30.0-gcc-9.2.0-h7kut76 2) julia/1.4.2-gcc-9.2.0-ncvp5em 3) ruby/2.6.2-gcc-9.2.0-vbzxuyf
The output shows the list of currently loaded modules.
Module Unload
To remove a specific module which you no longer need or you want to discard it, run the below command.
Syntax: module unload <software-name>
module unload perl
Output:
Currently Loaded Modules:
1) ruby/2.6.2-gcc-9.2.0-vbzxuyf 2) mpi/openmpi-x86_64
Module perl
has been successfully removed from your environment.
You have to run the module list command after the unload command to view the list of the currently loaded modules. |
Module Purge
If you want to get rid of all the loaded modules in your environment, use the below command
Syntax: module purge
module purge
Output:
No modules loaded
The above output after running the module purge
and list
command shows that all the loaded modules have been removed from the environment successfully.
Note that Module unload is used to remove a specific module from your environment and module purge is used to get rid of all the modules. |
Module Help
This command helps to explore all the options, commands and arguments associated with the module command.
Syntax: module help
module help
Output:
Usage: module [options] sub-command [args ...]
Options:
-h -? -H --help This help message
-s availStyle --style=availStyle Site controlled avail style: system (default: system)
--regression_testing Lmod regression testing
-D Program tracing written to stderr
--debug=dbglvl Program tracing written to stderr (where dbglvl is a number 1,2,3)
--pin_versions=pinVersions When doing a restore use specified version, do not follow defaults
-d --default List default modules only when used with avail
-q --quiet Do not print out warnings
--expert Expert mode
-t --terse Write out in machine readable format for commands: list, avail, spider, savelist
--initial_load loading Lmod for first time in a user shell
--latest Load latest (ignore default)
--ignore_cache Treat the cache file(s) as out-of-date
--novice Turn off expert and quiet flag
--raw Print modulefile in raw output when used with show
-w twidth --width=twidth Use this as max term width
...
...
...
Picking Modules When There are Multiple Directories in MODULEPATH
When there are multiple directories specified in MODULEPATH, the rules get more complicated on what module file to load. Lmod uses the following rules to locate a module file:
-
If the user specify the full name of a module, the exact module will load.
As an example, suppose you have the following module tree:
---------- /home/user/modulefiles ----------- abc/11.1 abc/11.2 ---------- /opt/apps/modulefiles ------------ abc/10.1 ---------- /opt/apps/mfiles ----------------- abc/12.0 abc/12.1 (D) abc/12.2
If a user type:
module load abc/10.1
This will load
abc/10.1
module. -
If the user didn’t specify the version (only the name). A marked default which will be loaded.
For example, when a user type:
module load abc
This will load
abc/12.1
, because it’s marked as default. -
If the user specified part of the version, and none of it matches the default, then the module with highest version will be selected.
For example:
module load abc/11
Then
abc/11.2
will be loaded because it’s the highest of theabc/11.*
modulefiles.This work doesn’t work is if a version matches is marked default. For example:
module load abc/12
Then
abc/12.1
will load notabc/12.2
because12.1
is the marked defaultFor information how Lmod marks a version as default, see Lmod documentation