Search and Use Modules
You can use Modules on Discovery to use available Software. To deal with modules on Discovery, you need to make sure that the Module is available. If the Module is available, you can load it in your environment and use it to run your jobs.
After completing this tutorial, you’ll be able to search for the intended Module. Then, you’ll be able to load the Module in environment.
Search for Modules
The first step in using a software module is searching for it. You should use the module spider command to search for modules and find what are available versions. For example, Discovery has different versions of Python.
To search if Python is available and what versions are available, run:
module spider python
Output:
--------------------------------------------------------------------------------------------------------------------------------------------------------------
python:
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Versions:
python/3.9.12-2022a-gcc_12.1.0-x3yusmt
python/3.9.12-2022a-gcc_12.1.0-ys2veed
python/3.10.8-2023a-gcc_7.5.0-enoj4ib
python/3.10.8-2023a-gcc_12.2.0-hvrnktz
--------------------------------------------------------------------------------------------------------------------------------------------------------------
For detailed information about a specific "python" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider python/3.10.8-2023a-gcc_12.2.0-hvrnktz
--------------------------------------------------------------------------------------------------------------------------------------------------------------
The above output shows that Python is available with different versions. You pick the version of the Module you are looking for.
Use Modules
To be able use a software through Modules, you need to load the module in you environment. To load a module, use the module load command.
Try to load a specific version of Python , python/3.9.12-2022a-gcc_12.1.0-x3yusmt for example, type:
$ module load python/3.9.12-2022a-gcc_12.1.0-x3yusmt
Lmod has detected the following error: These module(s) or extension(s) exist but can't be loaded as requested:
"python/3.9.12-2022a-gcc_12.1.0-x3yusmt"
Try: "module spider python/3.9.12-2022a-gcc_12.1.0-x3yusmt" to see how to load the module(s).
You can see the module didn’t load successfully. This is because the module depends on other Modules, and those modules should be loaded first.
To find what are the dependencies of a module, you can use the module spider command against the intended version of the module.
For example, to find the dependencies of the python/3.9.12-2022a-gcc_12.1.0-x3yusmt
module, run:
module spider python/3.9.12-2022a-gcc_12.1.0-x3yusmt
Output:
--------------------------------------------------------------------------------------------------------------------------------------------------------------
python: python/3.9.12-2022a-gcc_12.1.0-x3yusmt
--------------------------------------------------------------------------------------------------------------------------------------------------------------
You will need to load all module(s) on any one of the lines below before the "python/3.9.12-2022a-gcc_12.1.0-x3yusmt" module is available to load.
spack/2022a gcc/12.1.0-2022a-gcc_8.5.0-ivitefn
Help:
The Python programming language.
The above output shows that the dependencies of the Module that you tried to load. To be able to load your Module, you must get these modules loaded first. Then you can load your intended module.
module load spack/2022a gcc/12.1.0-2022a-gcc_8.5.0-ivitefn
Then, you load your module:
module load python/3.9.12-2022a-gcc_12.1.0-x3yusmt
Now you can check the Python version and confirm that the Python module loaded successfully:
$ python3 --version
Python 3.9.12
Reset Modules
After loading different modules in your envirounment, you may need to restore the your environment to a predefined state, typically the state upon login, or a state defined by system administrators as, the default. This can be accomplished using the module reset command. This command clears any modifications to the environment made by previously loaded modules, and reinitializes the environment by loading a set of default modules.
For example, in the previous section you loaded three modules spack/2022a
, gcc/12.1.0-2022a-gcc_8.5.0-ivitefn
, and python/3.9.12-2022a-gcc_12.1.0-x3yusmt
.
$ module list
Currently Loaded Modules:
1) os/rhel_8 (S) 3) gcc/12.1.0-2022a-gcc_8.5.0-ivitefn
2) spack/2022a 4) python/3.9.12-2022a-gcc_12.1.0-x3yusmt
Where:
S: Module is Sticky, requires --force to unload or purge
To reset your environment to default, run:
module reset
Check the loaded modules using the module list command:
$ module list
Currently Loaded Modules:
1) os/rhel_8 (S)
Where:
S: Module is Sticky, requires --force to unload or purge
The output shows that the environment has the default modules, all other loaded modules are unloaded.
Default Modules
Lmod uses the directories listed in MODULEPATH to find the modulefiles to load. Each Model can have a default Module that’s loaded if the user didn’t specify the version of the Module.
MODULEPATH
MODULEPATH is an environment variable used by Lmod to locate module files. It’s a colon-separated list of directories that contain module files.
When a user loads a module, Lmod searches for the module file in each directory listed in MODULEPATH.
Picking Modules When There are Multiple Directories in MODULEPATH
When there are multiple directories specified in MODULEPATH, the rules get more complicated on what module file to load. Lmod uses the following rules to locate a module file:
-
If the user specify the full name of a module, the exact module will load.
As an example, suppose you have the following module tree:
---------- /home/user/modulefiles ----------- abc/11.1 abc/11.2 ---------- /opt/apps/modulefiles ------------ abc/10.1 ---------- /opt/apps/mfiles ----------------- abc/12.0 abc/12.1 (D) abc/12.2
If a user type:
module load abc/10.1
This will load
abc/10.1
module. -
If the user didn’t specify the version (only the name). A marked default which will be loaded.
For example, when a user type:
module load abc
This will load
abc/12.1
, because it’s marked as default. -
If the user specified part of the version, and none of it matches the default, then the module with highest version will be selected.
For example:
module load abc/11
Then
abc/11.2
will be loaded because it’s the highest of theabc/11.*
modulefiles.This doesn’t work as if a version matches the Module marked default. For example:
module load abc/12
Then
abc/12.1
won’t loadabc/12.2
because12.1
is the marked defaultFor information how Lmod marks a version as default, see Lmod documentation
MODULEPATH Precedence
The precedence of directories in MODULEPATH is crucial because it determines the order in which modulefiles are found and loaded, impacting which versions of software are available to the user by default. The directories listed in MODULEPATH are searched in the order they are listed. This means that if two modules with the same name exist in two different directories in MODULEPATH, the module in the directory listed first will be found and used first. For example, consider the following value of MODULEPATH:
/home/modulefiles:/project/modulefiles
If there are modules with the same name in both /home/modulefiles
and /project/modulefiles
, the versions in /home/modulefiles
will be found first because it’s listed first in MODULEPATH.
Modifying MODULEPATH
When you add a directory to MODULEPATH, you can add it (at the beginning or end) and this affects the precedence. You can modify the MODULEPATH through one of the following ways:
module use
Command
The module use command is used to add directories to the MODULEPATH in your environment. When you add a directory using this command, you make all the modulefiles in that directory available for loading into your environment.
Syntax:
module use /path/to/modules/directory
For example to add /home/packages/modulefiles
to you MODULEPATH, run:
module use /home/packages/modulefiles
module use command affects only the current shell session. If you open a new terminal or log in again, you’ll need to re-run the module use command to add the directory back to the MODULEPATH.
|
Shell Commands
If you want to manipulate the MODULEPATH environment variable directly using shell commands, you can do so in your current shell session. These changes will only persist for the duration of the session and won’t affect other sessions or terminals.
You can either add a module path to MODULEPATH
either at the beginning. Adding a path at the beginning makes it to be searched first, if there are anther paths containing the same software module.
To add /home/packages/modulefiles
to the beginning of the existing MODULEPATH (giving it higher precedence), run:
export MODULEPATH=/home/packages/modulefiles:${MODULEPATH}
To add /home/packages/modulefiles
to the end of the existing MODULEPATH (giving it lower precedence), run:
export MODULEPATH=${MODULEPATH}:/home/packages/modulefiles
Configuration Files
To ensures that every time you login or open a new shell session that a module path has the highest precedence over the others, you can configure MODULEPATH within your configuration files, for example .bashrc
file.
For example, to add /home/packages/modulefiles
to the beginning of the existing MODULEPATH (giving it higher precedence), and make persist across terminal sessions and system reboots, follow the next steps:
-
Edit
.bashrc
file using any text editor, such asvim
.
vim ~/.bashrc
-
Add the following line to the
.bashrc
file, and then save and exit:
export MODULEPATH=/home/packages/modulefiles:${MODULEPATH}
-
After editing and saving your .bashrc file, you need to apply the changes. You can either close and reopen your terminal or source the .bashrc file directly in any existing terminal window:
source ~/.bashrc
To verify the changes, echo the MODULEPATH variable and check if the path /home/packages/modulefiles
is the first value.
echo $MODULEPATH