Search and Use Modules

You can use Modules on Discovery to use available Software. To deal with modules on Discovery, you need to make sure that the Module is available. If the Module is available, you can load it in your environment and use it to run your jobs.

After completing this tutorial, you’ll be able to search for the intended Module. Then, you’ll be able to load the Module in environment.

Search for Modules

The first step in using a software module is searching for it. You should use the module spider command to search for modules and find what are available versions. For example, Discovery has different versions of Python.

To search if Python is available and what versions are available, run:

module spider python

Output:

  --------------------------------------------------------------------------------------------------------------------------------------------------------------
  python:
--------------------------------------------------------------------------------------------------------------------------------------------------------------
     Versions:
        python/3.9.12-2022a-gcc_12.1.0-x3yusmt
        python/3.9.12-2022a-gcc_12.1.0-ys2veed
        python/3.10.8-2023a-gcc_7.5.0-enoj4ib
        python/3.10.8-2023a-gcc_12.2.0-hvrnktz

--------------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "python" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider python/3.10.8-2023a-gcc_12.2.0-hvrnktz
--------------------------------------------------------------------------------------------------------------------------------------------------------------

The above output shows that Python is available with different versions. You pick the version of the Module you are looking for.

Use Modules

To be able use a software through Modules, you need to load the module in you environment. To load a module, use the module load command.

Try to load a specific version of Python , python/3.9.12-2022a-gcc_12.1.0-x3yusmt for example, type:

$ module load python/3.9.12-2022a-gcc_12.1.0-x3yusmt
Lmod has detected the following error:  These module(s) or extension(s) exist but can't be loaded as requested:
"python/3.9.12-2022a-gcc_12.1.0-x3yusmt"
   Try: "module spider python/3.9.12-2022a-gcc_12.1.0-x3yusmt" to see how to load the module(s).

You can see the module didn’t load successfully. This is because the module depends on other Modules, and those modules should be loaded first.

To find what are the dependencies of a module, you can use the module spider command against the intended version of the module. For example, to find the dependencies of the python/3.9.12-2022a-gcc_12.1.0-x3yusmt module, run:

module spider python/3.9.12-2022a-gcc_12.1.0-x3yusmt

Output:

--------------------------------------------------------------------------------------------------------------------------------------------------------------
  python: python/3.9.12-2022a-gcc_12.1.0-x3yusmt
--------------------------------------------------------------------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "python/3.9.12-2022a-gcc_12.1.0-x3yusmt" module is available to load.

      spack/2022a  gcc/12.1.0-2022a-gcc_8.5.0-ivitefn

    Help:
      The Python programming language.

The above output shows that the dependencies of the Module that you tried to load. To be able to load your Module, you must get these modules loaded first. Then you can load your intended module.

module load spack/2022a  gcc/12.1.0-2022a-gcc_8.5.0-ivitefn

Then, you load your module:

module load python/3.9.12-2022a-gcc_12.1.0-x3yusmt

Now you can check the Python version and confirm that the Python module loaded successfully:

$ python3 --version
Python 3.9.12

Reset Modules

After loading different modules in your envirounment, you may need to restore the your environment to a predefined state, typically the state upon login or a state defined by system administrators as the default. This can be accomplished using the module reset command. This command clears any modifications to the environment made by previously loaded modules, and reinitializes the environment by loading a set of default modules.

For example, in the previous section you loaded three modules spack/2022a, gcc/12.1.0-2022a-gcc_8.5.0-ivitefn, and python/3.9.12-2022a-gcc_12.1.0-x3yusmt.

$ module list

Currently Loaded Modules:
  1) os/rhel_8   (S)   3) gcc/12.1.0-2022a-gcc_8.5.0-ivitefn
  2) spack/2022a       4) python/3.9.12-2022a-gcc_12.1.0-x3yusmt

  Where:
   S:  Module is Sticky, requires --force to unload or purge

To reset your environment to default, run:

module reset

Check the loaded modules using the module list command:

$ module list

Currently Loaded Modules:
  1) os/rhel_8 (S)

  Where:
   S:  Module is Sticky, requires --force to unload or purge

The output shows that the environment has the default modules, all other loaded modules are unloaded.

Default Modules

Lmod uses the directories listed in MODULEPATH to find the modulefiles to load. Each Model can have a default Module that’s loaded if the user didn’t specify the version of the Module.

MODULEPATH

MODULEPATH is an environment variable used by Lmod to locate module files. It’s a colon-separated list of directories that contain module files.

When a user loads a module, Lmod searches for the module file in each directory listed in MODULEPATH.

Picking Modules When There are Multiple Directories in MODULEPATH

When there are multiple directories specified in MODULEPATH, the rules get more complicated on what module file to load. Lmod uses the following rules to locate a module file:

If the user specify the full name of a module, the exact module will load.

As an example, suppose you have the following module tree:

---------- /home/user/modulefiles -----------
abc/11.1  abc/11.2


---------- /opt/apps/modulefiles ------------
abc/10.1

---------- /opt/apps/mfiles -----------------
abc/12.0   abc/12.1 (D) abc/12.2

If a user type:

module load abc/10.1

This will load abc/10.1 module.

If the user didn’t specify the version (only the name). A marked default which will be loaded.
For example, when a user type:
module load abc
This will load abc/12.1, because it’s marked as default.
If the user specified part of the version, and none of it matches the default, then the module with highest version will be selected.
For example:
module load abc/11
Then abc/11.2 will be loaded because it’s the highest of the abc/11.* modulefiles.

This doesn’t work as if a version matches the Module marked default. For example:
module load abc/12
Then abc/12.1 won’t load abc/12.2 because 12.1 is the marked default
For information how Lmod marks a version as default, see Lmod documentation

MODULEPATH Precedence

The precedence of directories in MODULEPATH is crucial because it determines the order in which modulefiles are found and loaded, impacting which versions of software are available to the user by default. The directories listed in MODULEPATH are searched in the order they are listed. This means that if two modules with the same name exist in two different directories in MODULEPATH, the module in the directory listed first will be found and used first. For example, consider the following value of MODULEPATH:

/home/modulefiles:/project/modulefiles

If there are modules with the same name in both /home/modulefiles and /project/modulefiles, the versions in /home/modulefiles will be found first because it’s listed first in MODULEPATH.

Modifying MODULEPATH

When you add a directory to MODULEPATH, you can add it (at the beginning or end) and this affects the precedence. You can modify the MODULEPATH through one of the following ways:

`module use` Command

The module use command is used to add directories to the MODULEPATH in your environment. When you add a directory using this command, you make all the modulefiles in that directory available for loading into your environment.

Syntax:

module use /path/to/modules/directory

For example to add /home/packages/modulefiles to you MODULEPATH, run:

module use /home/packages/modulefiles

module use command affects only the current shell session. If you open a new terminal or log in again, you’ll need to re-run the module use command to add the directory back to the MODULEPATH.

Shell Commands

If you want to manipulate the MODULEPATH environment variable directly using shell commands, you can do so in your current shell session. These changes will only persist for the duration of the session and won’t affect other sessions or terminals.

You can either add a module path to MODULEPATH either at the beginning. Adding a path at the beginning makes it to be searched first, if there are anther paths containing the same software module.

To add /home/packages/modulefiles to the beginning of the existing MODULEPATH (giving it higher precedence), run:

export MODULEPATH=/home/packages/modulefiles:${MODULEPATH}

To add /home/packages/modulefiles to the end of the existing MODULEPATH (giving it lower precedence), run:

export MODULEPATH=${MODULEPATH}:/home/packages/modulefiles

Configuration Files

To ensures that every time you login or open a new shell session that a module path has the highest precedence over the others, you can configure MODULEPATH within your configuration files, for example .bashrc file.

For example, to add /home/packages/modulefiles to the beginning of the existing MODULEPATH (giving it higher precedence), and make persist across terminal sessions and system reboots, follow the next steps:

Edit .bashrc file using any text editor, such as vim.

vim ~/.bashrc

Add the following line to the .bashrc file, and then save and exit:

export MODULEPATH=/home/packages/modulefiles:${MODULEPATH}

After editing and saving your .bashrc file, you need to apply the changes. You can either close and reopen your terminal or source the .bashrc file directly in any existing terminal window:

source ~/.bashrc

To verify the changes, echo the MODULEPATH variable and check if the path /home/packages/modulefiles is the first value.

echo $MODULEPATH