Conda

Conda is an open-source package management system and environment management system that runs on Windows, macOS, and Linux. Conda quickly installs, runs, and updates packages and their dependencies. Conda as a package manager helps you find and install packages.

For more information see conda documentation

How to Install Conda?

You can use the SStack tool to install Conda to manage software on discovery.

For example, to install Conda stack named 2024A in your home directory, run command:

sstack install -t conda -n 2024A

This creates a module file that you can load in your environment to be able to use the stack to manage software, as shown in the following:

module use "/home/tahat/sstack/modules"
module load conda/2024A

To check if conda is loaded and what version is it, run the following command:

conda --version

Output

conda 4.13.0

How to Use Conda to Manage Software?

You can use conda to install packages in your environment. For example, to install matplotlib, run command:

conda install matplotlib

Once the installation is completed successfully, you can check if the package is available. To check if matplotlib is available, run command:

conda list matplotlib

Output

# packages in environment at /home/tahat/sstack/stacks/conda/2024A:
#
# Name                    Version                   Build  Channel
matplotlib                3.5.2           py310hff52083_0    conda-forge
matplotlib-base           3.5.2           py310h5701ce4_0    conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge

Create a Virtual Environmnet

Conda allows you to create a new conda environment. The simplest way you can create a new conda environment is like so:

conda create -n <env-name>

For example, to create a new environment named myproject, run command:

conda create -n myproject

To check all environmnets, run command:

conda env list

Output

# conda environments:
#
base                  *  /home/tahat/sstack/stacks/conda/2022A
myproject                /home/tahat/sstack/stacks/conda/2022A/envs/myproject

The output shows that the created environmnet is available and you can use it. It also that you are in the base environmnet.

Create a Module for Virtual Environment

Once you create a conda environmnet, you can create a module file for it. This will add the environment to the module system where you can load it in your enviroument. A module named base was created when the conda stack was installed. To see if the base module is available, run command:

module avail base

-------------------------------------------------------- /home/tahat/sstack/stacks/conda/2024A/modules ---------------------------------------------------------
   base

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

(1) Automatically Generate Module Files

In addition to base, you can create module files for other environments by updating your SStack Conda with a --du flag. In this case, modulefiles will be created automatically.

Syntax

sstack update [OPTIONS]

OPTIONS:
- Required:
  - -t or --type: Type of stack (here it should be conda).
  - -n or --name: Name of stack (test-conda/ my-SStack-conda / etc..).
  - --du Don’t Update Stack Software.

For example:

sstack update -n test-conda -t conda --du

Output

conda/test-conda: Sucessfully Regenerated Module File!

+------------+-------+---------+------------------------------------------+
| Name       | Type  | Version | Path                                     |
+------------+-------+---------+------------------------------------------+
| test-conda | conda | 24.7.1  | /fs1/home/yuxiwang/sstack/rhel_8/stacks/ |
|            |       |         | conda/test-conda                         |
+------------+-------+---------+------------------------------------------+

Module Setup/Load Commands:

module use "/fs1/home/yuxiwang/sstack/rhel_8/modules"
module load conda/test-conda

After that, you can check if the modules are ready by:

module avail

To see the module for conda environmnet, conda needs to be loaded. In my example, conda/test-conda is loaded before using module avail.

Output

-------------------------------------- /fs1/home/yuxiwang/sstack/rhel_8/stacks/conda/test-conda/modules --------------------------------------
   base    my_env

-------------------------------------- /fs1/home/yuxiwang/sstack/rhel_8/modules --------------------------------------
   conda/test-conda (L)    spack/my_spack    spack/test-spack_2    spack/test-spack

-------------------------------------- /fs1/software/sstack/rhel_8/modules --------------------------------------
   conda/2022a    conda/2023a    conda/2024a (D)    custom/2022a    custom/2023a    custom/2024a (D)    spack/2022a    spack/2023a (D)    sstack/main (L)

-------------------------------------- /etc/modulefiles --------------------------------------
   pmix/2.2.5    pmix/3.2.3    pmix/4.1.2 (D)

-------------------------------------- /fs1/software/sstack/modules --------------------------------------
   os/centos_7_test (S)    os/centos_7 (S)    os/rhel_8_test (S)    os/rhel_8 (S,L,D)

-------------------------------------- /usr/share/lmod/lmod/modulefiles/Core --------------------------------------
   lmod    settarg

  Where:
   S:  Module is Sticky, requires --force to unload or purge
   L:  Module is loaded
   D:  Default Module

If the avail list is too long consider trying:

"module --default avail" or "ml -d av" to just list the default modules.
"module overview" or "ml ov" to display the number of modules for each name.

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

(2) Manually Generate Module Files

You can also manually create the modulefiles.

You need to create module file having the same name of your created environmnet in the same path of base module file (base.lua). To find where to add the module file and what contents you need to add to it, run command:

module --raw show base

Output

------------------------------------------------------------------------------------------------------------------------------------------------------------
   /home/tahat/sstack/stacks/conda/2024A/modules/base.lua:
------------------------------------------------------------------------------------------------------------------------------------------------------------
help([[For help contact hpc-team@nmsu.edu]])
whatis([[Name : conda Base Env]])
whatis([[Version: Latest]])
whatis([[Target: x86_64]])
whatis([[Short description :  This is a redundant module to load the base conda environment as an example.]])

family("conda_env")

-- Modify String In Quotes To Match Desired Environment Name or Path
local env_name = "base"
-- Don't forget to update line 2/3 aswell to update the Module Name/Version

execute{cmd="conda activate " .. env_name,modeA={"load"}}
execute{cmd="conda deactivate",modeA={"unload"}}

The above output shows that base.lua is in /home/tahat/sstack/stacks/conda/2024A/modules. You need to create a module file myproject.lua in the same path, as follows:

touch /home/tahat/sstack/stacks/conda/2024A/modules/myproject.lua

Now, you need to copy the content of the base.lua file into myproject.lua.

cp /home/tahat/sstack/stacks/conda/2024A/modules/base.lua /home/tahat/sstack/stacks/conda/2024A/modules/myproject.lua

Now you need to update myproject.lua to have the correct environment name env_name as commented in line 9 in the file.

vim /home/tahat/sstack/stacks/conda/2024A/modules/myproject.lua

Update line 2 and 10, with the correct env-name.

help([[For help contact hpc-team@nmsu.edu]])
whatis([[Name : conda myproject Env]])
whatis([[Version: Latest]])
whatis([[Target: x86_64]])
whatis([[Short description :  This is a redundant module to load the base conda environment as an example.]])

family("conda_env")

-- Modify String In Quotes To Match Desired Environment Name or Path
local env_name = "myproject"
-- Don't forget to update line 2/3 aswell to update the Module Name/Version

execute{cmd="conda activate " .. env_name,modeA={"load"}}
execute{cmd="conda deactivate",modeA={"unload"}}

Check if the created module is available:

module avail myproject

Output

-------------------------------------------------------- /home/tahat/sstack/stacks/conda/2024A/modules ---------------------------------------------------------
   myproject

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

For more information see conda documentation